[openGL] unproject problem

Hi,

today I was trying to unproject my mouse position to see where it is in world space.
Here it is the unproject function:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
//project is the same exact matrix passed to opengl
Vec3 Unproject( Renderer* renderer, Camera* camera, matrix4x4 project, matrix4x4 backward, Vec2 screenP, r32 distanceFromCameraZ)
{
    //compute ndc X and Y based on screen coordinates
    Vec2 screenCenter = 0.5f * renderer->screenDim;

    r32 ndcX = (screenP.x - screenCenter.x) * (2.0f / renderer->screenDim.x);
    r32 ndcY = (screenP.y - screenCenter.y) * (2.0f / renderer->screenDim.y);
    
    //let's see what's the clipZ of the targetZ
    Vec4 probeZ = V4( camera->Position -worldDistanceFromCameraZ * camera->ZAxis, 1.0f );

    //for debugging, let's force the worldZ of the probe to be on the ground plane, so that our points will always be on the ground plane
    probeZ.z = 0;
    probeZ = project * probeZ;
    r32 clipZ =probeZ.z;

    //"backward" perspective divide to obtain the clip coordinates
    Vec4 clip = clipZ * V4(ndcX, ndcY, 1.0f, 1.0f);

    //"backward" projection to obtain world coordinates
    Vec3 world = (backward * clip).xyz;

    return world;
}



It seems to work fine when the camera is perfectly aligned with the world Z Axis (0, 0, 1): (you can see the red rectangle perfectly fits the screen)

https://ibb.co/iejA9z


but as soon as I start rotating the camera it doesn't work anymore, as it seems it is still using the straight up camera.

https://ibb.co/msPXNK


Assuming the problem is not in the camera matrixes (checked multiple times both the forward and back matrixes), I think the problem should be _somewhere_ in this function, but I really can't see it.

Can you guys give me a little help with this?

Thank you,
Leonardo


Edited by erpeo93 on
Figured it out!

I have to grab the W coordinate from the "forward" projection as well as the Z coordinate:

1
2
3
4
5
6
7
    Vec4 probeZ = V4( camera->Position -worldDistanceFromCameraZ * camera->ZAxis, 1.0f );
    probeZ = project * probeZ;
    r32 clipZ =probeZ.z;
    r32 clipW = probeZ.w;

    //"backward" perspective divide to obtain the clip coordinates
    Vec4 clip = clipW * V4(ndcX, ndcY, clipZ, 1.0f);



Problem solved. :)


I'm assuming
1
matrix4x4 project
is the camera projection matrix.

What is parameter
1
matrix4x4 backward
?
Inverse camera view matrix (camera space to world space transform).

Edited by Mārtiņš Možeiko on