3d picking with gles

Question

I'm trying to do 3d picking when user taps android phone's screen.

It works sometimes but when i rotate the screen --> the ray doesnt hit the sphere

I'll paste the source code with what i have until now :

a) class renderer.java:

... // variables used
    public static   picking gPicker     = null;   
    public static   vector3 campos      = new vector3(0,0, 500);
    public static   vector3 sphpos      = new vector3(60,-40,0);
...




... // this function is called when user taps Screen 
public void screenWasTapped(vector2 touchcoords) 
{
    // get current time
    now = System.currentTimeMillis();

    // can shoot/tap every 100 msecs., not earlyer.
    if (now > mLastFiredTime + mInterval) 
    {
        // do we have a valid GL context?
        if(gGL!=null)
        {   
            // screen was tapped , so call "pick" function wich will check if ray hits sphere wich is located at "sphpos".
            gPicker.pick(gGL, gScreenWidth, gScreenHeight, touchcoords.x, touchcoords.y);

            // update last time
            mLastFiredTime = now;
        }
    }
}
....

These are the GL OnSurfaceChanged settings :

public void onSurfaceChanged(GL10 gl, int width, int height) 
{
    // To prevent divide by zero
    if (height == 0) height = 1;   
    float aspect = (float) width/height;

    // Set the viewport (display area) to cover the entire window
    gl.glViewport(0, 0, width, height);

    // Setup perspective projection, with aspect ratio matches viewport
    gl.glMatrixMode(GL10.GL_PROJECTION);
    gl.glLoadIdentity();                
    // Use perspective projection
    GLU.gluPerspective(gl, 60.0f, aspect, 0.1f, 5000.f);

    gl.glMatrixMode(GL10.GL_MODELVIEW); 
    gl.glLoadIdentity();                

    // Fast Perspective Calculations
    gl.glHint(GL10.GL_PERSPECTIVE_CORRECTION_HINT, GL10.GL_FASTEST); 
    gl.glEnable(GL10.GL_TEXTURE_2D);


    // Enable GL Blending.
    gl.glEnable(GL10.GL_BLEND);
    gl.glBlendFunc(GL10.GL_ONE, GL10.GL_ONE_MINUS_SRC_ALPHA);

    // Drawing order & culling face settings.
    gl.glFrontFace(GL10.GL_CW);     // Front face in counter-clockwise orientation
    gl.glEnable(GL10.GL_CULL_FACE);     // Enable cull face
    gl.glCullFace(GL10.GL_BACK);        // Cull the back face (don't display)


    gScreenWidth  = width;
    gScreenHeight = height; 
}

...and this is how i DRAW the scene:

public void onDrawFrame(GL10 gl) 
{
    gl.glClear(GL10.GL_COLOR_BUFFER_BIT | GL10.GL_DEPTH_BUFFER_BIT);

    gTimer.startupdate();
    float dt = gTimer.getElapsed();

    gl.glLoadIdentity();
    gl.glTranslatef(-campos.x, -campos.y, -campos.z);
    gl.glRotatef(tableRotXAngle, 1, 0, 0); 
    gl.glRotatef(tableRotYAngle, 0, 1, 0f);


        gl.glColor4f(0,1,1,1);
        gl.glPushMatrix();
        gl.glTranslatef(sphpos.x,sphpos.y,sphpos.y); 
            newgame.gPyramid.render(gl);            // draw pyramid at 3d location specified by "sphpos"
        gl.glPopMatrix();
        gl.glColor4f(1,1,1,1);



        doGameAI(dt);
        doDisplayHUD(gl);

    gTimer.stopupdate();
    gl.glFlush();
}

IMPORTANT NOTE here: it's a arcball camera rotation, the camera rotates looking always at the center coordinates (0,0,0) and translated 500 units back (see campos initialisation).

b) class "picker.java" :

    public class picking 
    {

   picking(GL10 gl)
   {
    MatrixGrabber mg = new MatrixGrabber();
    mg.getCurrentState(gl);
   }

(UPDATED)
public void pick(GL10 gl, int width, int height, float xTouch, float yTouch) 
{
    MatrixGrabber mg = new MatrixGrabber();
    mg.getCurrentState(gl);

    int[] viewport = {0, 0, width, height};

    float[] nearCoOrds  = new float[3];
    float[] farCoOrds   = new float[3];
    float[] temp        = new float[4];
 // float[] temp2       = new float[4];

    float winx = xTouch, winy =(float)viewport[3] - yTouch;

    displayMatrix(mg.mModelView);
    displayMatrix(mg.mProjection);



    int result = GLU.gluUnProject(winx, winy, 0.0f, mg.mModelView, 0, mg.mProjection, 0, viewport, 0, temp, 0);
    // Matrix.multiplyMV(temp2, 0, mg.mModelView, 0, temp, 0);
    if(result == GL10.GL_TRUE){
        nearCoOrds[0] = temp[0] / temp[3];
        nearCoOrds[1] = temp[1] / temp[3];
        nearCoOrds[2] = temp[2] / temp[3];

    }



    Log.d("redwing", " ---> near pos "  + Float.toString(nearCoOrds[0]) + "," 
                                        + Float.toString(nearCoOrds[1]) + "," 
                                        + Float.toString(nearCoOrds[2]));

    result = GLU.gluUnProject(winx, winy, 1.0f , mg.mModelView, 0, mg.mProjection, 0, viewport, 0, temp, 0);
    // Matrix.multiplyMV(temp2,0, mg.mModelView, 0, temp, 0);
    if(result == GL10.GL_TRUE){
        farCoOrds[0] = temp[0] / temp[3];
        farCoOrds[1] = temp[1] / temp[3];
        farCoOrds[2] = temp[2] / temp[3];
    }

    Log.d("redwing", " ---> far pos "   + Float.toString(farCoOrds[0]) + "," 
                                        + Float.toString(farCoOrds[1]) + "," 
                                        + Float.toString(farCoOrds[2]));



    vector3 rayDirection = new vector3(farCoOrds[0]-nearCoOrds[0], farCoOrds[1]-nearCoOrds[1], farCoOrds[2]-nearCoOrds[2]);

    Log.d("redwing", " ---> raylen ="+ Float.toString(rayDirection.getLength()));

    rayDirection.setUnit();

    Log.d("redwing", " ---> dir "   + Float.toString(rayDirection.x) + "," 
                                    + Float.toString(rayDirection.y) + "," 
                                    + Float.toString(rayDirection.z));



    if(rayHitSphere(renderer.campos, rayDirection, renderer.sphpos, 10))
    {
        Log.e("redwing", "****************");
        Log.e("redwing", "ray hits OBJECT!");
        Log.e("redwing", "****************");
    }


}      


boolean rayHitSphere(vector3 rayPos, vector3 rayDir, vector3 sphereCntr, float radius)
{
      vector3 w = rayPos.sub(sphereCntr);   
      float A = rayDir.dot(rayDir);         
      float B = 2*w.dot(rayDir);            
      float C = w.dot(w) - radius*radius;   

      float D = B*B-4.0f*A*C;

      if(D>=0.0f)
      return true;
      else 
      return false;
    }


void displayMatrix(float[] m, String tag)
{
    for(byte i=0;i<4;i++)
    {
        Log.v("redbase "+tag, "mtx" + Byte.toString(i) + " - "  + new DecimalFormat("#.####").format(m[i*4+0])+","
                                                                + new DecimalFormat("#.####").format(m[i*4+1])+","
                                                                + new DecimalFormat("#.####").format(m[i*4+2])+","
                                                                + new DecimalFormat("#.####").format(m[i*4+3]));
    }
}

}

Just don't know where i'm going wrong here .. :( Pleease help !!

     Here are some values from Eclipse console when User tapped screen:
     (UPDATED - i get a Hit!)
    <<< NOT ROTATED SCENE >>
    06-02 12:15:22.611: D/redwing(5524): screen resolution 480x764
    06-02 12:15:22.611: D/redwing(5524): tapped coordinate ( 317.07507,426.2164
    06-02 12:15:22.621: V/redbase(5524): mtx0 - 1,0,0,0
    06-02 12:15:22.631: V/redbase(5524): mtx1 - 0,1,0,0
    06-02 12:15:22.631: V/redbase(5524): mtx2 - 0,0,1,0
    06-02 12:15:22.631: V/redbase(5524): mtx3 - 0,0,-500,1
    06-02 12:15:22.641: V/redbase(5524): mtx0 - 2.7568,0,0,0
    06-02 12:15:22.641: V/redbase(5524): mtx1 - 0,1.7321,0,0
    06-02 12:15:22.641: V/redbase(5524): mtx2 - 0,0,-1,-1
    06-02 12:15:22.651: V/redbase(5524): mtx3 - 0,0,-0.2,0
    06-02 12:15:22.651: D/redwing(5524):  ---> near pos 0.011649653,-0.0066829454,499.9
    06-02 12:15:22.661: D/redwing(5524):  ---> far pos 581.6618,-333.6764,-4493.4097
    06-02 12:15:22.661: D/redwing(5524):  ---> raylen =5038.134
    06-02 12:15:22.661: D/redwing(5524):  ---> dir 0.115449525,-0.06622883,-0.991103
    06-02 12:15:22.661: E/redwing(5524): ****************
    06-02 12:15:22.661: E/redwing(5524): ray hits OBJECT!
    06-02 12:15:22.661: E/redwing(5524): ****************


    <<< ROTATED SCENE!!  UPDATED - Still NO HIT!  >>
    06-02 12:15:30.671: D/redwing(5524): screen resolution 480x764
    06-02 12:15:30.671: D/redwing(5524): tapped coordinate ( 337.04843,440.11392
    06-02 12:15:30.671: V/redbase(5524): mtx0 - 0.9684,0.1802,0.1725,0
    06-02 12:15:30.671: V/redbase(5524): mtx1 - 0,0.6913,-0.7226,0
    06-02 12:15:30.681: V/redbase(5524): mtx2 - -0.2495,0.6997,0.6695,0
    06-02 12:15:30.681: V/redbase(5524): mtx3 - 0,0,-500,1
    06-02 12:15:30.681: V/redbase(5524): mtx0 - 2.7568,0,0,0
    06-02 12:15:30.681: V/redbase(5524): mtx1 - 0,1.7321,0,0
    06-02 12:15:30.681: V/redbase(5524): mtx2 - 0,0,-1,-1
    06-02 12:15:30.681: V/redbase(5524): mtx3 - 0,0,-0.2,0
    06-02 12:15:30.681: D/redwing(5524):  ---> near pos 86.22043,-361.2092,334.655
    06-02 12:15:30.691: D/redwing(5524):  ---> far pos -144.60953,2943.3904,-3498.0571
    06-02 12:15:30.691: D/redwing(5524):  ---> raylen =5065.9
    06-02 12:15:30.691: D/redwing(5524):  ---> dir -0.04556544,0.65232235,-0.7565708

if i dont rotate the scene, then i HAVE a Hit !
but..if i do rotate the scene, then NO hit :(

score 0 · Answer 1 · answered Jun 02 '12 at 08:36

0

Not 100% positive I understand all the code, but it seems to me that you shouldn't be multiplying the result of gluUnproject with the modelview matrix?

If unproject gives the click location in world coordinates, and you are testing against a sphere position in world coordinates, I can't see why you're transforming the unproject by the modelview.

If you do want to convert the clicks to camera space, then your sphere location would need to be transformed as well, which I don't see being done.

answered Jun 02 '12 at 08:36

Tim

35,413
11
95
121

ok, then two steps todo: 1) transform sphere pos with modelview mtx and 2) delete Matrix.multiplyMV(temp2,0, mg.mModelView, 0, temp, 0); lines (there are 2), right ? ill come back now with the result. – redbase Jun 02 '12 at 08:38
@redbase : You should probably do one or the other, doing both will be as equally wrong as what you started with. Try just removing the modelview multiply of the unproject result, and leave the sphere pos as it is. – Tim Jun 02 '12 at 08:56
thanks tim for your answer, ive update above my pick function. please help. Now it seems not to work with both! Right now ill try do one or other as you said... let's see. – redbase Jun 02 '12 at 08:57
sorry for that! i've updated Pick function & results, but still no luck :(. – redbase Jun 02 '12 at 09:20

score 0 · Answer 2 · answered Jun 04 '12 at 00:58

0

From an initial walk-through of your code it looks like its an issue of screen resolution change. Your code seems to not transpose the width and height when the screen is rotated. This could be off, but it stood out as a possible issue.

answered Jun 04 '12 at 00:58

Edward Carmack

341
1
4

3d picking with gles

2 Answers2