[Mesa-dev] Bug in picking mode

[Holger Waechtler]

Hi Brian, hi everybody else,


Mesa reports wrong depth values in picking mode. Try the pickdepth demo
from the Red Book examples to see.

The problem is in feedback.c (gl_select_triangle, gl_select_line and
gl_select_points):

      gl_update_hitflag( ctx, VB->Win.data[v0][3] / DEPTH_SCALE );
      gl_update_hitflag( ctx, VB->Win.data[v1][3] / DEPTH_SCALE );
      gl_update_hitflag( ctx, VB->Win.data[v2][3] / DEPTH_SCALE );

should be:

      gl_update_hitflag( ctx, VB->Win.data[v0][2] / DEPTH_SCALE );
      gl_update_hitflag( ctx, VB->Win.data[v1][2] / DEPTH_SCALE );
      gl_update_hitflag( ctx, VB->Win.data[v2][2] / DEPTH_SCALE );


Complete patch is attached. Could you check it in, Brian ?


- Holger

*** feedback.c.old      Mon Jan 17 14:08:35 2000
--- feedback.c  Mon Jan 17 14:09:16 2000
***************
*** 297,305 ****
     struct vertex_buffer *VB = ctx->VB;
  
     if (gl_cull_triangle( ctx, v0, v1, v2, 0 )) {
!       gl_update_hitflag( ctx, VB->Win.data[v0][3] / DEPTH_SCALE );
!       gl_update_hitflag( ctx, VB->Win.data[v1][3] / DEPTH_SCALE );
!       gl_update_hitflag( ctx, VB->Win.data[v2][3] / DEPTH_SCALE );
     }
  }
  
--- 297,305 ----
     struct vertex_buffer *VB = ctx->VB;
  
     if (gl_cull_triangle( ctx, v0, v1, v2, 0 )) {
!       gl_update_hitflag( ctx, VB->Win.data[v0][2] / DEPTH_SCALE );
!       gl_update_hitflag( ctx, VB->Win.data[v1][2] / DEPTH_SCALE );
!       gl_update_hitflag( ctx, VB->Win.data[v2][2] / DEPTH_SCALE );
     }
  }
  
***************
*** 309,316 ****
  {
     struct vertex_buffer *VB = ctx->VB;
  
!    gl_update_hitflag( ctx, VB->Win.data[v0][3] / DEPTH_SCALE );
!    gl_update_hitflag( ctx, VB->Win.data[v1][3] / DEPTH_SCALE );
  }
  
  void gl_select_points( GLcontext *ctx, GLuint first, GLuint last )
--- 309,316 ----
  {
     struct vertex_buffer *VB = ctx->VB;
  
!    gl_update_hitflag( ctx, VB->Win.data[v0][2] / DEPTH_SCALE );
!    gl_update_hitflag( ctx, VB->Win.data[v1][2] / DEPTH_SCALE );
  }
  
  void gl_select_points( GLcontext *ctx, GLuint first, GLuint last )
***************
*** 320,326 ****
  
     for (i=first;i<=last;i++) 
        if (VB->ClipMask[i]==0) 
!          gl_update_hitflag( ctx, VB->Win.data[i][3] / DEPTH_SCALE);
  }
  
  
--- 320,326 ----
  
     for (i=first;i<=last;i++) 
        if (VB->ClipMask[i]==0) 
!          gl_update_hitflag( ctx, VB->Win.data[i][2] / DEPTH_SCALE);
  }