opengl glReadPixels endianness/endless problem
  • Order of the Butterfly
    Order of the Butterfly
    Posts: 186 from 2003/10/23
    hi :D

    for picking an object in my game i use the simple opengl color picking method.

    for now it only work on pc , and all the example founded in internet refer only to little-endian machines.


    first i draw a polygon with this code :
    -------------------------------------
    int nIndex ;
    nIndex = 1000;
    glColor3ub ( nIndex % 256, (nIndex>>8) % 256, (nIndex>>16) % 256);
    glBegin(GL_QUADS);
    glVertex3f( -0.5f, -0.5f, +0.5f);
    glVertex3f( +0.5f, -0.5f, +0.5f);
    glVertex3f( +0.5f, +0.5f, +0.5f);
    glVertex3f( -0.5f, +0.5f, +0.5f);
    glEnd();
    ----------------------------------------------------

    my picking routine is this :
    -----------------------------------------------

    unsigned char *pRGB;

    pRGB = malloc(sizeof(unsigned char)*4);

    glReadPixels( mousex, viewport[3]-mousey, 1, 1, GL_RGBA, GL_UNSIGNED_BYTE, pRGB);

    ColorNumber = (pRGB[ 0 ]) + (pRGB[ 1 ]*256) + (pRGB[ 2 ]*256*256);
    ----------------------------------------
    this routine work only on Pc, on the peg the number returned isn't 1000.

    so there is involved the little/big-endian problem.

    i can't find a solution to obtain the same color number i obtain on the Pc (in this example 1000)

    and no i don't want to study the ray-plane intersection

    anyone want to help ?? please :D
    I'm nerdy in the extreme
    And whiter than sour cream

    White&Nerdy 2006 Al Yankovic
  • »15.06.11 - 22:13
    Profile
  • Order of the Butterfly
    Order of the Butterfly
    Yomgui
    Posts: 348 from 2004/8/31
    From: Québec - Canada
    Taken from blender code:

    Code:

    unsigned int col;

    (...)

    glReadPixels(x, y, 1, 1, GL_RGBA, GL_UNSIGNED_BYTE, &col);

    (...)

    if(ENDIAN_ORDER==B_ENDIAN) SWITCH_INT(col);


    It's not a bug, it's the working way of glReadPixels() by default.
    And now... next project!
  • »16.06.11 - 08:04
    Profile Visit Website
  • Order of the Butterfly
    Order of the Butterfly
    Posts: 186 from 2003/10/23
    no no it's not a bug of glReadPixels, it's a bug in my code :P :P

    i will start to dig the blender source, thank for the hint :D
    I'm nerdy in the extreme
    And whiter than sour cream

    White&Nerdy 2006 Al Yankovic
  • »16.06.11 - 09:16
    Profile
  • Paladin of the Pegasos
    Paladin of the Pegasos
    Jupp3
    Posts: 1193 from 2003/2/24
    From: Helsinki, Finland
    Quote:

    pRGB = malloc(sizeof(unsigned char)*4);

    Why this instead of unsigned char pRGB[4]? I wouldn't use malloc, especially if that's "per-pick"
    Quote:


    glBegin(GL_QUADS);
    glVertex3f( -0.5f, -0.5f, +0.5f);
    glVertex3f( +0.5f, -0.5f, +0.5f);
    glVertex3f( +0.5f, +0.5f, +0.5f);
    glVertex3f( -0.5f, +0.5f, +0.5f);
    glEnd();


    I'd highly recommend using vertex arrays instead of slow, deprecated and often impractical immediate mode. Also, if some day tinygl is improved to support VBO, then you can get some serious speedup from modifying the vertex array code to use VBO instead (it's basically only a few additional function calls around what you have in vertex arrays.)
  • »16.06.11 - 11:41
    Profile Visit Website
  • Order of the Butterfly
    Order of the Butterfly
    Posts: 186 from 2003/10/23
    yes, with the vertex array i can have a speedup, but for now i want the program to work :D (for ex. i haven't splitted the quad in triangle )

    the idea of the non pointer pRGB[4] is good, i don't allocate it per- pick and in this manner i don't need to reverse the endianness.

    but i have another problem :D :D

    here a snippet of the code

    same routine for drawing the quad
    so..
    -------------------
    nIndex=1000;

    glColor3ub ( nIndex % 256, (nIndex>>8) % 256, (nIndex>>16) % 256);
    -------------------
    the picking routine is
    -------------------


    unsigned char pRGB[4];
    glReadPixels(mousex, viewport[3]-mousey, 1, 1, GL_RGBA, GL_UNSIGNED_BYTE, &pRGB);
    ColorNumber = (pRGB[ 0 ]) + (pRGB[ 1 ]<<8) + (pRGB[ 2 ]<<16 );

    printf(" colore %u ,%u ,%u\n",nIndex %256,(nIndex >>8) % 256, (nIndex >>16) % 256);
    printf(" colore %u ,%u ,%u\n",pRGB[ 0 ],pRGB[ 1 ], pRGB[ 2 ]);

    --------------

    an unfortunatly i obtain

    nIndex = 1000;
    ColorNumber = 743;

    colore 232,3,0 <---- nIndex R,G,B
    colore 231,2,0 <---- pRGB R,G,B

    so the RGB fouded lack 1 unity :( (only on the R and G component)

    i dunno why
    I'm nerdy in the extreme
    And whiter than sour cream

    White&Nerdy 2006 Al Yankovic
  • »17.06.11 - 08:50
    Profile
  • Order of the Butterfly
    Order of the Butterfly
    Posts: 186 from 2003/10/23
    if anyone want to dig my source , here it is :D :D
    ------------------------------

    #include <math.h>
    #include <stdio.h>
    #include <stdlib.h>
    #include <string.h>
    #include <GL/gl.h>
    #include "SDL/SDL.h"
    #include "SDL/SDL_opengl.h"


    #define RENDER 1 // No ray-plane intersection (i haven't studied the vectors properly )
    #define SELECT 2 // So old-good color selection

    #define MORPHOS 1
    #define SWITCH_INT(a) { char s_i, *p_i;p_i= (char *)&(a);s_i=p_i[0]; p_i[0]=p_i[3]; p_i[3]=s_i;s_i=p_i[1]; p_i[1]=p_i[2]; p_i[2]=s_i; }

    int mode = RENDER;

    int FRAMES_PER_SECOND = 50; //The frames per second const
    int frame = 0; //Keep track of the current frame
    int startTicks = 0;
    int currentTicks=0;

    int SCREENW=800;
    int SCREENH=600;

    int mousex,mousey;

    unsigned int nIndex;


    void DrawSelection() {

    glDisable(GL_LIGHTING);
    glDisable( GL_TEXTURE_2D );
    nIndex=10000;

    glColor3ub ( nIndex % 256, (nIndex>>8) % 256, (nIndex>>16) % 256);
    glBegin(GL_QUADS);
    glVertex3f( -0.5f, -0.5f, +0.5f);
    glVertex3f( +0.5f, -0.5f, +0.5f);
    glVertex3f( +0.5f, +0.5f, +0.5f);
    glVertex3f( -0.5f, +0.5f, +0.5f);
    glEnd();

    }



    void mouseMove (int x, int y) {

    static int x_pos = 0;
    static int y_pos = 0;

    x_pos = x;
    y_pos = y;

    }

    void ProcessPick() {

    unsigned int col;
    GLint viewport[4];

    glGetIntegerv(GL_VIEWPORT,viewport);

    glReadPixels( mousex, viewport[3]-mousey, 1, 1, GL_RGB, GL_UNSIGNED_BYTE, &col);
    SWITCH_INT(col);

    printf(" color number %u\n",nIndex);
    printf(" color founded %u\n",col);

    printf(" color put %u ,%u ,%u\n",nIndex%256,(nIndex>>8) % 256, (nIndex>>16) % 256);
    printf(" color get %u ,%u ,%u\n",col%256,(col>>8) % 256, (col>>16) % 256);

    }


    int main(int argc, char *argv[]) {

    SDL_Surface *screen;

    int done;

    SDL_Event event;

    if ( SDL_Init(SDL_INIT_VIDEO) != 0 ) {
    printf("Unable to initialize SDL: %s\n", SDL_GetError());
    return 1;
    }

    SDL_GL_SetAttribute( SDL_GL_DOUBLEBUFFER, 1 ); // *new*

    screen = SDL_SetVideoMode( SCREENW, SCREENH, 32, SDL_OPENGL /*| SDL_FULLSCREEN*/); // *changed*
    if ( !screen ) {
    printf("Unable to set video mode: %s\n", SDL_GetError());
    return 1;
    }


    glViewport( 0, 0, SCREENW, SCREENH );
    glMatrixMode(GL_PROJECTION);
    glLoadIdentity();
    gluPerspective(60.0, 1.0 * SCREENW/SCREENH, 0.1, 400.0);
    glMatrixMode(GL_MODELVIEW);
    glLoadIdentity();


    glEnable (GL_DEPTH_TEST);

    glEnable (GL_COLOR_MATERIAL);

    glEnable(GL_BLEND);
    glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);

    gluLookAt(0,0,18,0.0,0.0,0.0,0.0,1.0,0.0);

    done=0;

    glClearColor( 0, 0, 0, 0);



    while(!done) {

    startTicks=SDL_GetTicks();


    // Clear the screen before drawing
    glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);



    while(SDL_PollEvent(&event)) {


    switch (event.type) {

    case SDL_MOUSEMOTION:
    mouseMove (event.button.x, event.button.y);
    mousex=event.button.x;
    mousey=event.button.y;
    break;

    case SDL_KEYDOWN:
    if (event.key.keysym.sym == SDLK_ESCAPE) done=1;
    break;

    case SDL_MOUSEBUTTONDOWN :
    if (SDL_GetMouseState (NULL,NULL) & SDL_BUTTON_LMASK) {
    mousex=event.button.x;
    mousey=event.button.y;
    mode = SELECT;
    }
    break;

    default:
    break;
    }
    }


    if (mode == SELECT) DrawSelection(); else DrawSelection();

    if (mode == SELECT) {
    ProcessPick();
    mode = RENDER;
    } else SDL_GL_SwapBuffers();





    if( currentTicks < 1000 / FRAMES_PER_SECOND ) SDL_Delay( ( 1000 / FRAMES_PER_SECOND ) - currentTicks );

    }

    SDL_Quit();

    return 0;
    }

    [ Edited by raistlin77it 17.06.2011 - 17:57 ]
    I'm nerdy in the extreme
    And whiter than sour cream

    White&Nerdy 2006 Al Yankovic
  • »17.06.11 - 16:52
    Profile
  • Order of the Butterfly
    Order of the Butterfly
    Posts: 186 from 2003/10/23
    BUMP :P
    I'm nerdy in the extreme
    And whiter than sour cream

    White&Nerdy 2006 Al Yankovic
  • »20.06.11 - 11:40
    Profile
  • Order of the Butterfly
    Order of the Butterfly
    Yomgui
    Posts: 348 from 2004/8/31
    From: Québec - Canada
    Seems there is a bug with function glColor3ub() than glReadPixels() !

    Replace the former by glColor3f() (don't forget to divide r,g,b channels by 255, it's a [0,1] range)
    And now... next project!
  • »23.06.11 - 01:00
    Profile Visit Website
  • Order of the Butterfly
    Order of the Butterfly
    Posts: 186 from 2003/10/23
    ummm, ok then, i will use only float. thanks
    I'm nerdy in the extreme
    And whiter than sour cream

    White&Nerdy 2006 Al Yankovic
  • »25.06.11 - 10:39
    Profile
  • Caterpillar
    Caterpillar
    AmiDARK
    Posts: 38 from 2011/10/29
    From: South France
    Do you use float ? or do you directly divide your rgb byte per 255 ?
  • »18.01.12 - 10:36
    Profile Visit Website
  • Order of the Butterfly
    Order of the Butterfly
    Posts: 186 from 2003/10/23
    i wanted to use unsigned byte, but as workaround i ended to use float (UByte/255)

    maybe kiero can help to fix this :D
    I'm nerdy in the extreme
    And whiter than sour cream

    White&Nerdy 2006 Al Yankovic
  • »18.01.12 - 16:31
    Profile
  • Caterpillar
    Caterpillar
    AmiDARK
    Posts: 38 from 2011/10/29
    From: South France
    I use glReadPixels in the AmiDARK Engine for the commande "DEGetImage" and it work perfectly so it's not the glReadPixels that cause that ...
  • »18.01.12 - 17:24
    Profile Visit Website
  • Paladin of the Pegasos
    Paladin of the Pegasos
    Jupp3
    Posts: 1193 from 2003/2/24
    From: Helsinki, Finland
    Not tested, but perhaps you could replace the immediate mode part (anything between glBegin() and glEnd(), including those lines) with:
    const GLfloat va[]=
    {-0.5f,-0.5f,+0.5f,
    +0.5f,-0.5f,+0.5f,
    +0.5f,+0.5f,+0.5f,
    -0.5f,+0.5f,+0.5f};

    glEnableClientState(GL_VERTEX_ARRAY);
    glVertexPointer(3, GL_FLOAT, 0, va);
    glDrawArrays(GL_TRIANGLE_FAN, 0, 4*3);
    glDisableClientState(GL_VERTEX_ARRAY);

    Of course you can skip last glDisableClientState(), as you probably want to keep that enabled anyway (you should of course disable it for color, normal etc. when you want to switch to "Giving a constant value with glColor*(), glNormal() etc.)
    GL_TRIANGLE_FAN instead of GL_QUADS because they do essentially the same thing (only with exactly one quad, as in this case), but unlike GL_QUADS, GL_TRIANGLE_FAN actually is implemented in OpenGL ES1, so at least that line should work on mobile devices too :-)
  • »19.01.12 - 10:29
    Profile Visit Website
  • Order of the Butterfly
    Order of the Butterfly
    Posts: 186 from 2003/10/23
    ok , thanks for the idea, but the problem is in the glColor3ub ( nIndex % 256, (nIndex>>8) % 256, (nIndex>>16) % 256); function of morphos

    infact if u draw a quad with nIndex=255 (255,0,0)
    and another quad with nIndex=256 (0,1,0)

    the 2 quads have the same color when u pick the pixels. (stested with glreadbuffer, and with a screenshot)

    i have resolved the problem with a workaround ( a lookup table for colors) and my game works now

    maybe this glColor3ub problem can be fixed :D :D
    I'm nerdy in the extreme
    And whiter than sour cream

    White&Nerdy 2006 Al Yankovic
  • »19.01.12 - 13:59
    Profile