TinyGL: FBO, Shaders and futur improvements
  • Priest of the Order of the Butterfly
    Priest of the Order of the Butterfly
    beworld
    Posts: 538 from 2010/2/10
    From: FRANCE
    Context:
    As you know, i'm working on SDL2 and games porting.
    Some games from SDL2 ports, examples: Doom3, sm64, ScummMV, PRBoom, etc... could use missing features (as Framebuffer objects or shaders) that allow them better rendering and gaming experience.

    Goal:
    Have a better TinyGL/OpenGL support

    Remark:
    I'm waiting for your opinion on this suject to know how many of us will find interesting.
    All arguments feeding this topic could lead to new features and improvements so do not hesitate to share your ideas.

    Depending on the number of responses and needs, we could open a Bounty for MorphOS Team developers.
    IMac G5 2.1,PowerMac 2.7,PowerBook 1.5
    My MOS ports
  • »16.02.22 - 13:28
    Profile Visit Website
  • MDW
  • Order of the Butterfly
    Order of the Butterfly
    MDW
    Posts: 388 from 2003/7/25
    From: Wroclaw/Poland
    For me TinyGL is the most important feature of MorphOS. I very like coding C/C++ with old OpenGL. MorphOS platform is perfect target for me. :)
    I am still in non-shaders age. However I will go to next step of my retro-3D hobby when TinyGL allows for it.
    So I am interested in add improvements to TinyGL. In my opinion this is very good idea.
  • »16.02.22 - 13:58
    Profile Visit Website
  • Paladin of the Pegasos
    Paladin of the Pegasos
    Papiosaur
    Posts: 1673 from 2003/4/10
    From: France
    Quote:

    MDW a écrit :
    For me TinyGL is the most important feature of MorphOS. I very like coding C/C++ with old OpenGL. MorphOS platform is perfect target for me. :)
    I am still in non-shaders age. However I will go to next step of my retro-3D hobby when TinyGL allows for it.
    So I am interested in add improvements to TinyGL. In my opinion this is very good idea.


    Yes me too, after Wayfarer and Iris projects, i think its the next step for MorphOS to have a best OS ;-)
  • »16.02.22 - 14:01
    Profile Visit Website
  • Paladin of the Pegasos
    Paladin of the Pegasos
    NewSense
    Posts: 1285 from 2012/11/10
    From: Manchester, UK/GB
    I too would welcome improvements with TinyGl and OpenGL, specifically I am aware of the shaders problems, but if the Framebuffer objects issue is also compounding these issues, then of course they too need addressing for MorphOS, as it will allow some game conversions to work as intended.

    I'm not an avid gamer, but some games, and game emulator(s), i.e. ScummVM, and the games they support, seem to be worthy of converting successfully for MorphOS so we can use all the games that they allow us to enjoy playing. 8-D
    MacMini 1.5GHz,64MB VRAM, PowerBooks A1138/9 (Model 5,8/9),PowerMac G5 2.3GHz(DP), iMac A1145 2.1GHz 20", all with MorphOS v3.17,Airport,Bluetooth,A1016 Keyboard,T-RB22 Mouse,DVD-RW-DL,MiniMax,Firewire/USB2 & MacOSX 10.4/5.
  • »17.02.22 - 00:44
    Profile
  • Yokemate of Keyboards
    Yokemate of Keyboards
    Andreas_Wolf
    Posts: 11720 from 2003/5/22
    From: Germany
    > game emulator(s), i.e. ScummVM

    ScummVM is as much an emulator as the original executables of the supported games are.

    > and the games they support, seem to be worthy of converting
    > successfully for MorphOS so we can use all the games

    Isn't the purpose of having ScummVM on a platform that the supported games do not need to be converted to that platform?
  • »17.02.22 - 02:38
    Profile
  • Priest of the Order of the Butterfly
    Priest of the Order of the Butterfly
    eliot
    Posts: 560 from 2004/4/15
    Oh yes, this would be great!
    Ermentrud also uses this in SDL2.
    I have a working modified version here, but I have some problems with alpha channel in bitmaps.
    regards
    eliot
  • »17.02.22 - 07:26
    Profile
  • Order of the Butterfly
    Order of the Butterfly
    MoerBoer
    Posts: 175 from 2019/10/15
    I would contribute too ( I have an original Doom III CD lying here )
  • »17.02.22 - 07:29
    Profile
  • Priest of the Order of the Butterfly
    Priest of the Order of the Butterfly
    beworld
    Posts: 538 from 2010/2/10
    From: FRANCE
    Quote:

    eliot a écrit :
    Oh yes, this would be great!
    Ermentrud also uses this in SDL2.
    I have a working modified version here, but I have some problems with alpha channel in bitmaps.


    MorphOS 3.16 fix somes part of blending problem.
    Your fix present in your github is correct.
    But in main git, i see unsing of SDL_CreateTextue with SDL_TEXTUREACCESS_TARGET flag and need FBOs in OpenGL mode.



    [ Edité par beworld 18.02.2022 - 07:17 ]
    IMac G5 2.1,PowerMac 2.7,PowerBook 1.5
    My MOS ports
  • »17.02.22 - 07:38
    Profile Visit Website
  • Order of the Butterfly
    Order of the Butterfly
    Zetec-s
    Posts: 188 from 2008/7/10
    From: Cheshire, UK
    I am definitely up for supporting this with a donation, as I believe it is an important development.

    I mentioned before, I do not know if this is outside of the scope of what you are suggesting, but if TinyGL could be brought to the point where it is compatible with OpenGL 2.0 then we could potentially get Virtual Grand Prix 3 ported over.

    MorphOS is behind Amiga OS 4 on the graphics side and anything that can be done to redress this balance I am fully in support with. Not that I have anything against OS4 it would just be good to get the same ports that they do, some of the time. I also appreciate we are not talking about Nova here so I am realistic with my expectations.
    PowerMac G5 Quad 2.5Ghz/2GB MorphOS 3.17 Registered
    Powerbook 1.67Ghz/1.5GB MorphOS 3.17 Registered
    MacMini 1.5Ghz/1GB MorphOS 3.17 Registered
    Efika 5200B 400Mhz/128MB MorphOS 2.3 Registered
  • »17.02.22 - 08:48
    Profile
  • Priest of the Order of the Butterfly
    Priest of the Order of the Butterfly
    polluks
    Posts: 718 from 2007/10/23
    From: Gelsenkirchen,...
    Quote:

    Andreas_Wolf schrieb:
    > game emulator(s), i.e. ScummVM

    ScummVM is as much an emulator as the original executables of the supported games are.

    > and the games they support, seem to be worthy of converting
    > successfully for MorphOS so we can use all the games

    Isn't the purpose of having ScummVM on a platform that the supported games do not need to be converted to that platform?


    I suppose he means to support TinyGL for the ScummVM engines.
    Pegasos II G4: MorphOS 3.9, Zalman M220W · iMac G5 12,1 17", MorphOS 3.17
    Power Mac G3: OSX 10.3 · PowerBook 5,8: OSX 10.5, MorphOS 3.17
  • »18.02.22 - 12:43
    Profile
  • Paladin of the Pegasos
    Paladin of the Pegasos
    koszer
    Posts: 1185 from 2004/2/8
    From: Poland
    One big question remains however: is there a single programmer willing to take the aformentioned bounty?
  • »21.02.22 - 06:14
    Profile
  • Butterfly
    Butterfly
    Posts: 72 from 2020/12/24
    Before taking the task to modernize the whole library I guess the most important part must be fix some of the functions that don't work as expected.

    As demonstrated with "Doom3", texture mapping and vbo's fail to work properly. You can live without texture mapping with shaders but surely vbo needs to be fixed or further progress doesn't make sense for 95% of projects out there.

    Also I suspect Stencil support is not ok and that is essential for modern shadow render code ( Doom3 fails at that ) and for example some of the tricks Scummvm does with OpenGL games (see Grimm Fandango drawing objects in bad order).

    Also documentation about the intricacies of Tinygl could be helpfull too at the time of debugging.
  • »21.02.22 - 08:16
    Profile
  • Priest of the Order of the Butterfly
    Priest of the Order of the Butterfly
    polluks
    Posts: 718 from 2007/10/23
    From: Gelsenkirchen,...
    @Cowcat
    Full acknowledge!
    Pegasos II G4: MorphOS 3.9, Zalman M220W · iMac G5 12,1 17", MorphOS 3.17
    Power Mac G3: OSX 10.3 · PowerBook 5,8: OSX 10.5, MorphOS 3.17
  • »22.02.22 - 20:12
    Profile
  • Priest of the Order of the Butterfly
    Priest of the Order of the Butterfly
    Cool_amigaN
    Posts: 703 from 2011/11/30
    This is a very important thread/initiative imo and hope it materialises sooner than later as everyday that passes by, our tinygl gets lost deeper into obscurity. Hope whatever needs to done in order to have even more ports of higher quality to be done properly asap :) Will support the project financially without a second thought.
    Amiga gaming Tribute: Watch, rate, comment :)
  • »22.02.22 - 20:46
    Profile Visit Website
  • MorphOS Developer
    bigfoot
    Posts: 430 from 2003/4/11
    Quote:

    Cowcat wrote:
    As demonstrated with "Doom3", texture mapping and vbo's fail to work properly. You can live without texture mapping with shaders but surely vbo needs to be fixed or further progress doesn't make sense for 95% of projects out there.


    I'm not aware of any problems with texture mapping or VBOs. At least Fodquake uses VBOs, and that's working right. If you know what doesn't work, or better, if you can supply me with a small test app that reproduces a problem, I'd be happy to fix it.

    Quote:

    Cowcat wrote:
    Also I suspect Stencil support is not ok and that is essential for modern shadow render code ( Doom3 fails at that ) and for example some of the tricks Scummvm does with OpenGL games (see Grimm Fandango drawing objects in bad order).


    You are quite right there, and I didn't realise that stencil buffer support had been left in a limbo since the TinyGL overhaul to use native drivers and support hardware TCL.

    I've just added the required code to TinyGL and to the R200 and R300(/R400/R500) drivers to support stencil buffers, and I wrote some unit tests to try to make sure that the functionality works as expected, and the unit tests are now passing on both drivers.

    If you find any stencil buffer issues after the next MorphOS release, please let me know!

    Quote:

    Cowcat wrote:
    Also documentation about the intricacies of Tinygl could be helpfull too at the time of debugging.


    Anything in particular you've got in mind?
    I rarely log in to MorphZone which means that I often miss private messages sent on here. If you wish to contact me, please email me at [username]@asgaard.morphos-team.net, where [username] is my username here on MorphZone.
  • »24.02.22 - 12:59
    Profile Visit Website
  • Butterfly
    Butterfly
    Posts: 72 from 2020/12/24
    @Bigfoot

    For VBO and texture mapping, well, try my infamous Doom3 port. ;)

    I complained about that before: TexGen functions fail totally even for some old opengl examples. I had to do a workaround in Doom3 for that among others.

    VBO "kinda works" but not with failures: Just right the level starts and having TGLDEBUG enabled you can catch a suspicious error about "disabling vbo support" of something like that.

    And reallocating new buffers locks up for seconds the game. Glitches and so on in game I don't know if they are related to vbo's or not.

    I compiled some examples using vbos and found some curious behaviour that differs from "standard way":

    Some raw example:

    Normal DrawArrays function (that works ok):

    Code:
    void drawVertexArray()
    {
    glEnableClientState(GL_VERTEX_ARRAY);
    glEnableClientState(GL_COLOR_ARRAY);

    // Set axes data
    glVertexPointer(nCoordsComponents, GL_FLOAT, 0, ave);
    glColorPointer(nColorComponents, GL_FLOAT, 0, ace);

    // Draw axes
    glDrawArrays(GL_LINES, 0, nLines*nVerticesPerLine);

    // Set pyramid data
    glVertexPointer(nCoordsComponents, GL_FLOAT, 0, pve);
    glColorPointer(nColorComponents, GL_FLOAT, 0, pce);

    // Draw pyramid
    glDrawArrays(GL_TRIANGLES, 0, nFaces*nVerticesPerFace);

    glDisableClientState(GL_VERTEX_ARRAY);
    glDisableClientState(GL_COLOR_ARRAY);
    }


    Now VBO way:

    Code:
    void drawVertexBufferObject()
    {
    vboIds = new GLuint[3];
    glGenBuffersARB(4,vboIds);

    glEnableClientState(GL_VERTEX_ARRAY);
    glEnableClientState(GL_COLOR_ARRAY);

    // Set axes data
    glBindBufferARB(GL_ARRAY_BUFFER, vboIds[0]); // coordinates
    glBufferDataARB(GL_ARRAY_BUFFER, sizeof(ave), ave, GL_STATIC_DRAW);
    glVertexPointer(nCoordsComponents, GL_FLOAT, 0, 0);

    glBindBufferARB(GL_ARRAY_BUFFER, vboIds[1]); // color
    glBufferDataARB(GL_ARRAY_BUFFER, sizeof(ace), ace, GL_STATIC_DRAW);
    glColorPointer(nColorComponents, GL_FLOAT, 0, 0);

    // Draw axes
    glDrawArrays(GL_LINES, 0, nLines * nVerticesPerLine);


    // Set pyramid data
    glBindBufferARB(GL_ARRAY_BUFFER, vboIds[2]); // coordinates
    glBufferDataARB(GL_ARRAY_BUFFER, sizeof(pve), pve, GL_STATIC_DRAW);
    glVertexPointer(nCoordsComponents, GL_FLOAT, 0, 0);

    glBindBufferARB(GL_ARRAY_BUFFER, vboIds[3]); // color
    glBufferDataARB(GL_ARRAY_BUFFER, sizeof(pce), pce, GL_STATIC_DRAW);
    glColorPointer(nColorComponents, GL_FLOAT, 0, 0);

    // Draw pyramid
    glDrawArrays(GL_TRIANGLES, 0, nFaces * nVerticesPerFace);

    glDisableClientState(GL_VERTEX_ARRAY);
    glDisableClientState(GL_COLOR_ARRAY);

    // Disable the VBO
    glBindBufferARB(GL_ARRAY_BUFFER, 0);
    }


    VBO function doesn't draw the pyramid unless vertex and color states are disabled and enabled again after drawing "axes". Kinda not the way to work.


    So if Stencil is fixed, SDL2-GL/ScummVM/etc. should work OK. :)

    Documentation? Well besides differences from others opengl variations, a little more debugging output for those functions or some kind of "debug" driver outputing more information.

    By the way, the Warp3d debug error messages still hilarious. :)


    [ Edited by Cowcat 26.02.2022 - 07:33 ]
  • »25.02.22 - 20:03
    Profile
  • MorphOS Developer
    bigfoot
    Posts: 430 from 2003/4/11
    Quote:

    Cowcat wrote:I complained about that before: TexGen functions fail totally even for some old opengl examples. I had to do a workaround in Doom3 for that among others.


    Not to me ;) But yes, TexGen is currently pretty much completely broken. Fixing it would require a larger overhaul of some parts of TinyGL.

    Quote:

    Cowcat wrote:
    VBO "kinda works" but not with failures: Just right the level starts and having TGLDEBUG enabled you can catch a suspicious error about "disabling vbo support" of something like that.


    Do you mean Disabling VBO rendering due to incompatible parameters? If so, that happens if the array format is not GL_FLOAT, or if a misaligned offset/stride has been given. It's only disabled for that specific array, and only for as long as it has incompatible data assigned to it.

    Quote:

    Cowcat wrote:
    And reallocating new buffers locks up for seconds the game. Glitches and so on in game I don't know if they are related to vbo's or not.


    Have you confirmed that the second long lock-ups are due to VBOs? Or is it just a hunch?

    Quote:

    Cowcat wrote:I compiled some examples using vbos and found some curious behaviour that differs from "standard way":


    I had to use a bit of Google-Fu, but I'm guessing you're referring to this example code on GitHub. I tried it out, and indeed, when switching to VBO mode on MorphOS, the pyramid disappears. I then tried the exact same example on Linux, and when switching to VBO mode, the pyramid also disappears there. So I think MorphOS is actually doing it right in this case ;)
    I rarely log in to MorphZone which means that I often miss private messages sent on here. If you wish to contact me, please email me at [username]@asgaard.morphos-team.net, where [username] is my username here on MorphZone.
  • »26.02.22 - 07:36
    Profile Visit Website
  • Butterfly
    Butterfly
    Posts: 72 from 2020/12/24
    Quote:

    Do you mean Disabling VBO rendering due to incompatible parameters? If so, that happens if the array format is not GL_FLOAT, or if a misaligned offset/stride has been given. It's only disabled for that specific array, and only for as long as it has incompatible data assigned to it.


    It explains game still works after that. Could be that misaligment screws all what is going after that. Who knows.

    Quote:

    Have you confirmed that the second long lock-ups are due to VBOs? Or is it just a hunch?


    Yep. At the time buffers are "enlarged" I had output telling that: It's is done from time to time where is needed but newer allocations increase the time spent doing it.

    Check out inside idVertexCache::Alloc function on github.

    https://github.com/Cowcat5150/dhewm3/blob/master/neo/renderer/VertexCache.cpp

    The OpenGL VBO example you tested is the one. Weird: In my Linux box works. Whatever.
  • »26.02.22 - 09:45
    Profile
  • MorphOS Developer
    bigfoot
    Posts: 430 from 2003/4/11
    Quote:

    Cowcat wrote:
    Quote:

    Have you confirmed that the second long lock-ups are due to VBOs? Or is it just a hunch?


    Yep. At the time buffers are "enlarged" I had output telling that: It's is done from time to time where is needed but newer allocations increase the time spent doing it.


    Which hardware is this running on? How large is your VBO? The only thing I can think of is that the sum of the sizes of the framebuffer, depth buffer, textures and vertex buffers runs close to or exceeds the size of the Radeon video memory.
    I rarely log in to MorphZone which means that I often miss private messages sent on here. If you wish to contact me, please email me at [username]@asgaard.morphos-team.net, where [username] is my username here on MorphZone.
  • »28.02.22 - 02:17
    Profile Visit Website
  • MorphOS Developer
    bigfoot
    Posts: 430 from 2003/4/11
    Quote:

    Cowcat wrote:
    The OpenGL VBO example you tested is the one. Weird: In my Linux box works. Whatever.


    I found a silly bug in the example code which is why it didn't work for me on Linux. I fixed the bug, and indeed it still didn't render the pyramid for me on MorphOS. Found the bug in MorphOS and fixed that too, so now the (fixed) example works on MorphOS too.

    On line 468 and 469 of main.cpp, it allocates space for just 3 vertex buffers, but on lines 491 through 493 of main.cpp it uses the 4th index of the array that only holds 3 elements, thus reading past the end of the allocated memory. If it worked for you on Linux, it was by luck.
    I rarely log in to MorphZone which means that I often miss private messages sent on here. If you wish to contact me, please email me at [username]@asgaard.morphos-team.net, where [username] is my username here on MorphZone.
  • »28.02.22 - 04:07
    Profile Visit Website
  • Butterfly
    Butterfly
    Posts: 72 from 2020/12/24
    Quote:

    I found a silly bug in the example code which is why it didn't work for me on Linux. I fixed the bug, and indeed it still didn't render the pyramid for me on MorphOS. Found the bug in MorphOS and fixed that too, so now the (fixed) example works on MorphOS too.


    Ups, yep I had the code fixed before and forgot that. But if that allowed to fix a Morphos tinygl bug then nice.


    Quote:

    Which hardware is this running on?


    Doom3 I have it "running" on MacMini G4 with 32mg VRAM.

    Before the "is not possible on your system":

    Game allocates 2mg for VBOs at the beginning and then increases as needed. I have the lowest possible configuration to save vram (from textures to effects).

    Game has render problems showed as glitches as "black triangles" from frame to frame depending of movement and actions. But the thing is people with higher ATI GFX cards with lots of VRAM have the same issues.

    Maybe you can tell about those glitches from this video:
    https://www.youtube.com/watch?v=M-DrfK5ZJ2Y
  • »28.02.22 - 07:38
    Profile
  • MorphOS Developer
    bigfoot
    Posts: 430 from 2003/4/11
    Quote:

    Cowcat wrote:
    Ups, yep I had the code fixed before and forgot that. But if that allowed to fix a Morphos tinygl bug then nice.


    Well, that'd explain why it worked for you then :) But this bug might also be related to whatever VBO bad rendering problems you might be having. It could cause a VBO to not be initialised correctly (on the hardware side) for the first frame after it was created.

    Quote:

    Cowcat wrote:
    Quote:

    Which hardware is this running on?


    Doom3 I have it "running" on MacMini G4 with 32mg VRAM.

    Before the "is not possible on your system":


    Sorry, I wasn't planning on saying any such thing :) It was just to see if my theory about getting close to the total VRAM capacity of the graphics card was plausible or not. With 32MB of VRAM, then yes, it's plausible. If you had been using a graphics card with 128MB, then less plausible. This is just about the "game freezes when increasing VBO size" issue, nothing else, BTW.

    But again, this was just to see if my theory was plausible. Even if my theory is right, it's an issue that needs to be fixed. Won't be a quick fix, though, unfortunately.
    I rarely log in to MorphZone which means that I often miss private messages sent on here. If you wish to contact me, please email me at [username]@asgaard.morphos-team.net, where [username] is my username here on MorphZone.
  • »28.02.22 - 07:48
    Profile Visit Website
  • Butterfly
    Butterfly
    Posts: 72 from 2020/12/24
    Quote:

    Sorry, I wasn't planning on saying any such thing :) It was just to see if my theory about getting close to the total VRAM capacity of the graphics card was plausible or not.


    Oh,no !! No problem, no offense ;) Just that I'm stubborn for making things ouside the usual.

    I have some testing code that makes the lagging disappear but makes the game a little slower but still glitches here and there as before.

    Maybe latest Tinygl upgrades improve things.
  • »28.02.22 - 07:55
    Profile
  • MorphOS Developer
    bigfoot
    Posts: 430 from 2003/4/11
    Pushing the limits is perfectly fine :)

    PM (or email) me your email address and I can hook you up with an updated TinyGL + drivers for testing.
    I rarely log in to MorphZone which means that I often miss private messages sent on here. If you wish to contact me, please email me at [username]@asgaard.morphos-team.net, where [username] is my username here on MorphZone.
  • »28.02.22 - 08:04
    Profile Visit Website
  • Butterfly
    Butterfly
    Posts: 72 from 2020/12/24
    Nice !
  • »28.02.22 - 08:07
    Profile