JIT JS Bounty - update
  • Yokemate of Keyboards
    Yokemate of Keyboards
    takemehomegrandma
    Posts: 2720 from 2003/2/24
    Isn't the problems with that related to the whole endianness situation in Webkit?

    There has been talk about setting up a bounty for someone to bite the challenge of fixing the JS endian stuff, I don't know how realistic it is to expect anything from a thing like that, but if such a bounty would be set up, maybe these funds could be transfered over to that one? It's a related thing after all...?
    MorphOS is Amiga done right! :-)
    MorphOS NG will be AROS done right! :-)
  • »12.01.16 - 11:38
    Profile
  • Yokemate of Keyboards
    Yokemate of Keyboards
    takemehomegrandma
    Posts: 2720 from 2003/2/24
    ...or otherwise, wasn't there some kind of "general bounty"/reward system set up where people could donate money and it will be distributed amongst active MorphOS developers as some kind of appreciation gesture? Or maybe I recall wrong? Maybe that would be an option?
    MorphOS is Amiga done right! :-)
    MorphOS NG will be AROS done right! :-)
  • »12.01.16 - 11:46
    Profile
  • Paladin of the Pegasos
    Paladin of the Pegasos
    pampers
    Posts: 1061 from 2009/2/26
    From: Tczew, Poland
    Quote:

    Isn't the problems with that related to the whole endianness situation in Webkit?


    Mark said that it isn't related.

    Quote:

    and it will be distributed amongst active MorphOS developers as some kind of appreciation gesture


    Personally I dislike "overall" bounties. I prefer to pay for certain thing.
    MorphOS 3.x
  • »12.01.16 - 11:54
    Profile Visit Website
  • MorphOS Developer
    bigfoot
    Posts: 510 from 2003/4/11
    Quote:

    pampers wrote:
    I was talking with bigfoot and he came back to developing it but he cannot promise any time frame. I'd say bigfoot will speak for himself in this topic soon.


    I'm not sure there's really that much to say. When I took the bounty, I made an estimate for how much time it would take to complete the JIT implementation for PPC32. In fact, I finished that task a bit faster than I had originally estimated. What I didn't realise was just how broken Webkit was internally, and how much of a problem it would pose.

    The problem is that the JIT engine internally expects a little endian memory layout. This is different from other endian problems in that the JIT engine itself isn't what isn't endian safe, it's the code that the JIT engine generates that isn't.

    Webkit's Javascript engine internally treats all Javascript values as 64 bit floating point values. This not only includes actual floating point values, but also integers, strings, objects and so on. This means that every value that the JIT-generated code uses is a 64 bit value, even on 32 bit architectures. How this works is that there are certain bit encodings of floating point values that are illegal, and which will never be the result of a normal floating point arithmetic operation. These encodings are then used to indicate if a value is actually something else than a floating point value.

    Now, the problem is that we're on a 32 bit CPU, so the integer part of the CPU, which is what is used for doing these tests, and for passing data around, can only load 32 bits at a time. The JIT engine then generates code that expects the upper 32 bits of the 64 bit floating point value to be located at address + 4. This assumption is made roughly every 20-30 lines over tens of thousands of lines of code. Furthermore, it expects 64 bit values to be passed and returned in 2 32 bit registers, which is perfectly normal, but it of course expects the order of this register pair to be swapped, while at the same time expecting 32 bit return values to reside in the register that also holds the lower 32 bits of a 64 bit return value. This assumption is made for every single interaction between JIT-generated Javascript code and the compiled Odyssey binary. Every time your Javascript code calls for example Math.Floor(), this assumption is made in the JIT-generated code.

    So what's the status of all this? I found and fixed probably 99% of all of the above issues. Yet at least two issues remain that are rather tricky to track down, and those two (or more) issues cause most large websites to fail, because one of the pieces of Javascript tha triggers one of the remaining issue is a Javascript framework that's used by just about everyone.

    So, as Pampers mentioned, I still want to finish this, but with the amount of time I ended up spending on fixing endian issues in the JIT engine, I overshot my initial time estimate by at least a factor of 5, and I had to focus on other things to get an income. Thus, I'm still (obviously very slowly) working on this, but I (also obviously) no longer expect to get paid for this work, and thus if anyone wants their money back or wants to assign their money to another project, then of course it is no problem.
    I rarely log in to MorphZone which means that I often miss private messages sent on here. If you wish to contact me, please email me at [username]@asgaard.morphos-team.net, where [username] is my username here on MorphZone.
  • »12.01.16 - 14:41
    Profile Visit Website
  • Paladin of the Pegasos
    Paladin of the Pegasos
    pampers
    Posts: 1061 from 2009/2/26
    From: Tczew, Poland
    For me (as a donor, not as a bounty organizer) I can wait with no probs.
    MorphOS 3.x
  • »12.01.16 - 15:27
    Profile Visit Website
  • Yokemate of Keyboards
    Yokemate of Keyboards
    Zylesea
    Posts: 2057 from 2003/6/4
    Quote:

    bigfoot schrieb:
    Quote:

    pampers wrote:
    I was talking with bigfoot and he came back to developing it but he cannot promise any time frame. I'd say bigfoot will speak for himself in this topic soon.


    I'm not sure there's really that much to say. When I took the bounty, I made an estimate for how much time it would take to complete the JIT implementation for PPC32. In fact, I finished that task a bit faster than I had originally estimated. What I didn't realise was just how broken Webkit was internally, and how much of a problem it would pose.

    The problem is that the JIT engine internally expects a little endian memory layout. This is different from other endian problems in that the JIT engine itself isn't what isn't endian safe, it's the code that the JIT engine generates that isn't.

    Webkit's Javascript engine internally treats all Javascript values as 64 bit floating point values. This not only includes actual floating point values, but also integers, strings, objects and so on. This means that every value that the JIT-generated code uses is a 64 bit value, even on 32 bit architectures. How this works is that there are certain bit encodings of floating point values that are illegal, and which will never be the result of a normal floating point arithmetic operation. These encodings are then used to indicate if a value is actually something else than a floating point value.

    Now, the problem is that we're on a 32 bit CPU, so the integer part of the CPU, which is what is used for doing these tests, and for passing data around, can only load 32 bits at a time. The JIT engine then generates code that expects the upper 32 bits of the 64 bit floating point value to be located at address + 4. This assumption is made roughly every 20-30 lines over tens of thousands of lines of code. Furthermore, it expects 64 bit values to be passed and returned in 2 32 bit registers, which is perfectly normal, but it of course expects the order of this register pair to be swapped, while at the same time expecting 32 bit return values to reside in the register that also holds the lower 32 bits of a 64 bit return value. This assumption is made for every single interaction between JIT-generated Javascript code and the compiled Odyssey binary. Every time your Javascript code calls for example Math.Floor(), this assumption is made in the JIT-generated code.



    I am rather clueless on low level programming - but what about using the 128 Bit registers of Altivec/VMX. Would be an Altivec/VMX only JIT then, but there are only very few non Altivec/VMX machines with MorphOS support (IIRC Peg1+e few PegII and Efika).
    --
    http://via.bckrs.de

    Whenever you're sad just remember the world is 4.543 billion years old and you somehow managed to exist at the same time as David Bowie.
    ...and Matthias , my friend - RIP
  • »12.01.16 - 18:35
    Profile Visit Website
  • Paladin of the Pegasos
    Paladin of the Pegasos
    pampers
    Posts: 1061 from 2009/2/26
    From: Tczew, Poland
    Zylsea, what about Sam460 and upcomming X5000?
    MorphOS 3.x
  • »12.01.16 - 20:32
    Profile Visit Website
  • Yokemate of Keyboards
    Yokemate of Keyboards
    Zylesea
    Posts: 2057 from 2003/6/4
    Quote:

    pampers schrieb:
    Zylsea, what about Sam460 and upcomming X5000?


    Indeed forgot these, but I doubt these machines will contribute much to the userbase.
    --
    http://via.bckrs.de

    Whenever you're sad just remember the world is 4.543 billion years old and you somehow managed to exist at the same time as David Bowie.
    ...and Matthias , my friend - RIP
  • »12.01.16 - 20:39
    Profile Visit Website
  • Yokemate of Keyboards
    Yokemate of Keyboards
    Andreas_Wolf
    Posts: 12150 from 2003/5/22
    From: Germany
    > there are only very few non Altivec/VMX machines with MorphOS support
    > (IIRC Peg1+e few PegII and Efika).

    ...plus one or two Sam460 and an unknown number of future Cyrus/X5000 :-)

    Edit: pampers beat me to it.
  • »12.01.16 - 20:44
    Profile
  • Moderator
    guruman
    Posts: 461 from 2003/7/21
    As a donor I am more than comfortable to keep the money at morphos.pl until the bounty has been fulfilled. Mark is a trustworthy, open and honest developer as can be seen by his answer in this very thread. I think him not expecting to be paid because the work took already 5 times more effort than he expected speaks volume. And pampers attitude as well makes me comfortable to keep it that way.
    From my point of view I would be available to make an additional small contribution instead, if this can help prioritize this work a bit, and in any case I do expect my part of the donation to be given to him when the work is completed anyway. The availability of a JIT JS engine for PPC would probably make the requirement of an additional bounty to fix the javascriptcore endianess bugs in OWB void, so I would instead invite the people discussing that aspect to join this thread and show support to Mark. Who is, after all, one of the few people still around that can solve such problems.

    Kind regards,
    Andrea

    [ Edited by guruman 13.01.2016 - 01:08 ]
  • »12.01.16 - 23:07
    Profile
  • Paladin of the Pegasos
    Paladin of the Pegasos
    NewSense
    Posts: 1513 from 2012/11/10
    From: Manchester, UK/GB
    I agree with guruman and am grateful for a clear and informative update from bigfoot who deserves to get paid the bounty for his work, ideally once the task has been completed, though as he has spent a lot more time on the bounty than even he anticipated it should maybe be part-paid now, and the remainder on full completion, but obviously the payment issue is not my responsibility.
    MacMini 1.5GHz,64MB VRAM, PowerBooks A1138/9 (Model 5,8/9),PowerMac G5 2.3GHz(DP), iMac A1145 2.1GHz 20", all with MorphOS v3.18+,Airport,Bluetooth,A1016 Keyboard,T-RB22 Mouse,DVD-RW-DL,MiniMax,Firewire/USB2 & MacOSX 10.4/5
  • »12.01.16 - 23:51
    Profile
  • Paladin of the Pegasos
    Paladin of the Pegasos
    Yasu
    Posts: 1724 from 2012/3/22
    From: Stockholm, Sweden
    I don't want to burden Bigfoot with more work, but are you capable of fixing the OWB javascriptcore big endian issue? I understand if you don't have time or if the job is too big, but would you be able to do it if someone put a gun to your head? :-)
    AMIGA FORUM - Hela Sveriges Amigatidning!
    AMIGA FORUM - Sweden's Amiga Magazine!

    My MorphOS blog
  • »13.01.16 - 16:24
    Profile Visit Website
  • Priest of the Order of the Butterfly
    Priest of the Order of the Butterfly
    ernsteiswuerfel
    Posts: 556 from 2015/6/18
    From: Funeralopolis
    Just read the thread and made a small donation as bigfoot still wants to finish this.
    Talos II. [Gentoo Linux] | PMac G5 11,2. PMac G4 3,6. PBook G4 5,8. [MorphOS 3.18 / Gentoo Linux] | A600GS
  • »13.01.16 - 18:36
    Profile
  • Paladin of the Pegasos
    Paladin of the Pegasos
    pampers
    Posts: 1061 from 2009/2/26
    From: Tczew, Poland
    @ernsteiswuerfel

    Donation where? Did you use button on the bounty page? If you did please cancel your transfer as I don't take any donations any more (as I don't own PayPal account) and you sent your donation to some dummy address - so you should be able to cancel it with no problem. Sorry for not taking that button down.
    MorphOS 3.x
  • »14.01.16 - 09:32
    Profile Visit Website
  • Moderator
    guruman
    Posts: 461 from 2003/7/21
    Quote:

    Yasu wrote:
    I don't want to burden Bigfoot with more work, but are you capable of fixing the OWB javascriptcore big endian issue? I understand if you don't have time or if the job is too big, but would you be able to do it if someone put a gun to your head? :-)

    Uhm, if you actually read carefully bigfoot post, you would see that it is basically what he is dealing with.
    Quote:

    bigfoot wrote:
    The JIT engine then generates code that expects the upper 32 bits of the 64 bit floating point value to be located at address + 4.

    This means that the JIT engine addresses the values using the little endian format. On the contrary, for big endian CPUs (like the PPC) the upper 32 bits are located at address and the lower 32 bits at address + 4. When bigfoot is done, we would have a working javascript again, and a JIT one at it... And I am confident that bigfoot is also making sure that either this engine can be merged with upstream WebKit or that at least synch with updated upstream WebKit releases can be performed.

    Mark is of course free to correct me where I am wrong.

    Kind regards,
    Andrea
  • »14.01.16 - 22:12
    Profile
  • Priest of the Order of the Butterfly
    Priest of the Order of the Butterfly
    ernsteiswuerfel
    Posts: 556 from 2015/6/18
    From: Funeralopolis
    Quote:

    pampers schrieb:
    @ernsteiswuerfel

    Donation where? Did you use button on the bounty page? If you did please cancel your transfer as I don't take any donations any more (as I don't own PayPal account) and you sent your donation to some dummy address - so you should be able to cancel it with no problem. Sorry for not taking that button down.

    Oops. Yes, I used the button. Thanks for the hint, I was able to cancel the donation now!
    Talos II. [Gentoo Linux] | PMac G5 11,2. PMac G4 3,6. PBook G4 5,8. [MorphOS 3.18 / Gentoo Linux] | A600GS
  • »14.01.16 - 22:49
    Profile
  • Yokemate of Keyboards
    Yokemate of Keyboards
    Andreas_Wolf
    Posts: 12150 from 2003/5/22
    From: Germany
    >> are you capable of fixing the OWB javascriptcore big endian issue?

    > if you actually read carefully bigfoot post, you would see that it is basically
    > what he is dealing with.

    No, it's not. The WebKit JavaScriptCore is the JavaScript interpreter, not the JavaScript JIT compiler bigfoot is working on. Or as bigfoot wrote in comment #5: "This is different from other endian problems in that the JIT engine itself isn't what isn't endian safe". So, JavaScriptCore itself is what isn't endian-safe. More there:

    https://morph.zone/modules/newbb_plus/viewtopic.php?topic_id=11347&forum=9&start=3
    https://morph.zone/modules/newbb_plus/viewtopic.php?topic_id=11362&forum=9&start=8
  • »14.01.16 - 23:32
    Profile
  • MorphOS Developer
    bigfoot
    Posts: 510 from 2003/4/11
    Thanks for the vote of confidence. I appreciate it!

    Quote:

    I am rather clueless on low level programming - but what about using the 128 Bit registers of Altivec/VMX. Would be an Altivec/VMX only JIT then, but there are only very few non Altivec/VMX machines with MorphOS support (IIRC Peg1+e few PegII and Efika).


    Although Altivec has 128 bit registers, it can't deal with 64 bit floating point values, which is an absolute must.

    Quote:

    I don't want to burden Bigfoot with more work, but are you capable of fixing the OWB javascriptcore big endian issue? I understand if you don't have time or if the job is too big, but would you be able to do it if someone put a gun to your head?


    Of course it's possible. It's always just a matter of time.

    Quote:

    This means that the JIT engine addresses the values using the little endian format. On the contrary, for big endian CPUs (like the PPC) the upper 32 bits are located at address and the lower 32 bits at address + 4. When bigfoot is done, we would have a working javascript again, and a JIT one at it... And I am confident that bigfoot is also making sure that either this engine can be merged with upstream WebKit or that at least synch with updated upstream WebKit releases can be performed.

    Mark is of course free to correct me where I am wrong.


    I haven't looked at the current Webkit at all, but any endian problems plaguing the interpreter are unlikely to be the same as the ones that are plaguing the JIT engine. Also, once I have fixed the remaining JIT problems, I will then have lots of work ahead of me merging my changes with newer versions of Webkit/Javascriptcore

    Quote:

    No, it's not. The WebKit JavaScriptCore is the JavaScript interpreter, not the JavaScript JIT compiler bigfoot is working on.


    Actually Javascriptcore is both the interpreter and the JIT engine. However, the problems I'm experiencing are unique to the JIT engine and thus unlikely to be what affects the interpreter in later Webkit versions.
    I rarely log in to MorphZone which means that I often miss private messages sent on here. If you wish to contact me, please email me at [username]@asgaard.morphos-team.net, where [username] is my username here on MorphZone.
  • »15.01.16 - 08:41
    Profile Visit Website
  • Moderator
    Develin
    Posts: 100 from 2003/4/19
    From: Karlstad, Sweden
    I'm confident that the result is wort waiting for so I would gladely keep my (small) donation where it is =)
  • »15.01.16 - 09:07
    Profile Visit Website
  • Paladin of the Pegasos
    Paladin of the Pegasos
    Acill
    Posts: 1926 from 2003/10/19
    From: Port Hueneme, Ca.
    I'd also say just leave the money where it is. Marko is a good guy and deserves to take the time he needs to complete this.
    Powermac Dual 2.0 GHZ G5 PCI-X (Registration #1894)
    Powerbook 1.67GHZ
    Powermac Dual 2.0 GHZ G5 PCIE (Registration #6130)
    A4000T CSPPC, Mediator
    Need Repairs, upgrades or a recap in the USA? Visit my website at http://www.acill.com
  • »15.01.16 - 19:58
    Profile Visit Website
  • Jim
  • Yokemate of Keyboards
    Yokemate of Keyboards
    Jim
    Posts: 4977 from 2009/1/28
    From: Delaware, USA
    Quote:

    Andreas_Wolf wrote:
    > I haven't looked at the current Webkit at all [...]

    The Leopard WebKit author asks:

    "Will Mark (Bigfoot) Olsen's JavaScriptCore PowerPC JIT for MorphOS every arrive?"
    https://sourceforge.net/p/leopard-webkit/tickets/65/#2521


    Yeah...Mark has taken on a lot of work.
    He has never promised anything he didn't deliver, he's probably one of our very best programmers (if not the best) and certainly the most prolific.
    This particular project is pretty complicated.
    "Never attribute to malice what can more readily explained by incompetence"
  • »12.12.16 - 12:18
    Profile
  • Order of the Butterfly
    Order of the Butterfly
    r-tea
    Posts: 306 from 2005/3/27
    From: Poland, Zdzies...
    Quote:

    Andreas_Wolf wrote:
    > I haven't looked at the current Webkit at all [...]

    The Leopard WebKit author asks:

    "Will Mark (Bigfoot) Olsen's JavaScriptCore PowerPC JIT for MorphOS every arrive?"
    https://sourceforge.net/p/leopard-webkit/tickets/65/#2521


    Why he's so interested in this?
    Mac mini G4@1,5GHz silent upgrade + Xerox Phaser 3140 + EPSON Perfection 1240U
    Commodore C64C + 2 x 1541II + Datasette + SD-Box

    I miss draggable screens... and do you? I know I'm in a minority unfortunately.
  • »17.12.16 - 20:44
    Profile
  • Yokemate of Keyboards
    Yokemate of Keyboards
    Andreas_Wolf
    Posts: 12150 from 2003/5/22
    From: Germany
    >> The Leopard WebKit author asks: [...]

    > Why he's so interested in this?

    Bigfoot's JavaScriptCore JIT compiler for PPC should also be usable with the Leopard WebKit browser (or any other WebKit-based browser on any PPC OS).
  • »18.12.16 - 09:09
    Profile
  • Jim
  • Yokemate of Keyboards
    Yokemate of Keyboards
    Jim
    Posts: 4977 from 2009/1/28
    From: Delaware, USA
    Quote:

    Andreas_Wolf wrote:
    >> The Leopard WebKit author asks: [...]

    > Why he's so interested in this?

    Bigfoot's JavaScriptCore JIT compiler for PPC should also be usable with the Leopard WebKit browser (or any other WebKit-based browser on any PPC OS).


    Neat that something Mark is doing for our little "hobbyist" oriented OS could be applied to other operating systems that run on the same ISA.

    Maybe someone should have posted to that forum that the work would be done in two weeks. ;-)
    "Never attribute to malice what can more readily explained by incompetence"
  • »18.12.16 - 11:40
    Profile