Will donate to bounty to get rid obnoxious Webkit RAM bug?
  • Order of the Butterfly
    Order of the Butterfly
    Raf_MegaByte
    Posts: 430 from 2004/10/10
    From: Nella grande r...
    OWB for MorphOS is afflicted by obnoxious bug that slowly erodes RAM. During intense websurfing or after watching 2 or 3 videos all system RAM got occupied at 95% or more forcing user to close OWB and loading it again forcing it release memory. Or worse, Webkit fails to allocate n-th bytes letting OWB going berserk bomb without freeing memory so it is necessary to reset whole machine.
    But OWB is not guilty for that.
    The bug lies in Webkit as I heard. This fact is hidden on MacOS or Windows as they benefit of virtual memory, so Webkit could run with no apparent fault as it will just messing with an amount of memory as big as whole hard disks installed on these system.
    It is no mecessary to spot this bug on these systems, hence thus developers of Webkit never had interest in correcting the bug. unfortunately this fact is obnoxious on our 1 GB RAM MacMinis, and 1,5 GB RAM Pegasos or Powerbook machines.
    I wonder if there is people in whole Amigaland who want to donate in a bounty open to any Amiga developers who will inspect behaviour of OWB with webkit html rendering engine in order to locate the bug and get rid of it once and for all.
    I will donate for it, if a bounty will be issued, and you?

    [ Edited by Raf_MegaByte 08.05.2015 - 18:22 ]
    Bill Gates "Think!", Steve Jobs: "Think different!" So... Let these guy continue blabbering thinking and enjoy computing! We are on Amiga!
  • »08.05.15 - 16:28
    Profile
  • Moderator
    hooligan
    Posts: 1948 from 2003/2/23
    From: Lahti, Finland
    This is the main reason I don't browse with MorphOS. I used to spend a couple dozen minutes in the mornings with coffee to browse news.. ran out of memory 2-4 times every morning.
    www.mikseri.net/hooligan <- Free music
  • »08.05.15 - 18:00
    Profile Visit Website
  • Order of the Butterfly
    Order of the Butterfly
    Raf_MegaByte
    Posts: 430 from 2004/10/10
    From: Nella grande r...
    You are lucky. I have to close OWB almost any hour. In this moment I have 5 pages open, RAM at 77,9 % RAM in use and rising.
    Problems of Bomb allocation could start anytime occurring from 83% RAM in use. System starts to not respond GUI with 95%RAM in use.

    [ Edited by Raf_MegaByte 09.05.2015 - 08:42 ]
    Bill Gates "Think!", Steve Jobs: "Think different!" So... Let these guy continue blabbering thinking and enjoy computing! We are on Amiga!
  • »08.05.15 - 18:06
    Profile
  • Butterfly
    Butterfly
    eliyahu
    Posts: 67 from 2011/4/21
    @thread

    i also have this issue whether running odyssey in MOS or AOS4, although it doesn't seem nearly as problematic when i turn JS off. it could well be fixed in more recent webkit builds. but since fab's code is private, only he can integrate a new version.

    -- eliyahu
  • »08.05.15 - 18:29
    Profile
  • Priest of the Order of the Butterfly
    Priest of the Order of the Butterfly
    Cego
    Posts: 733 from 2006/5/28
    From: Germany
    i would also donate to fix this bug.
    Pegasos II G4 @1.0GHz, 1GB DDR Ram, Radeon 9200Pro, 240GB SSD+160GB HD, MorphOS 3.18, AmigaOS4.1 FE, Debian 8
  • »08.05.15 - 19:03
    Profile
  • Priest of the Order of the Butterfly
    Priest of the Order of the Butterfly
    Cego
    Posts: 733 from 2006/5/28
    From: Germany
    and i hope that the JIT for javascript will come with the next OWB release :)
    Pegasos II G4 @1.0GHz, 1GB DDR Ram, Radeon 9200Pro, 240GB SSD+160GB HD, MorphOS 3.18, AmigaOS4.1 FE, Debian 8
  • »08.05.15 - 19:04
    Profile
  • Order of the Butterfly
    Order of the Butterfly
    pegasos-sigi2
    Posts: 265 from 2006/8/31
    A possible fix for eating up to much memory:

    Open the settings window of OWB and change the slider with the label "active connections".
    I change this to "20" and OWB eats only after 20 parallel tabs my memory.
    Lower than 20 connections are no problem by me.
    Spreedy - The spreedsheet editor of MorphOS
    Calimero - Do you know about the new DTP Program for MorphOS ?
  • »08.05.15 - 19:57
    Profile
  • Order of the Butterfly
    Order of the Butterfly
    Raf_MegaByte
    Posts: 430 from 2004/10/10
    From: Nella grande r...
    Quote:

    pegasos-sigi2 wrote:
    A possible fix for eating up to much memory:

    Open the settings window of OWB and change the slider with the label "active connections".
    I change this to "20" and OWB eats only after 20 parallel tabs my memory.
    Lower than 20 connections are no problem by me.


    My usual maximum is set up to only 16 connections! Sigh! TT_TT

    [ Edited by Raf_MegaByte 08.05.2015 - 21:28 ]
    Bill Gates "Think!", Steve Jobs: "Think different!" So... Let these guy continue blabbering thinking and enjoy computing! We are on Amiga!
  • »08.05.15 - 20:26
    Profile
  • Order of the Butterfly
    Order of the Butterfly
    In_Correct
    Posts: 245 from 2012/10/14
    From: DFW, TX, USA
    Quote:

    Raf_MegaByte wrote:
    OWB for MorphOS is afflicted by obnoxious bug that slowly erodes RAM. During intense websurfing or after watching 2 or 3 videos all system RAM got occupied at 95% or more forcing user to close OWB and loading it again forcing it release memory. Or worse, Webkit fails to allocate n-th bytes letting OWB going berserk bomb without freeing memory so it is necessary to reset whole machine.
    But OWB is not guilty for that.
    The bug lies in Webkit as I heard. This fact is hidden on MacOS or Windows as they benefit of virtual memory, so Webkit could run with no apparent fault as it will just messing with an amount of memory as big as whole hard disks installed on these system.
    It is no mecessary to spot this bug on these systems, hence thus developers of Webkit never had interest in correcting the bug. unfortunately this fact is obnoxious on our 1 GB RAM MacMinis, and 1,5 GB RAM Pegasos or Powerbook machines.
    I wonder if there is people in whole Amigaland who want to donate in a bounty open to any Amiga developers who will inspect behaviour of OWB with webkit html rendering engine in order to locate the bug and get rid of it once and for all.
    I will donate for it, if a bounty will be issued, and you?


    Yes, I will donate to this bounty. I will always donate to MorphOS and AROS if it increases development for those operating systems. :-)

    Currently I will not be able to donate much to bounties. My budget for this is spent on purchasing computers for these OS.
    :-) I Support Quark Microkernel. :-D
  • »09.05.15 - 00:08
    Profile Visit Website
  • Order of the Butterfly
    Order of the Butterfly
    Raf_MegaByte
    Posts: 430 from 2004/10/10
    From: Nella grande r...
    Quote:

    In_Correct wrote:
    Currently I will not be able to donate much to bounties. My budget for this is spent on purchasing computers for these OS.


    Me too is low on budget, but I think that to get rid of this obnoxius bug I could donate from 5 upto 10US$.

    That's why bounties exist, to donate a minimum amount of money each other in order to hit a precise target.

    I spotted a certain interest on this topic so next logic step is create the bounty..



    How to made one? Please help me. I am newbie to that. Any secure site to use?



    P.S.I created a parallel discussion on Amigaworld too.

    User Broadblues who is also a programmer suggested that the problem could lie into OWB system of caching sites in RAM.
    Interesting.

    http://amigaworld.net/modules/newbb/viewtopic.php?topic_id=40194&start=0&post_id=759337&order=0&viewmode=flat&pid=759333&forum=2#759337

    And user Deadwood suggested that pressing F12 makes appear a requester to clean the caches, but I don't see it happening on OWB MorphOS!

    http://amigaworld.net/modules/newbb/viewtopic.php?topic_id=40194&start=0&post_id=759337&order=0&viewmode=flat&pid=759333&forum=2#759333

    [ Edited by Raf_MegaByte 09.05.2015 - 08:39 ]
    Bill Gates "Think!", Steve Jobs: "Think different!" So... Let these guy continue blabbering thinking and enjoy computing! We are on Amiga!
  • »09.05.15 - 04:11
    Profile
  • jPV
  • Yokemate of Keyboards
    Yokemate of Keyboards
    jPV
    Posts: 2096 from 2003/2/24
    From: po-RNO
    It's not that bad on the pages I visit, or maybe because I just don't have too many tabs opened all the time... it goes hours or sometime whole day without restarting.

    BTW. has anyone tried how different Cache Models behave with OWB and memory usage? You can select them in Settings->Content->Cache Model. I have Primary Web Browser selected. Or if you even disable whole Page Cache there?


    [ Edited by jPV 09.05.2015 - 09:54 ]
  • »09.05.15 - 07:53
    Profile Visit Website
  • Fab
  • MorphOS Developer
    Fab
    Posts: 1331 from 2003/6/16
    @Raf_MegaByte

    You seem rather confused about what causes that leak... So let's rule out all the unrelated factors :

    - It has *nothing* to do with network connections setting
    - The problem doesn't reside in WebKit caching system
    - I developed the F12 key cache purge option on MorphOS, so of course it works, but it only purges what can be purged (images, fonts, css, scripts, internal caches), which leads us to the real leak reason : JavascriptCore memory handling.

    JavascriptCore allocate (lots of) memory blocks in virtual memory and these blocks can only be released when all runtime references created inside these blocks (javascript variables, objects, whatever) are released, which is unfortunately not often the case, due to several reasons (also related to bad javascript programming habits, actually).

    I don't know if they fixed this since the latest build i tried. It seems unlikely.

    That said, i can still use OWB for days without exhausting my memory, and that includes youtube videos, facebook and other resource hungry pages. And when the memory requester appears, pressing 'quit' most often works, actually.
  • »09.05.15 - 09:44
    Profile Visit Website
  • Order of the Butterfly
    Order of the Butterfly
    In_Correct
    Posts: 245 from 2012/10/14
    From: DFW, TX, USA
    Quote:

    Raf_MegaByte wrote:
    Quote:

    In_Correct wrote:
    Currently I will not be able to donate much to bounties. My budget for this is spent on purchasing computers for these OS.


    Me too is low on budget, but I think that to get rid of this obnoxius bug I could donate from 5 upto 10US$.

    That's why bounties exist, to donate a minimum amount of money each other in order to hit a precise target.

    I spotted a certain interest on this topic so next logic step is create the bounty..



    How to made one? Please help me. I am newbie to that. Any secure site to use?



    P.S.I created a parallel discussion on Amigaworld too.

    User Broadblues who is also a programmer suggested that the problem could lie into OWB system of caching sites in RAM.
    Interesting.

    http://amigaworld.net/modules/newbb/viewtopic.php?topic_id=40194&start=0&post_id=759337&order=0&viewmode=flat&pid=759333&forum=2#759337

    And user Deadwood suggested that pressing F12 makes appear a requester to clean the caches, but I don't see it happening on OWB MorphOS!

    http://amigaworld.net/modules/newbb/viewtopic.php?topic_id=40194&start=0&post_id=759337&order=0&viewmode=flat&pid=759333&forum=2#759333


    MorphZone has a bounty manager. I do not remember the forum user who is assigned as bounty manager. But I know there is one.

    And, on top of page:

    http://bounties.morph.zone/

    [ Edited by In_Correct 09.05.2015 - 03:47 ]
    :-) I Support Quark Microkernel. :-D
  • »09.05.15 - 09:46
    Profile Visit Website
  • Order of the Butterfly
    Order of the Butterfly
    Raf_MegaByte
    Posts: 430 from 2004/10/10
    From: Nella grande r...
    @Fab

    Quote:

    Fab wrote:
    @Raf_MegaByte
    You seem rather confused about what causes that leak... So let's rule out all the unrelated factors :

    It has *nothing* to do with network connections setting



    It was not me, it was pegasos_sigi2

    Quote:


    The problem doesn't reside in WebKit caching system



    It was suggested by Broadblues

    Quote:


    I developed the F12 key cache purge option on MorphOS, so of course it works, but it only purges what can be purged (images, fonts, css, scripts, internal caches), which leads us to the real leak reason : JavascriptCore memory handling.



    Oh, so F12 works automatically without prompting anything. Words of Deadwood confused me.

    Infact actually I was at 71,4% memory usage, I pressed F12 and it flushed memory down to 65%.

    it is not so many memory but is a palliative, thanks.

    Quote:


    That said, i can still use OWB for days without exhausting my memory, and that includes youtube videos, facebook and other resource hungry pages. And when the memory requester appears, pressing 'quit' most often works, actually.


    I know it works many often, but it is a mandatory issue, not a solution.

    And, not to lack of respect, but when memory requester prompts, it is not a good view specialy when I have guests at home laughing about how pitiful is MorphOS, neither good for anyone wants to approach MorphOS coming from OSX, for example.

    [ Edited by Raf_MegaByte 09.05.2015 - 12:47 ]
    Bill Gates "Think!", Steve Jobs: "Think different!" So... Let these guy continue blabbering thinking and enjoy computing! We are on Amiga!
  • »09.05.15 - 11:30
    Profile
  • Acolyte of the Butterfly
    Acolyte of the Butterfly
    Georg
    Posts: 111 from 2004/4/7
    Quote:

    Fab wrote:

    JavascriptCore allocate (lots of) memory blocks in virtual memory and these blocks can only be released when all runtime references created inside these blocks (javascript variables, objects, whatever) are released, which is unfortunately not often the case, due to several reasons (also related to bad javascript programming habits, actually).



    Would it be possible to (optionally) have a stronger isolation of the tabs from each other. So (not knowing the code), if at the moment it is like one single ~browser "object" and multiple ~ŧab/page "objects" you change that to one ~browser "object" per open tab. So when closing tab -> browser object killed -> more/all memory gets freed just like what happens now when quitting Odyssey.
  • »09.05.15 - 14:23
    Profile
  • Fab
  • MorphOS Developer
    Fab
    Posts: 1331 from 2003/6/16
    Quote:

    Georg wrote:

    Would it be possible to (optionally) have a stronger isolation of the tabs from each other. So (not knowing the code), if at the moment it is like one single ~browser "object" and multiple ~ŧab/page "objects" you change that to one ~browser "object" per open tab. So when closing tab -> browser object killed -> more/all memory gets freed just like what happens now when quitting Odyssey.




    I actually tried to use one instance of the js virtual machine per browser object, but it didn't work as expected. By design, in WebKit1, the VM is shared by all instances, and it relies on that. In WebKit2, that's different, each browser is isolated in a thread, with its own js VM instance, so it wouldn't be a problem there. At task destruction, all the memory could easily be released. But porting WebKit2 is quite some work.


    [ Edited by Fab 10.05.2015 - 22:25 ]
  • »09.05.15 - 23:17
    Profile Visit Website
  • Order of the Butterfly
    Order of the Butterfly
    Raf_MegaByte
    Posts: 430 from 2004/10/10
    From: Nella grande r...
    @Fab: two questions:

    1) Could promised Javascript JIT resolve many of the RAM occupation issues?

    2) OWB coul create a huge void file on Hard Disk and use it as fake Virtual Memory?
    Even if limited to 4 GB filesize of many of Amiga Filesystems, it sure will multiply existing available Memory for OWB tabs, and space on file could be easily wiped again and again "on the fly" to make it free for any new surfing occurrence.

    [ Edited by Raf_MegaByte 10.05.2015 - 08:11 ]
    Bill Gates "Think!", Steve Jobs: "Think different!" So... Let these guy continue blabbering thinking and enjoy computing! We are on Amiga!
  • »10.05.15 - 07:10
    Profile
  • Order of the Butterfly
    Order of the Butterfly
    pegasos-sigi2
    Posts: 265 from 2006/8/31
    FAB tell you a possible fix for memorytrashing in the last text.
    The fix calls "WebKitV2". But the porting to MorphOS if needs more time and
    is more complex.








    Quote:

    Raf_MegaByte wrote:
    @Fab: two questions:

    1) Could promised Javascript JIT resolve many of the RAM occupation issues?

    2) OWB coul create a huge void file on Hard Disk and use it as fake Virtual Memory?
    Even if limited to 4 GB filesize of many of Amiga Filesystems, it sure will multiply existing available Memory for OWB tabs, and space on file could be easily wiped again and again "on the fly" to make it free for any new surfing occurrence.
    Spreedy - The spreedsheet editor of MorphOS
    Calimero - Do you know about the new DTP Program for MorphOS ?
  • »10.05.15 - 09:21
    Profile
  • Paladin of the Pegasos
    Paladin of the Pegasos
    Yasu
    Posts: 1724 from 2012/3/22
    From: Stockholm, Sweden
    @Fab

    Is porting to v2 at all realistic? If yes, will you do it for a bounty? And how much?
    AMIGA FORUM - Hela Sveriges Amigatidning!
    AMIGA FORUM - Sweden's Amiga Magazine!

    My MorphOS blog
  • »10.05.15 - 09:43
    Profile Visit Website
  • Paladin of the Pegasos
    Paladin of the Pegasos
    redrumloa
    Posts: 1424 from 2003/4/13
    Quote:

    Fab wrote:
    That said, i can still use OWB for days without exhausting my memory, and that includes youtube videos, facebook and other resource hungry pages. And when the memory requester appears, pressing 'quit' most often works, actually.


    You must be the only MorphOS user on the planet who can go days without needing to restart OWB. I average 2-3 times an hour on my G5.
  • »10.05.15 - 13:05
    Profile
  • Moderator
    Kronos
    Posts: 2325 from 2003/2/24
    @red

    7 hours uptime, and sofar no need to restart OWB after visiting plenty sites.

    60% (about 1GB) of RAM used, so it probraly won't last for days, but 1 day is no problem.

    I can't even imagine what crappy sites I would need on constant reload to run into trouble several times an hour.
  • »10.05.15 - 13:39
    Profile
  • Paladin of the Pegasos
    Paladin of the Pegasos
    Jupp3
    Posts: 1193 from 2003/2/24
    From: Helsinki, Finland
    This definitely affects every webkit-based browser out there. Sure, on many platforms it only means that leaked memory will slowly pile up in swap (which doesn't really slow down system that much, as once there, it never needs to be accessed). Sure, it goes "via ram" (swapped out when some other program needs it), but that's about it.

    Basically means that if you are low on memory + swap or do some REALLY heavy browsing, you have to restart browser once or twice a day.

    This far, I intentionally left out one very major hardware group: mobile phones & tablets. There, these memory issues are much worse, due to less ram & swap.

    The point is, if it was easy to fix, some of the "big players" (Apple, Google, KDE, Opera...) would have already done that. If they can't, what hope would a lone MorphOS coder have?

    Of course Firefox engine is "at least as bad", and considering even Apple picked 3rd party solution, rather than creating an own closed one should prove how big task creating a proper web engine from scratch would be.

    As for (not) checking memory allocations, why isn't it done then?

    Simple answer: (on some platforms, namely those it was developed on), the only difference memory allocations would do is increase binary size (and memory requirements) slightly.

    Consider following (slightly technical):
    Quote:


    // Assuming size is >0 and either there is or isn't enough memory available:
    if((allocation=malloc(size)))
    {
    // MorphOS never ends up here, unless there was enough memory for allocation.
    allocation[0]=value;

    // On linux, if there was not enough memory available, kernel notices it during the previous call, and kills a seemingly random task (which might or might not be the one that caused running out of memory) to free memory.

    // If allocation was successful, MorphOS always reaches this point without problems.
    }
    else
    {
    // Memory allocation failed.
    // MorphOS ends up here, if there wasn't enough memory available.
    // Linux practically never ends up here, especially on 64bit systems.
    // Linux CAN end up here, if user tries to make an allocation, that exceeds the virtual memory the task has left. This has nothing to do with available memory, and is basically "how many different addresses can be represented by 32/64/whatever bits. On 32bits that's 4 gigabytes (- some used by kernel), on 64 bit... Well, each additional bit doubles the amount.
    }



    That assuming "default" linux settings. As you can see, the allocation check doesn't really make much difference there.

    Fab:
    Would it be possible to make download manager a somewhat separate program, that would crash independently from OWB? (Meaning, if I have to quit OWB, downloads would still keep running)
  • »10.05.15 - 13:40
    Profile Visit Website
  • Yokemate of Keyboards
    Yokemate of Keyboards
    Andreas_Wolf
    Posts: 12157 from 2003/5/22
    From: Germany
    > if it was easy to fix, some of the "big players" (Apple, Google, KDE, Opera...)
    > would have already done that.

    These "big players" don't use WebKit1 anymore anyway but WebKit2 or Blink*, so their browsers are not affected. There is no incentive for them to fix this WebKit1 issue at hand, whether they are able to or not.

    * Blink itself doesn't have multi-process architecture, but existing browsers using Blink implement this feature at browser level instead of engine level.
  • »10.05.15 - 18:32
    Profile
  • Order of the Butterfly
    Order of the Butterfly
    Raf_MegaByte
    Posts: 430 from 2004/10/10
    From: Nella grande r...
    Quote:

    Yasu wrote:
    @Fab

    Is porting to v2 at all realistic? If yes, will you do it for a bounty? And how much?


    You stole the words from my mouth.

    So then let's create a bounty for Webkit_2 then. ^_^
    :)
    Bill Gates "Think!", Steve Jobs: "Think different!" So... Let these guy continue blabbering thinking and enjoy computing! We are on Amiga!
  • »10.05.15 - 23:29
    Profile
  • MorphOS Developer
    jacadcaps
    Posts: 3108 from 2003/3/5
    From: Canada
    Quote:

    Raf_MegaByte wrote:
    So then let's create a bounty for Webkit_2 then. ^_^
    :)


    You do realize that MorphOS or any other amigaoid OS won't benefit that much from WebKit2? Even if each tab is a (set of) separate processes, there still won't be a way to clean it up nicely if it crashes (you'd be surprised how often Chrome crashes and reloads a page before you even notice), and there still won't be a way to regain the lost memory. Also mind that if you run out of memory when just using a single website like Facebook with current Odyssey, you would still run out of memory with a WebKit2 based one.
  • »11.05.15 - 05:32
    Profile Visit Website