Bug in picture datatype system - please help confirm it
  • Order of the Butterfly
    Order of the Butterfly
    ChrisH
    Posts: 167 from 2009/11/26
    I have found that if I load pictures of <=256 colours via datatypes, then I get a crash when I dispose of the datatype object.

    The same code works fine on AROS & OS4, and also for 24-bit pictures on MorphOS! The only way I've found is to stop the crashing is to either use PDTA_DESTMODE,PMODE_V42 or to not provide PDTA_SCREEN.

    Can anyone suggest what I might be doing wrong? As far as I can tell, I am not accessing the internals of the bitmap (which V43 disallows).

    Here is the offending E code:
    Code:
    datatypeObject := NewDTObjectA(file, [DTA_GROUPID,GID_PICTURE, PDTA_DESTMODE,PMODE_V43, PDTA_SCREEN,window.wscreen, PDTA_USEFRIENDBITMAP,useFriendBitmap, TAG_END]:tagitem)

    GetDTAttrsA(datatypeObject, [PDTA_BITMAPHEADER,ADDRESSOF bitmapheader, TAG_END]:tagitem)

    DoDTMethodA(datatypeObject, NIL, NIL, [DTM_PROCLAYOUT, NIL, TRUE]:gplayout)

    GetDTAttrsA(datatypeObject, [PDTA_DESTBITMAP,ADDRESSOF bitmap, TAG_END]:tagitem)

    and
    Code:
    WaitBlit()
    DisposeDTObject(datatypeObject)


    [ Edited by ChrisH 20.10.2011 - 17:43 ]
    Author of the PortablE programming language.
    It is pitch black. You are likely to be eaten by a grue...
  • »08.10.11 - 13:53
    Profile Visit Website
  • Order of the Butterfly
    Order of the Butterfly
    ChrisH
    Posts: 167 from 2009/11/26
    I thought the crash went away 'by itself' (or else I didn't test it right), but then it came back :-( . The crash is present in the latest preview of PictureAlbum, at least when I run it on MorphOS v2.7 on my Efika.

    I've now been able to PROVE that the crash is down to very specific corruption of my program's memory, when I perform DisposeDTObject() on a suitable datatype object. The corruption is actually a value in my own memory (which happens to be a pointer) getting decremented by one!

    If I check the value of the location before & after DisposeDTObject(), then I see it reduced by one. And if I add a dummy object member before the one that gets corrupted, then my program doesn't crash. I think this conclusively proves that the corruption is not caused by my program's code, but rather by DisposeDTObject().

    I guess I need to report this bug, but making a simple test case would be rather hard. Can anyone else confirm that PictureAlbum crashes for them when they view pictures with <= 256 colours?

    EDIT: It will probably help (show the crash) if you reset your machine BEFORE testing PictureAlbum.

    [ Edited by ChrisH 20.10.2011 - 17:43 ]
    Author of the PortablE programming language.
    It is pitch black. You are likely to be eaten by a grue...
  • »20.10.11 - 17:11
    Profile Visit Website
  • Order of the Butterfly
    Order of the Butterfly
    ChrisH
    Posts: 167 from 2009/11/26
    OK, I've been able to create a very simple test case, which I would really appreciate someone else trying out & reporting back. You can get it from here:
    http://cshandley.co.uk/temp/MOS_DT_crash_test.lha

    Reboot (please!), then unpack it to RAM, open the Shell, then run it by typing:
    MOS_DT_crash_test

    For me it reports the following:
    Quote:

    Loaded picture.
    After; byte 27 has value 74 (expected 75)
    Finished test.


    This indicates that the byte at offset 27 has been corrupted by DisposeDTObject(). The source code is included, but you cam view it here too:
    http://cshandley.co.uk/temp/MOS_DT_crash_test.cpp

    (I should note that I don't expect the test to ALWAYS see the corruption, because it relies upon MorphOS's TLSF memory system to allocate the memory block directly after whatever the datatype allocated (or perhaps in the same location as a something it allocated & then freed). This seems likely to happen, at least after a reboot when there is little external fragmentation.)

    [ Edited by ChrisH 20.10.2011 - 23:02 ]
    Author of the PortablE programming language.
    It is pitch black. You are likely to be eaten by a grue...
  • »20.10.11 - 22:49
    Profile Visit Website
  • Yokemate of Keyboards
    Yokemate of Keyboards
    Andreas_Wolf
    Posts: 12078 from 2003/5/22
    From: Germany
    > Reboot (please!), then unpack it to RAM, open the Shell, then run it by typing:
    > MOS_DT_crash_test
    >
    > For me it reports the following: [...]

    Same here (Mac mini, MorphOS 2.7):

    Code:
    Loaded picture.
    After; byte 27 has value 74 (expected 75)
    Finished test.
  • »20.10.11 - 23:17
    Profile
  • Order of the Butterfly
    Order of the Butterfly
    ChrisH
    Posts: 167 from 2009/11/26
    @Andreas_Wolf
    Thanks. I'll try reporting this through "official channels" (never done so before, so don't know what to expect).
    Author of the PortablE programming language.
    It is pitch black. You are likely to be eaten by a grue...
  • »20.10.11 - 23:21
    Profile Visit Website
  • MorphOS Developer
    kiero
    Posts: 129 from 2003/2/28
    no need. we will check it out. thanks
  • »21.10.11 - 07:22
    Profile Visit Website
  • MorphOS Developer
    Piru
    Posts: 576 from 2003/2/24
    From: finland, the l...
    I'm unable to reproduce this issue (Mac mini, MorphOS 2.7). I also tried with Wipeout (SDK:Tools/Wipeout) and was unable to generate hits.

    Can you please verify if you get any Wipeout hits? Also, just to make sure no buggy external components are involved: Does this memory trashing also happen when booted from original MorphOS 2.7 CD?
  • »22.10.11 - 11:16
    Profile
  • Order of the Butterfly
    Order of the Butterfly
    ChrisH
    Posts: 167 from 2009/11/26
    @Piru
    My test doesn't always notice the problem, since it depends on how the memory is allocated. And in fact running the test now doesn't give a positive result for me either, so you are probably just unlucky. Andreas_Wolf also saw the problem, so I think it is real. Perhaps I should have loaded the picture several times, to "guarantee" it spotted the problem.

    However, PictureAlbum always crashes for me when viewing a few 8-bit pictures, so you can try that as well.

    I had to download Wipeout. It didn't report a problem for me either (while my test was not reporting a problem anyway), but I wouldn't necessarily have expected to: All that seems to happen is that the picture datatype system decrements a value in a memory block it doesn't own (any longer?), while WipeOut checks something different (allocations & deallocations).

    My Efika is almost a totally clean MorphOS v2.7 install, so I doubt a third party component is to blame. But I will try booting off USB when I get the chance (no CD drive).

    [ Edited by ChrisH 22.10.2011 - 15:45 ]
    Author of the PortablE programming language.
    It is pitch black. You are likely to be eaten by a grue...
  • »22.10.11 - 15:44
    Profile Visit Website
  • MorphOS Developer
    Piru
    Posts: 576 from 2003/2/24
    From: finland, the l...
    Well I am extremely unlucky then, I ran it dozens of times (after clean reboot and otherwise). I'll try PictureAlbum next I guess.

    Ok, update: PictureAlbum nukes with "application meditating" crash. Is this how it b0rks when the trashing occurs?

    [ Edited by Piru 22.10.2011 - 20:13 ]
  • »22.10.11 - 18:04
    Profile
  • MorphOS Developer
    Piru
    Posts: 576 from 2003/2/24
    From: finland, the l...
    Okay I rewrote the POC app a bit, and now I can reproduce the issue. This one will be interesting to track down...
  • »22.10.11 - 18:50
    Profile
  • Acolyte of the Butterfly
    Acolyte of the Butterfly
    Georg
    Posts: 106 from 2004/4/7
    Why?

    Without even looking at or testing any code I bet the decrement comes from ReleasePen() and once you know that ...

    [ Edited by Piru 23.10.2011 - 02:46 ]
  • »22.10.11 - 19:59
    Profile
  • MorphOS Developer
    Piru
    Posts: 576 from 2003/2/24
    From: finland, the l...
    I was able to locate and fix the bug. It wasn't the actual datatype system, but rather something deeper (in certain situations palette indexed bitmap would end up with a stale pointer that would be referenced at FreeBitMap).

    The bug has been fixed in the development tree now and will be included in the next MorphOS release (3.0).
  • »23.10.11 - 00:43
    Profile
  • MorphOS Developer
    Piru
    Posts: 576 from 2003/2/24
    From: finland, the l...
    Quote:

    Why?

    Because it wasn't very obvious. It took me all this time to track it down.

    Quote:

    Without even looking at or testing any code I bet the decrement comes from ReleasePen() and once you know that ...

    It didn't.


    PS. Sorry for editing your post. For some reason I have some sort of edit powers here and accidently hit edit instead of reply. Oops :)

    [ Edited by Piru 23.10.2011 - 02:49 ]
  • »23.10.11 - 00:48
    Profile
  • Yokemate of Keyboards
    Yokemate of Keyboards
    Andreas_Wolf
    Posts: 12078 from 2003/5/22
    From: Germany
    > For some reason I have some sort of edit powers here and accidently
    > hit edit instead of reply.

    OS developers with surprising moderating power seems to be the new trend in Amiga land ;-)
  • »23.10.11 - 00:53
    Profile
  • Order of the Butterfly
    Order of the Butterfly
    ChrisH
    Posts: 167 from 2009/11/26
    @Piru
    OK, thanks, great to hear it is fixed. Can you suggest any work-around I can use? At the moment I am simply not deallocating datatype objects (for <= 256 colour pictures), which of course will cause a memory leak...
    Author of the PortablE programming language.
    It is pitch black. You are likely to be eaten by a grue...
  • »24.10.11 - 18:11
    Profile Visit Website