Lha bails out with CRC error on unpacking
  • Paladin of the Pegasos
    Paladin of the Pegasos
    Intuition
    Posts: 1110 from 2013/5/24
    From: Nederland
    Quote:

    asrael22 wrote:
    No. What are you implying?



    That the version of lha that congress with MorphOS is incompatible with archives the Mac version creates. MorphOS itself is however compatible with many 68k binaries and you previously stated that the 68k version is compatible with the archives the Mac version creates.

    From there it's not a great leap of logic to suggest trying the 68k binary is it?
    1.67GHz 15" PowerBook G4, 1GB RAM, 128MB Radeon 9700M Pro, 64GB SSD, MorphOS 3.15

    2.7GHz DP G5, 4GB RAM, 512MB Radeon X1950 Pro, 500GB SSHD, MorphOS 3.9
  • »26.04.17 - 11:35
    Profile
  • Order of the Butterfly
    Order of the Butterfly
    asrael22
    Posts: 404 from 2014/6/11
    From: Germany
    Quote:

    Intuition wrote:
    Quote:

    asrael22 wrote:
    No. What are you implying?



    That the version of lha that congress with MorphOS is incompatible with archives the Mac version creates. MorphOS itself is however compatible with many 68k binaries and you previously stated that the 68k version is compatible with the archives the Mac version creates.

    From there it's not a great leap of logic to suggest trying the 68k binary is it?


    I don't want to use a 68k binary for unpacking an lha archive on MorphOS.
    Nor would I want to go and download something that can do such a fundamentally important task as unpacking a LHA archive.
    Even if is has been created on a different system.
    Why not package up an LHA version into MorphOS that works.


    Manfred

    [ Edited by asrael22 26.04.2017 - 14:31 ]
  • »26.04.17 - 12:18
    Profile
  • Paladin of the Pegasos
    Paladin of the Pegasos
    Intuition
    Posts: 1110 from 2013/5/24
    From: Nederland
    Quote:

    asrael22 wrote:
    Quote:

    Intuition wrote:
    Quote:

    asrael22 wrote:
    No. What are you implying?



    That the version of lha that congress with MorphOS is incompatible with archives the Mac version creates. MorphOS itself is however compatible with many 68k binaries and you previously stated that the 68k version is compatible with the archives the Mac version creates.

    From there it's not a great leap of logic to suggest trying the 68k binary is it?


    I don't want to use a 68k binary for unpacking an lha archive on MorphOS.
    Nor would I want to go and download something that can do such a fundamentally important task as unpacking a LHA archive.
    Even if is has been created on a different system.
    Why not package up an LHA version into MorphOS that works.


    Manfred


    In gave you a solution that solves your problem, up to you if you'd rather just whine about it.
    1.67GHz 15" PowerBook G4, 1GB RAM, 128MB Radeon 9700M Pro, 64GB SSD, MorphOS 3.15

    2.7GHz DP G5, 4GB RAM, 512MB Radeon X1950 Pro, 500GB SSHD, MorphOS 3.9
  • »26.04.17 - 15:52
    Profile
  • Priest of the Order of the Butterfly
    Priest of the Order of the Butterfly
    connor
    Posts: 570 from 2007/7/29
    Quote:

    asrael22 wrote:
    I can try all that. But why the heck is the version shipped with MorphOS not able to unpack this?


    Manfred


    Maybe because lha on different platforms was continued by different people. The programs have the same name but not always the same features or bugfixes. Also your version on OSX is verys old (lha 1.14). But it seems there is no newer one.
    Maybe you can give your archive to the MOS team and they can have a look if our lha can be fixed or if your OSX lha needs a fix instead.

    Quote:

    asrael22 wrote:
    OK, but why does m68k Lha 2.15 unpack this but not 2.7.12?
    Is 2.7.12 an older version than AmigaOS 3 m68k 2.15?



    Yes. 7 is smaller than 15, so 2.7.12 is older than 2.15.


    Quote:

    asrael22 wrote:

    I don't want to use a 68k binary for unpacking an lha archive on MorphOS.
    Nor would I want to go and download something that can do such a fundamentally important task as unpacking a LHA archive.
    Even if is has been created on a different system.
    Why not package up an LHA version into MorphOS that works.


    Manfred


    LHA of MOS works very well with tens of thousands of archives. I wouldn't say it does not work. You just use one at least 13 year old version on a never well supported platform that can create an incompatible archive.
  • »26.04.17 - 16:42
    Profile
  • Order of the Butterfly
    Order of the Butterfly
    asrael22
    Posts: 404 from 2014/6/11
    From: Germany
    Quote:

    connor wrote:
    Quote:

    asrael22 wrote:
    I can try all that. But why the heck is the version shipped with MorphOS not able to unpack this?


    Manfred


    Maybe because lha on different platforms was continued by different people. The programs have the same name but not always the same features or bugfixes. Also your version on OSX is verys old (lha 1.14). But it seems there is no newer one.
    Maybe you can give your archive to the MOS team and they can have a look if our lha can be fixed or if your OSX lha needs a fix instead.

    Quote:

    asrael22 wrote:
    OK, but why does m68k Lha 2.15 unpack this but not 2.7.12?
    Is 2.7.12 an older version than AmigaOS 3 m68k 2.15?



    Yes. 7 is smaller than 15, so 2.7.12 is older than 2.15.


    Quote:

    asrael22 wrote:

    I don't want to use a 68k binary for unpacking an lha archive on MorphOS.
    Nor would I want to go and download something that can do such a fundamentally important task as unpacking a LHA archive.
    Even if is has been created on a different system.
    Why not package up an LHA version into MorphOS that works.


    Manfred


    LHA of MOS works very well with tens of thousands of archives. I wouldn't say it does not work. You just use one at least 13 year old version on a never well supported platform that can create an incompatible archive.


    The solution is simple.
    Just include a PPC build of 2.15 into MorphOS.

    Manfred
  • »26.04.17 - 17:17
    Profile
  • MorphOS Developer
    Piru
    Posts: 587 from 2003/2/24
    From: finland, the l...
    EDIT: The actual issue here was the incompatibility between UNIX and Amiga LhA. See post here.


    Quote:

    What might be the problem there?

    MorphOS LhA doesn't currently support "lh7" compression method (which is the default of unix LhA).
    Quote:

    Are there any cmd line options I could try?

    If you wish to use MorphOS LhA to decompress the archives, you can -o6 option with UNIX LhA to force the use of the supported compression method "lh6".

    For example:
    lha ao6 mystuff.lha mystuff


    [ Edited by Piru 27.04.2017 - 22:45 ]
  • »26.04.17 - 17:48
    Profile
  • MorphOS Developer
    Piru
    Posts: 587 from 2003/2/24
    From: finland, the l...
    Quote:

    The solution is simple.

    I'm afraid it is not.
    Quote:

    Just include a PPC build of 2.15 into MorphOS.

    For some reason Amiga LhA is not open source.

    Anyway Amiga LhA has some issues. It doesn't handle large files and archive sizes for example. Just try using >4GB archive with LhA 2.15 and see it fail miserably.

    [ Edited by Piru 26.04.2017 - 20:57 ]
  • »26.04.17 - 17:49
    Profile
  • Order of the Butterfly
    Order of the Butterfly
    asrael22
    Posts: 404 from 2014/6/11
    From: Germany
    Quote:

    Piru wrote:
    Quote:

    The solution is simple.

    I'm afraid it is not.
    Quote:

    Just include a PPC build of 2.15 into MorphOS.

    For some reason Amiga LhA is not open source.


    Uhm. That's unfortunate.
    What about the LHA Unix version? Couldn't that be used instead? AROS seems to use it, too.

    Alright, the lh7 compression it is which makes the incompatibility.
    But I'm wondering why some files can be uncompressed. But at certain files there is this CRC error.


    Manfred
  • »26.04.17 - 18:18
    Profile
  • MorphOS Developer
    Piru
    Posts: 587 from 2003/2/24
    From: finland, the l...
    Quote:

    asrael22 wrote:
    What about the LHA Unix version? Couldn't that be used instead?

    No, it lacks support for amiga attributes and comments, among other things.

    Quote:

    AROS seems to use it, too.

    And thus AROS has a problem where their LhA doesn't support amiga attributes and comments.

    Quote:

    But I'm wondering why some files can be uncompressed.

    No lh7 compressed files can be decompressed. The MorphOS LhA doesn't even try.
  • »26.04.17 - 18:37
    Profile
  • Order of the Butterfly
    Order of the Butterfly
    asrael22
    Posts: 404 from 2014/6/11
    From: Germany
    Quote:

    Piru wrote:
    Quote:

    asrael22 wrote:
    What about the LHA Unix version? Couldn't that be used instead?

    No, it lacks support for amiga attributes and comments, among other things.


    OK, that's a point.

    Quote:

    Piru wrote:
    Quote:

    But I'm wondering why some files can be uncompressed.

    No lh7 compressed files can be decompressed. The MorphOS LhA doesn't even try.



    I don't understand.
    I have created an archive on macOS with LHA 1.14i. If it compresses files with lh7 as default, then MorphOS shouldn't be able to uncompress any of the files?
    But it does. Some files are uncompressed, like 20 or 30, don't know exactly.
    Then it stops with error, always at the same file.


    Manfred
  • »26.04.17 - 19:03
    Profile
  • Yokemate of Keyboards
    Yokemate of Keyboards
    Andreas_Wolf
    Posts: 12150 from 2003/5/22
    From: Germany
    > 2.7.12 is older than 2.15.

    Code:
    > version c:lha full
    LhA 2.7.12 (26.07.13)

    Code:
    > version ram:lha_68k full
    LhA 2.15 68000+ Jan 3 2011

    ;-)

    > one at least 13 year old version

    "Latest commit [...] on Oct 15, 2016"
    https://github.com/jca02266/lha

    That's still version 1.14i as far as I can see.

    > on a never well supported platform

    LHa for UNIX not supporting macOS well? :-)
  • »26.04.17 - 19:38
    Profile
  • Order of the Butterfly
    Order of the Butterfly
    asrael22
    Posts: 404 from 2014/6/11
    From: Germany
    Quote:

    Intuition wrote:
    In gave you a solution that solves your problem, up to you if you'd rather just whine about it.



    Yes, you gave me a solution. Thanks for that.
    If I whine, then not because I'm unable to uncompress this archive.
    But for a negative user experience of MorphOS.
    I whine because I'd like that fixed in future versions of MorphOS.


    Manfred

    [ Edited by asrael22 26.04.2017 - 22:22 ]
  • »26.04.17 - 20:18
    Profile
  • Yokemate of Keyboards
    Yokemate of Keyboards
    Andreas_Wolf
    Posts: 12150 from 2003/5/22
    From: Germany
    > I have created an archive on macOS with LHA 1.14i. If it compresses files with lh7 as
    > default, then MorphOS shouldn't be able to uncompress any of the files? But it does.
    > Some files are uncompressed [...]. Then it stops with error, always at the same file.

    The 6th byte of the archive tells the compression method used.


    Edit: That's not quite correct. The 6th byte of the archive tells the compression method used for the first object in the archive. Each object in the archive is preceded by such an indicator as different objects may have been processed using different methods.

    [ Edited by Andreas_Wolf 27.04.2017 - 10:03 ]
  • »26.04.17 - 20:29
    Profile
  • MorphOS Developer
    Piru
    Posts: 587 from 2003/2/24
    From: finland, the l...
    Quote:

    asrael22 wrote:
    Quote:

    Piru wrote:
    Quote:

    But I'm wondering why some files can be uncompressed.

    No lh7 compressed files can be decompressed. The MorphOS LhA doesn't even try.



    I don't understand.

    I have created an archive on macOS with LHA 1.14i. If it compresses files with lh7 as default, then MorphOS shouldn't be able to uncompress any of the files?


    There is no code there to decompress lh7.

    Maybe you're confused by for example directories and such being created (directories obviously are not compressed), or empty or very small files being there included without compression (lh0).
  • »26.04.17 - 22:58
    Profile
  • Order of the Butterfly
    Order of the Butterfly
    asrael22
    Posts: 404 from 2014/6/11
    From: Germany
    OK, Andreas mentioning the 6th byte led me to have a closer look.

    So here are the last console lines of the uncompress attempt:
    Code:

    Extracting: ( 19510/ 19510) B/BadCat/data/bike.sam
    Extracting: ( 3632/ 3632) B/BadCat/data/boing.sam
    Extracting: ( 0/ 0) B/BadCat/data/c/
    Extracting: ( 2324/ 2324) B/BadCat/data/c/Run
    Extracting: ( 0/ 0) B/BadCat/data/devs/
    Extracting: ( 232/ 232) B/BadCat/data/devs/system-configuration
    Extracting: ( 2992/ 2992) B/BadCat/data/down.sam
    Extracting: ( 2188/ 2188) B/BadCat/data/exe
    Extracting: ( 200/ 200) B/BadCat/data/hall of fame
    Extracting: ( 7444/ 7444) B/BadCat/data/hey.sam
    *** Error on file 'i' : Failed CRC Check
    WARNING: Skipping corrupt/extraneous data
    264 files extracted, 1 file failed.

    Operation not entirely successful.


    The next file of name 'i' is weird but it is correct. The next file in the archive is a file named 'i'.

    Then I looked with a hex editor:
    The 6th byte of the archive file is "d" => "-lhd-". Not sure what that means.

    Finding the file "hey.sam" in the hext code reveils that it is compressed with -lh5-.
    Code:
    -lh5-Ô����ÑIΩ# ˜U��g 
    �hey.sam


    Then, searching for the next occurence of "-lh" should find the 'i' file.
    And indeed.
    But it seems to be compressed with -lh5- as well.
    Code:
    -lh5-y3��Ëg��»IΩ# ÔjU��}�i�BˇB



    That leads me to think there might be a bug in MorphOS lha?



    Manfred
  • »27.04.17 - 07:04
    Profile
  • Yokemate of Keyboards
    Yokemate of Keyboards
    Andreas_Wolf
    Posts: 12150 from 2003/5/22
    From: Germany
    > The 6th byte of the archive file is "d" => "-lhd-". Not sure what that means.

    'lhd' means the respective object (first object in the archive in this case) is a directory. As you figured out already despite my confusing comment, each object in the archive is preceded by such an indicator as different objects may have been processed using different methods.
  • »27.04.17 - 08:13
    Profile
  • MorphOS Developer
    Piru
    Posts: 587 from 2003/2/24
    From: finland, the l...
    @asrael22

    Okay that should not happen.

    I managed to find the "i" file myself, and problem is reproduced. I will need to look into this issue.

    [ Edited by Piru 27.04.2017 - 17:24 ]
  • »27.04.17 - 13:33
    Profile
  • MorphOS Developer
    Piru
    Posts: 587 from 2003/2/24
    From: finland, the l...
    Some extensive background - some of it technical - follows. So bear with me, this is rather interesting story. I had merrily forgotten all about it, but it has already been 14 years since I last laid eyes on that particular code fragment...

    Issue: Amiga and UNIX LhA archives are incompatible. That is you can have a old Amiga LhA archive that cannot be decompressed with UNIX LhA. Similarly, you can have a UNIX LhA archive that cannot be decompressed with Amiga LhA. The actual difference is regarding what values the algorithm can expect to find from the dictionary: ' ' (space) or 0 (zero). Amiga LhA uses zero, UNIX LhA uses space.

    I believe the problem was solved at some point by making sure that Amiga LhA compressor creates archives that are compatible with the UNIX LhA in all situations. I believe this might actually result in slightly larger archives vs the older incompatible compressor. However, this cannot possibly fix the already existing incompatible archives.

    Sometime around the same time a decision was made to modify the Amiga LhA decompressor to work similar to the UNIX LhA: That is, it would fail with those older archives.

    Now, here comes the dilemma: The new LhA would create archives that are always ok, regardless of which LhA you used to decompress them. However: It would now itself fail to decompress the old incompatible Amiga LhA archives that had been created earlier! I didn't like that much, since for example it lead to the situation where many existing Amiga LhA archives would just fail to extract. So I made it so that MorphOS LhA worked with the Amiga LhA archives in this corner case, instead of the UNIX ones.

    Now, this meant that occasionally you would run into UNIX LhA archives that MorphOS LhA command would fail to extract. And this is exactly what is happening here.

    To remedy the situation I decided to create somewhat of a hack:
    1. By default I now expect to get UNIX like archive at decompress.
    2. If an CRC error occurs I will try to decompress it again with the old AMIGA LhA specific setup. If the checksum is correct now the extract succeeds.

    This LhA version (2.7.14) will be available in the next MorphOS release.
  • »27.04.17 - 19:25
    Profile
  • Order of the Butterfly
    Order of the Butterfly
    asrael22
    Posts: 404 from 2014/6/11
    From: Germany
    OK, thanks for this information.

    It's unfortunate that the source code for Amiga LhA is not free.
    The m68k version can unpack it. Possibly they have applied a similar 'hack'.
    Hacks are not nice but sometimes necessary.
    Regarding point 1: the majority of archives would be 'Amiga' ones.
    When you expect a UNIX archive, wouldn't you have a higher number of CRCs (and double decompress operations) than when expecting an Amiga archive?


    Manfred
  • »27.04.17 - 20:12
    Profile
  • MorphOS Developer
    Piru
    Posts: 587 from 2003/2/24
    From: finland, the l...
    Quote:

    asrael22 wrote:
    OK, thanks for this information.

    It's unfortunate that the source code for Amiga LhA is not free.
    The m68k version can unpack it.

    Yep. The Amiga LhA decided to be compatible with UNIX LhA instead of old Amiga Archives.

    Quote:

    Regarding point 1: the majority of archives would be 'Amiga' ones.
    When you expect a UNIX archive, wouldn't you have a higher number of CRCs (and double decompress operations) than when expecting an Amiga archive?

    Nope. 99% of Amiga LhA archives have been created with LhA that never generates archives which could show the issue. The Amiga LhA compression algorithm has been tuned to avoid that.

    Amiga LhA archives that show the issue are actually quite rare, so the performance impact should be limited.
  • »27.04.17 - 20:20
    Profile
  • Priest of the Order of the Butterfly
    Priest of the Order of the Butterfly
    polluks
    Posts: 803 from 2007/10/23
    From: Gelsenkirchen,...
    For lh7 try lhasa
    Pegasos II G4: MorphOS 3.9, Zalman M220W · iMac G5 12,1 17", MorphOS 3.18
    Power Mac G3: OSX 10.3 · PowerBook 5,8: OSX 10.5, MorphOS 3.18
  • »27.04.17 - 21:58
    Profile
  • Yokemate of Keyboards
    Yokemate of Keyboards
    Andreas_Wolf
    Posts: 12150 from 2003/5/22
    From: Germany
    > For lh7 try lhasa

    ...or XAD that comes with MorphOS :-)
  • »27.04.17 - 22:17
    Profile
  • jPV
  • Yokemate of Keyboards
    Yokemate of Keyboards
    jPV
    Posts: 2096 from 2003/2/24
    From: po-RNO
    XAD is also used when you browse into an archive from Ambient, you can double click the archive and copy files from it that way too.

    I think that XAD is somehow underrated and ignored many times, but I find it pretty great invention.

    You can unarchive about any archive with the same command ("XADUnFile archive dest:" in the simpliest form) and no need to re-learn syntax of every format specific command every time you happen to need it :) XAD can unpack zip, gzip, tar, lzx, lha, bzip2, rar, xz, and about 100 other formats just with a single command... and XAD has also made possible to browsing into all kinds of archives for example with Ambient and DOpus... we would miss a lot without it.
  • »28.04.17 - 08:34
    Profile Visit Website