OK, Andreas mentioning the 6th byte led me to have a closer look.

So here are the last console lines of the uncompress attempt:
Code:

The next file of name 'i' is weird but it is correct. The next file in the archive is a file named 'i'.

Then I looked with a hex editor:
The 6th byte of the archive file is "d" => "-lhd-". Not sure what that means.

Finding the file "hey.sam" in the hext code reveils that it is compressed with -lh5-.
Code:

Then, searching for the next occurence of "-lh" should find the 'i' file.
And indeed.
But it seems to be compressed with -lh5- as well.
Code:


That leads me to think there might be a bug in MorphOS lha?



Manfred

> The 6th byte of the archive file is "d" => "-lhd-". Not sure what that means.

'lhd' means the respective object (first object in the archive in this case) is a directory. As you figured out already despite my confusing comment, each object in the archive is preceded by such an indicator as different objects may have been processed using different methods.

@asrael22

Okay that should not happen.

I managed to find the "i" file myself, and problem is reproduced. I will need to look into this issue.

[ Edited by Piru 27.04.2017 - 17:24 ]

Some extensive background - some of it technical - follows. So bear with me, this is rather interesting story. I had merrily forgotten all about it, but it has already been 14 years since I last laid eyes on that particular code fragment...

Issue: Amiga and UNIX LhA archives are incompatible. That is you can have a old Amiga LhA archive that cannot be decompressed with UNIX LhA. Similarly, you can have a UNIX LhA archive that cannot be decompressed with Amiga LhA. The actual difference is regarding what values the algorithm can expect to find from the dictionary: ' ' (space) or 0 (zero). Amiga LhA uses zero, UNIX LhA uses space.

I believe the problem was solved at some point by making sure that Amiga LhA compressor creates archives that are compatible with the UNIX LhA in all situations. I believe this might actually result in slightly larger archives vs the older incompatible compressor. However, this cannot possibly fix the already existing incompatible archives.

Sometime around the same time a decision was made to modify the Amiga LhA decompressor to work similar to the UNIX LhA: That is, it would fail with those older archives.

Now, here comes the dilemma: The new LhA would create archives that are always ok, regardless of which LhA you used to decompress them. However: It would now itself fail to decompress the old incompatible Amiga LhA archives that had been created earlier! I didn't like that much, since for example it lead to the situation where many existing Amiga LhA archives would just fail to extract. So I made it so that MorphOS LhA worked with the Amiga LhA archives in this corner case, instead of the UNIX ones.

Now, this meant that occasionally you would run into UNIX LhA archives that MorphOS LhA command would fail to extract. And this is exactly what is happening here.

To remedy the situation I decided to create somewhat of a hack:
1. By default I now expect to get UNIX like archive at decompress.
2. If an CRC error occurs I will try to decompress it again with the old AMIGA LhA specific setup. If the checksum is correct now the extract succeeds.

This LhA version (2.7.14) will be available in the next MorphOS release.

OK, thanks for this information.

It's unfortunate that the source code for Amiga LhA is not free.
The m68k version can unpack it. Possibly they have applied a similar 'hack'.
Hacks are not nice but sometimes necessary.
Regarding point 1: the majority of archives would be 'Amiga' ones.
When you expect a UNIX archive, wouldn't you have a higher number of CRCs (and double decompress operations) than when expecting an Amiga archive?


Manfred

Quote:
Yep. The Amiga LhA decided to be compatible with UNIX LhA instead of old Amiga Archives.

Quote:
Nope. 99% of Amiga LhA archives have been created with LhA that never generates archives which could show the issue. The Amiga LhA compression algorithm has been tuned to avoid that.

Amiga LhA archives that show the issue are actually quite rare, so the performance impact should be limited.

For lh7 try lhasa

> For lh7 try lhasa

...or XAD that comes with MorphOS :-)

XAD is also used when you browse into an archive from Ambient, you can double click the archive and copy files from it that way too.

I think that XAD is somehow underrated and ignored many times, but I find it pretty great invention.

You can unarchive about any archive with the same command ("XADUnFile archive dest:" in the simpliest form) and no need to re-learn syntax of every format specific command every time you happen to need it :) XAD can unpack zip, gzip, tar, lzx, lha, bzip2, rar, xz, and about 100 other formats just with a single command... and XAD has also made possible to browsing into all kinds of archives for example with Ambient and DOpus... we would miss a lot without it.

Quote:

If I'm not completely mistaken, then 'The Unarchiver' on macOS (which I'm using regularly) uses XAD as well.


Manfred

> If I'm not completely mistaken, then 'The Unarchiver' on macOS [...] uses XAD as well.

Yes, it uses libxad.

XAD is indeed nice, but ugly slow for some reason. Results from G5 2GHz.

http://aminet.net/game/demo/ToT_Amiga_V0.57.lha

Time LhA x ToT_Amiga_V0.57.lha RAM:
Time Elapsed: 6.6433s

Time XADUnFile ToT_Amiga_V0.57.lha RAM:
Time Elapsed: 28.0831s

This happens also for other archives, not just lha. Also with the quiet option which can help in certain cases (when unpacking many small files).

> XAD is [...] ugly slow for some reason. [...] This happens also for other archives, not just lha.

I just tried with bzip2 archives and found that other than with LhA archives, XAD is only marginally slower than Bz2 command.

Quote:

I got the same times for bzip2. xz and gz was a bit slower when using xad. Looks like xad struggles when untaring. Also I remember xad was a lot slower with some random zip from the internet, packing tot with zip2.3 makes no diff though.

Ram Disk:>Time XADUnTar Q talesoftamar_amiga.tar.bz2 RAM:
Processed 2286 files and 66 directories.
There are 2560 additional tar-bytes at fileend.

Time Elapsed: 60.5093s

Ram Disk:> Time tar xf talesoftamar_amiga.tar.bz2

Time Elapsed: 60.9077s

Ram Disk:> Time tar xf talesoftamar_amiga.tar.xz

Time Elapsed: 24.9833s

Ram Disk:> Time XADUnTar Q talesoftamar_amiga.tar.xz ram:
Processed 2286 files and 66 directories.
There are 2560 additional tar-bytes at fileend.

Time Elapsed: 30.0210s

Ram Disk:> Time tar xf talesoftamar_amiga.tar.gz

Time Elapsed: 22.8524s

Ram Disk:> Time XADUnTar Q talesoftamar_amiga.tar.gz ram:
Processed 2286 files and 66 directories.
There are 2560 additional tar-bytes at fileend.

Time Elapsed: 19.6157s

Ram Disk:> Time XADUnTar Q talesoftamar_amiga.tar ram:
Processed 2286 files and 66 directories.
There are 2560 additional tar-bytes at fileend.

Time Elapsed: 13.9966s

Ram Disk:> Time tar xf talesoftamar_amiga.tar

Time Elapsed: 4.9367s

> I got the same times for bzip2.

So it's not XAD that is inherently slow but just XAD's built-in LhA client.

> gz was a bit slower when using xad. Looks like xad struggles when untaring.
> Also I remember xad was a lot slower with some random zip from the internet

Does MorphOS come with commands for unpacking gz, tar and zip (except from XAD, that is)? I could find them in MOSSYS:C only for bzip2, lzma, rar and xz.

Quote:
tar and gzip are part of the SDK

@asrael22:
You didn't need any hex editor, option "vv" is your friend


[ Editiert durch polluks 03.05.2017 - 20:44 ]

> tar and gzip are part of the SDK

Thanks, found them. Now when I attempt to unpack a tar.bz2 archive with tar, it says it needs lbzip2 (tar doesn't need it for gz and xz archives, though). Where can I find lbzip2?

Edit: The error message is somewhat misleading. What tar needs is the bzip2 executable in the GG:bin/ path (which is not created when the SDK is not installed). After creating the respective assignment, it works well.

[ Edited by Andreas_Wolf 02.05.2017 - 20:36 ]