Unicode in disk and file names
  • Priest of the Order of the Butterfly
    Priest of the Order of the Butterfly
    connor
    Posts: 607 from 2007/7/29
    I get sometimes USB drives with files from friends or send by email. When they have disk or file names like “René” then the names cannot be shown in Ambient and Shell and Requesters. It shows “Rene\0301” and such. Worse for longer names of files that have complete words or titles in Unicode. Then it looks like \0301\0452\0110 and so on. Often they do not even fit in the place because for every singel character it takes 5 characters in this code. Then there is no file ending shown and I have not even an idea which file type it is because Ambient does not always detect images, sound files, text files right. Then I double click on them and some times an image opens but sometimes only hex view in editor. This makes the files useless for me on MorphOS. I want to work with the files and use all MorphOS programs but with the code names it is not possible. Also I fear when I rename just one character that they whole name will be destroyed for all other operating systems and when I send them back to other persons for them the files become useless. Can the next version of MorphOS please support Unicode correctly for file names in Ambient and Shell?

    I have found here old threads about Unicode, Asian fonts and all that from 8-9 years ago. Still, MorphOS NG (I hope they will never call it that way but just 4.0) is not available. So it would be nice to see improvements here.
  • »21.03.25 - 09:19
    Profile
  • Priest of the Order of the Butterfly
    Priest of the Order of the Butterfly
    polluks
    Posts: 817 from 2007/10/23
    From: Gelsenkirchen,...
    We have some Unicode support in the locales. Your feature request is not Ambient related, it should be part of the filesystem IMHO.
    Pegasos II G4: MorphOS 3.9, Zalman M220W · iMac G5 12,1 17", MorphOS 3.18
    Power Mac G3: OSX 10.3 · PowerBook 5,8: OSX 10.5, MorphOS 3.18
  • »21.03.25 - 10:26
    Profile
  • Priest of the Order of the Butterfly
    Priest of the Order of the Butterfly
    connor
    Posts: 607 from 2007/7/29
    I did not say it is Ambient related. I clearly wrote "file names" and included "Ambient", "Shell", "Requesters" because these are the programs where I can see and use the filenames. So yes, it is related to the filesystem AND the correct display in the programs that make the filesystem visible. Ambient and Shell are such programs.
  • »21.03.25 - 10:40
    Profile
  • Priest of the Order of the Butterfly
    Priest of the Order of the Butterfly
    polluks
    Posts: 817 from 2007/10/23
    From: Gelsenkirchen,...
    At least we have another FAT implementation in XAD. You can try to access USB via rawdisk:...
    Pegasos II G4: MorphOS 3.9, Zalman M220W · iMac G5 12,1 17", MorphOS 3.18
    Power Mac G3: OSX 10.3 · PowerBook 5,8: OSX 10.5, MorphOS 3.18
  • »23.03.25 - 22:16
    Profile
  • Priest of the Order of the Butterfly
    Priest of the Order of the Butterfly
    KennyR
    Posts: 891 from 2003/3/4
    From: #AmigaZeux, Gu...
    Quote:

    connor wrote:
    I have found here old threads about Unicode, Asian fonts and all that from 8-9 years ago. Still, MorphOS NG (I hope they will never call it that way but just 4.0) is not available. So it would be nice to see improvements here.


    It's a twofold problem: the first is that the filesystem (not MorphOS) isn't correctly translating unicode letters to whatever character set is in use. For words like René that's easy. In fact for OS 2.x and up it was easy, using FFS-INTL (DOS\3 and up).

    For Korean or Cyrillic, not so easy. That's the second problem. Unicode is huge, and some writing formats go right to left. Both things are not necessarily compatible with the original design of MorphOS (and AmigaOS, which is tried to be compatible with).

    (Edit: Character sets go back to AmigaOS 1.3 and were part of FFS v34, although they just supported Latin character sets at that point to my knowledge and not Russian or Greek.)

    [ Edited by KennyR 23.03.2025 - 22:57 ]
  • »23.03.25 - 22:29
    Profile
  • Priest of the Order of the Butterfly
    Priest of the Order of the Butterfly
    KennyR
    Posts: 891 from 2003/3/4
    From: #AmigaZeux, Gu...
    Quote:

    polluks wrote:
    At least we have another FAT implementation in XAD. You can try to access USB via rawdisk:...


    It wouldn't help. They still have to be mounted using the same filesystem.

    It's probably the MorphOS port of FUSE that underlies both FAT32 and NTFS that's not translating properly.
  • »23.03.25 - 22:59
    Profile
  • Priest of the Order of the Butterfly
    Priest of the Order of the Butterfly
    connor
    Posts: 607 from 2007/7/29
    Like Kenny said it does not help.
    I tried it with RADWISK: When I put the USB0 from “Disks” in FileImageCtrl then it says it mounted FILE0: but this does not show up in Ambient or in Shell. When I do the same with the rawdisk from “Devices” then it also just pops up “File1:” but does not show in Ambient. And also not with the partition00. In Shell I can go to File0: etc. but all the names are the same for the disk and the files, the same like in Ambient already directly when I’m not using rawdisk.
    So Ambient cannot show multiple devices with the same name at all. That's also a problem when I have an old and a new harddisk inside for backups or reinstall and they have the same partition names. Shell at least can least them with info command.
    And also files from my NAS I cannot mount with USB but go through SMB. There also the filenames and directory names are with \0123 etc.
    Funny thing: I got another USB drive from René and this one Ambient shows as “Ren” without the “é”.
  • »25.03.25 - 09:34
    Profile
  • Priest of the Order of the Butterfly
    Priest of the Order of the Butterfly
    connor
    Posts: 607 from 2007/7/29
    Quote:

    KennyR wrote:
    It's a twofold problem: the first is that the filesystem (not MorphOS) isn't correctly translating unicode letters to whatever character set is in use. For words like René that's easy. In fact for OS 2.x and up it was easy, using FFS-INTL (DOS\3 and up).


    So maybe SFS does not have this international mode=?
    Quote:



    For Korean or Cyrillic, not so easy. That's the second problem. Unicode is huge, and some writing formats go right to left. Both things are not necessarily compatible with the original design of MorphOS (and AmigaOS, which is tried to be compatible with).

    (Edit: Character sets go back to AmigaOS 1.3 and were part of FFS v34, although they just supported Latin character sets at that point to my knowledge and not Russian or Greek.)


    For Arabian language it is more difficult because they write from right to left. Could it be made working at least for writings from left to right? That would already be many languages. Or will it break all cpomatibility with programs? And we have to wait for MorphOS 4 (wait, wait, wait ...).
  • »25.03.25 - 09:39
    Profile
  • Priest of the Order of the Butterfly
    Priest of the Order of the Butterfly
    KennyR
    Posts: 891 from 2003/3/4
    From: #AmigaZeux, Gu...
    Quote:

    connor wrote:
    So maybe SFS does not have this international mode=?



    It had from the start, as did PFS. The only reason FFS needed an international mode was because only the basic ANSI character set was used at the start.

    No filesystem on Amiga ever supported unicode directly.

    Quote:

    For Arabian language it is more difficult because they write from right to left. Could it be made working at least for writings from left to right? That would already be many languages. Or will it break all cpomatibility with programs? And we have to wait for MorphOS 4 (wait, wait, wait ...).


    ?taht did ti fi ,ti dnatsrednu ot elba eb uoy dluow tuB
  • »25.03.25 - 11:03
    Profile
  • MorphOS Developer
    jacadcaps
    Posts: 3180 from 2003/3/5
    From: Canada
    Ambient and perhaps other things like ASL requesters could be made to show Unicode, provided file systems get updated to handle additional packets.

    But... there's no way to do this with so that old programs would display things correctly (think Dopus4 if you're into that). You'd still end up in a situation where you would not be able to rename a file using Dopus4 without losing information...
  • »25.03.25 - 11:09
    Profile Visit Website
  • Priest of the Order of the Butterfly
    Priest of the Order of the Butterfly
    connor
    Posts: 607 from 2007/7/29
    Quote:

    KennyR wrote:
    Quote:

    connor wrote:
    So maybe SFS does not have this international mode=?



    It had from the start, as did PFS. The only reason FFS needed an international mode was because only the basic ANSI character set was used at the start.

    No filesystem on Amiga ever supported unicode directly.

    Quote:

    For Arabian language it is more difficult because they write from right to left. Could it be made working at least for writings from left to right? That would already be many languages. Or will it break all cpomatibility with programs? And we have to wait for MorphOS 4 (wait, wait, wait ...).


    ?taht did ti fi ,ti dnatsrednu ot elba eb uoy dluow tuB

    )-: yletelpmoC
  • »25.03.25 - 13:40
    Profile
  • Priest of the Order of the Butterfly
    Priest of the Order of the Butterfly
    connor
    Posts: 607 from 2007/7/29
    Quote:

    jacadcaps wrote:
    Ambient and perhaps other things like ASL requesters could be made to show Unicode, provided file systems get updated to handle additional packets.

    But... there's no way to do this with so that old programs would display things correctly (think Dopus4 if you're into that). You'd still end up in a situation where you would not be able to rename a file using Dopus4 without losing information...


    That would be okay for me. I don't use such old programs. You can write a warning in the release readme that renaming the files with not supported programs (e.g. those that are not on the Boot CD), especially old programs will probrbly loose or break the information. And maybe then they will be updated also when the tools on board can handle it. But if I could open and edit images, PDFs, videos, music files with the programs we have: Vpdf, MPlayer, Multiview, Jukebox etc., If I could list them in the MUICON Shell that would be a great improvement.
  • »25.03.25 - 13:47
    Profile
  • MorphOS Developer
    jacadcaps
    Posts: 3180 from 2003/3/5
    From: Canada
    Yea well ... MUICON would be one of the things that couldn't be fixed to handle Unicode filenames because it needs to be able to pass them to whatever the tool you're gonna run in a shell with those.
  • »25.03.25 - 14:53
    Profile Visit Website
  • Priest of the Order of the Butterfly
    Priest of the Order of the Butterfly
    KennyR
    Posts: 891 from 2003/3/4
    From: #AmigaZeux, Gu...
    Quote:

    connor wrote:
    That would be okay for me. I don't use such old programs. You can write a warning in the release readme that renaming the files with not supported programs (e.g. those that are not on the Boot CD), especially old programs will probrbly loose or break the information. And maybe then they will be updated also when the tools on board can handle it. But if I could open and edit images, PDFs, videos, music files with the programs we have: Vpdf, MPlayer, Multiview, Jukebox etc., If I could list them in the MUICON Shell that would be a great improvement.


    But do you really need Unicode? MorphOS and (AmigaOS) character maps do support all special characters used in Latin languages, Greek, possibly Cyrillic (not sure, but Russian, Serbian and Bulgarian keymaps apparently exist). It's just one filesystem not translating them properly.

    I mean if you want to name your files using Egyptian hieroglyphs, Dravidian languages or Tolkien elvish runes Unicode is great, but otherwise it's a lot easier just to translate a small subset of commonly used letters like the accented e in Rene. Probably not worth having uncertain compatibility which would result in HUGE problems.
  • »25.03.25 - 17:09
    Profile
  • Yokemate of Keyboards
    Yokemate of Keyboards
    Andreas_Wolf
    Posts: 12281 from 2003/5/22
    From: Germany
    >>> I get sometimes USB drives with files from friends or send by email.

    >> [...]

    > But do you really need Unicode?

    I guess he could simply refuse receipt of those files ;-)

    > Probably not worth having uncertain compatibility which would result
    > in HUGE problems.

    File name compatibility already *is* the problem at hand here, as far as I understand the issue.
  • »25.03.25 - 18:12
    Profile
  • Priest of the Order of the Butterfly
    Priest of the Order of the Butterfly
    KennyR
    Posts: 891 from 2003/3/4
    From: #AmigaZeux, Gu...
    Quote:

    Andreas_Wolf wrote:
    File name compatibility already *is* the problem at hand here, as far as I understand the issue.


    The filesystem can still track the file, even if the shown filename isn't what the user expects. When you start messing with software that is aware of Unicode renaming files that are then being opened by software that isn't, that may no longer be true.

    [ Edited by KennyR 25.03.2025 - 20:36 ]
  • »25.03.25 - 20:36
    Profile