Suggestions for Unicode
  • Order of the Butterfly
    Order of the Butterfly
    asrael22
    Posts: 404 from 2014/6/11
    From: Germany
    Hi.

    How far is the unicode support in MorphOS?
    What are the suggestions to deal with unicode texts?
    Would http://aminet.net/package/util/libs/UniLibDev be an option?


    Manfred
  • »09.02.16 - 09:06
    Profile
  • MorphOS Developer
    itix
    Posts: 1516 from 2003/2/24
    From: Finland
    MorphOS has Unicode support through locale.library and charsets.library. Keymaps and Intuition support Unicode for the input. Multiview can display Unicode text and AmigaGuide files.
    1 + 1 = 3 with very large values of 1
  • »09.02.16 - 09:43
    Profile
  • Order of the Butterfly
    Order of the Butterfly
    asrael22
    Posts: 404 from 2014/6/11
    From: Germany
    OK. So if I were to build an application that deals a lot with UTF8 encoded texts.
    Could I use system resources to load and display (MUI) those?


    Manfred

    [ Edited by asrael22 09.02.2016 - 12:12 ]
  • »09.02.16 - 09:52
    Profile
  • Order of the Butterfly
    Order of the Butterfly
    pegasos-sigi2
    Posts: 265 from 2006/8/31
    Quote:

    itix wrote:
    MorphOS has Unicode support through locale.library and charsets.library. Keymaps and Intuition support Unicode for the input. Multiview can display Unicode text and AmigaGuide files.



    Ährm... AmigaGuide can't display chinese unicode text files.
    The charsets.library can't decode all UTF- 32 chars (japanese, korean, chinese, arab and so on.)
    The only piece of software thats handle correct Unicode is the clipboard.
    I know. The charsets.library docde only to local charsets, but thats not full Unicode handling.

    The only lib to decode and display full unicode is cairo.library.
    Spreedy - The spreedsheet editor of MorphOS
    Calimero - Do you know about the new DTP Program for MorphOS ?
  • »09.02.16 - 17:50
    Profile
  • Order of the Butterfly
    Order of the Butterfly
    pegasos-sigi2
    Posts: 265 from 2006/8/31
    Quote:

    asrael22 wrote:
    OK. So if I were to build an application that deals a lot with UTF8 encoded texts.
    Could I use system resources to load and display (MUI) those?


    Manfred


    Calimero can load UTF-8 Text (my own software).
    Spreedy - The spreedsheet editor of MorphOS
    Calimero - Do you know about the new DTP Program for MorphOS ?
  • »09.02.16 - 17:54
    Profile
  • MorphOS Developer
    Nadir
    Posts: 157 from 2003/3/17
    Scintilla (and therefore Scribble) supports unicode when using TTEngine for rendering (that's an option in the settings). Unfortunately, there is no support for glyph substitution so it might not be completely perfect but it does work for common cases at least.
  • »09.02.16 - 18:50
    Profile
  • Order of the Butterfly
    Order of the Butterfly
    asrael22
    Posts: 404 from 2014/6/11
    From: Germany
    Quote:

    pegasos-sigi2 wrote:
    Quote:

    asrael22 wrote:
    OK. So if I were to build an application that deals a lot with UTF8 encoded texts.
    Could I use system resources to load and display (MUI) those?


    Manfred


    Calimero can load UTF-8 Text (my own software).


    Nice, how do you do it?


    Manfred
  • »09.02.16 - 19:47
    Profile
  • Order of the Butterfly
    Order of the Butterfly
    asrael22
    Posts: 404 from 2014/6/11
    From: Germany
    Quote:

    Nadir wrote:
    Scintilla (and therefore Scribble) supports unicode when using TTEngine for rendering (that's an option in the settings). Unfortunately, there is no support for glyph substitution so it might not be completely perfect but it does work for common cases at least.

    So, it's required some use fonts that can actually display unicode.
    What about reading from file? I read into a char array, what di I do with it to be treated as unicode text?


    Manfred
  • »09.02.16 - 19:48
    Profile
  • Yokemate of Keyboards
    Yokemate of Keyboards
    takemehomegrandma
    Posts: 2720 from 2003/2/24
    Quote:

    pegasos-sigi2 wrote:
    Quote:

    itix wrote:
    MorphOS has Unicode support through locale.library and charsets.library. Keymaps and Intuition support Unicode for the input. Multiview can display Unicode text and AmigaGuide files.



    Ährm... AmigaGuide can't display chinese unicode text files.
    The charsets.library can't decode all UTF- 32 chars (japanese, korean, chinese, arab and so on).



    Neither emojis AFAIK (and there is no support for color fonts AFAIK which would be nice in emoji contexts). A lot of texting and even writing takes place on phones or tablets today, using e-mail applications, chat applications, or even text documents. And people use emojis! An e-mail sent from one iPad to another will properly display any emojis inside it, but open up the same mail on MorphOS, what will you see?

    So when discussing future unicode support, please don't forget the emojis! 😉👍 (<- I put two of them right there!)

    http://apps.timwhitlock.info/emoji/tables/unicode
    https://en.m.wikipedia.org/wiki/Apple_Color_Emoji
    MorphOS is Amiga done right! :-)
    MorphOS NG will be AROS done right! :-)
  • »09.02.16 - 21:45
    Profile
  • MorphOS Developer
    itix
    Posts: 1516 from 2003/2/24
    From: Finland
    @asrael22
    Quote:


    OK. So if I were to build an application that deals a lot with UTF8 encoded texts.
    Could I use system resources to load and display (MUI) those?



    MUI has no Unicode support so you can only use UTF8 encoded text through your MUI custom class you write for your application. For example while Odyssey Web Browser and Multiview can display Unicode text this is limited to the text display area only. If the web site title has code points not supported by local codepage you just get question marks to OWB titlebar.

    The built-in graphics.library/Text() can't render Unicode glyphs so you have to use freetype.library or ttengine.library to render Unicode.

    Quote:

    pegasos-sigi2 wrote:
    Quote:

    itix wrote:
    MorphOS has Unicode support through locale.library and charsets.library. Keymaps and Intuition support Unicode for the input. Multiview can display Unicode text and AmigaGuide files.



    Ährm... AmigaGuide can't display chinese unicode text files.



    I have not tried (I think I have tried only Japanese) but the text rendering part is the problem.

    Quote:


    The charsets.library can't decode all UTF- 32 chars (japanese, korean, chinese, arab and so on.)
    The only piece of software thats handle correct Unicode is the clipboard.
    I know. The charsets.library docde only to local charsets, but thats not full Unicode handling.

    The only lib to decode and display full unicode is cairo.library.



    Which is also part of the OS. There is also iconv.library which provides more encodings than charsets.library (but at the expense of bigger memory footprint).

    It is also possible to create Cairo.mcc...
    1 + 1 = 3 with very large values of 1
  • »10.02.16 - 06:35
    Profile
  • Order of the Butterfly
    Order of the Butterfly
    asrael22
    Posts: 404 from 2014/6/11
    From: Germany
    Quote:

    pegasos-sigi2 wrote:
    Quote:

    itix wrote:
    MorphOS has Unicode support through locale.library and charsets.library. Keymaps and Intuition support Unicode for the input. Multiview can display Unicode text and AmigaGuide files.



    Ährm... AmigaGuide can't display chinese unicode text files.
    The charsets.library can't decode all UTF- 32 chars (japanese, korean, chinese, arab and so on.)
    The only piece of software thats handle correct Unicode is the clipboard.
    I know. The charsets.library docde only to local charsets, but thats not full Unicode handling.

    The only lib to decode and display full unicode is cairo.library.


    OK, if charsets.library can decode UTF-8 chars then that should be fine.
    I don't think I'm dealing with any UTF-32 stuff.


    Manfred
  • »10.02.16 - 14:45
    Profile
  • Order of the Butterfly
    Order of the Butterfly
    asrael22
    Posts: 404 from 2014/6/11
    From: Germany
    Quote:

    itix wrote:
    MorphOS has Unicode support through locale.library and charsets.library. Keymaps and Intuition support Unicode for the input. Multiview can display Unicode text and AmigaGuide files.


    Is it possible to embed a MultiView component into an application?


    Manfred
  • »10.02.16 - 14:54
    Profile
  • MorphOS Developer
    itix
    Posts: 1516 from 2003/2/24
    From: Finland
    Quote:

    asrael22 wrote:
    Quote:

    pegasos-sigi2 wrote:
    Quote:

    itix wrote:
    MorphOS has Unicode support through locale.library and charsets.library. Keymaps and Intuition support Unicode for the input. Multiview can display Unicode text and AmigaGuide files.



    Ährm... AmigaGuide can't display chinese unicode text files.
    The charsets.library can't decode all UTF- 32 chars (japanese, korean, chinese, arab and so on.)
    The only piece of software thats handle correct Unicode is the clipboard.
    I know. The charsets.library docde only to local charsets, but thats not full Unicode handling.

    The only lib to decode and display full unicode is cairo.library.


    OK, if charsets.library can decode UTF-8 chars then that should be fine.
    I don't think I'm dealing with any UTF-32 stuff.


    Charsets.library can convert UTF-8 to an OS supported 8-bit charset or to UTF-16/32 respectively. You find supported charsets from libraries/charsets.h header.
    1 + 1 = 3 with very large values of 1
  • »10.02.16 - 15:24
    Profile
  • MorphOS Developer
    itix
    Posts: 1516 from 2003/2/24
    From: Finland
    Quote:

    asrael22 wrote:
    Quote:

    itix wrote:
    MorphOS has Unicode support through locale.library and charsets.library. Keymaps and Intuition support Unicode for the input. Multiview can display Unicode text and AmigaGuide files.


    Is it possible to embed a MultiView component into an application?



    Unfortunately no. Multiview.mcc has been planned but that plan has existed five years now and it still does not exist...
    1 + 1 = 3 with very large values of 1
  • »10.02.16 - 15:25
    Profile
  • MorphOS Developer
    Nadir
    Posts: 157 from 2003/3/17
    However, you can embed a scintilla.mcc view that will allow you to display Unicode text easily. All the material, including amigaguide docs, are in the SDK. You just need to pass the text to scintilla and set the rendering engine to TTEngine as described in the docs. If you don't want editing, then make the view read-only.

    Nicholai
  • »10.02.16 - 16:28
    Profile
  • MorphOS Developer
    Nadir
    Posts: 157 from 2003/3/17
    .

    [ Edited by Nadir 10.02.2016 - 20:05 ]
  • »10.02.16 - 16:28
    Profile
  • MorphOS Developer
    Nadir
    Posts: 157 from 2003/3/17
    .

    [ Edited by Nadir 10.02.2016 - 20:05 ]
  • »10.02.16 - 16:28
    Profile
  • MorphOS Developer
    itix
    Posts: 1516 from 2003/2/24
    From: Finland
    Quote:

    asrael22 wrote:
    Quote:

    Nadir wrote:
    Scintilla (and therefore Scribble) supports unicode when using TTEngine for rendering (that's an option in the settings). Unfortunately, there is no support for glyph substitution so it might not be completely perfect but it does work for common cases at least.

    So, it's required some use fonts that can actually display unicode.
    What about reading from file? I read into a char array, what di I do with it to be treated as unicode text?



    Read into char array, then load a font and render text using TTEngine. IIRC TTEngine accepts UTF-8 but check this from the docs first. TTEngine is bit clumsy but can render Unicode glyphs in Amiga way.
    1 + 1 = 3 with very large values of 1
  • »10.02.16 - 19:55
    Profile
  • MorphOS Developer
    Nadir
    Posts: 157 from 2003/3/17
    Yes, UTF8 works fine with TTEngine, that's what I use in Scribble/Scintilla. There are some limitations in both the design and the features of that library but it's the best we have for rendering at the moment without going too deep into Linux domains
  • »11.02.16 - 01:19
    Profile
  • Order of the Butterfly
    Order of the Butterfly
    asrael22
    Posts: 404 from 2014/6/11
    From: Germany
    Thanks, that's great to know.
    I'll give this a try.


    Manfred
  • »11.02.16 - 07:09
    Profile
  • Priest of the Order of the Butterfly
    Priest of the Order of the Butterfly
    Cool_amigaN
    Posts: 745 from 2011/11/30
    I have properly displayed Greek characters only on MPlayer (as subtitles), on OWB partially (only when writing on forums/googling but not displaying properly inherited menus from forums/site news) and on Calimero (read/write). I would love to know whether there is a solution for OWB which is a lot annoying (can't navigate easily).
    Amiga gaming Tribute: Watch, rate, comment :)
  • »11.02.16 - 08:53
    Profile Visit Website
  • Order of the Butterfly
    Order of the Butterfly
    pegasos-sigi2
    Posts: 265 from 2006/8/31
    Quote:

    Cool_amigaN wrote:
    I have properly displayed Greek characters only on MPlayer (as subtitles), on OWB partially (only when writing on forums/googling but not displaying properly inherited menus from forums/site news) and on Calimero (read/write). I would love to know whether there is a solution for OWB which is a lot annoying (can't navigate easily).


    Install the font "Arial Unicode MS" from Windows. Thats a full size Unicode font.
    Then OWB displays all languages "Greek, Chinese, Japanese, Korean and so on...) And Calimero, too ;-)
    Spreedy - The spreedsheet editor of MorphOS
    Calimero - Do you know about the new DTP Program for MorphOS ?
  • »12.02.16 - 15:42
    Profile
  • Priest of the Order of the Butterfly
    Priest of the Order of the Butterfly
    Cool_amigaN
    Posts: 745 from 2011/11/30
    @pegasos-sigi2

    Hey, thanks for your suggestion. I downloaded the font but it seems that's almost 23MB, wtf, is this correct? After rebooting, OWB re-initialized fonts but yet again, the same problem occurs. I have also placed encoding to ISO 8859-7 and selected in every category within OWB Arial unicode MS as default font, but it is still a no go. Any ideas?
    Amiga gaming Tribute: Watch, rate, comment :)
  • »15.02.16 - 13:51
    Profile Visit Website