wxString: UTF8, normalization, grapheme cluster iterator

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

wxString: UTF8, normalization, grapheme cluster iterator

Gerald Brandt
Hi,

How complete is the UTF8 handling in wxString. It looks like it doesn't
do normalization or know about graphemes. Are there plans to deal with that?

I know ICU can handle all of the above, but it's quite a pain to use.
Does anyone use any other libraries or have some helpful pointers?

Thanks,

Gerald



--
Please read http://www.wxwidgets.org/support/mlhowto.htm before posting.

To unsubscribe, send email to [hidden email]
or visit http://groups.google.com/group/wx-users
Reply | Threaded
Open this post in threaded view
|

Re: wxString: UTF8, normalization, grapheme cluster iterator

Frédéric
> How complete is the UTF8 handling in wxString. It looks like it doesn't
> do normalization or know about graphemes. Are there plans to deal with that?
>
> I know ICU can handle all of the above, but it's quite a pain to use.
> Does anyone use any other libraries or have some helpful pointers?

Note that wxString is not UTF-8 by default. In particular, it is not
UTF-8 on Windows.

--
Please read http://www.wxwidgets.org/support/mlhowto.htm before posting.

To unsubscribe, send email to [hidden email]
or visit http://groups.google.com/group/wx-users
Reply | Threaded
Open this post in threaded view
|

Re: wxString: UTF8, normalization, grapheme cluster iterator

Vadim Zeitlin-4
In reply to this post by Gerald Brandt
On Thu, 27 Sep 2018 09:51:40 -0500 Gerald Brandt wrote:

GB> How complete is the UTF8 handling in wxString.

 Non-existent.

GB> I know ICU can handle all of the above, but it's quite a pain to use.
GB> Does anyone use any other libraries or have some helpful pointers?

 I've started using https://github.com/JuliaStrings/utf8proc recently. It's
a C library but seems of good quality and provides all I need (which is not
much) while being much simpler to integrate than ICU.

 Regards,
VZ

--
TT-Solutions: wxWidgets consultancy and technical support
               http://www.tt-solutions.com/

attachment0 (203 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: wxString: UTF8, normalization, grapheme cluster iterator

jon bird-2
I use boost locale on top of ICU. It's monstrous but that's fine for what I'm developing. I'm in no way proficient enough to understand half of what the boost documentation tells me but it wasn't too hard to use the utf conversions and comparison (case insensitive) algorithms it provides.


On 27 September 2018 21:41:08 BST, Vadim Zeitlin <[hidden email]> wrote:
On Thu, 27 Sep 2018 09:51:40 -0500 Gerald Brandt wrote:

GB> How complete is the UTF8 handling in wxString.

Non-existent.

GB> I know ICU can handle all of the above, but it's quite a pain to use.
GB> Does anyone use any other libraries or have some helpful pointers?

I've started using https://github.com/JuliaStrings/utf8proc recently. It's
a C library but seems of good quality and provides all I need (which is not
much) while being much simpler to integrate than ICU.

Regards,
VZ

--
Please read http://www.wxwidgets.org/support/mlhowto.htm before posting.
 
To unsubscribe, send email to [hidden email]
or visit http://groups.google.com/group/wx-users
Reply | Threaded
Open this post in threaded view
|

Re: wxString: UTF8, normalization, grapheme cluster iterator

peter.koch.larsen
In reply to this post by Gerald Brandt


torsdag den 27. september 2018 kl. 16.51.43 UTC+2 skrev gbr:
Hi,

How complete is the UTF8 handling in wxString. It looks like it doesn't
do normalization or know about graphemes. Are there plans to deal with that?

I know ICU can handle all of the above, but it's quite a pain to use.
Does anyone use any other libraries or have some helpful pointers?

If I were to get serious about Unicode I would certainly take a look at a library from Zach Laine. Google his name!
I believe Zach is an extraordinaly good developer and would expect his library to be both good and complete.
He has made a talk about it (two, actually) on boostcon if I remember correctly. Watch and check.

/Peter

Thanks,

Gerald



--
Please read http://www.wxwidgets.org/support/mlhowto.htm before posting.
 
To unsubscribe, send email to [hidden email]
or visit http://groups.google.com/group/wx-users
Reply | Threaded
Open this post in threaded view
|

Re: wxString: UTF8, normalization, grapheme cluster iterator

Gerald Brandt



On 2018-09-28 10:01 AM, [hidden email] wrote:


torsdag den 27. september 2018 kl. 16.51.43 UTC+2 skrev gbr:
Hi,

How complete is the UTF8 handling in wxString. It looks like it doesn't
do normalization or know about graphemes. Are there plans to deal with that?

I know ICU can handle all of the above, but it's quite a pain to use.
Does anyone use any other libraries or have some helpful pointers?

If I were to get serious about Unicode I would certainly take a look at a library from Zach Laine. Google his name!
I believe Zach is an extraordinaly good developer and would expect his library to be both good and complete.
He has made a talk about it (two, actually) on boostcon if I remember correctly. Watch and check.

/Peter


This looks GREAT so far. Delving in deeper...

Thank you!

Gerald

--
Please read http://www.wxwidgets.org/support/mlhowto.htm before posting.
 
To unsubscribe, send email to [hidden email]
or visit http://groups.google.com/group/wx-users
Reply | Threaded
Open this post in threaded view
|

Re: wxString: UTF8, normalization, grapheme cluster iterator

Gerald Brandt
In reply to this post by peter.koch.larsen



On 2018-09-28 10:01 AM, [hidden email] wrote:


torsdag den 27. september 2018 kl. 16.51.43 UTC+2 skrev gbr:
Hi,

How complete is the UTF8 handling in wxString. It looks like it doesn't
do normalization or know about graphemes. Are there plans to deal with that?

I know ICU can handle all of the above, but it's quite a pain to use.
Does anyone use any other libraries or have some helpful pointers?

If I were to get serious about Unicode I would certainly take a look at a library from Zach Laine. Google his name!
I believe Zach is an extraordinaly good developer and would expect his library to be both good and complete.
He has made a talk about it (two, actually) on boostcon if I remember correctly. Watch and check.


https://www.youtube.com/watch?v=944GjKxwMBo

--
Please read http://www.wxwidgets.org/support/mlhowto.htm before posting.
 
To unsubscribe, send email to [hidden email]
or visit http://groups.google.com/group/wx-users
Reply | Threaded
Open this post in threaded view
|

Re: wxString: UTF8, normalization, grapheme cluster iterator

peter.koch.larsen
I was just about to copy the link to you. You can find Zach and his libraries on github.

fredag den 28. september 2018 kl. 22.16.49 UTC+2 skrev gbr:



On 2018-09-28 10:01 AM, <a onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;" href="javascript:" target="_blank" rel="nofollow" gdf-obfuscated-mailto="_fSV6Sn4AQAJ">peter.ko...@... wrote:


torsdag den 27. september 2018 kl. 16.51.43 UTC+2 skrev gbr:
Hi,

How complete is the UTF8 handling in wxString. It looks like it doesn't
do normalization or know about graphemes. Are there plans to deal with that?

I know ICU can handle all of the above, but it's quite a pain to use.
Does anyone use any other libraries or have some helpful pointers?

If I were to get serious about Unicode I would certainly take a look at a library from Zach Laine. Google his name!
I believe Zach is an extraordinaly good developer and would expect his library to be both good and complete.
He has made a talk about it (two, actually) on boostcon if I remember correctly. Watch and check.


<a onmousedown="this.href=&#39;https://www.youtube.com/watch?v\x3d944GjKxwMBo&#39;;return true;" onclick="this.href=&#39;https://www.youtube.com/watch?v\x3d944GjKxwMBo&#39;;return true;" href="https://www.youtube.com/watch?v=944GjKxwMBo" target="_blank" rel="nofollow">https://www.youtube.com/watch?v=944GjKxwMBo

--
Please read http://www.wxwidgets.org/support/mlhowto.htm before posting.
 
To unsubscribe, send email to [hidden email]
or visit http://groups.google.com/group/wx-users