Page 9 of 10

Re: Irrlicht i18n (Unicode, FreeType, etc.)

PostPosted: Wed May 21, 2014 6:20 pm
by hendu
The lineskip is handled wrong (ascender / 64 ??). See http://hg.libsdl.org/SDL_ttf/file/997ba ... /SDL_ttf.c for how to get the lineskip properly for both bitmap and vector fonts.

It would also be good to include a way to get the font's lineskip once it's calculated and stored in the class.

Re: Irrlicht i18n (Unicode, FreeType, etc.)

PostPosted: Wed May 21, 2014 6:31 pm
by Nalin
That's really neat!

CuteAlien wrote:- No dependency on ustring (I don't think string-conversions should happen inside the font-class itself)

How will you be able to draw unicode glyphs then? Are you just forgoing unicode entirely with your version?

Re: Irrlicht i18n (Unicode, FreeType, etc.)

PostPosted: Wed May 21, 2014 7:41 pm
by CuteAlien
I expect the string to correspond already to the encoding requested from freetype when drawing is requested. Unless I'm missing something you can pass through utf-16 using wchar_t for example (just will waste 2 bytes on Linux). I think any kind of encoding conversions should already happen earlier - not when you want to draw the font. But maybe I'm missing something there - I didn't dig too deep into the ustring stuff for now as I just wanted to get it working with my game (which converts utf-8 to ucs-2 on loading and so I didn't want to risk to have another conversion in drawing).

edit: removed stuff about newlines - I was wrong about that.

@hendu: I'm not allowed to look at SDL implementations - they have an incompatible license. I add their code I'll have to gpl everything.

Re: Irrlicht i18n (Unicode, FreeType, etc.)

PostPosted: Wed May 21, 2014 7:58 pm
by Nalin
CuteAlien wrote:I expect the string to correspond already to the encoding requested from freetype when drawing is requested. Unless I'm missing something you can pass through utf-16 using wchar_t for example (just will waste 2 bytes on Linux). I think any kind of encoding conversions should already happen earlier - not when you want to draw the font. But maybe I'm missing something there - I didn't dig too deep into the ustring stuff for now as I just wanted to get it working with my game (which converts utf-8 to ucs-2 on loading and so I didn't want to risk to have another conversion in drawing).

I merged a lot of your bug fixes and performance improvements into my code. I'll release a new version once I get back home at the end of the week (I can't test to see if I didn't make any mistakes at my parent's house).

Anyway, the problem is in on line 511: https://code.google.com/p/irr-playgroun ... nt.cpp#511
I guess it isn't really a problem if you are only sticking to UCS-2. That limits you to unicode plane 0 (http://en.wikipedia.org/wiki/Plane_%28Unicode%29)
Anything above plane 0 will use UTF-16 surrogate pairs and will be comprised of two wchar_t's on Windows (wchar_t is 16-bits and plane 1+ characters are 32-bit).

Re: Irrlicht i18n (Unicode, FreeType, etc.)

PostPosted: Wed May 21, 2014 8:44 pm
by CuteAlien
Ah ok. Yeah - that won't work without doing a conversion like you did or using UCS-4 type instead. Note that I don't think ustring is a bad idea - I just wanted a quick solution for now which works without having to change too much. So yeah - my comment above that I'd prefer converting it before draw is probably not working for chinese and similar (or is chinese available with ucs-2 already? not sure... so let's say it won't work for klingon to be safe).

Re: Irrlicht i18n (Unicode, FreeType, etc.)

PostPosted: Wed May 21, 2014 9:56 pm
by Nalin
Actually, I believe the majority of usable Chinese characters are in plane 0. Just some of the older characters are in later planes. It should be fine for the majority of actual text.

Re: Irrlicht i18n (Unicode, FreeType, etc.)

PostPosted: Thu May 22, 2014 12:12 am
by christianclavet
Wow! Thank! That's really great news!

Re: Irrlicht i18n (Unicode, FreeType, etc.)

PostPosted: Thu May 22, 2014 7:14 am
by hendu
/*
SDL_ttf: A companion library to SDL for working with TrueType (tm) fonts
Copyright (C) 2001-2013 Sam Lantinga <slouken@libsdl.org>

This software is provided 'as-is', without any express or implied
warranty. In no event will the authors be held liable for any damages
arising from the use of this software.

Permission is granted to anyone to use this software for any purpose,
including commercial applications, and to alter it and redistribute it
freely, subject to the following restrictions:

1. The origin of this software must not be misrepresented; you must not
claim that you wrote the original software. If you use this software
in a product, an acknowledgment in the product documentation would be
appreciated but is not required.
2. Altered source versions must be plainly marked as such, and must not be
misrepresented as being the original software.
3. This notice may not be removed or altered from any source distribution.
*/


That is a BSD license.

Re: Irrlicht i18n (Unicode, FreeType, etc.)

PostPosted: Thu May 22, 2014 9:46 am
by CuteAlien
@hendu: Ah cool. Well, still not exactly compatible with zlib, but anyway - your report is as always helpful :-)
I didn't notice that problem.

edit: Forget anything I wrote earlier about newlines - seems I was wrong and Irrlicht handles those in fonts + in gui elements (got confused by that).

Re: Irrlicht i18n (Unicode, FreeType, etc.)

PostPosted: Thu May 22, 2014 11:04 am
by superpws
CuteAlien wrote:@hendu: Ah cool. Well, still not exactly compatible with zlib, but anyway - your report is as always helpful :-)
I didn't notice that problem.


The latest version has zlib license. See the changes:
https://www.libsdl.org/projects/SDL_ttf/

Re: Irrlicht i18n (Unicode, FreeType, etc.)

PostPosted: Thu May 22, 2014 11:46 am
by CuteAlien
@superpws: wow, didn't hear about that

Re: Irrlicht i18n (Unicode, FreeType, etc.)

PostPosted: Thu May 22, 2014 5:48 pm
by christianclavet
Checked the Freetype licence, and it seem compatible with ZLIB.
http://git.savannah.gnu.org/cgit/freetype/freetype2.git/tree/docs/FTL.TXT

Re: Irrlicht i18n (Unicode, FreeType, etc.)

PostPosted: Fri May 23, 2014 2:05 pm
by thanhle
http://www.libsdl.org/

From their frontpage: SDL 2.0 is distributed under the zlib license.

Re: Irrlicht i18n (Unicode, FreeType, etc.)

PostPosted: Thu Jul 31, 2014 1:31 am
by christianclavet
HI, Learned more about the character encoding and just figured out that when I open XML files in UTF8 the string is still in UTF8 in memory and have to convert it to ANSI (The Irrlicht font system).

Can I use the irrUString.h classes to convert the UTF8 encoded string coming from IRRxml to something I could use directly with Irrlicht like core::stringw? (So it display accented character proper (latin1) with the default Irrlicht ANSI font? I found a hacky way to do it on windows, because I treat all characters that are more than one bytes and calculate an offset to get to latin 1. On windows it work, because the "char" is 2 bytes, But I think it's completely different on Linux (4 bytes and perhaps another endianess). So I was thinking to use the class to convert...

I see that I could use perhaps use it to save back in UTF8, (test.toUTF8()) but could I load (test.FromUTF8() for example?

Re: Irrlicht i18n (Unicode, FreeType, etc.)

PostPosted: Thu Jul 31, 2014 3:54 am
by Nalin
christianclavet wrote:HI, Learned more about the character encoding and just figured out that when I open XML files in UTF8 the string is still in UTF8 in memory and have to convert it to ANSI (The Irrlicht font system).

Can I use the irrUString.h classes to convert the UTF8 encoded string coming from IRRxml to something I could use directly with Irrlicht like core::stringw? (So it display accented character proper (latin1) with the default Irrlicht ANSI font? I found a hacky way to do it on windows, because I treat all characters that are more than one bytes and calculate an offset to get to latin 1. On windows it work, because the "char" is 2 bytes, But I think it's completely different on Linux (4 bytes and perhaps another endianess). So I was thinking to use the class to convert...

I see that I could use perhaps use it to save back in UTF8, (test.toUTF8()) but could I load (test.FromUTF8() for example?

It automatically assumes const char* strings to be in UTF8. The string is stored UTF-16 internally. The toWCHAR_s() function will convert to a core::stringw. However, there is no guarantee your font will display correctly. It really depends on the code point of your accented characters and whether or not the default font classes are capable of drawing that glyph. It has been a long time since I looked at the bitmapped font classes in Irrlicht so I can't say off the top of my head if it would work. Plus, if you are using a glyph that is represented by two wchar_t's, Irrlicht's default font classes will not be able to draw it at all.