Irrlicht i18n (Unicode, FreeType, etc.)

Announce new projects or updates of Irrlicht Engine related tools, games, and applications.
Also check the Wiki

Re: Irrlicht i18n (Unicode, FreeType, etc.)

Postby Nalin » Sun May 20, 2012 6:58 pm

christianclavet wrote:Edit: Reading from of this thread, I think I will have to patch the XML loader, as the data came from there.

Correct. Because of how Irrlicht's XML loader was programmed, there is no way to extend or replace it without modifying the Irrlicht source code. You could write your own loader, but then you wouldn't be able to use Irrlicht's serialization functions. I suppose I should re-write the XML loading code one of these days and submit it as a patch, but, for now, you either have to patch Irrlicht or forgo unicode XML files.
Nalin
 
Posts: 194
Joined: Thu Mar 30, 2006 12:34 am
Location: Lacey, WA, USA

Re: Irrlicht i18n (Unicode, FreeType, etc.)

Postby christianclavet » Tue May 22, 2012 12:41 am

I tried my "trick" that I've done with TinyXML to correct the strings and it seem to work also with IRRXml. My primary interest is to at least have accented characters working (LATIN1 would be fine). So I will have something while Irrlicht support that fully. (By the way, great work on this! I just hope the DEVs will add theses into Irrlicht one day..)

As for my "trick", I will have to check how the string come out on Linux as the Irrlicht string will surely be different on that platform as on Windows. My sources is UTF8 xml files. Thanks Nalin.

If you're curious here what I've done:

cpp Code: Select all
core::stringw winconvert(core::stringw str)
// Convert accents from loaded XML files (irrXML)
// WARNING: Tested only on windows
// might not work on Linux or other platform.
{
        bool debug = false;
        core::stringw textline = L"";
        core::stringw text = L"";
        u32 base = 0;
       
        char test2 = ' ';
 
        for (u32 a=0; a<str.size(); a++)
        {
                // Get the character first
                text = str.subString(a,1);
               
                // Then check this character directly (convert to unsigned 32bit)
                base=(u32)text[0];
 
                if (base<256) // Standard characters
                {
                        textline+=text;
                }
               
                // All characters after 256 are ignored except thoses
                // Character higher are re-aligned from the offset to match LATIN1
 
                // Reference to the table is here:
                // http://www.utf8-chartable.de/unicode-utf8-table.pl
               
                const u32 offset=65216;
 
                core::stringw replace = L" ";
                if ((base>255) && ((base-offset)<255))
                {
                                replace[0]=(base-offset);
                                textline+=replace;
                }
               
        }
       
        return textline;
}


Then when I want to "convert" the irrXML string to a valid Irrlicht stringw:
(Here getting the stringw into "description" and sending it to a listbox for verification):

cpp Code: Select all
description = xml->getAttributeValue("description");                                                   
list->addItem(winconvert(description).c_str());


Please note that not all of the accented characters are "parsed" as this was only used to test. I think there could be an even better way to get this, but I have to know how Irrlicht string table is build (is it matching the Ascii Latin1 table?). If it's matching I could simply put the "resulting number from the offset" and convert it to ascii back into the string. Right now I'm checking PER character and I think it work fine but not hyper efficient.

EDIT:Cleaned the code having found a better way to get the proper LATIN1 characters. The character is then replaced in the string.
Last edited by christianclavet on Thu May 24, 2012 8:21 pm, edited 1 time in total.
User avatar
christianclavet
 
Posts: 1638
Joined: Mon Apr 30, 2007 3:24 am
Location: Montreal, CANADA

Re: Irrlicht i18n (Unicode, FreeType, etc.)

Postby Nalin » Tue May 22, 2012 3:53 pm

If you aren't relying on Irrlicht's built-in serialization to create entitites, and you are just using the XML loader raw, just pass the data you receive to the ustring class. Irrlicht's XML loader won't mangle the bytes at all, so you can just directly pass the data into the ustring class and it will be able to parse the data. If you call the char* constructor, it will attempt to load the data as UTF-8. You can also use the loadDataStream.

If you are using UTF-8 files, they should be the same on both Windows and Linux. The difference comes with the wchar_t data type. It is 16-bit on Windows and 32-bit on Linux. Irrlicht writes XML files using wchar_t by default, so you have to watch out for that. ustring will load wchar_t data using the platform specified type. EG, on Windows, it tries to read it as UTF-16, while on Linux it tries to read it as UTF-32.
Nalin
 
Posts: 194
Joined: Thu Mar 30, 2006 12:34 am
Location: Lacey, WA, USA

Re: Irrlicht i18n (Unicode, FreeType, etc.)

Postby Nalin » Thu Jun 21, 2012 8:21 am

Just some small updates:
  • xml_io.patch: Updated to the latest trunk revision.
  • irrUString.h: Changed u32 to size_t.
  • CXMLWriterUTF8.zip: Merged CXMLWriter.cpp performance improvements from trunk.
Nalin
 
Posts: 194
Joined: Thu Mar 30, 2006 12:34 am
Location: Lacey, WA, USA

Re: Irrlicht i18n (Unicode, FreeType, etc.)

Postby feelthat » Tue Feb 19, 2013 2:54 am

can it done on ios and android?
feelthat
 
Posts: 194
Joined: Sat Feb 02, 2013 5:27 am

Re: Irrlicht i18n (Unicode, FreeType, etc.)

Postby Nadro » Tue Feb 19, 2013 3:22 am

I can confirm that this add-on works on iOS (I use it in my project for true type fonts).
Nadro
 
Posts: 1647
Joined: Sun Feb 19, 2006 9:08 am
Location: Warsaw, Poland

Re: Irrlicht i18n (Unicode, FreeType, etc.)

Postby feelthat » Tue Feb 19, 2013 12:02 pm

if could, would u send me some build step and example?

email: fatalfeel@yahoo.com.tw

Nadro wrote:I can confirm that this add-on works on iOS (I use it in my project for true type fonts).
feelthat
 
Posts: 194
Joined: Sat Feb 02, 2013 5:27 am

Re: Irrlicht i18n (Unicode, FreeType, etc.)

Postby vroad » Tue Feb 19, 2013 1:27 pm

http://irrlicht.svn.sourceforge.net/vie ... es/ogl-es/

There is OpenGL-ES version of Irrlicht.
If you don't have svn client, you can download as tarball from sourceforge's browser.
It also contains project for Android and iOS.
This addon should works well on those platforms as it works on desktop.
vroad
 
Posts: 18
Joined: Sun Jan 03, 2010 1:30 pm

Re: Irrlicht i18n (Unicode, FreeType, etc.)

Postby feelthat » Tue Feb 19, 2013 6:57 pm

I use Proton 3D,

After download http://irrlicht.svn.sourceforge.net/vie ... es/ogl-es/

I did not see any utf8 loader?

Did u mean irrlicht/branches/ogl-es/ include i18n ????


vroad wrote:https://irrlicht.svn.sourceforge.net/svnroot/irrlicht/branches/ogl-es

There is OpenGL-ES version of Irrlicht.
If you don't have svn client, you can download as tarball from sourceforge's browser.
It also contains project for Android and iOS.
This addon should works well on those platforms as it works on desktop.
feelthat
 
Posts: 194
Joined: Sat Feb 02, 2013 5:27 am

Re: Irrlicht i18n (Unicode, FreeType, etc.)

Postby feelthat » Wed Feb 20, 2013 3:02 am

For UTF8 on mobile compare

1. IrrFreeType

2. IrrlichtML 1.7.1

3. Irrlicht i18n

which one is good on mobile?
feelthat
 
Posts: 194
Joined: Sat Feb 02, 2013 5:27 am

Re: Irrlicht i18n (Unicode, FreeType, etc.)

Postby Nadro » Wed Feb 20, 2013 7:11 am

I use ogl-es branch + add-ons from this topic (Irrlicht i18n), but as I said only TT was important for me, I didn't test UTF8 features from this project but I think that it should works properly.
Nadro
 
Posts: 1647
Joined: Sun Feb 19, 2006 9:08 am
Location: Warsaw, Poland

Re: Irrlicht i18n (Unicode, FreeType, etc.)

Postby feelthat » Wed Feb 20, 2013 11:55 am

in first page of this project

/*I have a keyboard that has non-ASCII characters. When I type into a text box in Irrlicht, it doesn't show the proper character! Why? (ALSO: Why can't a paste unicode text into Irrlicht?)
These patches don't solve this issue. If you want to be able to input unicode characters into Irrlicht, please take a look at MadHyde's patches*/

so Can input i18n unicode words in edit box
like Japanese Chinese

Nadro wrote:I use ogl-es branch + add-ons from this topic (Irrlicht i18n), but as I said only TT was important for me, I didn't test UTF8 features from this project but I think that it should works properly.
feelthat
 
Posts: 194
Joined: Sat Feb 02, 2013 5:27 am

Re: Irrlicht i18n (Unicode, FreeType, etc.)

Postby vroad » Wed Feb 20, 2013 1:59 pm

Unfortunately, typing with unicode characters is only supported on Windows/Linux.
This is OS-Specific part. Other futures should work on every platforms.
vroad
 
Posts: 18
Joined: Sun Jan 03, 2010 1:30 pm

Unicode, FreeType on ios android help

Postby feelthat » Wed Mar 06, 2013 3:33 am

Dear All:

Recently I use Freetype2 to impelement UTF8 words in Irrlicht 1.73

Do you have any other idea, Maybe use ios or android system utf8 on
Irrlicht 1.73?

thanks first by jesse stone
feelthat
 
Posts: 194
Joined: Sat Feb 02, 2013 5:27 am

Re: Irrlicht i18n (Unicode, FreeType, etc.)

Postby CuteAlien » Wed May 21, 2014 2:08 pm

I've started working with this ttf implementation. It really is a lot faster than the other implemenation which I used so far (because this one does put characters on pages while the other used one character per texture).

I modified it somewhat to adapt it to my needs (and getting it a little closer to Irrlicht).
Changes I made are:
- No dependency on ustring (I don't think string-conversions should happen inside the font-class itself)
- Reworked that allocation stuff. That had some memory leaks. SGUITTGlyph uses now the rule of three (destructor, operator=, copy constructor are all needed when one is needed) and so I could get rid of the custom allocator. Also set_free_when_destroyed isn't about pointers of elements but about array-memory itself (yeah confusing stuff).
- remove textscenenodes. I get that they are useful, I just don't think they belong in that class.
- Font should probably not grab()/drop() the driver (we have no weak pointers unfortunately). I rewrote that stuff some more to pass the exact types needed.
- Avoid a few memory allocations in draw (not a good place to use a map as allocations are more expensive than linearly going over a small array).
- Minor stuff tidying up I changed on the way while hunting some independent bug (font-textures sometimes had been complete white, but that turned out to be an opengl-texture bug in svn trunk which will be fixed soon).

My version is here:
https://code.google.com/p/irr-playgroun ... TTFont.cpp
https://code.google.com/p/irr-playgroun ... UITTFont.h
https://code.google.com/p/irr-playgroun ... se/ttf.cpp (an example how to use it)

Will still need some work probably until it can be added to Irrlicht directly. Especially have to think about handling the freetype face management (might move that to an own class so we can get the font constructor back into public).

But all in all a very nice implementation, thanks for it!
IRC: #irrlicht on irc.freenode.net
Code snippets, patches&stuff: http://www.michaelzeilfelder.de/irrlicht.htm
Free racer created with Irrlicht: http://www.irrgheist.com/hcraftsource.htm
User avatar
CuteAlien
Admin
 
Posts: 8192
Joined: Mon Mar 06, 2006 2:25 pm
Location: Tübingen, Germany

PreviousNext

Return to Project Announcements

Who is online

Users browsing this forum: No registered users and 1 guest