i18n Issues

I want to keep the directory and file structure  for a Karma lesson as simple as possible. Ideally, the html markup, css, and javascript should not be locale-specific. All translated text should be in .po files. I have been looking around at different i18n strategies for web sites and they all appear to only facilitate i18n for text, not for audio neither for right-to-left reading layouts. I haven’t looked carefully at Adobe Flex’s i18n mechanism. Perhaps it provides support for those aspects of i18n.

I have done some research on i18n support for websites and desktop software. I want to thank Sayamindu Dasgupta for being extremely helpful. He sent me many links but the best is perhaps the GNU manual on Gettext.

Various i18n frameworks handle i18n text quite well but they all seem to have 3 significant holes.

1. They lack native digit support. Most people don’t know this but many countries use different characters for the digits 0-9

For example, in Nepali script the digit 5 is ५ . Javascript lets you print ५ as a string, but there is no obvious way to do the following:

var myScore = 555;

$(“scoreBox”).html(myscore);

where myscore displays using the Nepali digits ५५५

2. They lack an easy way support to right-to-left reading layouts. It seems to me that most web frameworks require you to use different css files for different locales. This isn’t very elegant.

3. Sound — No frameworks have an easy mechanism to link to different sound files depending on locale. This doesn’t map seamlessly to the gettext paradigm as .po files would have to hold links to external sound files. It isn’t feasible to embed binary sound files directly into .po files.

Here are some ideas I have for the above problems. Please comment!

Supporting Native Digits

I hoped their would already be simple javascript library for this but they all seems to focus on changing the periods and commas according to locale, such as jquery-i18n. Sayamindu directed me to this feature of GlibC which does appear to convert the digits to the correct localized character. I still haven’t dug into the GLibC library to understand how it works its magic. I the meantime I have a cruder idea on how to output native digits.

I could write  simple javascript method that parses the digits to be printed and adds each digit to the unicode base for a given language. For example,

Devanagari numerals (Nepali, Hindi) use the range 0966 – 096F for 0-9 and Arabic uses 06F0-06F9. I could simply  add the base for the particular writing script and print out the result as a unicode string.

function kL10nDigit (digit, lang) {

baseNoLang = lookupBaseNumber(lang);

var myDigit = baseNoLang + digit;

return  “\u” + myDigit;

}

var myscore = 555;

var listMyscore = makeIntoList(myscore);

$(“scoreBox”).html($.map(myscore, kL10nDigit));

The map function here returns the 555 as the unicode string “\u0970\u0970\u0970” or ५५५ and $(“scoreBox”).html(….) sets the content of the scorebox to that value.

Some Nepalis I have met argue that Nepali kids don’t really need to learn the Devaganari digits and I have met others who feel very passionately that it is essential. This is just another example of how many people have very different but strongly held beliefs about what seem like a small detail to many of us. As software developers, it isn’t my role to decide what is taught but to empower educators and communities to make those decisions for themselves.

Right-to-Left Layouts

Most websites seem to use separate CSS stylesheets when changing between left-to-right and right-to-left layouts. I would really like to avoid this if at all possible. Redundant stylesheets means redundant code, which means codebase fragmentation and more room for errors to creep in. I would really like to label a set of HTML elements and be able to simply reverse the order according to locale.

For Example:

The main CSS file — karma.css

.horizontal_elements { direction: ltr; }

The HTML — index.html

<div class=”horizontal_elements”>

<button id=”btn1″ /> <button id=”btn2″ /> <button id=”btn3″ />

</div>

For a right-to-left script, I want to reverse the order of the buttons but not the order of all the elements on the page. That would turn the page effectively upside down. Ideally, I could just change the direction property in .horizontal_elements to “rtl” . Unfortunately that doesn’t work and I don’t know why.

I could write a simple javascript method that checks if the locale is right-to-left and if true reverses the order of all elements w/in a div elemen having the class “horizontal_elements” . This is feasible but it would be much nicer to do it by changing the “direction” property for the <div> element.

Sound

Sorry, don’t have time to write about that this morning! next blog post 😉

Miscellaneous

GNU gettext seems to assume that the first version of a piece of software will have an English language locale. Can we use the html element id as the msgid in the .pot file?

What about text that shouldn’t be locale specific? for example, portions of an English lesson that are meant to be in English regardless of the locale?

A key priority, trying to adhere as closely as possible to the Model-View-Controller design pattern. Why? Because it is easy to read and understand code that follows MVC and this pattern makes lets teams easily divide up work between less technical designers and their programmer colleagues.

UPDATE #1:

Thanks to Guy Sheffer and Tzafrir Cohen for showing me how to use the “dir” attribute in <html dir=”rtl”> to set the text direction for document. That certainly works for buttons but doesn’t seem to work for <label> or <span> elements. Will have to investigate further.

UPDATE #2:

I have found that <label> and <span> element do change direction properly is the language embedded in them contains text written RTL instead of English, which I was using.

Initial Thoughts on i18n for Karma

Subzero and I have lately been discussing i18n (internationalization) strategies for Karma. Please bear w/ me as my i18n knowledge is quite limited and I am just now wading into this important, if complex, area. Subzero and I need to spend some quality time discussing the issue with Sayamindu but first I need to do some self-study on the matter.

So far I can identify four different aspects of a Karma lesson* that need i18n. Perhaps only 3 of them should be.

1. Text
2. Audio
3. Programmatically generated numbers and text, for example the score in a game
4 images

I am not certain that images can or should be internationalized in an automated way. That said, certain images are not appropriate for different locales. For instance, it may not be a good idea to use images of pigs and dogs in certain muslim countries. Similarly, you have be careful depicting cows in countries with large hindu populations. Still, it would be great to abstract image-local-house to an image of a house typical of the locale but I am sure how feasible that is.

For a Karma activity, we would like to put all of the presentation stuff goes into the CSS, including dimensions of elements, images used, layout. Then we would basically use the html to link in the css and javascript files. The HTML file would hold html elements as place holders but not contain any actual text.

Here is a file layout that I have dreamed up for a karma lesson. Anything common to Karma should start with a lowercase “k”

index.html               # links in css, js, and placeholder html elements
css/klessonx.css      #css specific to the lesson
css/karma.css        #global karma css styles, default common styles
images/                  # images , how could these be localized?
js/klessonx.js        #javascript specific to the lesson
js/karma.js             #javascript utilities common to Karma lessons
js/jquery.js             #javascript dependencies
js/. . . ..js                #other javascript dependencies
locale/text/ne.po 
                  sp.po
                  en.po    #pootle localization files
locale/sounds/ne/correctAnswer.ogg   #actual audio is in Nepali
                       en/correctAnswer.ogg   #now in English
                       sp/correctAnswer.ogg   #now in Spanish

Localization in Sugar only covers text strings and not sounds. How can we extend the current paradigm to cover audio files as well?

So far, I have it my mind that the universal identifier for an audio, text, or image element is the html element “id” attribute. This scheme will work best if the “id” attribute is in a language most widely understood. For better worse, that language is English. My idea is that the audio files should all match the target html element id but be stored in folder for the appropriate locale.

This does present a file management problem in that it makes it hard to track down audio files when they are stored on a central server. In general, I am in favor of putting metadata in filenames for image and audio files, i.e. christmas_50px_100px.jpg or en_CorrectAnswer.ogg, sp_CorrectAnswer.ogg.

Karma would place the localized text by reading parsing the .po file for the current locale and inserting the localized strings each time the page is loaded.

Questions I have:
1) How would this work together w/ pootle and gettext?
2) What are some pitfalls I haven’t thought of?
3) How could we ensure dynamically generated numbers are presented in the locale without making the programmer’s job too hard?

And a request: Pls, pls direct me to great i18n resources and guides. If there is already an established standard for this, I will be happy to copy it.

* A Karma lesson is much more granular than a whole Sugar activity. In general, we expect a number of Karma lessons to be aggregated into sugar bundles much like gcompris bundles its “lessons.”

UPDATE

I have been thinking about what text and audio might we might not want to internationalize. For example an English language lesson for Nepali students might have some text in Nepali and some in English. Similarly, upper-level math classes may use Nepali text but Roman Numerals, NOT Nepali numerals. This page from the W3C on declaring intended language in HTML markup seems relevant

canvas – js interaction

Hi.

I have been working on “Quadrilaterals” activity from OLE Nepal.

here is the flash version:

go to http://karma.sugarlabs.org/framework_original/MenuStage.html
choose something like “3″ (on the top)
choose the first button on your right
click on the activity’s image (wait, it’s loading)
click on title
click on the “2” button

and here is the Karma version (under development):
http://karma.sugarlabs.org/quadrilaterals/index.html

Working with it I found troubles and… yes, solutions, also I got some new ideas, lets start:

There is one little (big) difference between “Quadrilaterals” and “Can and Can’t“: buttons.
“Can and can’t”: I used the jQuery click event to handle clicks on the div html tag for the white row.
“Quadrilaterals”: I’ve implemented a solution based on canvas, these are images that behave as normal buttons. The advantage is that it permits interaction between code and mouse events (i.e. mouseup, mousedown, mouse over). Disadvantage: the click area is limited to a specific region (circles for now).

There are some other things that we have to improve:

• TextFields (I would like something like flash one, maybe it’s too much)
• draw rectangles, circles, etc.. It’s posible with the actual js code, but It would be really good to write less do more.
• to separate the code and the design of the buttons (from now Karma buttons).
• layers? well… div is the unqiue way, so something like Karma.addLayer( "layer"); would be useful

I’m working around all this, so keep tuned, there is a lot more to see 😉

think again

Hi guys.

Working with animations, I realized some things that I had not stopped to think about them until now.

jQuery uses an animate effect, usually we alter the property values of the element we want to affect.

animate syntax is really simple:
animate (params, [duration], [easing], [callback])

The point is that it sets the duration (call it t) and not speed (v). If we want to move an item from one position to another we fix this little detail setting the duration (t) depending on the speed and distance: t = d / v. The problem is that we always have to adjust the settings .. and what would happen if you don’t know the distance?.

I think a better approach is to set the speed. Following Flash dogma we have frames per second (fps), but here (jQuery – Karma) we have (at this point) no frames, in the real life we have meters per second (mps) and others….perhaps something like “pixels” per second might work?

uhh??

Animation with JavaScript: that little beast

Hi guys.

In the past post I told you about “can and can’t” audio troubles. And here I told you about animation problems using jQuery. The final “can and can’t” activity’s version is approaching..

I have spent some days wondering how to join animation version + everything else version. All the possible ways I figured end in a bunch of code. That’s not the only thing and unfortunately there are more to worry about it.

First, what do we want?
Take a look about the original activity following these steps:

go to http://karma.sugarlabs.org/framework_original/MenuStage.html
choose english
choose something like “3”
click “can & can’t” image
click “can & can’t” title

The Karma version principal problems:

• how to run parallel animations?. In fact, parrallel js code?
  I have been using setTimeOut but this solution stinks.
  Bryan and kbrosnan (#firefox) has directed me to web workers. I’m new with this but I’m excited, it seems useful!.
• how to get something like layers?. It’s hard to work in the same layer.
  I have been using div html tag and z-index property, it works, but.. does it exists something better?
• js framerate?
  Inspired by processingjs I wrote a minimalist draw function but… seems that I need to adjust the framerate depending of the cpu, I wonder if it exists a way to get the cpu lag…:S

what do you think?
any advise is welcomed. 🙂

Making Progress with Hudson

It took me a while to gain momentum, but I now think I am getting the hang of Hudson. The key tool is really nose, which makes it simple stupid to run tests and output the results in a format that Hudson can read.

sudo easy_install nose
nosetests

This should output “Ran 0 tests in 0 seconds”

The next step is to output the test results in a format compatible w/ JUnit test reports and thus w/ Hudson

nosetests --with-xunit

This outputs the results to nosetests.xml . Then you can get a nice pictures like this:

Check out the cool graph!

Now that is readable!

Automating Builds, Testing, and QA for E-Paath and (eventually) Karma

I feel bad that lately I haven’t spent much time assisting Subzero w/ the coding of Karma. My current excuse is that I am working on a related if not immediate goal. Here in Nepal, we need to roll out a new build of the E-Paath learning activity. E-Paath is a huge activity , comprising more than 100 individual lessons. If I stick E-Paath into a single XO bundle, it is almost 1 GB in size. Testing E-Paath is also a major hassle since there are so many individual lessons and our eventual users are not particularly sophisticated. We have a fairly complex build process and we really need a straightforward testing and bundling process.

The holy grail for me is a web application that will tell me which tests failed in which build on which day. This sounds simple, but it is actually quite a complex problem.

We have to automate QA, regression testing, and reporting the results to a web page. Testing by hand is far to time-consuming. I believe that these problems will later affect organizations that use Karma to build large suites of lessons tuned to their locale and educational context. I do see a standard build and testing process to be important parts of Karma in the long-run. I really want developers with little knowledge of systematic testing, build processes, or QA to be able to re-use the hard work we are having to do here to get these processes together from scratch.

While unit-testing, QA, regression testing are in Karma’s future, they are in E-Paath’s immediate present. I have to get this stuff working ASAP for E-Paath.

E-Paath is different from most large software projects. The individual lessons are particularly dependent on each other. They are dependent on the “framework” that we use for a common layout for the lessons. Changes to the framework affect all of the lessons but changes to a particular lesson generally do not affect the other lessons.

I have tried to set up buildbot and found it quite complex to set up. It seems like a great tool to distribute running builds but that is not currently one of my requirements. I spent about a day w/ it last week before becoming quite frustrated. It seems like a great piece of software but its complex set up process implies to me that it will require a fair bit of maintenance.

Last Friday, I happened Martin Fowler’s essay on Continuous Integration. Really a good read. Soon after someone on the #buildbot IRC channel told me that Hudson may be a better fit for my particular needs. It is super-easy to test out hudson. Just download hudson and type in
sudo java -jar hudson.war

I have successfully created and run some test builds with hudson. So far it is quite easy. The next big step is to run regression and unit tests and output the results to a lovely chart like this:

Warning — the above is an example from the Hudson demo. I haven’t gotten it running yet.

Putting the test results into the proper format is a non-trivial problem. You have to output your unit test results into a schema compatible with the JUnit test result format. I am a complete n00b when it comes to unit testing. I think I did a tutorial all of of 8 months ago. I have mucked around with pyunit and XMLTestRunner without much success for several hours this morning.

Now i have shifted my focus to nose which has a handy --with-xunit switch that could make outputting reports to Hudson a cinch! I will let you know my results.

“can and can’t” that is the question

Hi guys.

I have been working with the “Can and can’t activity”, the good part is that I have done almost everything, the bad part is that I work in different versions: one around animation and the other about everything else, long story.

In this post I’m talking about “everything else”, specially the sound part:

jQuery has its own sound plugin, but it’s useless:

This is a simple sound player, it uses flash to play the sounds, but DOES NOT provide a flash interface for any controls, everything is controlled via jQuery

Bryan recently sent me a link about nice space invaders game 🙂
I was guessing how it plays sound files, so I peeked a little. It uses soundmanager2, it works, yes!!, but it depends too on a swf file (soundmanager2.swf).

what about swf?, why not to use a flash based solution?
well, if you don’t have the flash plugin the sounds don’t play (I tested it with ff3.5 beta4+), here you can find more information about it. Also, I remember you.. we’re creating something that must be flash independent.

By the way, here is another flash audio wrapper: JS Sound Kit

Also, if you want something harder….there’re java applet solutions :S

At this point, do you have any good suggestion??

well, do you remember HTML5??
I hope, well one of the good new things is the tag:

The audio element is used to embed sound content in an HTML or XHTML document. The audio element was added as part of HTML 5.

Here you can se the “can’t and can activity” with audio support. 🙂

It has some points to need to improve:

• [Bug] the canvas text moves after redrawing the rectangle.
• [Improve] preload all the images and sound files

The bad part about html5 is that it only load-plays .ogg files. By the way in unix you can use mp321ogg to convert mp3 to ogg.

what do you think?