New feature on Locale : Mark for review

We just  released a new feature that should make your translation process even more effective: you can now mark some of your translation entries to be reviewed. It’s particularly useful when you modify your default locale and need all other locales to be modified as well on a specific string.

Click the cog icon next to the translation key and then click “Mark for review” from the dropdown.

Image Markdown Review

It will mark all translations not in the default locale as incomplete, like this:

Image Markdown review 2

Happy translating!

Posted in LocaleApp | Leave a comment

Locale enters Heroku private beta

HerokuConnecting your Heroku-hosted Rails app to Locale and streamlining the entire localization process is now even easier.

The Locale add-on for Heroku entered private beta today. Members of the Heroku private beta program are encouraged to check it out and contact us with any questions or issues.

More information about setting up the Locale Heroku add-on.

Posted in LocaleApp | Tagged , | Leave a comment

YAML Security

As you’re probably aware, the recent serious vulnerability in the Ruby on Rails framework was caused by the way loading YAML works in Ruby.

As Locale both loads your YAML files and sends content which you then load into your application via YAML this affects us more than most Rails applications.

On learning of the vulnerability we immediately applied the patch so that we wouldn’t be affected by the Rails bug.

We then checked the content in the database for any potentially insecure YAML and thankfully found none.

The next step was to audit the code everywhere we accept YAML input and ensure that potentially insecure content is never loaded by Ruby.

This was all implemented and deployed before the first exploits became widely available and we’ll continue to monitor the content we receive to make sure there are no problems.

We’ve updated the localeapp gem so that it won’t load anything that looks suspicious. This has been released as the 0.6.9 version which you should update to as soon as you can.

Posted in LocaleApp | Leave a comment

New on Locale: Translation history

We’ve just released the first version of translation history on Locale. Just click the gear icon next to any key:

You’ll see who did what to the key, and when:

As ever, we’d welcome your feedback!

Posted in LocaleApp | Tagged | Leave a comment

Avoiding the Tar Pits of Localization with Jeff Casimir

In this Ruby HangoutJeff Casimir of JumpstartLab talks about:

  • The tar pits of localization
  • Writing better code
  • Fixing magic data
  • White labeling your app

Contents

  1. Internationalization
  2. Localization
  3. Locale Selection
  4. Bonus: White labeling
  5. Summary
  6. Q and A


1. Internationalization

Even if you’re only going to support one language, internationalization will help you to build a better application.

We’re about to embark on a dangerous journey ..

Internationalization & Localization, i18n, l10n

As a developer considering internationalization and localization, you’re like this mammoth. You’re out walking and you think: “I’ll walk through this water – how deep could it be?” .. and pretty soon, you’re stuck in the tar pit.

The tar pits of localization

That’s how internationalization is. It starts out easy enough, and eventually you’ll be drinking tar.

i18n & l10n

Internationalization, i18n is the one-time process of preparing an application to support more than one locale. In a Rails application, that typically involves going through your controllers, models and views, pulling out all the content strings and replacing them with references to a look-up file or some other means of finding the translated text.

Localization, l10n is the many-time process of taking an application which has already been internationalized and creating a locale for it – US Spanish or Brazilian Portuguese, for example.

Why businesses care
Localization provides a better user experience. The more users you have, the more customers you have. The more customers you have, the more money you make.

Why developers care
Developers typically try to stay away from localization. The tar pit is scary, and making money is generally somebody else’s problem.

The main reason developers care about internationalization is that it results in better code.

Magic Data
Internationalization takes a scalpel to your code.  I’ve written many controller actions like this – something I call magic data.

Magic data

Magic data leads to several problems, one of which is copy-edit commits. That’s where you change a line of code only to fulfil a marketing or copy change demand.

For example when a submit button changes to save, or a flash message changes from your article was created to your article was saved.

That kind of stuff pollutes your code. It looks like you’re making functionality changes – but you’re just changing a bunch of strings.

When you look at a controller action like this, you’re just looking for the magic data, the strings.

It’s one of the reasons I like to use symbols in Rails whenever possible because it separates functional code from magic data. Whenever you see strings, they should be removed. So how do you remove them?

Establish a look-up dictionary
This is what a YAML-based dictionary in Rails looks like.

YAML based dictionary

With this dictionary established in config/locales in a default Rails application, you can utlise those keys with the t helper.

You can access the YAML hierarchy via a dot notation. So this means a top level key of article having a child key of deleted.

YAML hierarchy

The t helper works in a view or a controller context. If you’re not in either of those contexts, you can speak directly to the i18n class, call t on it, and pass in your key that way.

However, that’s typically a warning sign. If you’re calling the translate helper from the model layer, you’re probably doing a bad thing because translation is about presentation and models shouldn’t be presenting things.

I’m not saying it’s wrong to use I18n.t – but do it with hesitation.

Structuring the keys
The first rule is to keep them short – and you do that by avoiding snake-casing the text.

I’ve often seen the text snippet turned into the key by replacing all the spaces with underscores. This isn’t a maintainable way to go. Instead you should focus on the core meaning of the text.

Previously, you saw a key called article.deleted rather than article_has_been_deleted – because has_been is not core to the meaning.

Instead, just focus on the central idea – because that text is very likely to change, but the core meaning of the key is unlikely to change. Also, forget about reusability.

Look-up dictionaries are not programming. Resist the urge to do fancy things with your keys and translations – interpolation, automatic pluralization and singularisation.

The more complex you make your keys and translations, the further down in the tar pit you’ll find yourself. As far as possible, use simple keys and simple strings.

If you want to do something more complicated, the Draper gem can be helpful in handling complex logic at the view layer.

I was inspired to write this gem from the Microsoft community. They have this idea of view models and Draper helps create models that deal with presentation concerns at the view layer.

Magic data in a controller

So, we remove magic data from the controller and use the t helper to replace them with keys. This is what a nicely internationalized controller action looks like.

Magic data replaced by keys

When you look down at the model layer, even more magic data tends to spring up. These are all spots that could be dealt with using internationalization.

Magic data at the model layer

The numbers here are probably better handled in configuration files because they’re unlikely to change between different locales.

If they’re not going to change across different locales then they belong in a config file.

If they are going to change – like that message in line 3 – those belong in a locale file.

In this case, ActiveRecord can help you out – especially when you’re setting a specific validation message.

When an ActiveRecord model fails validation, ActiveRecord will attempt a sequence of look-ups in the current locale.

ActiveRecord look-up hiererchy

This is a description of that look-up hierarchy – the most useful of which is activerecord.errors.messages

In your YAML file, under the messages child key, you can define keys for each built-in validation and Rails will automatically use those strings.

Rails guide messages

One gotcha is that you have to define the keys by what the Rails guide calls message rather than what it calls validation.

In this example, your key would be blank, not presence

YAML structure for ActiveRecord messages

Putting it all together, when I structure my keys like this, I can delete the message line from my model, leaving just validates_presence_of :title

Removing magic data from models

A word of warning, this is an example of where you’ll be tempted to get fancy. You’ll say “I can get the attribute name and dynamically inject it into the message.”

Don’t do it. You’ll regret it.

Just keep it simple. It works fine in English because title is one word. In other languages, it might be two words, you might also have a capitalisation problem, etc etc.

This leads down a road of pain. Just keep the messages simple and use the string you define in the YAML file.

You can also define the print name of your attributes if you follow this kind of hierarchy:

Automatic attribute naming

So, here I’ve given the article model a string of Title and the comment model a string of Your comment.

If you’re using label tags like you should be, along with any other form helpers built in to Rails, they will make use of these strings automatically – you don’t have to monkey around with your forms.

In this example, if you’re just outputting a label, it will show the string Your comment instead of body on the comment form.

By pulling these strings out of my functional code, I have one less reason to change my functional code, which improves its durability, which makes it better.

Even if you’re only going to support one language, internationalization will help you to build a better application.


2. Localization

Let’s say you have a blogging app, and you want to present the articles in multiple languages. Pretty soon you’re going to run in to the problems of creating multiple translations, managing the import and export of the content and keeping these translations in sync.

This is where you start to enter the nightmare mode of localization because of this scenario:

  • Write a blog post in English
  • Send it off for translation to Japanese
  • Make a couple of revisions in the English text
  • Receive back the Japanese translation of the original English text (minus the revisions)

Now the Japanese translation is a slightly different version than the English one – and we’re only talking about two languages. When you’re supporting 12 or 20 languages, this becomes a tremendous issue.

One solution is CopyCopter, formerly a commercial product by ThoughtBot in Boston. It didn’t really work out for them so they open-sourced it. Now you can run your own server.

The model of CopyCopter is that you keep your keys and strings in a separate app. You run a rake task on your primary app to fetch copy from that remote app. Your translators just interact with the remote app, without needing access to your primary app.

A second option is to run it right in your primary application with a gem called Globalize3, primarily authored by Sven Fuchs who also wrote the i18n gem.

Globalize3 works but it makes you realize what a complicated problem this is, because it is a complicated solution.

You end up creating tables for your fields so each field in your primary table will have its own sub-table where each row is a different language – so that they can all be time-stamped and versioned.  Your database schema will be bananas, you’ll have tables proliferating like bunnies.

This third option is my favourite one. Since I discovered this app, I will not use anything else.

To clarify, I have no affiliation with these people other than thinking that they’re bad-asses!

Locale has a gem that you install in your app which talks to their app and manages your translations. It is amazing, so please go out and use it because these people are doing bad-ass work.

i18n, l10n, Rails internationalization and localization

Honestly, I don’t care how much it costs. Once you try those other options, you will start throwing your money at these people to save you from the tar pit.


3. Locale Selection

How do we determine what locale your user actually wants?  The Locale_Setter gem helps you do this, and there are five broad strategies for deciding which locale to display:

  • Geo-location
  • Browser preferences
  • URL parameters
  • Account preferences
  • Default settings

1. Geolocation - sucks, don’t do it.

Think about when you’re travelling – does your current physical location on the planet change the language you want to read on the web? Geo-location is stupid. Please don’t tie a person’s location to the language they speak.

2. Browser preferences – deep down in your browser preferences there are language settings which you can access. Those preferences are submitted with every request made on the web.

If you go to a Rails view template and use the debug helper to output request.env – you’ll see http_accept_language buried in the output.

One of my favourite debugging or investigation techniques is from a controller you can call render: text and then pass it some text. It’ll spit that raw text back to your browser without any wrapping html.

http_accept_language

Here’s a slightly more complicated result:
en-US,en;q=0.8,es;q=0.2

There are three separate comma-separated parameters in this second example.

It means that the user:

  • Prefers US English
  • Understands 80% of generic English
  • Understands 20% of generic Spanish

Because the user submits this information of every request, you can just parse it out with a regex and pull out the locale names.

Regex to parse http_accept_language

You can probably get away with assuming that the language they submit first is the one they want most.

LocaleSetter will give you back an ordered list of locales.

LocaleSetter produces an ordered list of preferred locales

Once you know what language the user wants, you can query the i18n library to see what locales are available.

Querying i18n for available locales

Then the challenge is to match the user’s preferences to the available locales. You find the locales they want and convert them to symbols and compare them to the supported locales (because they’re stored as an array of symbols), and then find the first match in order of their preference (not yours).

This seems like a simple problem. This matcher module takes in requested as an array of the symbols the user has asked for, and matches that against your available locales.

Matching requested and available locales

The single ampersand operator finds the intersection of two sets – the elements that are common to both – and orders them by their position in the first set.

In this example requested will take order priority over i18n.available_locales.

3. URL parameters  can also be used to determine which locale to serve.

Locale parameters in the URL

What’s good about using url parameters is that it makes it really easy to switch locales for development and debugging.

It’s easy to pull the locale parameter out of the url with a simple:
before_filter :set_locale to get back the params hash and look for the locale key. If it’s nil, we leave the current locale unchanged.

Getting the locale from the url

There is a danger here. You can query the symbol module for all the symbols currently defined on a system – and you’ll get back a very large array of hundreds, maybe thousands of symbols.

If you set the locale to some string that the user passed in:
garbage_from_a_user, and then query for it, you will find that it is now set as a symbol.

Dangers of user submitted strings in urls

The danger is that there’s now one more symbol in the symbol table than there used to be. This opens you up to a denial of service attack because it allows users to spam your urls, generate symbols, fill your symbol table and crash your application.

A safer option is to match more intelligently by pulling out the params locale and sending it to the params module which triggers the matcher.

LocaleSetter matcher

Things get interesting in the matcher where instead of symbolising the input and comparing it to the available locales, we stringify the available locales.

Here in the available method, we map to strings which is safer because strings are garbage collected – and they’re our own strings anyway.

We map those strings and compare them to the requested strings.

Mapping requested strings to available strings

Isn’t it wasteful to create these strings?

Yes, but Rails creates thousands of strings – something like 16,000 objects on a Hello World request. So the strings we’re creating here are statistically insignificant – don’t worry about it.

There’s another problem with these locale-embedded links. You embed the locale in the url, click any link on the page and now your the locale is lost to subsequent requests.

Thankfully, Rails provides an easy way to handle this, you just need the default_url_options method.

default_url_options

This allows you to manipulate any url generated from a path helper. If you are a good citizen of your view templates, and you’re only ever using path helpers and never writing your own a tags, path helpers will automatically embed the locale in every url.

But, you probably have a primary locale which serves the majority of your users – so why bother embedding the locale in this default locale? Here, we don’t embed the locale if we’re using the default locale.

4. User preference  You might store locale in the user table and the locale_setter gem checks whether the current user has a preferred locale.

If they have, the gem will pull it out and attempt to match it against the available locales with the matcher module.

Selecting locale from user preferences

Putting it all together. How do we prioritise these locale selection options? Here’s my recommended look-up chain:

  1. URL parameter – this allows you to override your own user preference, despite being logged in, and manipulate the url to control your locale
  2. User preference – If users are logged-in and they haven’t monkeyed with the url, go ahead and use their stored preference.
  3. Browser settings
  4. Default locale
  5. No geolocation

The most interesting part here is the set_locale method which prioritises the params, then the user, then the http and then the default locale.

LocaleSetter set_locale
LocaleSetter set_locale

4. i18n also makes white boxing (white labeling) easy.

Here’s a strategy you could use to get one application serving two different clients

At the top of your application layout, you’ll have lines like this that specify what javascript and stylesheet to use. Those are just strings and all internationalization and localization do is output strings.

White boxing / labeling an app with i18n

You can use the t helper here and make locales per client and set stylesheet and javascript files per client / locale / subdomain, for example.

White boxing / labeling an app with i18n

White boxing / labeling an app with i18n

5. Summary

  • Step 1: Internationalize – carve up your code, pull out the magic data, pull out the strings and get them into lookup dictionaries.
  • Step 2: Localise your user content. Don’t do this on you’re own, you’ll hate life. Just use Locale, its awesome.
  • Step 3: Determine the locale and try out my gem
  • Step 4: Hack i18n beyond translations – anything that has to do with strings, you can do with i18n


6. Q and A

Q: Is there anything which will test localization files to ensure that all lookups are defined for all languages?

A: No, but by default the i18n gem will output the name of the key if the translation is missing. This is good because the app doesn’t crash but it’s bad because users see garbage. You can overrride that exception handler, and I recommend you do so – particularly at the CI level. Run CI through each of your locales and set your exception handler to raise an exception for missing translations. You will catch issues that happen because of translations.

Q: Is there a way to mount a locale using a url holder pattern like /locale/stuff?

A: Yes, it is possible but it might cause some issues. I’m not too wild about using subdomains either.

Q: How do you decide when to keep something in a model as a constant, as opposed to keeping it as a setting?

A: In my opinion it’s always wrong to keep it in the model. From a code quality standpoint it is always wrong to have numbers in the middle of your functional code – because that number has a domain meaning and it should have a name. It shouldn’t be 6. it should be front_page.articles_limit. The only reason we put that 6 there is because we’re lazy. You write that code once, but you read it and debug it many times. You’re better off typing the long thing. So, the first step is to go through code and move the integers up to constants. Once you see all the constants, you realise that if you wanted to change them you could change the source code or monkey patch it with an initialiser. That could work, but if you’re going to write an initialiser you might as well just write an initialiser. So, take those constants, bring them over to an initialiser and handle configuration data there.

Q: Whether you’re changing a constant in the model or data in the initialiser, you’re still making code changes?

A: Yes, same repository, but different conceptual area. We’re changing the parts of the code that change when configuration changes, but we leave the models alone which reflect business logic.

Posted in LocaleApp | Tagged , , , , , , , , | 9 Comments

New Translation Workflow Feature

Yesterday we launched an update for our most requested feature – translation workflow.

Now, when viewing content inside Locale, each key has a status icon displayed next to it – complete or incomplete.

Translation workflow

Locale automatically updates this status when content is translated, but developers and translators can manually toggle the status of a key to indicate that something needs doing.

By default, empty translations have a status of incomplete and get completed when content is entered.  We’ve updated this logic so that deleting the content also sets the translation to incomplete.

Translations can also be marked as incomplete manually.  You might want to do this if you’ve made a change in the default locale and want to signal to translators that they should re-evaluate the content.

Translators might want to do this themselves to indicate that their changes are ready to be reviewed.

We hope you enjoy this new translation workflow feature.

Please give us your feedback and keep on telling us what you need so that we can prioritise our ongoing development of Locale.

Posted in LocaleApp | Tagged , , , , | Leave a comment

Tigerlily’s Secret Social Sauce

TigerlilyThe Tigerlily platform is a powerful social marketing solution.  It allows companies to manage their engagement strategies.

Tigerlily is used across the world to manage conversations and run effective campaigns.

Still in startup mode, the Tigerlily team are the independent young guns in a market already crowded with corporate incumbents like Buddy Media, now owned by Salesforce, and Wildfire, now owned by Google.

Why do Tigerlily think they can take on this established market and win? How do they differentiate themselves?

Damien FischettiDamien Fischetti, Digital Marketing Manager at Tigerlily explains:

As a startup, being agile is one of our key differentiators. We can afford to seek out and pursue new opportunities that older, more established companies might overlook. Targeting specific foreign market segments is one area where Tigerlily is taking the lead.

In order to reach new markets, Tigerlily needs to speak their languages – and they’re using Locale to make that happen.

Damien continues:

We’re developing a new site for Tigerlily incorporating a number of new languages. As a marketer, it’s my job to write the original copy and get it professionally translated. Using Locale to achieve these tasks means we save a lot of development time, streamline the process and get to market more quickly.

As social communication increasingly tends towards hyper-local, Damien believes that their ability to localize quickly and accurately is a major competitive advantage – one that will increase with time.

Now that Locale enables localization based on language and region, Tigerlily can easily and independently target markets in Brazilian Portuguese and Portuguese Portuguese, for example.

If you’re looking for a platform to help maximise your social marketing strategy, check out Tigerlily.

If you’re looking to quickly and authentically communicate in new markets, check out Locale.

Posted in LocaleApp | Tagged , , , , , , , | Leave a comment

45,264 Locales

language and region We just upgraded how locales are defined.

Instead of the 90 or so predefined locales we had previously, we now support 184 languages and 246 regions.

A locale is a combination of a language and a region, and you can now combine them in whatever way you need for your projects.

We’re pretty sure that with 45,264 potential combinations, everyone will be able to create the locales they need!

Existing locales for existing projects haven’t changed at all.  This new method of specifying locales is available when starting a new project or adding a new locale to an existing one.

For ISO standard lovers ..

Locales are now made of an ISO 639-1 language code and an optional ISO 3166-1 alpha-2 region code. Combined, we call this a BCP47 tag.

Language and region definitions now take place in the bcp47 gem, so check that out if you feel like contributing.

Posted in LocaleApp | Tagged , , , , , | Leave a comment

Ubilabs: Displaying store locations in your language

UbilocalUbilabs, the company behind Ubilocal, specialise in Google Map applications.

Their clients include Swiss Post, Deutsche Telekom and the German social business network XING.

Over the years, one particular client request kept cropping up again and again .. “We want an easy way of mapping our store and branch locations.”

Last year, Ubilabs decided that it was the right time to develop a SaaS solution to address this market. Their goals were to make it super easy and fast to use, while being versatile enough to satisfy different use cases.

The solution: A recently launched Rails app called Ubilocal. It literally takes only minutes from sign-up to creating a store locator.

Product Manager, Michael Pletziger commented:

Locale really helped us develop Ubilocal as a multilingual web application. We tried different tools and methods to get control of our YAML files, but everything else was very frustrating to work with.  Locale is the first tool that actually worked – and is accepted both by developers and translators. It’s really fast too.

Available initially in English and German, more languages will be added to Ubilocal as the product and market expand – something that Locale makes very easy.

If you want a simple application to map your store of branch locations, check out Ubilocal.

If you’re tired of managing your Rails language files manually, check out Locale.

Posted in LocaleApp | Leave a comment

Tutor With Me – localization and faster development

TutorWithMeTutor With Me is a Rails platform connecting teachers, tutors, and trainers with students.

It facilitates the connection between those who want to learn and those who teach.

tutor profile

Tutors can publicise their lesson plans, set their own rates and connect with students through a virtual classroom environment.


Students
can find a tutor that fits their needs and budget, and can study from anywhere.

Based in Montreal, Canada, Tutor With Me was founded in 2011, and conceived as a multi-lingual platform from the start.

However, localization quickly became difficult to manage – particularly in the early stages of app development when localization significantly slowed progress.

In early 2012, the Tutor With Me development team began to use Locale to solve the problems associated with multi-language support and content management.

Andrew Gardener, Tutor With Me co-founder & director of development explains:

Locale is a powerful tool that made it possible for us to organize and manage content more efficiently. Since we plan to offer our services in multiple languages in the near future, Locale makes the whole process easier, and allows our translators to work in an user-friendly environment.

If you’d like to offer online tutoring at your own rates or need help with a certain topic, check out Tutor With Me.

If localization is hampering your Rails development progress, check out Locale.

Posted in LocaleApp | Leave a comment