User Data Translations

Introduction

United Nations

UN being an international organization has a strong reason to have Indico pretty much fully translated to UN official languages.

  • Arabic, Chinese, English, French, Russian and Spanish.
  • Soon Indico.UN will be open for many UN regional duty stations in different parts of the world.

i18n Translation

We are already able to translate every static text of Indico i.e.

  • WTForm Labels, description
  • Labels of Buttons
  • Instructions of Human Activities such as tooltip (title)
  • Text in Links’ URL
  • Messages displayed to the end-user that are embedded in the application (validation messages, widgets’ confirmation, and other messages)
  • Text literals found in all types of expressions (including Unescaped Expressions and JavaScript)

Our requirement

  • Translated titles majorly Categories and Events
  • Translated list choices/options of the dropdown widgets such as countries or any other table, if directly being populated from the Database. (probably, this will be done on UN plugin)

Indico Core models -

Model columns
Category title, description
Event title, description, contact_title
RegistrationForm title, description
  • ARABIC: Page layout compliant with the script writing direction (Left to right or right to left) (CSS)

Its Complicated, is it?

We recognise that user data translation adds complexity, and we understand that you, as the Indico core team, are not yet ready to pursue this.

As a result, we tried something similar to TranslationHybrid and have a draft PR on our GitHub repository. We believe that simply adding a Mutable JSONB blob column and replacing the String column with hybrid_property adds no further complexity and is easily maintainable.

  • We don’t need to change any WTForm, Templates, etc.
  • We don’t need to change most of our sqlalchemy queries.
  • If the title is edited from the form, then the value is placed with current session’s language i.e key-value pair. (setter of TranslationHybrid)

Mystery begins

  • The query.options(load_only('title'))
    Most of the Indico Core’s queries on Category and Event involve load only, For example:
_category_query_options = (joinedload('children').load_only('id', 'title_translations', 'protection_mode'),
                               undefer('attachment_count'), undefer('has_events'))
_event_query_options = (joinedload('person_links'), joinedload('series'), undefer_group('series'),
                        joinedload('label'),
                        load_only('id', 'category_id', 'created_dt', 'start_dt', 'end_dt', 'timezone',
                                      'protection_mode', 'title_translations', 'type_', 'series_pos', 'series_count',
                                      'own_address', 'own_venue_id', 'own_venue_name', 'label_id', 'label_message',
                                      'visibility'))

  • Notice I changed title to title_translations
  • We are not yet able to think of a cleaner way to patch or change these queries to replace title with title_translations
  • if we rename the column title instead of title_translations then, all the other queries would fail.
  • Also, if we rename the hybrid property we will have to patch WTFORMS, controllers, and templates.

Open for discussions

We would be grateful, If the Indico.core team can assist us in finding a solution. We are open to discuss these and schedule a developers-only meeting to find a common solution.

Thanks in advance
Vasant Vohra
Indico.UN Team

As I had mentioned on IRC, doing this in the core in a fully transparent way would add a huge amount of complexity - not only in the backend but also in the frontend which would need to provide a way to add additional language versions for all the translated strings. I’m extremely hesitant about considering this as core functionality - in the end, the vast majority of events have one main language, and thus do not need translations of user-provided text.

But I do have an idea on how you could somewhat easily add this functionality without patching large amounts of core code:

  • Create a (database-backed) translation dictionary for the event which stores a source string, translated string and language code, and some UI to manage it. Since you know which parts you usually want translated, you can even have a button that populates the strings, or fully automate this whenever changes to relevant parts like the event title are detected.
  • Register a Jinja extension that applies a custom filter on all interpolated variables. You can check what we did in past Indico versions to make sure everything is unicode to get an idea how this is done. This custom filter would simply check if a given string is in the event’s dictionary and in that case use the translated version (or fall back to the original one if there is no translation).
  • Load the dictionary data early enough so you just need one query to load everything for the event; one of the early RH signals and a check whether you inherit from an event-related (or, for event titles, category-related) RH should do the job. Checking whether the RH has a self.event attribute could be an option too.

For translating category titles you could do the same, just on the category level. Or keep the model generic enough so you can use the same one in events and categories (event id and category id, with exactly one of them being populated - we do something similar e.g. for badge/poster templates).


By the way, since you mentioned the need for Indico to be translated to “Arabic, Chinese, English, French, Russian and Spanish”: The French translation is actively maintained by us (our section leader, who happens to be French, takes care of that); and English being the source language is always up to date as well. We have someone contribution to the Chinese translation (which is also included in the core), but currently there are no Arabic (probably particularly tricky as it’s RTL - there’s a thread in the i18n area of the forum on this topic) or Russian translations; and the Spanish one is rather outdated/incomplete. So contributions to those translations would be greatly appreciated. @plourenco is managing the translation teams (we use Transifex for it), so if you have someone willing to help with those translations please reach out to him in order to get access to the Indico project on Transifex!