Plugin translation, msg extraction from templates

Hi,

First, is there anything particular (anything written?) I should be aware of concerning setting up a plugin for translation?

Second, running each step in the translation process manually, I notice that extraction does not pick out the msgs marked with {% trans %} and {% endtrans %} in templates. I.e. a command such as the following:

$ indico i18n extract-messages    --input-paths {plugin-dir}  --sort-by-file --output-file {plugin-dir}/translations/messages.pot

will not pick out these msgs. Any ideas what I may be missing?

Third, the above command also gives me absolute location of each msg in the generated pot files, but I see in the stock indico pot file the location is relative e.g.

    #: indico/web/templates/footer.html:13
    #, python-format
    msgid "Powered by %(indico)s"
    msgstr ""

Abs path is obviously a problem, as it is installation dependent! How do I generate relative msg paths?

Thanks! Mario

indico i18n isn’t for plugins but only for the core. Check what we do in https://github.com/indico/indico-plugins/blob/master/manage-i18n.sh for our own plugins; should be easy to adapt.

BTW, the path in the pot files wouldn’t matter - it’s only informational and has no technical purpose.

Thanks, will look thru your manage-i18n.sh.

Concerning the path, yes, it is only context info. But, have the full path is just total diff noise! How do you generate the partial path only?

If you use the commands from the script it will already be the case. I think we simply use a relative path there.

OK, using the pybabel set of commands, and relative paths, corrects the path issue. The other issue – of msgids in templates not being picked up when updating – is a question of babel configuration (i.e. need to explicitly update the plugin babel.cfg, similarly but separately from the core’s babel.cfg).

Another Problem:

  • a plugin inherits a template from core e.g. header.html
  • in the inherited template (so, in the plugin) a new i18n msgid is marked e.g. {% trans %}In Plugin Inherited Template{% endtrans %}
  • the whole translation process is run for the plugin i.e. msgs properly extracted, pot and locale .po files properly updated, and the specific message above is then translated in say fr_FR, then the .po file compiled to .mo files, server restarted…
  • with fr_FR as selected locale, accessing pages containing this above msg will NOT show the french translation of the msg.
  • BUT now if you push the fr_FR translation of this msg to the core’s .po file for fr_FR, and recompile the .mo, restart the server… and redo previous step (access the same page in fr_FR), then the msg IS displayed in french.

Obvious questions:

  1. Why in inherited templates does babel not look for msg translations in the inheriting plugins?
  2. Is there any manageable way to tell babel to do exactly that?
  3. Or, to generalize further, why is there different translation spaces (core, per plugin) in the first place? Msgids do not come with a qualifier for this? Msgids are defined in a flat space… !
  4. Shouldn’t plugin translations then simply be appended/merged into a singular translation space for each application instance?
  5. Is there a way to tell indico to always get translations from ONE place? Specifically, a given plugin?
  6. Better still, is there a way to specify a cascade of translation catalogs that should ALWAYS be used by Indico, in the order specified and irrespective of the context Indico may think it is in?

Are you using the template override folder? If yes, I think we might not taking the plugin context into account there. Fixing that is likely tricky; I’ll have a look when I’m back in the office on monday (currently at EuroPython :snake:).

We’re using separate gettext domains for this. So no, the space isn’t fully flat.

No. Besides the obvious reason of plugins being loaded at runtime, and gettext catalogs being compiled at build time, merging translations could be problematic. Imagine a somewhat common word that has a different meaning in the context of a plugin. Ideally we’d use gettext context for that but at least the Jinja i18n system doesn’t support this. I think there were more reasons, but it’s been almost 5 years that we did this, so I don’t really remember them.

No, but being able to provide its own core translation sounds like an interesting feature, maybe similar to how template overrides work.

Thanks for the after hours reply :wink: Will appreciate any closer investigation when you get the opportunity; for now i’ll just like to address a couple aspects you brought up about the “big picture”.

  1. plugins are loaded at runtime – just like like everything else e.g. modules are loaded at runtime? Does not mean we modify them in any way dynamically at runtime (altho it is possible), we still prepare them before hand and then run our application… concerning plugins, all is prepared and decided beforehand, then the app config adjusted, then the app restarted – what is the problem to to include a merged build step for all i18n domains into one at this same step?

  2. A common word that has a different meaning in context of a plugin – first of all, how often does this actually happen? Then, what’s wrong with just differentiating it e.g. “Search” if used in context A could imply one thing and in context B could imply another, fine, then set the msgids to say “Search A” and “Search B”, and you have full control on the translation of each. As opposed to imposing a catalog domain for whatever Indico thinks is a different context, and not allowing to merge those contexts, and then not correctly determining the right context when requesting a translation?

Enjoy Basel and Europython :wink:

But the msgids are by definition the english string (since we don’t “translate” to english), so that’s not really an option unless changing what’s shown to the user is acceptable.

Not “translating” msgids to English is just a lazy convention – and given you use final display English as msgid! But there is no constraint about not translating any given msgid to English, should it be or whenever desired. And, between the possible occasional translation of a msgid to EN or always constraining a different catalog for each app and plugin (that is currently disfunctional), the choice seems very easy to me…

In any case, have patched our plugin’s handling of i18n catalogs such that all app/plugin catalogs are dynamically merged into one per language, and so translation lookup at runtime never fails on a msgid that has been translation in at least one app or any of the currently loaded plugin catalogs.