Localization en_US issue in 2.3.3

Administrators
1

I’m not sure if we did something wrong in recent upgrade to 2.3.3, any input on solving the mystery is welcome.

After our standard upgrade from 2.3.2 to 2.3.3 some of our users complained that they cannot interact with the pages, buttons or menus did not do any action. For many other users things were working fine. After some debugging we found that for problematic users browser fails to load en_US.js and en_US-react.js files, but in successful cases the files that were loaded successfully were named en_GB.js and en_GB-react.js. Turns out that users that have en_US in their language settings all have problems now. As a workaround I updated database for those unlucky users and replaced en_US with en_GB. I tried to understand how en_US.js is served, but apparently it is something dynamically generated, I’m not sure I can follow how that request is handled.

Did anything related to localization changed in 2.3.3? (And I hope US was not banned from Indico entirely?)

2

en_US should never have been in the database because there was never such a locale. I think there was a bug at some point where the locale ended up being stored nonetheless when a user’s preferred locale was en_US.

When changing the logic that handles accesses to the JS locale files fail properly with a 404 when trying to retrieve invalid locale files (instead of caching a minimal valid locale file for any weird random locale), this unfortunately broke the JS for people with such a locale set in the account/session.

Here’s a script you can paste in indico shell to fix this:

from indico.util.i18n import get_all_locales


valid_locales = set(get_all_locales())

for locale in valid_locales:
    query = UserSetting.query.filter(UserSetting.module == 'users',
                                     UserSetting.name == 'lang',
                                     UserSetting.value[()].astext != locale,
                                     db.func.lower(UserSetting.value[()].astext) == locale.lower())
    count = query.count()
    print('Incorrect versions of {}: {}'.format(locale, count))
    if count:
        query.update({UserSetting.value: locale}, synchronize_session=False)


query = UserSetting.query.filter(UserSetting.module == 'users',
                                 UserSetting.name == 'lang',
                                 UserSetting.value[()].astext.notin_(valid_locales))
count = query.count()
if count:
    invalid_locales = {us.value for us in query}
    print('Invalid locales: {} ({})'.format(count, ', '.join(sorted(invalid_locales))))
    query.update({UserSetting.value: 'en_GB'}, synchronize_session=False)

db.session.commit()

Afterwards, force a logout for all users (since the bad locale may be stored in their session as well). The easiest way to do this is to purge the redis cache using redis-cli -n 1 flushdb

3

Thanks! How is locale for new users configured? We have ~20% of users with en_US and remaining with en_GB (mostly). Our indico.conf has DEFAULT_LOCALE = 'en_US', is it what determines it? Should not Indico complain about this wrong configuration?

4

Ah, that explains it. Yes, I think it would make sense to actively reject invalid locales at startup or log a warning and fallback to the default en_GB in that case.

5

Yes, I think that would be a good idea and would have prevented this. So to confirm, this “broke” for us with Do not cache JS i18n data for invalid locales by ThiefMaster · Pull Request #4766 · indico/indico · GitHub ?

6

yes, that’s the change

7

Excellent. Thanks for the incredibly quick help.