Need help upgrading from Indico 0.99

[tak0222]

Need help upgrading from Indico 0.99

Hello,

My name is Takashi Ichihara and I am the Indico admin at RIKEN.
We are running Indico version 0.99 at https://indico2.riken.jp/indico/
on Scientific Linux 6.2 (CentOS 6.2).
We would like to upgrade to the latest version of indigo.

On our test server of ContOS 7.4, I tried to install indigo 1.2.2
following to the last paragraph (let’s try installing it from Git:) of

Need help upgrading from Indico 0.98

TL;DR: Working commands are down there at “So, let’s try installing it from Git”. Looks like easy_install broke when PyPI switched to SSL-only (it’s a legacy tool that usually nobody should be using anymore). I just played around a bit in a CentOS6 docker container and the Python packaging infrastructure seems to be utterly broken on 2.6 - but this makes sense, since Python 2.6 has reached EOL quite some time ago, the core utilities (like setuptools and pip) are dropping support for it, and th…

However, the last command
“python setup.py develop_config”
failed with following errors:

Full path [/root]: /opt/indico
Creating directories... Done!
error: indico/locale/es_ES/LC_MESSAGES/messages.po:28: placeholders are incompatible
error: indico/locale/es_ES/LC_MESSAGES/messages.po:1098: placeholders are incompatible
error: indico/locale/es_ES/LC_MESSAGES/messages.po:1350: placeholders are incompatible
error: indico/locale/es_ES/LC_MESSAGES/messages.po:1477: positional format placeholders are unbalanced
error: indico/locale/es_ES/LC_MESSAGES/messages.po:2316: placeholders are incompatible
 :
error: indico/locale/es_ES/LC_MESSAGES/messages.po:26465: incompatible format for placeholder 1: 's' and 'u' are not compatible
error: indico/locale/fr_FR/LC_MESSAGES/messages.po:18284: incompatible format for placeholder 1: 'c' and 'e' are not compatible
error: indico/locale/fr_FR/LC_MESSAGES/messages.po:24728: placeholders are incompatible
error: indico/locale/fr_FR/LC_MESSAGES/messages.po:24743: placeholders are incompatible
error: invalid command 'compile_catalog_js'
(indico-env)[root@m indico]# 

Is there any suggestion to fix this problem?

[pferreir]

@tak0222

error: invalid command ‘compile_catalog_js’

Weird. Are you sure you’ve checked out v1.2.2?

[tak0222]

Yes, I did. (From the log file):

(indico-env)[root@m ~]# cd indico
(indico-env)[root@m indico]# git checkout v1.2.2
Note: checking out 'v1.2.2'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b new_branch_name

HEAD is now at b34329d... Bump version to 1.2.2
(indico-env)[root@m indico]# sed -i -E "s/cssmin==.*/cssmin/" requirements.txt
(indico-env)[root@m indico]# python setup.py develop_config
Indico 1.2.2
running develop_config
running egg_info
creating indico.egg-info

[tak0222]

Hi,
I temporary put this log at the URL:
https://ribf.riken.jp/~ttest/misclog/20180201-indico122-CentOS74.txt

[ThiefMaster]

OK, this happens when babel is not installed. Please run pip install -r requirements.txt before python setup.py install. I’ve also updated the other post. I guess when testing it I had it already installed from a previous attempt.

[tak0222]

Thank you. I did it. But, It seems to have problems…
https://ribf.riken.jp/~ttest/misclog/20180201-indico122-CentOS74-2nd.txt

[ThiefMaster]

Looks OK to me. The placeholder errors can be ignored. Since it’s a dev setup, the config file is taken from etc/ within the source folder (ie the one from where you were running python setup.py ...

[tak0222]

Thank you again.

Although the destination folders (/opt/indico/) are empty,
I noticed everything is stored at the source folder as you pointed out.

I will continue.

[tak0222]

Hi,
The migration of indico DB from 0.99 to 1.2 seems to be successful.
It took 15 minutes. I will prepare new indico 2.0 server and migrate the DB.

@tak0222

(indico-env)[root@r indico]# zdaemon -C ./etc/zdctl.conf start
. daemon process started, pid=19226

(indico-env)[root@r indico]# python ./bin/migration/migrate.py --prev-version 0.99
/tmp/indico-env/lib/python2.7/site-packages/pkg_resources/init.py:193: RuntimeWarning: You have iterated over the result of pkg_resources.parse_version. This is a legacy behavior which is inconsistent with the new version class introduced in setuptools 8.0. In most cases, conversion to a tuple is unnecessary. For comparison of versions, sort the Version instances directly. If you have another use case requiring the tuple, please file a bug with the setuptools project describing that need.
stacklevel=1,

This script will migrate your Indico DB to a new version. We recommend that
this operation be executed while the web server is down, in order to avoid
concurrency problems and DB conflicts.

Are you sure you want to execute the migration now? [y/N] y

Executing migration...

Probing DB connection... DONE!

# Reloading all plugins (0.97)
Traceback (most recent call last):
  File "/usr/lib64/python2.7/logging/handlers.py", line 929, in emit
    smtp = smtplib.SMTP(self.mailhost, port, timeout=self._timeout)
  File "/usr/lib64/python2.7/smtplib.py", line 255, in __init__
    (code, msg) = self.connect(host, port)
  File "/usr/lib64/python2.7/smtplib.py", line 315, in connect
    self.sock = self._get_socket(host, port, self.timeout)
  File "/usr/lib64/python2.7/smtplib.py", line 290, in _get_socket
    return socket.create_connection((host, port), timeout)
  File "/usr/lib64/python2.7/socket.py", line 571, in create_connection
    raise err
error: [Errno 111] Connection refused
Logged from file bot.py, line 25
Traceback (most recent call last):
  File "/usr/lib64/python2.7/logging/handlers.py", line 929, in emit
    smtp = smtplib.SMTP(self.mailhost, port, timeout=self._timeout)
  File "/usr/lib64/python2.7/smtplib.py", line 255, in __init__
    (code, msg) = self.connect(host, port)
  File "/usr/lib64/python2.7/smtplib.py", line 315, in connect
    self.sock = self._get_socket(host, port, self.timeout)
  File "/usr/lib64/python2.7/smtplib.py", line 290, in _get_socket
    return socket.create_connection((host, port), timeout)
  File "/usr/lib64/python2.7/socket.py", line 571, in create_connection
    raise err
error: [Errno 111] Connection refused
Logged from file bot.py, line 25
Traceback (most recent call last):
  File "/usr/lib64/python2.7/logging/handlers.py", line 929, in emit
    smtp = smtplib.SMTP(self.mailhost, port, timeout=self._timeout)
  File "/usr/lib64/python2.7/smtplib.py", line 255, in __init__
    (code, msg) = self.connect(host, port)
  File "/usr/lib64/python2.7/smtplib.py", line 315, in connect
    self.sock = self._get_socket(host, port, self.timeout)
  File "/usr/lib64/python2.7/smtplib.py", line 290, in _get_socket
    return socket.create_connection((host, port), timeout)
  File "/usr/lib64/python2.7/socket.py", line 571, in create_connection
    raise err
error: [Errno 111] Connection refused
Logged from file bot.py, line 25
Traceback (most recent call last):
  File "/usr/lib64/python2.7/logging/handlers.py", line 929, in emit
    smtp = smtplib.SMTP(self.mailhost, port, timeout=self._timeout)
  File "/usr/lib64/python2.7/smtplib.py", line 255, in __init__
    (code, msg) = self.connect(host, port)
  File "/usr/lib64/python2.7/smtplib.py", line 315, in connect
    self.sock = self._get_socket(host, port, self.timeout)
  File "/usr/lib64/python2.7/smtplib.py", line 290, in _get_socket
    return socket.create_connection((host, port), timeout)
  File "/usr/lib64/python2.7/socket.py", line 571, in create_connection
    raise err
error: [Errno 111] Connection refused
Logged from file bot.py, line 25
Traceback (most recent call last):
  File "/usr/lib64/python2.7/logging/handlers.py", line 929, in emit
    smtp = smtplib.SMTP(self.mailhost, port, timeout=self._timeout)
  File "/usr/lib64/python2.7/smtplib.py", line 255, in __init__
    (code, msg) = self.connect(host, port)
  File "/usr/lib64/python2.7/smtplib.py", line 315, in connect
    self.sock = self._get_socket(host, port, self.timeout)
  File "/usr/lib64/python2.7/smtplib.py", line 290, in _get_socket
    return socket.create_connection((host, port), timeout)
  File "/usr/lib64/python2.7/socket.py", line 571, in create_connection
    raise err
error: [Errno 111] Connection refused
Logged from file bot.py, line 25
Traceback (most recent call last):
  File "/usr/lib64/python2.7/logging/handlers.py", line 929, in emit
    smtp = smtplib.SMTP(self.mailhost, port, timeout=self._timeout)
  File "/usr/lib64/python2.7/smtplib.py", line 255, in __init__
    (code, msg) = self.connect(host, port)
  File "/usr/lib64/python2.7/smtplib.py", line 315, in connect
    self.sock = self._get_socket(host, port, self.timeout)
  File "/usr/lib64/python2.7/smtplib.py", line 290, in _get_socket
    return socket.create_connection((host, port), timeout)
  File "/usr/lib64/python2.7/socket.py", line 571, in create_connection
    raise err
error: [Errno 111] Connection refused
Logged from file bot.py, line 25
Traceback (most recent call last):
  File "/usr/lib64/python2.7/logging/handlers.py", line 929, in emit
    smtp = smtplib.SMTP(self.mailhost, port, timeout=self._timeout)
  File "/usr/lib64/python2.7/smtplib.py", line 255, in __init__
    (code, msg) = self.connect(host, port)
  File "/usr/lib64/python2.7/smtplib.py", line 315, in connect
    self.sock = self._get_socket(host, port, self.timeout)
  File "/usr/lib64/python2.7/smtplib.py", line 290, in _get_socket
    return socket.create_connection((host, port), timeout)
  File "/usr/lib64/python2.7/socket.py", line 571, in create_connection
    raise err
error: [Errno 111] Connection refused
Logged from file bot.py, line 25
Traceback (most recent call last):
  File "/usr/lib64/python2.7/logging/handlers.py", line 929, in emit
    smtp = smtplib.SMTP(self.mailhost, port, timeout=self._timeout)
  File "/usr/lib64/python2.7/smtplib.py", line 255, in __init__
    (code, msg) = self.connect(host, port)
  File "/usr/lib64/python2.7/smtplib.py", line 315, in connect
    self.sock = self._get_socket(host, port, self.timeout)
  File "/usr/lib64/python2.7/smtplib.py", line 290, in _get_socket
    return socket.create_connection((host, port), timeout)
  File "/usr/lib64/python2.7/socket.py", line 571, in create_connection
    raise err
error: [Errno 111] Connection refused
Logged from file bot.py, line 25
  DONE

# Initializing/updating index catalog (0.98b)
  DONE

# Moving from OAI Private Harvesting to a more general IP-based ACL. (1.0)
  DONE

# Removing old CSS Templates from events (1.0)
  DONE

# Tasks: 1. Moving support info fields from conference to a dedicated class 2. Update non inherited children list 3. Update Vidyo indexes (1.0)
  DONE2512 99.880573%] 995 RIKEN VTX Software Meeting                            

# Changing Vidyo Room Names (1.0)
  DONE

# Indexing Conference Title (1.1)
  DONE2512 99.880573%] 995 RIKEN VTX Software Meeting                            

# Convert Avatar.linkedTo structure to use OOTreeSets and import linkedTo information into Redis (if enabled) (1.1)
Note: Some links might point to broken objects which will be skipped automatically.
  Done   
  DONE

# Import linkedTo information into Redis (1.1)
  Redis not configured, skipping
  DONE

# Add Category Suggestion Task to scheduler (Redis needed) (1.2)
  Redis not configured, skipping
  DONE

# Tasks: 1. Removing Video Services from core 2. Migrates old AbstractField to new AbstractField subclasses 3. Add download e-ticket PDF link to the menu (1.2)
  DONE2512 99.880573%] 995 RIKEN VTX Software Meeting                            

# Generate the new password with a salt (1.2)
  Processing 1711/1711 
  DONE

# Remove the NiceIdentities from the avatars (1.2)
  DONE

# Convert all LDAP identities to lowercase (1.2)


  DONE

# Makes sure that all the secondary emails are lower case (otherwise it would be difficult to use indexes) (1.2)
  DONE

# Fix indexes encoding. They may be in unicode and they have to be encoded in utf-8 (1.2)
  DONE

Database Migration successful!

(indico-env)[root@ri indico]#

[tak0222]

Hi,
Migration from indico 0.99 to indico 2.0 seems to be successful.
Thank you very much for your help.

[tak0222]

Hi,
We noticed one problem in the migration from indigo 0.99 to 2.0.

In Indico 2.0, the following URL request of indigo 0.99 style
https://hostname/conferenceDisplay.py?confId=1805
is redirected to
https://hostname/event/1805/
This is OK.

However, our top page of the indigo 0.99 was
https://hostname/indico/

So, in Indico 2.0 (+apache), the requested URL of
https://hostname/indico/conferenceDisplay.py?confId=1805
is not redirected to
https://hostname/event/1805/.
and results
Not Found (The page you are looking for doesn’t exist.)

In Indico 2.0 (+apache), is there any apache configuration to redirect the request
https://hostname/indico/conferenceDisplay.py?confId=1805
to
https://hostname/event/1805/
(ignore the first “indico/” string in the requested URL)

Than you for any advice.

[bpedersen2]

The following trwo rewriterules should work (assuming the host changed as well),

RewriteRule event/(.*)$ https://<hostname>/event/$1 [R=permanent,L]
RewriteRule indico/(.*)$ https://<hostname>/$1 [R=permanent,L]

With the host unchanged:

RewriteRule indico/(.*)$ /$1 [R=permanent,L]

should do the trick.

[tak0222]

Thank you very much. This works for us.

RewriteRule indico/(.*)$ /$1 [R=permanent,L]

[tak0222]

Hi,
After updating indico from 2.0 to 2.0.1, can I do the process

indico-migrate postgresql:///indico file:///opt/indico-legacy/db/Data.fs …

again from new /db/Data.fs of Indico 1.2.2 on indico 2.0.1
after cleaning the postgresql database of indico 2.0.1 ?

[ThiefMaster]

If you already have 2.0 running, you do not need to migrate again - you can simply follow the upgrade instructions to update 2.0 to 2.0.1 (same for any future updates).

If you do want to migrate your ZODB again for some reason, then yes, deleting the postgres database, recreating it with the extensios, and running indico-migrate again would work.

[tak0222]

Thanks. Since our indico 0.99 is still running, we will decide the day of the migration
(from 0.99 to 2.0) and migrate again from ZODB.

[tak0222]

Hi,
After the (test) migration from indico 0.99 to 2.0, I noticed that all of the registration page (registration form) in indico 2.0 are empty, even if the registration is in open status.

Therefore, I created a test conference page, and created a test Registration Form and then tried to configure the Registration Form. However, there is no items in the Configure questions in form “test of registration” page as attached.
Where is it wrong? Thank you for any suggestion.

[kolodzie]

We have noticed problems like this one before and as far as I remember the problem was in the Apache
config, namely the RewriteRules. Could you please share the relevant part of your Apache config?

[ThiefMaster]

This is usually caused by Indico being accessed by a URL that does not match BASE_URL in indico.conf - that results in the AJAX requests loading the templates for the registration form to fail due to Same-Origin-Policy violations (you can probably see messages about that in your browser’s developer console).

The solution for this is to use the correct url in indico.conf and then reload/restart uwsgi so the updated config is actually used.

[tak0222]

Thanks.
Concerning to the apache configuration, I almost exactly follows the page:
https://docs.getindico.io/en/stable/installation/production/centos/apache/#centos-apache-web

In /opt/indico/etc/indico.conf

BASE_URL = 'https://indico3.riken.jp'

and the requested url was

https://indico3.riken.jp/event/2728/registrations/86/

By the way, when I click “configure” the Registration Form, apache logs say,
(404 176)

/opt/indico/log/apache/access.log
134.160.x.x - - [23/Feb/2018:17:14:37 +0900] "GET /event/2728/manage/registration/86/form/ HTTP/1.1" 200 49606 "https://indico3.riken.jp/event/2728/manage/registration/86/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) Gecko/20100101 Firefox/52.0"
134.160.x.x - - [23/Feb/2018:17:14:37 +0900] "GET /assets/js-vars/user.js HTTP/1.1" 200 90 "https://indico3.riken.jp/event/2728/manage/registration/86/form/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) Gecko/20100101 Firefox/52.0"
134.160.x.x - - [23/Feb/2018:17:14:38 +0900] "GET /modules/registration/form/tpls/registrationform.tpl.html HTTP/1.1" 404 176 "https://indico3.riken.jp/event/2728/manage/registration/86/form/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) Gecko/20100101 Firefox/52.0"

/opt/indico/log/apache/error.log
(no new entry)

I temporary opened 443/tcp port of this test server to any.