Issues creating 3.2.7 DB

Mark_Golder · August 22, 2023, 2:17pm

OK

I have a version of indico 3.1 up and running in production – no problem.

I have taken a copy of 3.2.7 codebase and trying to create a new version.

Starting from scratch, new folder structure etc. I get errors when trying to do a db prepare on the new DB.

My DB name is EMS4 and this is reflected in the indico.conf

(env) mark.golder@GL1269-MGOLDER Events_Management_System % indico setup wizard --dev
Enter the base directory where Indico will be installed.                                                                                                                                                                                                                                               
Indico root path: /Users/mark.golder/Indico_Local                                                                                                                                                                                                                                                      
Config file already exists. If you really want to run this wizard again,
delete
/Users/mark.golder/Indico_Local/Events_Management_System/indico/indico.conf
Aborted!
(env) mark.golder@GL1269-MGOLDER Events_Management_System % vi indico/indico.conf
(env) mark.golder@GL1269-MGOLDER Events_Management_System % indico db prepare
Traceback (most recent call last):
  File "/Users/mark.golder/Indico_Local/env/bin/indico", line 33, in <module>
    sys.exit(load_entry_point('indico', 'console_scripts', 'indico')())
  File "/Users/mark.golder/Indico_Local/env/lib/python3.9/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/Users/mark.golder/Indico_Local/env/lib/python3.9/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/Users/mark.golder/Indico_Local/env/lib/python3.9/site-packages/click/core.py", line 1686, in invoke
    sub_ctx = cmd.make_context(cmd_name, args, parent=ctx)
  File "/Users/mark.golder/Indico_Local/env/lib/python3.9/site-packages/click/core.py", line 943, in make_context
    self.parse_args(ctx, args)
  File "/Users/mark.golder/Indico_Local/env/lib/python3.9/site-packages/click/core.py", line 1644, in parse_args
    rest = super().parse_args(ctx, args)
  File "/Users/mark.golder/Indico_Local/env/lib/python3.9/site-packages/click/core.py", line 1404, in parse_args
    parser = self.make_parser(ctx)
  File "/Users/mark.golder/Indico_Local/env/lib/python3.9/site-packages/click/core.py", line 1315, in make_parser
    for param in self.get_params(ctx):
  File "/Users/mark.golder/Indico_Local/Events_Management_System/indico/cli/util.py", line 114, in get_params
    return self._impl.get_params(ctx)
  File "/Users/mark.golder/Indico_Local/env/lib/python3.9/site-packages/werkzeug/utils.py", line 106, in __get__
    value = self.fget(obj)  # type: ignore
  File "/Users/mark.golder/Indico_Local/Events_Management_System/indico/cli/util.py", line 99, in _impl
    return getattr(import_module(module), name)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 790, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/Users/mark.golder/Indico_Local/Events_Management_System/indico/cli/database.py", line 21, in <module>
    from indico.core.db.sqlalchemy.migration import PluginScriptDirectory, migrate, prepare_db
  File "/Users/mark.golder/Indico_Local/Events_Management_System/indico/core/db/sqlalchemy/migration.py", line 18, in <module>
    from indico.core.db.sqlalchemy.util.management import create_all_tables, get_all_tables
  File "/Users/mark.golder/Indico_Local/Events_Management_System/indico/core/db/sqlalchemy/util/management.py", line 12, in <module>
    from indico.core.db.sqlalchemy.protection import ProtectionMode
  File "/Users/mark.golder/Indico_Local/Events_Management_System/indico/core/db/sqlalchemy/protection.py", line 20, in <module>
    from indico.core.db.sqlalchemy.principals import EmailPrincipal, PrincipalType
  File "/Users/mark.golder/Indico_Local/Events_Management_System/indico/core/db/sqlalchemy/principals.py", line 20, in <module>
    class PrincipalType(IndicoIntEnum):
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/enum.py", line 215, in __new__
    enum_member = __new__(enum_class)
TypeError: object.__new__(PrincipalType) is not safe, use int.__new__()
(env) mark.golder@GL1269-MGOLDER Events_Management_System %

any ideas?

Mark

ThiefMaster · August 22, 2023, 2:22pm

Strange error, never seen it.

What’s the exact python 3.9.x version you’re using (python --version)?

Mark_Golder · August 22, 2023, 2:41pm

‘’‘’
(env) mark.golder@GL1269-MGOLDER Events_Management_System % python3 --version

Python 3.9.1rc1

(env) mark.golder@GL1269-MGOLDER Events_Management_System %
‘’’

Mark_Golder · August 22, 2023, 2:42pm

I can upgrade if you wish? it is a weird one I know, reading up on it I get python posts from 2.6/7 LOL

ThiefMaster · August 22, 2023, 2:44pm

Yeah some 3.9 release candidate isn’t a version I really want to bother testing with. Please try with the latest 3.9.x version.

Mark_Golder · August 22, 2023, 2:53pm

sure - no probs

ThiefMaster · August 22, 2023, 3:38pm

FWIW on the latest 3.9.17 in a fresh Docker container (running ubuntu) I was not able to reproduce your problem. So it’s either related to your old Python version or something weird with your macos Python…

Mark_Golder · August 24, 2023, 2:41pm

OK

I can create an empty DB using docker and 3.9.17 no problem, The code version is 3.2.7 tag from master. when I try this against a 3.1 DB which has all alembics added to it I get the following. It is a pain trying to dump / restore the DB from external docker to internal and ended up making a different names one (called EMS - original 'good one is called ‘indico’).
I will be something I am doing wrong, but any ideas - i.e. can I just dump 3.21 onto 3.2 DB, or use a 3,1 DB, with append alembics, against 3.2.7 without any additional manipulation?

sqlalchemy.exc.InternalError: (psycopg2.errors.InFailedSqlTransaction) current transaction is aborted, commands ignored until end of transaction block

[SQL: SELECT indico.settings.id AS indico_settings_id, indico.settings.module AS indico_settings_module, indico.settings.name AS indico_settings_name, indico.settings.value AS indico_settings_value 
FROM indico.settings]
(Background on this error at: https://sqlalche.me/e/14/2j85)

ThiefMaster · August 24, 2023, 2:44pm

Not sure I understand your question correctly, but if you want to run 3.2 on a 3.1 DB that won’t work - you need to run indico db upgrade to upgrade the DB to 3.2.

PS: You need to use 3 backticks (`) and not 3 single quotes (') when posting errors, code, etc. to get the proper display.

Mark_Golder · August 24, 2023, 2:51pm

sorry about quotes. I did do there indico db upgrade on the 3.2 database, which is external to docker, running from a 3.2.7 codebase, and it picked up the two alembic migrations I was missing, and the alembic DB table reflects this. My aim was to dump restore this onto the 3.2.7 database I had in my docker container, but it didn’t like that, and I ended up just restoring the external dump to a different names dv in docker. They both are up to date with all the appropriate alembic migrations. However, the docker container gives the error earlier when I point that to the newly created database I restored. I have an empty 3.2.x database in docker and that works fine (I created a new user etc. on startup, it is just this darn database dump to the new one!).
I will get there, I was just wondering ifs there was anything else to do apart from the alembic updates (indico db upgrade)
No worries, will keep at it

bpedersen2 · August 28, 2023, 10:23am

It would help if you give the complete steps you try to for your dump/restore.

What I would expect to work:
pg_dump --clean indico > dump.sql
on the old db

psql indico < dump.sql
on the new database.
After that run indico db upgrade in the docker container, just to be sure…