Deployment process from DEV to PROD VM

(.venv) [indico@centos ~]$ pip uninstall -y indico
Can't uninstall 'indico'. No files were found to uninstall.
(.venv) [indico@centos ~]$ 

Then just try installing the wheel…

 [indico@centos ~]$ pip install indico-2.2.dev0+20190423.1332.df161ea-py2-none-any.whl 
Requirement already satisfied: indico==2.2.dev0+20190423.1332.df161ea from file:///opt/indico/indico-2.2.dev0%2B20190423.1332.df161ea-py2-none-any.whl in ./.venv/lib/python2.7/site-packages
(.venv) [indico@centos ~]$ 

yesterday i got LOADS more output and ity took minutes, not less than a second LOL

Just trying to figure this all out – assuming a wheel just installs to a folder structure, could it be that the folder structure on the dev machine and the prod machine are different so something has happened on the original pip install that has messed things up or am I talking rubbish? – it is just the setup of VM is quite different between Prod and DEV

The wheel is basically a zip file containing the python files - but unless there’s something seriously wrong in your changes (that fail in some what not causing an exception), or you changed something vital in setup.py, I don’t see how you end up with the problems you have.

If you don’t mind sharing your code, maybe you could attach the wheel here so we can have a look if there’s anything obviously wrong with it?

The output of ls -al ~/.venv/lib/python2.7/site-packages/indico/ and ls -dal ~/.venv/lib/python2.7/site-packages/indico* could be interesting as well.

ok - that would be great - thanks

fist the ls’s and then the 40MB file in second post

(.venv) [indico@centos ~]$ 
(.venv) [indico@centos ~]$ ls -al ~/.venv/lib/python2.7/site-packages/indico/
total 40
drwxr-xr-x.  11 indico apache   200 Apr 24 11:54 .
drwxr-xr-x. 246 indico apache 12288 Apr 25 10:34 ..
drwxr-xr-x.   2 indico apache  4096 Apr 24 11:53 cli
drwxr-xr-x.   8 indico apache  4096 Apr 24 11:53 core
-rw-r--r--.   1 indico apache     0 Apr 24 11:52 __init__.py
-rw-r-----.   1 indico apache   151 Apr 24 11:54 __init__.pyc
drwxr-xr-x.   7 indico apache   130 Apr 24 11:53 legacy
-rw-r--r--.   1 indico apache     0 Apr 24 11:52 logging.yaml.sample
drwxr-xr-x.   3 indico apache    92 Apr 24 11:53 migrations
drwxr-xr-x.  22 indico apache  4096 Apr 24 11:53 modules
drwxr-xr-x.   3 indico apache   141 Apr 24 11:53 testing
drwxr-xr-x.   5 indico apache   114 Apr 24 11:53 translations
drwxr-xr-x.   5 indico apache  4096 Apr 24 11:53 util
drwxr-xr-x.   9 indico apache  4096 Apr 24 11:54 web
(.venv) [indico@centos ~]$ 
(.venv) [indico@centos ~]$ ls -dal ~/.venv/lib/python2.7/site-packages/indico/
drwxr-xr-x. 11 indico apache 200 Apr 24 11:54 /opt/indico/.venv/lib/python2.7/site-packages/indico/
(.venv) [indico@centos ~]$ 
(.venv) [indico@centos ~]$ 

will not let me attach whl to message - email address???

Try again, I just added whl to the allowed extensions.

haha - it is now too big - it is 40 MB (zipped 34) - max allowed is 4.

raised to 50MB -> try now

my files is 40,581 KB - but it still says it is too big - will give it another go. - a 7z of it is 34,453 kb

nope - gets to about 87% and fails - allow a zip?

ok just use something like wetransfer and post the link or try emailing it to me. not sure if the cern mail server allows attachments that big, but my private one should be fine with it -> adrian@planetcoding.net

cheers - sent a zip to email - hope it gets there

Your wheel looks fine to me. I don’t see why it would cause the weird issues you are having.

PS: Some unrelated thing I noticed while looking at your wheel: there’s no need to change DEFAULTS entries in the code; those config options can (and should) be set directly in indico.conf instead.

You could try recreating the whole virtualenv:

deactivate
rm -rf ~/.venv
virtualenv ~/.venv
source ~/.venv/bin/activate
pip install -U pip setuptools
pip install /path/to/your/indico*.whl

ok - did that - is started to install, a lot of cashed files being used and requirements already satified, and then this…



ollecting Pillow==5.4.1 (from indico==2.2.dev0+20190423.1332.df161ea)
  Using cached https://files.pythonhosted.org/packages/0d/f3/421598450cb9503f4565d936860763b5af413a61009d87a5ab1e34139672/Pillow-5.4.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting PyYAML==5.1 (from indico==2.2.dev0+20190423.1332.df161ea)
ERROR: Exception:
Traceback (most recent call last):
  File "/opt/indico/.venv/lib/python2.7/site-packages/pip/_internal/cli/base_command.py", line 178, in main
    status = self.run(options, args)
  File "/opt/indico/.venv/lib/python2.7/site-packages/pip/_internal/commands/install.py", line 352, in run
    resolver.resolve(requirement_set)
  File "/opt/indico/.venv/lib/python2.7/site-packages/pip/_internal/resolve.py", line 131, in resolve
    self._resolve_one(requirement_set, req)
  File "/opt/indico/.venv/lib/python2.7/site-packages/pip/_internal/resolve.py", line 294, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/opt/indico/.venv/lib/python2.7/site-packages/pip/_internal/resolve.py", line 242, in _get_abstract_dist_for
    self.require_hashes
  File "/opt/indico/.venv/lib/python2.7/site-packages/pip/_internal/operations/prepare.py", line 353, in prepare_linked_requirement
    progress_bar=self.progress_bar
  File "/opt/indico/.venv/lib/python2.7/site-packages/pip/_internal/download.py", line 873, in unpack_url
    unpack_file_url(link, location, download_dir, hashes=hashes)
  File "/opt/indico/.venv/lib/python2.7/site-packages/pip/_internal/download.py", line 807, in unpack_file_url
    unpack_file(from_path, location, content_type, link)
  File "/opt/indico/.venv/lib/python2.7/site-packages/pip/_internal/utils/misc.py", line 628, in unpack_file
    flatten=not filename.endswith('.whl')
  File "/opt/indico/.venv/lib/python2.7/site-packages/pip/_internal/utils/misc.py", line 505, in unzip_file
    zip = zipfile.ZipFile(zipfp, allowZip64=True)
  File "/usr/lib64/python2.7/zipfile.py", line 766, in __init__
    self._RealGetContents()
  File "/usr/lib64/python2.7/zipfile.py", line 807, in _RealGetContents
    raise BadZipfile, "File is not a zip file"
BadZipfile: File is not a zip file

Is this of any help (lets hope so).

I never saw this error, but it sounds like the file it downloaded was corrupted.

Did you simply try running it again? If that doesn’t help, use rm -rf ~/.cache/pip to clear pip’s cache so it re-downloads everything.

ok
cleared out cashe - fine
ran complete setup again - seemed to install ok with no errors

end of script below

  Building wheel for scandir (setup.py) ... done
  Stored in directory: /opt/indico/.cache/pip/wheels/91/95/75/19c98a91239878abbc7c59970abd3b4e0438a7dd5b61778335
  Building wheel for infinity (setup.py) ... done
  Stored in directory: /opt/indico/.cache/pip/wheels/b9/7a/91/cf9903933d1788bd68870c9b3d3b114909eb574f7f8c4b9a68
  Building wheel for pycparser (setup.py) ... done
  Stored in directory: /opt/indico/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511
Successfully built Markdown alembic terminaltables blinker WTForms-Components SQLAlchemy pytest-localserver translitcodec pyatom Flask-BabelEx pyPdf html2text pojson zope.interface colorclass simplejson termcolor PyYAML Mako billiard node-semver simplegeneric validators intervals speaklater functools32 pyrsistent scandir infinity pycparser
Installing collected packages: Markdown, six, webencodings, html5lib, lxml, itsdangerous, click, MarkupSafe, Jinja2, Werkzeug, Flask, SQLAlchemy, Mako, python-editor, python-dateutil, alembic, atomicwrites, pluggy, attrs, scandir, pathlib2, py, more-itertools, funcsigs, pytest, terminaltables, dnspython, idna, email-validator, pytz, billiard, vine, amqp, kombu, celery, redis, psycopg2, urllib3, chardet, certifi, requests, oauthlib, requests-oauthlib, pynpm, node-semver, pywebpack, pyparsing, packaging, blinker, backports.shutil-get-terminal-size, simplegeneric, enum34, ipython-genutils, decorator, traitlets, pygments, ptyprocess, pexpect, wcwidth, prompt-toolkit, pickleshare, ipython, ipaddress, WTForms, Flask-WTF, bleach, validators, infinity, intervals, WTForms-Components, pytest-localserver, coverage, pytest-cov, marshmallow, flask-marshmallow, translitcodec, pycparser, cffi, bcrypt, freezegun, qrcode, Flask-PluginEngine, marshmallow-enum, indico-fonts, Babel, marshmallow-sqlalchemy, Flask-Multipass, pyatom, speaklater, Flask-BabelEx, pbr, mock, pytest-mock, pyPdf, functools32, pyrsistent, jsonschema, pycountry, html2text, simplejson, webargs, Flask-SQLAlchemy, Flask-Migrate, polib, argparse, pojson, hiredis, zope.interface, icalendar, Pillow, reportlab, colorclass, Flask-OAuthlib, flask-webpackext, contextlib2, xlsxwriter, termcolor, PyYAML, indico
Successfully installed Babel-2.6.0 Flask-0.12.4 Flask-BabelEx-0.9.3 Flask-Migrate-2.4.0 Flask-Multipass-0.2.2 Flask-OAuthlib-0.9.5 Flask-PluginEngine-0.3.1 Flask-SQLAlchemy-2.3.2 Flask-WTF-0.14.2 Jinja2-2.10 Mako-1.0.9 Markdown-2.3.1 MarkupSafe-1.1.1 Pillow-5.4.1 PyYAML-5.1 SQLAlchemy-1.3.1 WTForms-2.2.1 WTForms-Components-0.10.3 Werkzeug-0.14.1 alembic-1.0.8 amqp-2.4.2 argparse-1.4.0 atomicwrites-1.3.0 attrs-19.1.0 backports.shutil-get-terminal-size-1.0.0 bcrypt-3.1.6 billiard-3.5.0.5 bleach-3.1.0 blinker-1.4 celery-4.2.1 certifi-2019.3.9 cffi-1.12.3 chardet-3.0.4 click-7.0 colorclass-2.2.0 contextlib2-0.5.5 coverage-4.5.3 decorator-4.4.0 dnspython-1.16.0 email-validator-1.0.3 enum34-1.1.6 flask-marshmallow-0.10.0 flask-webpackext-1.0.0 freezegun-0.3.11 funcsigs-1.0.2 functools32-3.2.3.post2 hiredis-1.0.0 html2text-2018.1.9 html5lib-1.0.1 icalendar-4.0.3 idna-2.8 indico-2.2.dev0+20190423.1332.df161ea indico-fonts-1.1 infinity-1.4 intervals-0.8.1 ipaddress-1.0.22 ipython-5.8.0 ipython-genutils-0.2.0 itsdangerous-1.1.0 jsonschema-3.0.1 kombu-4.4.0 lxml-4.3.2 marshmallow-3.0.0rc4 marshmallow-enum-1.4.1 marshmallow-sqlalchemy-0.16.1 mock-2.0.0 more-itertools-5.0.0 node-semver-0.6.1 oauthlib-2.1.0 packaging-19.0 pathlib2-2.3.3 pbr-5.1.3 pexpect-4.7.0 pickleshare-0.7.5 pluggy-0.9.0 pojson-0.7 polib-1.1.0 prompt-toolkit-1.0.16 psycopg2-2.7.7 ptyprocess-0.6.0 py-1.8.0 pyPdf-1.13 pyatom-1.4 pycountry-18.12.8 pycparser-2.19 pygments-2.3.1 pynpm-0.1.1 pyparsing-2.4.0 pyrsistent-0.14.11 pytest-4.3.1 pytest-cov-2.6.1 pytest-localserver-0.5.0 pytest-mock-1.10.1 python-dateutil-2.8.0 python-editor-1.0.4 pytz-2019.1 pywebpack-1.0.0 qrcode-6.1 redis-3.2.1 reportlab-3.5.13 requests-2.21.0 requests-oauthlib-1.1.0 scandir-1.10.0 simplegeneric-0.8.1 simplejson-3.16.0 six-1.12.0 speaklater-1.3 termcolor-1.1.0 terminaltables-3.1.0 traitlets-4.3.2 translitcodec-0.4.0 urllib3-1.24.2 validators-0.12.5 vine-1.3.0 wcwidth-0.1.7 webargs-5.2.0 webencodings-0.5.1 xlsxwriter-1.1.5 zope.interface-3.8.0
(.venv) [indico@centos ~]$ 
(.venv) [indico@centos ~]$ 
(.venv) [indico@centos ~]$ 
(.venv) [indico@centos ~]$ 

didn’t / haven’t left env

tried chrome from centos desktop - still get internal server error!

will try logging off / on again and restarting celery.service - which didnt start last time as stated in errors earlier

Celery doesn’t really matter for this - except for periodic tasks and email sending everything in Indico works fine without celery (so while testing you can even leave it stopped).

Does indico shell show anything now?