New plugin for S3 storage available

Recently we released a plugin indico-plugin-storage-s3 than adds an S3 storage backend to Indico.

We are using it ourselves on a production system with a Ceph-based S3-like storage, so we think it’s stable - but if someone wants to try it e.g. with the real Amazon S3 we’d love to get some feedback.

If you want to try it out, simply pip install indico-plugin-storage-s3, and add storage_s3 to PLUGINS in indico.conf. Then add an entry like this one to STORAGE_BACKENDS:

's3': 's3:bucket=my-indico-test-bucket'

Run indico s3 create_bucket if the bucket does not exist yet and you want Indico to create it.

Now all you need to do is telling indico to use s3 as its active storage backend by adding this option to indico.conf:

ATTACHMENT_STORAGE = 's3'

Note: You do this at your own risk. We do not provide a script to move data from S3 to the local filesystem (but the other way round - see indico s3 migrate --help after enabling the plugin). We recommend you to test it on a local development instance and not your main production server.

For S3 authentication and config (e.g. to use subdomains for bucket names), the preferred way is to use the standard files, i.e. ~/.aws/credentials and ~/.aws/config, but you can also specify all settings in the storage backend entry like this:

's3': 's3:bucket=my-indico-test-bucket,access_key=12345,secret_key=topsecret'

For advanced configuration options such as automatically using a new bucket every week/month/year, please look at the code. We didn’t have time to write documentation for the plugin yet:

1 Like