Error generating document

EISSeuropa · July 2, 2024, 11:01am

Hello,

I’m running into an error when trying to generate documents for my conference attendants:

ERROR errors.py:110 – Could not load template preview data: {‘registration’: {‘transaction’: {‘data’: [‘Field may not be null.’]}}}

Full details below:

2024-07-02 10:50:48,619  ca59a322dd0347bf  1       indico.flask - ERROR errors.py:110 -- Could not load template preview data: {'registration': {'transaction': {'data': ['Field may not be null.']}}}

Traceback (most recent call last):
  File "/opt/indico/.venv/lib/python3.12/site-packages/indico/modules/receipts/util.py", line 265, in get_safe_template_context
    return tds.load(dumped)
           ^^^^^^^^^^^^^^^^
  File "/opt/indico/.venv/lib/python3.12/site-packages/marshmallow/schema.py", line 722, in load
    return self._do_load(
           ^^^^^^^^^^^^^^
  File "/opt/indico/.venv/lib/python3.12/site-packages/marshmallow/schema.py", line 909, in _do_load
    raise exc
marshmallow.exceptions.ValidationError: {'registration': {'transaction': {'data': ['Field may not be null.']}}}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/indico/.venv/lib/python3.12/site-packages/flask/app.py", line 880, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/indico/.venv/lib/python3.12/site-packages/flask/app.py", line 865, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/indico/.venv/lib/python3.12/site-packages/indico/web/flask/util.py", line 80, in wrapper
    return obj().process()
           ^^^^^^^^^^^^^^^
  File "/opt/indico/.venv/lib/python3.12/site-packages/indico/web/rh.py", line 299, in process
    res = self._do_process()
          ^^^^^^^^^^^^^^^^^^
  File "/opt/indico/.venv/lib/python3.12/site-packages/indico/web/rh.py", line 267, in _do_process
    rv = self._process()
         ^^^^^^^^^^^^^^^
  File "/opt/indico/.venv/lib/python3.12/site-packages/webargs/core.py", line 649, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/indico/.venv/lib/python3.12/site-packages/indico/modules/receipts/controllers/event.py", line 220, in _process
    html_sources = {registration: _compile_receipt_for_reg(registration) for registration in self.registrations}
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/indico/.venv/lib/python3.12/site-packages/indico/modules/receipts/controllers/event.py", line 216, in _compile_receipt_for_reg
    safe_ctx = get_safe_template_context(self.event, registration, custom_fields)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/indico/.venv/lib/python3.12/site-packages/indico/modules/receipts/util.py", line 269, in get_safe_template_context
    raise RuntimeError(f'Could not load template preview data: {exc}') from exc
RuntimeError: Could not load template preview data: {'registration': {'transaction': {'data': ['Field may not be null.']}}}

The template I am using is the following:


{% set personal_data = registration.personal_data %}
{% set txn = registration.transaction %}

<h1>Certificate 
  of Participation</h1>

<aside id="title">
  <div class="title">
    <!-- Title -->
    <h2>{{ event.title }}</h2>
    <!-- Date(s) -->
    {% if event.start_dt.date() != event.end_dt.date() %}
      {{ event.start_dt | format_date('dd MMM') }}
      -
      {{ event.end_dt | format_date('dd MMM') }}
    {% else %}
      {{ event.start_dt | format_date('dd MMM YYYY') }}
    {% endif %}

    <!-- Venue -->
    {% if custom_fields.venue %}
      - {{ custom_fields.venue }}
    {% elif event.venue_name %}
      - {{ event.venue_name }}
    {% endif %}

    {% if custom_fields.add_url %}
      <p>{{ event.url }}</p>
    {% endif %}
  </div>

  <!-- Logo -->
  {% if custom_fields.logo %}
    <img src="{{ custom_fields.logo }}">
  {% endif %}
</aside>

<aside id="addresses">
  <!-- Address of organizer -->
  <address id="from">
    {{ custom_fields.address_from }}
  </address>

  <!-- Address of participant -->
  <address id="to">
    <strong>{{ personal_data.first_name }} {{ personal_data.last_name }}</strong>
    {%- if custom_fields.add_affiliation %}
      {{ personal_data.affiliation }}
    {%- endif %}
    {%- if custom_fields.address_to_override %}
      {{ custom_fields.address_to_override }}
    {% elif personal_data.address %}
      {{ personal_data.address }}
      {{ personal_data.country }}
    {% else %}
      {{ personal_data.country }}
    {% endif %}
  </address>
</aside>

This is to certify that {{ registration.personal_data.title }} {{ registration.personal_data.first_name }} {{ registration.personal_data.last_name }}
affiliated with {{ registration.field_data[4].friendly_value }} has registered and taken part in {{ event.title }}, that took place at {{ event.venue_name }}, {{ event.address }}
on

    {% if event.start_dt.date() != event.end_dt.date() %}
      {{ event.start_dt | format_date('dd MMM') }}
      -
      {{ event.end_dt | format_date('dd MMM YYYY') }}.
    {% else %}
      {{ event.start_dt | format_date('dd MMM YYYY') }}.
    {% endif %}

<aside id="signature">
  <div class="signature">
    Prof. Nicolas Blarel
    <br>
    Conference Coordinator
    <br>
    European Initiative for Security Studies
  </div>
</aside>

  </tbody>

ThiefMaster · July 2, 2024, 11:15am

I can’t reproduce this. Can you please provide the full template config (HTML, CSS and YAML)?

PS: This looks like a bad idea. Use {{ personal_data.affiliation }} for this.

{{ registration.field_data[4].friendly_value }}

The numeric indexes should not be used for anything, they are simply shown because field_data is a list and lists have numeric indexes. I’m pretty sure they are not reliable

EISSeuropa · July 2, 2024, 12:21pm

Sure, thank you for looking into it!

Here is the CSS:

@page {
  margin: 3cm;

  @bottom-center {
    color: #a9a;
    content: 'Authenticated and produced by Indico';
    font-size: 9pt;
  }
}

html {
  color: #14213d;
  font-family: sans-serif;
  font-size: 11pt;
  line-height: 1.6;
}

body {
  margin: 0;
}

aside {
  display: flex;
  margin: 2em 0 4em;
  width: 100%;
}

aside#title {
  flex: 2;
  align-items: center;
}

aside#title div.title {
  flex: 2;
}

aside#addresses {
  justify-content: space-between;
}

aside img {
  max-width: 5cm;
  max-height: 5cm;
  margin-left: 0.5cm;
}

address {
  display: block;
  white-space: pre-line;
}

h1 {
  color: #18597d;
  font-size: 40pt;
  margin: 0;
}

aside address#from {
  color: #a9a;
  flex: 1;
}

aside address#to {
  text-align: right;
  flex: 1;
}

dl#information {
  position: absolute;
  right: 0;
  text-align: right;
  top: 0;
}

dt,
dd {
  display: inline;
  margin: 0;
}

dt {
  color: #a9a;
}

dt::before {
  content: '';
  display: block;
}

dt::after {
  content: ':';
}

table {
  border-collapse: collapse;
  width: 100%;
}

tr.total {
  padding-top: 5cm;
}

tr.total td {
  margin-top: 2cm;
  padding-top: 0.25cm;
  border-top: 0.2mm solid #a9a;
}

th {
  border-bottom: 0.2mm solid #a9a;
  color: #a9a;
  font-size: 10pt;
  font-weight: 400;
  padding-bottom: 0.25cm;
  text-transform: uppercase;
}

td {
  padding-top: 2mm;
}

td:last-of-type {
  color: #18597d;
  font-weight: bold;
  text-align: right;
}

th,
td {
  text-align: center;
}

th:first-of-type,
td:first-of-type {
  text-align: left;
}

th:last-of-type,
td:last-of-type {
  text-align: right;
}

And here is the YAML:

custom_fields:
  - name: address_from
    type: textarea
    attributes:
      label: Organizer Address
      description: |
        The address of the entity who is issuing this receipt.

  - name: address_to_override
    type: textarea
    attributes:
      label: Override participant address
      description: |
        A custom address to use instead of the participant's address from
        their registration.

  - name: venue
    type: input
    attributes:
      label: Venue
      description: |
        A custom venue name instead of the event's venue.

  - name: fee_category
    type: input
    attributes:
      label: Fee category
      description: |
        A custom type of registration fee like "Early Bird", shown in parentheses
        behind the "Registration fee" text.

  - name: receipt_number
    type: input
    attributes:
      label: Receipt number
      description: |
        A custom receipt number instead of the auto-generated one.

  - name: logo
    type: image
    attributes:
      label: Logo
      description: |
        A logo image to be shown on the receipt.

  - name: add_url
    type: checkbox
    attributes:
      label: Add event URL
      description: |
        Whether to include the URL of the event on the receipt.

  - name: add_affiliation
    type: checkbox
    attributes:
      label: Add registrant affiliation
      description: |
        Whether to include the affiliation of the registrant between their name
        and address.

EISSeuropa · July 2, 2024, 12:23pm

Thanks, I’ve made the change!

ThiefMaster · July 2, 2024, 12:28pm

Does the registration for which you’re trying to generate the document have any payment transaction associated? Can you please run this code in indico shell?

reg = Registration.get(REGID)
reg.transaction
reg.transaction.__dict__  # this one only if the previous one returned something

Then share the output (censor any personal data of course). The REGID can be found in the URL (/event/EVENTID/manage/registration/REGFORMID/registrations/REGID/)

EISSeuropa · July 2, 2024, 1:02pm

Yes we have payments associated but most I have manually validated. But I’m not sure why it is preventing creating the document, the template does not include references to pricing.

I was trying to generate documents for 75 people, and after a bit of trial and error I have found at least one problematic registration. Here are the outputs below:

In [1]: reg = Registration.get(190)

In [2]: reg.transaction
Out[2]: <PaymentTransaction(99, registration_id=190, stripe, 40.00, eur, 2024-03-23 12:13:25.300606+00:00, status=successful)>

In [3]: reg.transaction.__dict__
Out[3]: 
{'_sa_instance_state': <sqlalchemy.orm.state.InstanceState at 0x7efdbd514890>,
 'amount': Decimal('40.00'),
 'status': <TransactionStatus.successful: 1>,
 'provider': 'stripe',
 'data': None,
 'registration_id': 190,
 'id': 99,
 'currency': 'eur',
 'timestamp': datetime.datetime(2024, 3, 23, 12, 13, 25, 300606, tzinfo=<UTC>)}

Nothing seems like personal data but let me know if you spot something sensitive.

ThiefMaster · July 2, 2024, 1:09pm

OK, it fails because the transaction’s data is None, this is very uncommon, but apparently the stripe plugin doesn’t bother storing any details about the transaction…

As a workaround you can run this query (it will not help for future registrations paid via stripe, but you can safely re-run it at any time):

PaymentTransaction.query.filter(PaymentTransaction.data == 'null').update({PaymentTransaction.data: {}})
db.session.commit()

EISSeuropa · July 2, 2024, 1:26pm

It worked indeed, thank you!