LDAP Groups filter error

We are using Shibboleth for authentication, with LDAP as an identity provider. We would just use shibboleth for the identity provider as well, but it seems that indico can’t get group information from shibboleth (even though our shibboleth does provide a multivalued ‘groups’ attribute.) Our shibboleth configuration uses a federated login, then supplements the attributes with data from this LDAP directory, which is why we can’t use LDAP for both authentication and identity.

Authentication works. Also, I am able to search for the LDAP groups, but indico claims that these groups have no members. When I use indico shell to try to investigate these groups, I get ldap.FILTER_ERROR.

The relevant part of our indico.conf file is:

AUTH_PROVIDERS = {
‘shib-sso’: {
‘type’: ‘shibboleth’,
‘title’: ‘SSO’,
‘attrs_prefix’: ‘’,
‘callback_uri’: ‘/login/shib-sso/shibboleth’,
}
}
IDENTITY_PROVIDERS = {
‘ldap’: {
‘type’: ‘ldap’,
‘title’: ‘LDAP’,
‘ldap’: {
‘uri’: ‘ldaps://OURHOST’,
‘bind_dn’: ‘XXXX’,
‘bind_password’: ‘XXXX’,
‘timeout’: 30,
‘verify_cert’: True,
‘page_size’: 1500,

 'uid': 'uid',
 'user_base': 'ou=People,cn=XXXX',
 'user_filter': '(objectClass=inetOrgPerson)',

 'gid': 'cn',
 'group_base': 'ou=Groups,cn=XXXX',
 'group_filter': '(objectClass=groupOfNames)',
 'member_of_attr': 'memberOf',                                                                                                                                                                                                                                                                                                     
'mapping': {                                                                                                                                                                                                                                                                                                          
       'first_name': 'givenName',
       'last_name': 'sn',
       'email': 'mail',
       'affiliation': 'o',
   },
},
'default_group_provider': True,
'synced_fields': {'first_name', 'last_name', 'affiliation'}

},
}

Attempts to list the group memberships in the gui produce “There are no users in this group.”
memberOf capability is enabled on the LDAP directory.

When I use indico shell I get:

In [1]: g = groupProxy(‘Project’, ‘ldap’)
In [2]: g.get_members()
Out [2]: set()
In [3]: u = User.get(2)
In [4]: u in g

ValueError: The filter supplied to the operation is invalid. (This is most likely due to a bad user or group filter.

I suspect that the LDAP setup is not using the correct attribute to search for users.

But I don’t understand the invalid filter part. I am able to search for groups and assign them permissions for events. But it has no effect.

Since the upgrade to 2.3.4 I also sometimes see such an error (with config unchanged).
The only change in flask-multipass where some unicode changes, so possibly something is wrongly encoded now?

Traceback (most recent call last):
  File "/usr/home/indicoadm/indicov2/.indicov2a/lib/python2.7/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/home/indicoadm/indicov2/.indicov2a/lib/python2.7/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/home/indicoadm/indicov2/.indicov2a/lib/python2.7/site-packages/indico/legacy/services/interface/rpc/json.py", line 22, in process
    rv['result'] = invoke_method(str(payload['method']), payload.get('params', []))
  File "/usr/home/indicoadm/indicov2/.indicov2a/lib/python2.7/site-packages/indico/legacy/services/interface/rpc/process.py", line 62, in invoke_method
    result = _process_request(method, copy.deepcopy(params))
  File "/usr/home/indicoadm/indicov2/.indicov2a/lib/python2.7/site-packages/indico/legacy/services/interface/rpc/process.py", line 52, in _process_request
    return handler(params).process()
  File "/usr/home/indicoadm/indicov2/.indicov2a/lib/python2.7/site-packages/indico/legacy/services/implementation/base.py", line 39, in process
    return self._getAnswer()
  File "/usr/home/indicoadm/indicov2/.indicov2a/lib/python2.7/site-packages/indico/legacy/services/implementation/search.py", line 48, in _getAnswer
    users = search_avatars(criteria, self._exactMatch, self._searchExt)
  File "/usr/home/indicoadm/indicov2/.indicov2a/lib/python2.7/site-packages/indico/modules/users/legacy.py", line 331, in search_avatars
    **{AVATAR_FIELD_MAP[k]: v for (k, v) in criteria.iteritems() if v})
  File "/usr/home/indicoadm/indicov2/.indicov2a/lib/python2.7/site-packages/indico/modules/users/util.py", line 284, in search_users
    for ident in identities:
  File "/usr/home/indicoadm/indicov2/.indicov2a/lib/python2.7/site-packages/flask_multipass/core.py", line 366, in search_identities
    for identity_info in provider.search_identities(provider.map_search_criteria(criteria), exact=exact):
  File "/usr/home/indicoadm/indicov2/.indicov2a/lib/python2.7/site-packages/flask_multipass/providers/ldap/providers.py", line 201, in search_identities
    yield IdentityInfo(self, identifier=user_data[self.ldap_settings['uid']][0], **user_data)
  File "/usr/local/lib/python2.7/contextlib.py", line 35, in __exit__
    self.gen.throw(type, value, traceback)
  File "/usr/home/indicoadm/indicov2/.indicov2a/lib/python2.7/site-packages/flask_multipass/providers/ldap/util.py", line 100, in ldap_context
    raise ValueError("The filter supplied to the operation is invalid. "
ValueError: The filter supplied to the operation is invalid. (This is most likely due to a bad user or group filter.

Could you try updating to the latest python-ldap version? pip install -U python-ldap

At least here:
Requirement already up-to-date: python-ldap in ./indicov2/.indicov2a/lib/python2.7/site-packages (3.3.1)

Try adding a raise before this line in your local flask-multipass installation. That should give you are more detailed error:

Seems not much more helpful, but printing the searchfilter in … ldap/operations.py:search gives a hint:
(&(\61\66\66\69\6c\69\61\74\69\6f\6e=Astra)(objectClass=person))

=> So the attribute name is wrongly encoded here

This seems to only occur if no mapping for ‘affiliation’ is supplied in the conf mapping dict, so probably the error lies in indicov2/.indicov2a/lib/python2.7/site-packages/flask_multipass/identity.py: map_search_criteria

So to reproduce:
configure ldap without a mapping for ‘affiliation’ and issue a search with affiliation(organization in the search dialog )

Already up-to-date:

Requirement already up-to-date: python-ldap in ./.venv/lib/python2.7/site-packages (3.3.1)

So your error is probably related to mine, but I do have affiliation mapped.

I can trigger the error by restricting a category to a particular group, then trying to enter that category. The group check triggers the error, with the same traceback you reported.

Also, it seems that any external search for users triggers it. Clearly there is something with the encoding of the user filter.

Did you recently upgrade indico (which would have upgraded flask-multipass as well)? If yes, it’d be interesting to know which versions you had before (only way to check it is the shell output from during the update…)

From my side: indico 2.3.4 , upgraded from 2.3.3 on March 11th
flask-multipass is now @ 0.3.5

I was more interested in the flask-multipass version - anywy, we changed the pin from >=0.3.1 to >-0.3.5 for this so there’s no way you had an even older multipass version, so my guess would be that this update pulled in a newer python-ldap version as well which somehow causes the problem for you.

Anyway, I’ll see if I can reproduce it locally somehow…

Yes, I upgraded indico to 2.3.4 from 2.3.3 just before trying this. Previously we did not use LDAP for the IDENTITY_PROVIDER, and did not even have python-ldap installed.

Can you give me some pointers on how to interrogate the user filter being used? My python debugging skills are not quite up to it.

ldap: Fix searching for binary (bytes) attributes · indico/flask-multipass@f26be36 · GitHub seems to be the root.

It seems all searches where the attribute is derived from an upstream object and is not mapped (like affiliation in my case) get wrongly identified as binary, not text type in _escape_filter_chars

Except that this commit is already part of 0.3.1 so you had it before…

Hmm, right. What’s strange is , that the search attribute name is escaped like this, not the value.

Trying to simplify, I did the following in indico shell. It fails trying to determine if a user belongs to a group. Here is the traceback.

In [2]: from indico.modules.groups import GroupProxy
In [3]: g = GroupProxy('Project', 'ldap')
In [5]: u = User.get(3)
In [6]: u in g
FILTER_ERROR                              Traceback (most recent call last)
<ipython-input-6-48a18c40ad4d> in <module>()
----> 1 u in g
> /opt/indico/.venv/lib/python2.7/site-packages/indico/modules/groups/core.pyc in __contains__(self, user)
     71         if not user:
     72             return False
---> 73         return self.has_member(user)
     74 
     75     @cached_property
/opt/indico/.venv/lib/python2.7/site-packages/indico/modules/groups/core.pyc in has_member(self, user)
    243             rv = False
    244         else:
--> 245             rv = any(x[1] in self.group for x in user.iter_identifiers(check_providers=True, providers={self.provider}))
    246         cache.set(key, rv, 1800)
    247         return rv
/opt/indico/.venv/lib/python2.7/site-packages/indico/modules/groups/core.pyc in <genexpr>(***failed resolving arguments***)
    243             rv = False
    244         else:
--> 245             rv = any(x[1] in self.group for x in user.iter_identifiers(check_providers=True, providers={self.provider}))
    246         cache.set(key, rv, 1800)
    247         return rv
/opt/indico/.venv/lib/python2.7/site-packages/indico/modules/users/models/users.pyc in iter_identifiers(self, check_providers, providers)
    590         if not check_providers:
    591             return
--> 592         for identity_info in multipass.search_identities(providers=providers, exact=True, email=self.all_emails):
    593             item = (identity_info.provider.name, identity_info.identifier)
    594             if item not in done:
/opt/indico/.venv/lib/python2.7/site-packages/flask_multipass/core.pyc in search_identities(self, providers, exact, **criteria)
    364             if not provider.supports_search:
    365                 continue
--> 366             for identity_info in provider.search_identities(provider.map_search_criteria(criteria), exact=exact):
    367                 yield identity_info
    368 
/opt/indico/.venv/lib/python2.7/site-packages/flask_multipass/providers/ldap/providers.pyc in search_identities(self, criteria, exact)
    197             if not search_filter:
    198                 raise IdentityRetrievalFailed("Unable to generate search filter from criteria", provider=self)
--> 199             for _, user_data in self._search_users(search_filter):
    200                 user_data = to_unicode(user_data)
    201                 yield IdentityInfo(self, identifier=user_data[self.ldap_settings['uid']][0], **user_data)
/opt/indico/.venv/lib/python2.7/site-packages/flask_multipass/providers/ldap/operations.py in search(base_dn, search_filter, attributes)
     93     while True:
     94         msg_id = connection.search_ext(base_dn, SCOPE_SUBTREE, filterstr=search_filter, attrlist=attributes,
---> 95                                        serverctrls=[page_ctrl], timeout=settings['timeout'])
     96         try:
     97             _, r_data, __, server_ctrls = connection.result3(msg_id, timeout=settings['timeout'])
/opt/indico/.venv/lib/python2.7/site-packages/ldap/ldapobject.pyc in search_ext(self, base, scope, filterstr, attrlist, attrsonly, serverctrls, clientctrls, timeout, sizelimit)
    857       RequestControlTuples(serverctrls),
    858       RequestControlTuples(clientctrls),
--> 859       timeout,sizelimit,
    860     )
    861 
/opt/indico/.venv/lib/python2.7/site-packages/ldap/ldapobject.pyc in _ldap_call(self, func, *args, **kwargs)
    338         self._trace_file.write('=> LDAPError - %s: %s\n' % (e.__class__.__name__,str(e)))
    339       try:
--> 340         reraise(exc_type, exc_value, exc_traceback)
    341       finally:
    342         exc_type = exc_value = exc_traceback = None
/opt/indico/.venv/lib/python2.7/site-packages/ldap/ldapobject.pyc in _ldap_call(self, func, *args, **kwargs)
    322     try:
    323       try:
--> 324         result = func(*args,**kwargs)
    325         if __debug__ and self._trace_level>=2:
    326           if func.__name__!="unbind_ext":
FILTER_ERROR: {'ctrls': [], 'result': -7, 'desc': u'Bad search filter'}

Sorry if my last post was spammy.
But seriously, is there a confirmed bug in how indico handles LDAP groups, and is that bug going to be addressed?

If the built-in functionality is broken with no likelihood of being fixed I’ll try to implement my own group sync with a python script.