Using LDAP to authenticate users

Credentials are needed to access Elasticsearch clusters if security is set. Credentials can be standard user/passwords stored inside Elasticsearch, or more complex solutions such as Active Directory and Lightweight Directory Access Protocol (LDAP) can be used.

You can configure the Elastic Stack security features to communicate with an LDAP server to authenticate users. LDAP stores users and groups hierarchically, similar to the way folders are grouped in a file system. An LDAP directory’s hierarchy is built from containers such as the organizational unit (ou), organization (o), and domain controller (dc).

The path to an entry is a Distinguished Name (DN) that uniquely identifies a user or group. User and group names typically have attributes such as a common name (cn) or unique ID (uid). A DN is specified as a string, for example "cn=admin,dc=example,dc=com" (white spaces are ignored).

The ldap realm supports two modes of operation, a user search mode and a mode with specific templates for user DNs.

Introduction to LDAP

The full name of LDAP is Lightweight Directory Access Protocol, Lightweight Directory Access Protocol. Simply put, LDAP is a protocol for accessing directory databases. Directory service data is also a kind of database. Compared with the well-known relational databases, such as MySQL and Oracle, this kind of database has only the following characteristics:

• It organizes data in a tree structure, similar to a file directory
• It is a database optimized for querying, browsing and searching, which means that LDAP is particularly readable, but the writing performance is poor, and it does not support transaction processing, rollback and other responsible functions

LDAP directory structure:

1. Target tree: as shown in the figure above, in a directory server, the entire directory information set can be expressed as a directory information tree, and each node in the tree is an entry.
2. Entry: each entry is a record, and each entry has its own uniquely distinguished name (DN). For example, each circle in the figure is a record.
3. DN, RDN: for example, the first leaf entry in the above figure has a uniquely distinguished name DN: uid=bob,ou=people,dc=acme,dc=org. Absolute paths are similar to relative paths for file directories. In addition to its DN, it also has an RDN. RDN has nothing to do with the directory structure, such as uid=bob,ou=people,dc=acme,dc=org mentioned earlier, its RDN is uid=bob.
4. Attribute: describe the specific information of the item. For example, 'uid=bill,ou=people,dc=acme,dc=org', it has attribute name as bill, attribute age as 11, attribute school as xx:

In this article, Elastic Stack 8.3.3 will be used for the demo, and the whole configuration to be used is as follows:

As shown above, two machines are used. One Ubuntu OS machine is used for installing the Elastic Stack, and the other one is used for installing the Apache Directory.

Installations

As shown in above configuration, Elastic Stack and Apache Directory are needed to to be installed.

Elastic Stack

If you have not installed Elasticsearch and Kibana yet, please refer to our official guides:

• Installing Elasticsearch
• Installing Kibana

According to Elastic's subscription at Subscriptions | Elastic Stack Products & Support | Elastic, LDAP is a feature to buy:

After installing Elasticsearch and Kibana, we need to activate the trial license as follows:

Apache Directory

Apache Directory Studio is a complete directory tools platform designed to work with any LDAP server, but it is specifically designed for use with ApacheDS. It is an Eclipse RCP application consisting of several Eclipse (OSGi) plug-ins that can be easily upgraded with other plug-ins. These plug-ins can even run within Eclipse itself. We can go to the address  Downloads — Apache Directory  to download and install according to your own platform:

We then need to use ApacheDS to create some users and groups as follows. We create the following ou under dc=example,dc=com:

• groups: This is a group that contains cn=user1 and cn=user2 entries. If you want to know how to create a group, please refer to the document "Preparing the LDAP environment".

 

• users: This is an ou. It contains some user information. If you want to know how to create a user, please read the article "Preparing the LDAP environment".

• usrs: This is an ou. It is used by the groups group: 

 

For each entry above, we set the uid and userPassword for it. For simplicity, in our case, set them all to 123456 as password。

In order to verify the correctness of a DN and user, we can use the following command to verify:

ldapsearch -x -D "cn=liuxg,ou=users,dc=example,dc=com"\
       -W -H ldap://ubuntu:10389 -b "dc=example,doc=com"\
       -s sub '(sAMAccountName=liuxg)'

where "ubuntu" is the hostname for the Apache Directory machine. The response is as follows:

$ ldapsearch -x -D "cn=liuxg,ou=users,dc=example,dc=com"\
>        -W -H ldap://ubuntu:10389 -b "dc=example,doc=com"\
>        -s sub '(sAMAccountName=liuxg)'
Enter LDAP Password: 
# extended LDIF
#
# LDAPv3
# base <dc=example,doc=com> with scope subtree
# filter: (sAMAccountName=liuxg)
# requesting: ALL
#
 
# search result
search: 2
result: 32 No such object
text: NO_SUCH_OBJECT: failed for MessageType : SEARCH_REQUEST
Message ID : 2
 
     SearchRequest
        baseDn : 'dc=example,doc=com'
        filter : '(|
 (sAMAccountName=liuxg)(objectClass=referral))'
        scope : whole subtree
 
        typesOnly : false
        Size Limit : no limit
        Time Limit 
 : no limit
        Deref Aliases : never Deref Aliases
        attributes : 
 
org.apache.directory.api.ldap.model.message.SearchRequestImpl@430edd43: ERR
 _268 Cannot find a partition for dc=example,doc=com
 
# numResponses: 1

In the above, we enter the password for liuxg account. If you see the outputs as shown above, it means that your DN works well.

Configure the ldap realm with user search

The ldap realm supports two modes of operation, a user search mode and a mode that provides specific templates for user DNs. LDAP user search is the most common mode of operation. In this mode, a specific user with permission to search the LDAP directory is used to search for the DN of the authenticating user based on the supplied username and LDAP attributes. Once found, the user is authenticated by attempting to bind to the LDAP server using the found DN and the provided password.

We first open the Elasticsearch configuration file config/elasticsearch.yml file:

config/elasticsearch.yml

xpack:
  security:
    authc:
      realms:
        ldap:
          ldap1:
            order: 0
            url: "ldap://ubuntu:10389"
            bind_dn: "ou=users,dc=example,dc=com"
            bind_password: "123456"
            user_search:
              base_dn: "dc=example,dc=com"
              filter: "(cn={0})"
            group_search:
              base_dn: "ou=groups,dc=example,dc=com"
            files:
              role_mapping: "/Users/liuxg/elastic/elasticsearch-8.3.3/config/role_mapping.yml"
            unmapped_groups_as_roles: false

We add the above code at the end of the file. The url needs to be configured according to your actual installation. The set password is 123456, which is configured in ApacheDS. Many people may think that it is not a good idea to configure plaintext passwords in elasticsearch.yml above. We can use the following command to add the bind_dn (as shown above) password to the Elasticsearch keystore:

bin/elasticsearch-keystore add xpack.security.authc.realms.ldap.ldap1.secure_bind_password

Once we configure it like this, then I don't need to configure it in elasticsearch.yml, and we can directly remove the bind_dn line.

We need to modify the above role_mapping configuration according to our Elasticsearch installation path.

role_mapping.yml

# Role mapping configuration file which has elasticsearch roles as keys
# that map to one or more user or group distinguished names
 
#roleA:   this is an elasticsearch role
#  - groupA-DN  this is a group distinguished name
#  - groupB-DN
#  - user1-DN   this is the full user distinguished name
 
superuser:
  - "cn=liuxg,ou=users,dc=example,dc=com"
  - "cn=xgliu,ou=users,dc=example,dc=com"
  - "employeeNumber=1,ou=users,dc=example,dc=com"

As shown above, superuser is a role already defined in Elasticsearch. This role can be preset, and it can also be defined by ourselves. If you don't know how to define roles yet, please refer to our global document at Defining roles | Elasticsearch Guide [8.5] | Elastic.

  - "cn=liuxg,ou=users,dc=example,dc=com"
  - "cn=xgliu,ou=users,dc=example,dc=com"
  - "employeeNumber=1,ou=users,dc=example,dc=com"

The users represented by these DNs are superuser users. We next restart Elasticsearch and log in to the Kibana interface:

Obviously, we can use liuxg:123456 to log in successfully. We can also use the following command to check: 

curl -k -u xgliu:123456 https://localhost:9200

$ curl -k -u xgliu:123456 https://localhost:9200
{
  "name" : "liuxgm.local",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "zWDMrYU2RJm9RugZCZGhsQ",
  "version" : {
    "number" : "8.3.3",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "801fed82df74dbe537f89b71b098ccaff88d2c56",
    "build_date" : "2022-07-23T19:30:09.227964828Z",
    "build_snapshot" : false,
    "lucene_version" : "9.2.0",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}

The xgliu account can also successfully access Elasticsearch.

Next, we show how to map the accounts in groups so that they also have the role of superuser. Of course, we can also make them map to any roles we want. However, the premise is that these roles must be preset or created by yourself. In the following, two methods are used to show the role mapping.

Kibana interface

In the above, we fill in the following values ​​in Value: 

cn=user*,ou=groups,dc=example,dc=com

where wildcard is used to match user1 and user2. They are used in the groups. Click Save role mapping button:

We can enter the following command in the console:

GET  /_security/role_mapping/

The above command returns the result:

{
  "users": {
    "enabled": true,
    "roles": [
      "superuser"
    ],
    "rules": {
      "all": [
        {
          "field": {
            "groups": "cn=user*,ou=groups,dc=example,dc=com"
          }
        }
      ]
    },
    "metadata": {}
  }
}

It shows that all users whose DN matches cn=user*,ou=groups,dc=example,dc=com will have the role of superuser. We next use jim:123456 to log in:
 

 

The above screen shows our login was successful. Of course, another user sue:123456 in this group can also log in successfully.

In order to demonstrate the use of the following API, we first delete the users role mapping we just created:

 

In this way, we don't have any role mapping, and of course neither jim nor sue can log in. If we log in with jim's account, we can see the following screen:

Use the API 

We use the following API to implement role mapping:

PUT /_security/role_mapping/admins
{
  "roles" : [ "superuser" ],
  "rules" : { "field" : {
    "groups" : "cn=user*,ou=groups,dc=example,dc=com" 
  } },
  "enabled": true
}

We can use the following command to view:

GET  /_security/role_mapping/

The above command generates:

{
  "admins": {
    "enabled": true,
    "roles": [
      "superuser"
    ],
    "rules": {
      "field": {
        "groups": "cn=user*,ou=groups,dc=example,dc=com"
      }
    },
    "metadata": {}
  }
}

It generates a role mapping called admins. We can go back to the previous role mapping interface:

A new role mapping for admins has been created. We can use a user defined in the group to log in. We try jim:123456 in the following:

As shown above, the login is successful again!

Refereneces:

• LDAP user authentication | Elasticsearch Guide [8.3] | Elastic