Introduction
This article describes setting up a Kerberos Key Distribution Center and a Kerberos Administration server on an embedded Linux IoT device (Raspberry Pi B2). It uses an OpenLDAP server to store credentials, keys and persistent data to operate Kerberos Services.
Background
This is a continuation of part 1 of this article published here. The following tasks are described in this article, these would complete the installation of Kerberos:
- Install Kerberos services
- Configure and deploy the Kerberos KDC
- Setup user principals and IoT device principals on Kerberos
- Configure and check Kerberos authentication on all devices
Overview of Kerberos
The Kerberos service provides a trusted third party authentication service in which each entity or principle (server or client) is a user to the Kerberos service and believe Kerberos’ judgment of its peer’s identity.
Kerberos uses shared secret-key symmetric encryption. For a user, the secret is the user’s password applied to a one-way function. Kerberos uses propagation mode CBC (cipher block chaining) encryption which propagates an error throughout the message if even a single bit is changed or tampered.
Kerberos keeps a database of the users and their secret keys. The secret keys are negotiated at registration. It also generates temporary session keys which are only used for one session. Kerberos can be configured to use only authentication features, or integrity protection, or both. Collision-resistant hashes are generated and used to protect the integrity of messages. Kerberos utilizes libraries for providing encryption, database administration, authentication, administration, database replication, and user/application programs. This can be mixed and matched as required to suit the software stack.
In the Kerberos database, records are held for each principal, containing the name, secret key, expiration date and other administrative data. Names are of the format name.instance@realm, where:
- primary name is the name of the user or service
- instance is the name of the machine where the server runs
- realm is the administrative domain that maintains the authentication data
There is one copy of the master database/server on which user administration can be done, but several slaves can be setup which provide a read-only database for authentication. The database can be replicated on these read-only slave machines, which receive periodic updates from the master machine.
Two types of credentials are used in Kerberos:
- tickets - these are used to securely pass user’s identity between the auth server and application
- authenticators - additional user information to prove the client’s identity
Tickets are secured with the server’ secret key and can be used by the user several times during a session and is described as:
Ticket = Encrypt serverKey (serverid, clientid, client network address, timestamp, ticket lifetime, client-server sessionKey)
An authenticator can only be used once during authentication, it is given as:
Ticket = Encrypt sessionKey (clientid, client network address, timestamp)
A principal needs to perform the following steps to use Kerberos authentication (here, server means the IoT device):
- Client sends a request for a Ticket-Granting-Server (TGS) ticket to the Kerberos server by sending the client name and TGS name.
- The Kerberos server authenticates the client and issues session keys for the TGS-client exchange, the TGS ticket encrypted using the TGS-Kerberos server keys; all encrypted using the client’s key.
- Request for a service ticket from the TGS by sending the service name, TGS ticket and the client authenticator, both encrypted in the TGS-client session key.
- The TGS server checks for the clients access controls and issues a client-server session ticket encrypted using the server key and the client-server session key, both encrypted using the TGS-client session key. This is used in all server accesses made by the client to the same server.
- An optional mutual authentication may be done where the server sends the client the timestamp value+1 encrypted in the server-client session key.
This process is illustrated in the following figure:
The other functions performed by Kerberos are:
- Administration functions: password change, add/delete principals, etc.
- Periodic Kerberos database replication onto the slave machines
- Interaction with other Kerberos realms for cross-realm trust propagation
Installation
Package Installation
Issue the following commands to install any outstanding package updates:
sudo apt-get update
sudo apt-get upgrade
Install Kerberos services for ldap-kdc backend, distribution center and administration server:
sudo apt-get install krb5-kdc-ldap krb5-kdc krb5-admin-server
The installation is accompanied by a configuration script to setup a Kerberos realm - krb5_newrealm which should be able to generate a Kerberos configuration file /etc/krb5.conf with the required settings for the realm. Hoever, we'll configure the kerberos file manually in this example.
Preparing the LDAP Datastore for Kerberos
Update the Access Control Lists (ACL) to restrict access to the Kerberos principal key attribute:
sudo ldapmodify -Q -Y EXTERNAL -H ldapi:/// -D cn=schema,cn=config -W -c
Enter LDAP Password:
dn: olcDatabase={1}mdb,cn=config
replace: olcAccess
olcAccess: to attrs=userPassword,shadowLastChange,krbPrincipalKey by dn="cn=admin,
dc=examplefirm,dc=com" write by anonymous auth by self write by * none
modifying entry "olcDatabase={1}mdb,cn=config"
[ctrl+D]
sudo ldapmodify -Q -Y EXTERNAL -H ldapi:/// -D cn=schema,cn=config -W -c
Enter LDAP Password:
dn: olcDatabase={1}mdb,cn=config
add: olcAccess
olcAccess: to dn.base="" by * read
modifying entry "olcDatabase={1}mdb,cn=config"
dn: olcDatabase={1}mdb,cn=config
add: olcAccess
olcAccess: to * by dn="cn=admin,dc=examplefirm,dc=com" write by * read
modifying entry "olcDatabase={1}mdb,cn=config"
[Ctrl+D]
The OpenLDAP directory is now ready to serve as a Kerberos principal database.
Kerberos Configuration
Configure the Key Distribution Center (KDC) server and admin server for the realm EXAMPLEFIRM.COM by editing the main configuration file /etc/krb5.conf and add the following options under the appropriate sections:
[libdefaults]
default_realm = EXAMPLEFIRM.COM
# any other configuration items
...
[realms]
EXAMPLEFIRM.COM = {
kdc = iot01.examplefirm.com
admin_server = iot01.examplefirm.com
default_domain = examplefirm.com
database_module = openldap_ldapconf
}
[domain_realm]
.examplefirm.com = EXAMPLEFIRM.COM
[dbdefaults]
ldap_kerberos_container_dn = dc=examplefirm,dc=com
[dbmodules]
openldap_ldapconf = {
db_library = kldap
ldap_kdc_dn = "cn=admin,dc=examplefirm,dc=com"
# this object needs to have read rights on
# the realm container, principal container and realm sub-trees
ldap_kadmind_dn = "cn=admin,dc=examplefirm,dc=com"
# this object needs to have read and write rights on
# the realm container, principal container and realm sub-trees
ldap_service_password_file = /etc/krb5kdc/service.keyfile
ldap_servers = ldapi:/// ldaps://iot01.examplefirm.com
ldap_conns_per_server = 5
}
[dbdefaults]
ldap_kerberos_container_dn = cn=krbContainer,dc=examplefirm,dc=com
Create the Kerberos Realm
We need the following OUs in the subtrees of the realm so that we can store principals prepresenting IoT devices under Computers and principals representing users under People:
Create the realm with both subtrees using this command:
sudo kdb5_ldap_util -D cn=admin,dc=examplefirm,dc=com create -subtrees ou=People,
dc=examplefirm,dc=com:ou=Computers,dc=examplefirm,dc=com -r EXAMPLEFIRM.COM -s -H ldapi:///
Password for "cn=admin,dc=examplefirm,dc=com":
Initializing database for realm 'EXAMPLEFIRM.COM'
You will be prompted for the database Master Password.
It is important that you NOT FORGET this password.
Enter KDC database master key:
Re-enter KDC database master key to verify:
Verify it has been setup by querying the principle sub-trees setup at this ldap server:
kdb5_ldap_util -D cn=admin,dc=examplefirm,dc=com -H ldapi:/// view -r EXAMPLEFIRM.COM
Password for "cn=admin,dc=examplefirm,dc=com":
Realm Name: EXAMPLEFIRM.COM
Subtree: ou=People,dc=examplefirm,dc=com
Subtree: ou=Computers,dc=examplefirm,dc=com
View the list of realms using this command:
kdb5_ldap_util -D cn=admin,dc=examplefirm,dc=com -H ldapi:/// list
Password for "cn=admin,dc=examplefirm,dc=com":
EXAMPLEFIRM.COM
Create a stash of the password used to bind to the LDAP server. This password is used by the ldap_kdc_dn and ldap_kadmin_dn ldap user DNs as setup in /etc/krb5.conf:
sudo kdb5_ldap_util -D cn=admin,dc=examplefirm,dc=com stashsrvpw
-f /etc/krb5kdc/service.keyfile cn=admin,dc=examplefirm,dc=com
Password for "cn=admin,dc=examplefirm,dc=com":
Password for "cn=admin,dc=examplefirm,dc=com":
Re-enter password for "cn=admin,dc=examplefirm,dc=com":
Check to make sure this file is only root accessible:
sudo ls -al /etc/krb5kdc/service.keyfile
-rw------- 1 root root 77 Sep 9 00:50 /etc/krb5kdc/service.keyfile
Create and Deploy Policies
Kerberos enables us to configure and enforce password and security configuration policies such as maximum age of tickets, minimum password length, password complexity, etc. We will configure some of these now.
Create a Kerberos default ticket policy for this realm using the following command:
sudo kdb5_ldap_util -D cn=admin,dc=examplefirm,
dc=com -H ldapi:/// create_policy -r EXAMPLEFIRM.COM -maxtktlife "8 day"
-maxrenewlife "1 week" -allow_postdated +needchange -allow_forwardable tktpolicy
To verify that this policy got added, get a list of all policies for the realm and query this policy:
kdb5_ldap_util -D cn=admin,dc=examplefirm,dc=com -H ldapi:/// -r EXAMPLEFIRM.COM list_policy
Password for "cn=admin,dc=examplefirm,dc=com":
tktpolicy
kdb5_ldap_util -D cn=admin,dc=examplefirm,dc=com -H ldapi:/// -r EXAMPLEFIRM.COM view_policy tktpolicy
Password for "cn=admin,dc=examplefirm,dc=com":
Ticket policy: tktpolicy
Maximum ticket life: 8 days 00:00:00
Maximum renewable life: 7 days 00:00:00
Ticket flags: DISALLOW_POSTDATED DISALLOW_FORWARDABLE REQUIRES_PWCHANGE
Configure a password policy using this command:
sudo kadmin.local
Authenticating as principal root/admin@EXAMPLEFIRM.COM with password.
kadmin.local: add_policy -maxlife "30 days" -minlength 8 -minclasses 4
-maxfailure 5 -failurecountinterval "6 hours" default
This command adds a password policy named “default” puts the following restrictions when setting or changing passwords:
- minimum password length = 8 characters
- password has at least 4 character classes: lower case, upper case, numbers and punctuation characters.
- Maximum number of failures tolerated before account lockout = 5
- Maximum life for a password = 30 days
- Time interval after which authentication failure count is reset to one failure, is set as "6 hours"
Verify this has been correctly configured by listing out the parameters using this command:
kadmin.local: get_policy default
Policy: default
Maximum password life: 3888000
Minimum password life: 0
Minimum password length: 8
Minimum number of password character classes: 4
Number of old keys kept: 1
Reference count: 0
Maximum password failures before lockout: 5
Password failure count reset interval: 0 days 06:00:00
Password lockout duration: 0 days 00:00:00
User Management
Adding New Users to Kerberos
With the kerberos service configured, we can now start adding Kerberos principals to the LDAP database. To add a principal, start the kadmin.local utility by entering:
sudo kadmin.local
Authenticating as principal root/admin@examplefirm.COM with password.
kadmin.local:
Enter the following command for creating user user01:
kadmin.local: addprinc -x dn="uid=user01,ou=People,dc=examplefirm,dc=com" user01
NOTICE: no policy specified for user01@EXAMPLEFIRM.COM; assigning "default"
Enter password for principal "user01@EXAMPLEFIRM.COM":
Re-enter password for principal "user01@EXAMPLEFIRM.COM":
Principal "user01@EXAMPLEFIRM.COM" created.
Repeat the same for creating user02:
kadmin.local: addprinc -x dn="uid=user02,ou=People,dc=examplefirm,dc=com" user02
NOTICE: no policy specified for user02@EXAMPLEFIRM.COM; assigning "default"
Enter password for principal "user02@EXAMPLEFIRM.COM":
Re-enter password for principal "user02@EXAMPLEFIRM.COM":
Principal "user02@EXAMPLEFIRM.COM" created.
After completing this step, the Kerberos services need to be restarted to ensure the settings have taken effect:
sudo service krb5-kdc restart
sudo service krb5-admin-server restart
Test Initialization of the User
Initialize a principle with Kerberos KDC and kadmin:
sudo kinit -V user01
Using default cache: /tmp/krb5cc_0
Using principal: user01@EXAMPLEFIRM.COM
Password for user01@EXAMPLEFIRM.COM:
Authenticated to Kerberos v5
View the ticket generated using klist:
sudo klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: user01@EXAMPLEFIRM.COM
Valid starting Expires Service principal
09/09/2016 23:51 10/09/2016 09:51 krbtgt/EXAMPLEFIRM.COM@EXAMPLEFIRM.COM
renew until 10/09/2016 23:51
This confirms the user is able to obtain a valid kerberos ticket and get authenticated.
Using Kerberos for Device Authentication
On each device which needs to use Kerberos, install the following packages to setup authentication via kerberos:
sudo apt-get install libpam-krb5 krb5-user
Update the PAM using this command:
sudo pam-auth-update
Select and enable Kerberos authentication on the screen.
In order to authenticate using Kerberos, the local system needs to first know the user login being attempted is a valid user - this is achieved either by adding a local user with the same name as the kerberos user, or, refer to an LDAP directory to validate user IDs.
If you opt for a local user option, create a local accounts on the IoT device and do not set the password - skip this step during user creation. This will make all users available in the local passwd file for the system to validate user IDs, then users will be able to login to the device using their kerberos password.
sudo adduser user01
Adding user `user01' ...
Adding new group `user01' (1001) ...
Adding new user `user01' (1001) with group `user01' ...
Creating home directory `/home/user01' ...
Copying files from `/etc/skel' ...
Current Kerberos password:
Current Kerberos password:
passwd: Authentication token manipulation error
passwd: password unchanged
Try again? [y/N] N
Changing the user information for user01
Enter the new value, or press ENTER for the default
Full Name []: user01
Room Number []:
Work Phone []:
Home Phone []:
Other []:
Is the information correct? [Y/n] Y
This option is better since it is easier to troubleshoot login problems with this setup.
For configuring the system to check and retrieve user IDs against an LDAP directory, complete the following configuration:
sudo apt-get install sssd
Add the following in a config file /etc/sssd/sssd.conf :
[sssd]
config_file_version = 2
services = nss,pam
domains = EXAMPLEFIRM.COM
[nss]
#debug_level = 0xFFF0
filter_users = root
filter_groups = root
[pam]
[domain/EXAMPLEFIRM.COM]
#debug_level = 0xFFF0
auth_provider = krb5
krb5_server = iot01.examplefirm.com
krb5_realm = EXAMPLEFIRM.COM
cache_credentials = true
# credential caching (above) is optional
access_provider = simple
chpass_provider = krb5
id_provider = ldap
ldap_uri = ldap:
ldap_search_base = dc=examplefirm,dc=com
ldap_id_use_start_tls = false
# set the above to true after installing TLS certificates for LDAP
ldap_sasl_mech = GSSAPI
sudo_provider = none
Then, fix file permissions so the config file is only accessible by root.
Restart the sssd daemon:
sudo service sssd restart
Create Device Principals
The "host" service is the name of the service provided by remote access services such as telnet or SSH. To support Kerberized remote login. Login to the kadmin prompt using kadmin.local and create a service principal with the name host/iot01.examplefirm.lan@EXAMPLEFIRM.COM.
While a user principal is able to supply a password which acts as its secret key during Kerberos authentication, but the application (IoT device) is not able to do so interactively. Hence, its secret key is created on the KDC and securely transmitted to the IoT device in a "keytab" file.
To do this, first we will add the server principal (this represents the IoT which uses Kerberos) in Kerberos using the kadmin utility:
sudo kadmin.local
kadmin.local: addprinc -randkey -x dn=cn=host,cn=iot01.examplefirm.com,ou=Computers,dc=examplefirm,dc=com host/iot01.examplefirm.com@EXAMPLEFIRM.COM
NOTICE: no policy specified for host/iot01.examplefirm.com@EXAMPLEFIRM.COM; assigning "default"
Principal "host/iot01.examplefirm.com@EXAMPLEFIRM.COM" created.
Dump these keys into a keytab file for storing at the IoT's /etc directory, save the IoT principal's credentials in a keytab file /etc/krb5.keytab:
kadmin.local: ktadd -k krb5-iot01.keytab host/iot01.examplefirm.com@EXAMPLEFIRM.COM
Entry for principal host/iot01.examplefirm.com@EXAMPLEFIRM.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:krb5-iot01.keytab.
Entry for principal host/iot01.examplefirm.com@EXAMPLEFIRM.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:krb5-iot01.keytab.
Entry for principal host/iot01.examplefirm.com@EXAMPLEFIRM.COM with kvno 2, encryption type des3-cbc-sha1 added to keytab WRFILE:krb5-iot01.keytab.
Entry for principal host/iot01.examplefirm.com@EXAMPLEFIRM.COM with kvno 2, encryption type arcfour-hmac added to keytab WRFILE:krb5-iot01.keytab
As you can see many different combinations of encryption+hash functions are used to store the key in the keytab file. This is done to ensure backward compatibility with older Kerberos installations. If you don't have any such backward compatibility, you should change the default encryption policy to retain only the most secure algorithms and key-sizes.
Transfer this file securely to the device iot01 and secure file by setting strict file permissions to 0700 and make it only root accessible:
scp krb5-iot01.keytab anotheruser@iot01.examplefirm.com:/etc/krb5.keytab
sudo chown root:root /etc/krb5.keytab
sudo chmod 0700 /etc/krb5.keytab
Verify this principal has been created properly by testing initializing the ticket for this principal using kinit:
sudo kinit -k -t /etc/krb5.keytab -p host/iot01.examplefirm.com@EXAMPLEFIRM.COM
View the kerberos ticket issued to this principal using klist:
sudo klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: host/iot01.examplefirm.com@EXAMPLEFIRM.COM
Valid starting Expires Service principal
17/09/2016 08:39 18/09/2016 08:39 krbtgt/EXAMPLEFIRM.COM@EXAMPLEFIRM.COM
Similarly, create Kerberos principals for other IoT's services too.
Kerberize the SSH Login
At the IoT device, edit the SSH daemon configuration file (usually /etc/ssh/sshd_config) to verify that the following options are enabled:
KerberosAuthentication yes
KerberosOrLocalPasswd yes
KerberosTicketCleanup yes
GSSAPIAuthentication yes
GSSAPICleanupCredentials yes
Next, restart the SSH service:
sudo service sshd restart
Edit the configuration file /etc/krb5.conf to add the following:
[libdefaults]
default_realm = EXAMPLEFIRM.COM
[realms]
EXAMPLEFIRM.COM = {
kdc = iot01.examplefirm.com
admin_server = iot01.examplefirm.com
default_domain = examplefirm.com
}
[domain_realm]
.examplefirm.com = EXAMPLEFIRM.COM
In some cases, logins could generate errors due to several reasons, some of the common ones are:
- Keytab was not the correct keytab for this principal so the device cant communicate with Keberos.
- Keys generated for the device's principal got stale and hence need to be re-generated
- The device and Kerberos server have a large time difference due to clock
Investigate and troubleshoot login issues by viewing events in the log file /var/log/syslog , a sample set of log events for a successful login are as follows:
Sep 4 21:06:36 iot01 krb5kdc[4041]: AS_REQ (6 etypes {18 17 16 23 25 26}) 127.0.0.1: ISSUE: authtime 1039591196, etypes {rep=18 tkt=18 ses=18}, user01@EXAMPLEFIRM.COM for krbtgt/EXAMPLEFIRM.COM@EXAMPLEFIRM.COM
Sep 4 21:06:37 iot01 sshd[4239]: pam_unix(sshd:session): session opened for user user01 by (uid=0)
Sep 4 21:06:37 iot01 systemd-logind[386]: New session c5 of user user01.
Sep 4 21:06:37 iot01 systemd: pam_unix(systemd-user:session): session opened for user user01 by (uid=0)
The fastest way to pinpoint the root cause while troubleshooting login problems are to check the following:
- The KDC and Admin server are reachable from the device and their system time difference is not more than 30 seconds (conservative estimate).
- The device and user principals can be successfuly issued tickets on the kerberos server itself (using kinit as shown in the article).
- Other users can login to the same device with their Kerberos IDs without any issues.
- User id and password are valid and password has not expired
Apart from SSH, many other network services also support Kerberization, such as - NFS, HTTP/HTTPS and RDBMS databases like Oracle database, PostgreSQL, etc. To deploy Kerberos for each service, an ipservice object needs to be created under the LDAP object of the device, and then a Kerberos principal created for this service.
Conclusion
This concludes the configuration and deployment of a working Kerberos authentication service on an IoT device for other IoT devices.
For production deployments, the following additional configuration tasks and environment setup are recommended:
- Setup LDAP on TLS (StartTLS) to enforce secure communication, change Kerberos to use ldaps instead of plain un-encrypted ldap.
- Setup replication of the LDAP server to another device for redundancy and business continuity,
- Setup secondary or more Kerberos servers on other devices configured as failovers.
- Use graphical tools such as Jxplorer or Apache DS studio to manage the LDAP directory, this makes it more convenient to administer large deployments.
- Automate user management using scripts triggered by the firm's processes of leavers/joiners.
- Synchronize all clocks on devices using an NTP service; its required since timestamps form an important role in the secure messages exchanged between the parties in the Kerberos protocol. The protocol will only tolerate a small time difference amongst the devices and the Kerberos server.
- The OS of each device hosting the kerberos and ldap servers must be hardened to secure it as well as to remove un-necessary services such as X-windows, etc.
- Operate all IoT devices with good quality reliable power supplies (with high power capacity so they remain cool when running) and high quality storage cards to avoid media corruption issues.
Appendix 1 - Managing User Principals
Within the kadmin.local prompt, use the following command to change the password for a user:
cpw user01
Use this command to change details of a principal, such as marking an account inactive, change names/emails/etc.:
modprinc user01
View Kerberos related details of a user:
getprinc user02
Principal: user02@EXAMPLEFIRM.COM
Expiration date: [never]
Last password change: Sun Sep 04 01:07:56 IST 2016
Password expiration date: [none]
Maximum ticket life: 1 day 00:00:00
Maximum renewable life: 0 days 00:00:00
Last modified: Sun Sep 04 01:07:56 IST 2016 (root/admin@EXAMPLEFIRM.COM)
Last successful authentication: [never]
Last failed authentication: [never]
Failed password attempts: 0
Number of keys: 4
Key: vno 1, aes256-cts-hmac-sha1-96, no salt
Key: vno 1, aes128-cts-hmac-sha1-96, no salt
Key: vno 1, des3-cbc-sha1, no salt
Key: vno 1, arcfour-hmac, no salt
MKey: vno 1
Attributes:
Policy: default
Users can reset their kerberos password at any time using the kpasswd utility. Since the password policy enforces password expiry after 30 days, and users don't get automated reminders every few days to reset their password as the password deadline approaches, it's recommended to setup alternate mechanisms to monitor password expiry of users and alert them in advance.
Points of Interest
Kerberos services can be configured to use many different back-ends as a data store. In this article, an LDAP has been used as the persistent storage for Kerberos keys and data.
History
- Original Version (30-Sep-2016)
- Updated with spelling corrections and sytax error corrections (3-Oct-2016)
- Updated to fix missing diagram and provide more detailed information on troubleshooting (4-Oct-2016)
