Configure Kerberos Authentication in Hortonworks Hadoop HDP 2.2

This is quick and short tutorial to install and configure Kerberos authentication in hortonworks Hadoop cluster hdp2.2.


Here is my setup environment:


Kerberos Server:

Kerberos Client:

Test Hadoop Hortonworks 2.2 Cluster:




Please ensure that Kerberos server and Client/Hadoop cluster should have each other’s entry in /etc/hosts file and they should be ping-able to each other.


Let’s get started!


Step 1: Install krb server packages on Kerberos Server


On execute below command:


yum –y install krb5-server krb5-libs krb5-auth-dialog krb5-workstation



Step 2: Edit /etc/krb5.conf and change the default REALM


Edit “/etc/krb5.conf” on



It should look like below:


[root@kerberos ~]# cat /etc/krb5.conf
default = FILE:/var/log/krb5libs.log
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log
default_realm =
dns_lookup_realm = false
dns_lookup_kdc = false
ticket_lifetime = 24h
renew_lifetime = 7d
forwardable = true
[realms] = {
kdc =
admin_server =
[domain_realm] = =  


Note – is my default realm

Step 3: Create Kerberos database


Run below command to create db on


/usr/sbin/kdb5_util create -s



Step 4: Start the Core Kerberos services


Execute below commands on


/etc/rc.d/init.d/krb5kdc start


/etc/rc.d/init.d/kadmin start



Step 5: Install and configure Kerberos Client


Use below command to install kerberos client on (Client machine)


yum install krb5-workstation


Note: Please copy modified krb5.conf obtained from step 2 to (Kerberos client and Hadoop cluster)



Step 6: Create the principals by following automated method



6.1 Go to Ambari server admin UI –> Admin –> Security –> Enable Security –> Enter your realm instead of EXAMPLE.COM (here we have used


6.2 Then Click Next –> Download CSV files containing list of nodes, principals & keytabs.


6.3 Then Go to Ambari server and execute below commands:


6.4 /var/lib/ambari-server/resources/scripts/ host-principal-keytab-list.csv >


6.5 Copy the generated to your Kerberos server. (Copy from to





6.6 Run with sudo. This creates a tar file for each node/host in your Hadoop cluster. Each tar contains the keytabs needed to be on that host.


6.7 Copy each tar file to the right host and unzip it to the root directory (it already contains the correct directory structure).


Note – Please ensure that your keytab files are there at correct location on Kerberos i.e. /etc/security/keytabs



Step 7: Please set permissions of your keytab files by running below script. 


Note – If you are using multi-node cluster then you need to run this script on each host. Please ignore errors if you get file not found.


Create (or give any favorite name to your script) on your home directory, copy all the below contents in it and run it on all the kerberos client machines.


chown root:hadoop /etc/security/keytabs
chmod 750 /etc/security/keytabs
chown ambari:ambari /etc/security/keytabs/ambari.keytab
chmod 400 /etc/security/keytabs/ambari.keytab 
chown hdfs:hadoop /etc/security/keytabs/nn.service.keytab 
chmod 400 /etc/security/keytabs/nn.service.keytab
chown root:hadoop /etc/security/keytabs/spnego.service.keytab 
chmod 440 /etc/security/keytabs/spnego.service.keytab
chown ambari-qa:hadoop /etc/security/keytabs/smokeuser.headless.keytab
chmod 440 /etc/security/keytabs/smokeuser.headless.keytab
chown hdfs:hadoop /etc/security/keytabs/hdfs.headless.keytab
chmod 440 /etc/security/keytabs/hdfs.headless.keytab
chown hbase:hadoop /etc/security/keytabs/hbase.headless.keytab
chmod 440 /etc/security/keytabs/hbase.headless.keytab
chown hdfs:hadoop /etc/security/keytabs/dn.service.keytab 
chmod 400 /etc/security/keytabs/dn.service.keytab
chown  mapred:hadoop /etc/security/keytabs/jhs.service.keytab 
chmod 400 /etc/security/keytabs/jhs.service.keytab 
chown root:hadoop /etc/security/keytabs/spnego.service.keytab 
chmod 440 /etc/security/keytabs/spnego.service.keytab
chown yarn:hadoop /etc/security/keytabs/rm.service.keytab 
chmod 400 /etc/security/keytabs/rm.service.keytab
chown yarn:hadoop /etc/security/keytabs/nm.service.keytab 
chmod 400 /etc/security/keytabs/nm.service.keytab
chown oozie:hadoop /etc/security/keytabs/oozie.service.keytab 
chmod 400 /etc/security/keytabs/oozie.service.keytab
chown root:hadoop /etc/security/keytabs/spnego.service.keytab 
chmod 440 /etc/security/keytabs/spnego.service.keytab
chown hive:hadoop /etc/security/keytabs/hive.service.keytab 
chmod 400 /etc/security/keytabs/hive.service.keytab
chown root:hadoop /etc/security/keytabs/spnego.service.keytab 
chmod 440 /etc/security/keytabs/spnego.service.keytab
chown hbase:hadoop /etc/security/keytabs/hbase.service.keytab 
chmod 400 /etc/security/keytabs/hbase.service.keytab
chown zookeeper:hadoop /etc/security/keytabs/zk.service.keytab 
chmod 400 /etc/security/keytabs/zk.service.keytab
chown nagios:nagios /etc/security/keytabs/nagios.service.keytab
chmod 400 /etc/security/keytabs/nagios.service.keytab
chown hdfs:hadoop /etc/security/keytabs/jn.service.keytab
chmod 400 /etc/security/keytabs/jn.service.keytab



Step 8: Verify that the correct keytab files and principals are associated with the correct service using the klist command. For example, on the NameNode:


klist –k -t /etc/security/keytabs/nn.service.keytab



Step 8: Click apply in Ambari server to apply the security settings.




Step 9: If zookeeper does not start then check this out (Hadoop / Ambari configuration, part 2 section)




Step 10:   Once your services are started, try running some Hadoop command by root user


[kuldeepk@myclient ~]# hadoop fs -ls /
ls: Failed on local exception: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]; Host Details : local host is: ""; destination host is: "":8020;


You got an error and Yes! It’s expected because root user does not have any valid TGT!


Step 11: Add principal for root user and get a ticket granting ticket


Run below commands on Kerberos server and remember password.


[root@kerberos ~]# kadmin.local
kadmin.local: addprinc
WARNING: no policy specified for; defaulting to no policy
Enter password for principal "":
Re-enter password for principal "":
Principal "" created.



Step12: Initiate a TGT and enjoy hadooping :-)


On Kerberos client run below command & enter password to get a TGT


[kuldeepk@myclient ~]$ kinit kuldeepk
Password for


Verify your ticket by klist command


[kuldeepk@myclient ~]$ klist
Ticket cache: FILE:/tmp/krb5cc_1003
Default principal:
Valid starting     Expires           Service principal
04/30/15 22:11:15 05/01/15 22:11:14 krbtgt/
       renew until 04/30/15 22:11:15
[kuldeepk@ myclient ~]$



Please comment below if you have any questions! Your Feedback is appreciated :-)

facebooktwittergoogle_plusredditpinterestlinkedinmailby feather


  • Hari Sekhon

    MIT KDCs are fine for very basic setups, VMs, small PoCs etc but it you have a lot of users you’ll need ID lookup information with an LDAP backend tied in as well.

    I recommend you just install FreeIPA it’s easier.

    I wrote a perl program to automated the generation of all the principals and distribution of all the keytabs to all the cluster nodes using the same CSV export file from Ambari, you can find it here:

    git clone
    cd toolbox

    ./ --help

  • Kuldeep Kulkarni

    Thats correct! For production cluster we should configure Kerberos with LDAP backend as local KDC has some limitations.

    Thanks for the very positive feedback, may be I can write next article on Kerberos configuration with LDAP backend using FreeIPA, and will use script :-)

  • movers sydney

    I must thank you for the efforts you have put in writing this site. I’m hoping to view the same high-grade blog posts by you in the future as well. In fact, your creative writing abilities has encouraged me to get my very own blog now ;)|

    • Esra

      Hi everyone:I’ve aldarey completed each step of these processes. AD Users are able to authenticate through SQUID to surf by internet, BUT!! after 2 hours -sometimes more or less- suddenly some users -one or two- couln’t surf by Internet Internet Explorer requests for new credentials (user/password).. Then AD users type them, but They aren’t able to surf by internet I checked that an AD user type user/password correctly but the prompt appears every time From cache.log these lines are recorded:: -2011/02/10 17:58:15| squid_kerb_auth: Got YR YIIGJgYGKwYBBQUCoIIGGjCCBhagMDAuBgkqhkiC9xIBAgIGCSqGSIb3EgECAgYKKwYBBAGCNwICHgYKKwYBBAGCNwICCqKCBeAEggXcYIIF2AYJKoZIhvcSAQICAQBuggXHMIIFw6ADAgEFoQMCAQ6iBwMFACAAAACjggS7YYIEtzCCBLOgAwIBBaENGwtHSUxBVExBLkNPTaIoMCagAwIBAqEfMB0bBEhUVFAbFVBFUFJPWFkwMS5HSUxBVExBLkNPTaOCBHEwggRtoAMCARKhAwIBAqKCBF8EggRb0CrGoBRyYtVTklzkviLTrUm2PaUeSEwiOnTMPXpXZpiqlXeIzc7EyCtLUu5YIjr8kfWunnManLdfPzFZzvlpwSV5XgpvYimVwwjijX+1M6ASwLQAoMuW7p9Km7ebFGenPsfD2stMiKvi+abaw0oqp3262aYsQKCcN+qDXyU7whJ2Li9tr3MeRnYEc6QLXMbB2CDThA7KkRf+HosCJD54W8Wofk3h0RKERMYpUkfugI2kfakkcZS/QVE241CMj8Y+iF6YwcQTOZ1FLikPveTuZu4wbkR2ywyI5/OYq1iU5oKRmh+iG3XIUMdv8vtJ3c74zrwfAS387D1wfPVw1mnoCDC6mg2NtQa2POrEsJV116fakrRt6dDRK8v6uKFXyytkZG5H+9R8cwaYXYMhGvMVAd46RB62PYerDon0V/UZh9A8Yhr9xfmnQekvIgcZYlD2a8c+SgwRf3EoOYTxa2XF93uzyLM7N/7uw+Ud0J0oSlTOOZ8N5awJ43ISC+xVsqX+lHlEDhpGhYjxfLK6EZoOxdJA9eoWWDMkndgsAhRnK06kuwZWLxGq9m1QjX1THqTy+CK0NcD7FVEJ7DpVVWYHEZoETpJN6QmwIZnPq5TsJcRzOJiL03VtKZiM01vtUGTIw5bXTQZMZFA16GlTK1j0jynjxTMDCMGzJ0prk7q6Z4jtvRga3EJdgRf0HmsUDG/r+/lDECkpPV3L7drYCdqJuyJxdMOfJM5sqNtGKK/qKYlilkfAhwOPqk/hLIue3My44Am7vj8IaJkExpK1DpN3fcRIFwUlwf81CxI1jBF4wTfHlRdYwXVgci1pxlA8wnlD24Ebiacwz9EwQfj2VkUrZoeAyt3/gZrhVxyDfSyvhQcjXcLbUDGdOPjKdki4BVGhIo3wGN+1ZZjGnitDusb4ONIiU5Md1g6d325smXiLWvSFtiBhbVBFB6CByXLx18JJ57DVSgxjkVO6nhTloltEAb8vpsXipgHkeOkmFAcAILqtugP2jFe5kwE/aZ5X9kZTXCiRaOqJItniATP2a3DPqSP0Bpp0mHx6xZv3ZzPy50rZ1aKOGj5rUlVW/YuuW2ULB2weyen44OMvkhceLFAsXppnlAmQUFB5xf5Xy2HjEuktOi8Ka7lfR2rbaYykVaEWE3DEHbd8iF8Ws1emBIaSLekMtc9HjOq4DWdXYTTOmxPLbIvuagmjGqffaDnLIBhjV7IG6q07sah0cTVGdylq/qkGTjyxi/F0f1Pk8uEncVLQ9Kib4lq9+h5OOykifeAtboTSf09skjpMB+G7qa3QD+YOiiqA4PpolEnVrHFCbMm69USlfcppzQ2zklQIEIFNOAzWRkktVgqy74B8I8SWsPLCWz54Y3IvdS8nQYElCPd821QDjAao9vFuYOnELLmpAG0R5sEODqb+EKzXecsXr5vxVoBn5m9TgfbI6XCxQVfADaaaA1O9yhd3JNI6IeH69cWH0mhil68HxQOkge4wgeugAwIBEqKB4wSB4NRMfqBGyTBUUvPKKu1cKscC63v+d5WPmFbyYUYdZ0EB/USQZWSnhWMvbEWFLiRYThJG6UL4op4nNGuw+GuW6PKQi+eB97o4NSGqWN2kvtHtc0YAGSmMma17xIkhUfRtd9j1k/+K6kIKCFN96hDiapeppnE3Tj5fy0MQf+atYDaKV35ai7guVPFxRZFfBvvRdkLntEPAqy4JjIL8ySeUUYaaqyYM97Z6Db43KodJceoECCRZZ6Y1SELwFcKpBoBlGj670kZH1hPKwsncy80TxHSjz1xU1hgYBDRbKjtYP0cH’ from squid (length: 2107).2011/02/10 17:58:15| squid_kerb_auth: gss_accept_sec_context() failed: Unspecified GSS failure. Minor code may provide more information. Key table entry not found -I’ve been using:+RHEL 5.0+SQUID 3.0+modules: (squid_kerb_auth1.0.7 and squid_kerb_ldap1.2.1a) from squid.confI’ve configured the option debug_options ALL,1 33,2 28,9 in order to get more records.. but all these records aren’t useful.Can somebody help?Thanks in advance.

  • DouglassPVampa

    Hi! I’m at work surfing around your blog from my new apple
    iphone! Just wanted to say I love reading your blog and look
    forward to all your posts! Carry on the fantastic work!

  • TarshaEMattingley

    you are in reality a just right webmaster. The website loading speed is amazing.
    It sort of feels that you’re doing any distinctive trick.

    Also, The contents are masterwork. you have performed a excellent activity on this subject!

  • ÿþS

    Hi, Neat post. There is a problem with your web site in internet explorer, would check this… IE still is the market leader and a huge portion of people will miss your great writing because of this problem.

  • CollenUDanehy

    Greetings from Ohio! I’m bored at work so I decided
    to browse your website on my iphone during lunch break. I enjoy the knowledge you present here and can’t wait to
    take a look when I get home. I’m surprised at how quick your
    blog loaded on my cell phone .. I’m not even using WIFI,
    just 3G .. Anyhow, superb site!

  • Visit Website

    I just want to tell you that I am just newbie to weblog and really loved you’re web page. Very likely I’m planning to bookmark your blog post . You certainly have exceptional articles and reviews. Kudos for sharing your web-site.

  • MurrayBFrancher

    I’m really inspired together with your writing abilities
    as smartly as with the structure on your weblog. Is that this a paid subject or did you customize it
    yourself? Either way keep up the nice high quality writing, it’s
    uncommon to see a great blog like this one nowadays..

  • view

    I simply want to tell you that I am beginner to weblog and seriously enjoyed your blog. Likely I’m planning to bookmark your blog post . You amazingly have amazing well written articles. Cheers for sharing your web site.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>