Automated Kerberos Installation and Configuration

Automated Kerberos Installation and Configuration – For this post, I have written a shell script which uses Ambari APIs to configure Kerberos on HDP Single or Multinode clusters. You just need to clone our github repository and modify property file according to your cluster environment, execute setup script and phew!! Within 5-10 minutes you should have your cluster completely secured by Kerberos! Cool isn’t it? :)

 

Detailed Steps(Demo on HDP Sandbox 2.4):

 

1. Clone our github repository on your local machine or one of the node in your Hadoop Cluster.

git clone https://github.com/crazyadmins/useful-scripts.git

Sample Output:

[root@sandbox ~]# git clone https://github.com/crazyadmins/useful-scripts.git
Initialized empty Git repository in /root/useful-scripts/.git/
remote: Counting objects: 29, done.
remote: Compressing objects: 100% (25/25), done.
remote: Total 29 (delta 4), reused 25 (delta 3), pack-reused 0
Unpacking objects: 100% (29/29), done.

 

2. Goto useful-scripts/ambari directory

[root@sandbox ~]# cd useful-scripts/ambari/
[root@sandbox ambari]# ls -lrt
total 16
-rw-r--r-- 1 root root 5701 2016-04-23 20:33 setup_kerberos.sh
-rw-r--r-- 1 root root 748 2016-04-23 20:33 README
-rw-r--r-- 1 root root 366 2016-04-23 20:33 ambari.props
[root@sandbox ambari]#

 

3. Copy setup_kerberos.sh and ambari.props to the host where you want to setup KDC Server

 

4. Edit and modify ambari.props file according to your cluster environment

Sample output for my Sandbox

[root@sandbox ambari]# cat ambari.props
CLUSTER_NAME=Sandbox
AMBARI_ADMIN_USER=admin
AMBARI_ADMIN_PASSWORD=admin
AMBARI_HOST=sandbox.hortonworks.com
KDC_HOST=sandbox.hortonworks.com
REALM=HWX.COM
KERBEROS_CLIENTS=sandbox.hortonworks.com
##### Notes #####
#1. KERBEROS_CLIENTS - Comma separated list of Kerberos clients in case of multinode cluster
#2. Admin princial is admin/admin and password is hadoop
[root@sandbox ambari]#

 

5. Start installation by simply executing setup_kerberos.sh

Notes:

1. Please run setup_kerberos.sh from KDC_HOST only, you don’t need to setup or configure KDC, this script will do everything for you.

2. If you are running script on Sandbox then please turn OFF maintenance mode for HDFS and turn ON maintenance mode for Zepplin Notebook before executing the script.

sh setup_kerberos.sh

 

Screenshots:

1. Before Script Execution
Automated Kerberos Installation and Configuration

 

 

2. Script execution is in progress

Automated Kerberos Installation and Configuration

 

3. Script finished

Automated Kerberos Installation and Configuration

 

 

4. Ambari UI shows Kerberos is enabled.

Automated Kerberos Installation and Configuration

 

 

Please comment if you have any feedback/questions/suggestions. Happy Hadooping!! :)

 

facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

3 comments

  • avimanyu

    I run your scripts as you mentioned in README successfully. But the kerberos is not enabled and keytab files are also not created. I am using Ambari 2.1.2.1 and HDP 2.3. Below are the logs:

    kerberos.props Kerb_setup.log payload setup_kerberos.sh
    [root@hdp1 scripts]# cat Kerb_setup.log

    2016-04-24,21:16:29 Installing kerberos RPMs
    Loaded plugins: fastestmirror
    Loading mirror speeds from cached hostfile
    * base: http://ftp.iitm.ac.in
    * extras: http://ftp.iitm.ac.in
    * updates: http://ftp.iitm.ac.in
    Setting up Install Process
    Package krb5-server-1.10.3-42z1.el6_7.x86_64 already installed and latest version
    Package krb5-libs-1.10.3-42z1.el6_7.x86_64 already installed and latest version
    Package krb5-workstation-1.10.3-42z1.el6_7.x86_64 already installed and latest version
    Nothing to do

    2016-04-24,21:16:35 Configuring Kerberos
    Loading random data
    Initializing database ‘/var/kerberos/krb5kdc/principal’ for realm ‘ARGOTECH.COM’,
    master key name ‘K/M@ARGOTECH.COM’

    2016-04-24,21:18:10 Starting KDC services
    Starting Kerberos 5 KDC: [ OK ]
    Starting Kerberos 5 Admin Server: [ OK ]

    2016-04-24,21:18:10 Creating admin principal
    Authenticating as principal root/admin@ARGOTECH.COM with password.
    Principal “admin/admin@ARGOTECH.COM” created.

    2016-04-24,21:18:11 Restarting kadmin
    Stopping Kerberos 5 Admin Server: [ OK ]
    Starting Kerberos 5 Admin Server: [ OK ]

    2016-04-24,21:18:14 Adding KERBEROS Service to cluster
    HTTP/1.1 201 Created
    User: admin
    Set-Cookie: AMBARISESSIONID=15p6n2gg03h781t4pdmtgnwx7s;Path=/;HttpOnly
    Expires: Thu, 01 Jan 1970 00:00:00 GMT
    Content-Type: text/plain
    Content-Length: 0
    Server: Jetty(8.1.17.v20150415)

    2016-04-24,21:18:14 Adding KERBEROS_CLIENT component to the KERBEROS service
    HTTP/1.1 201 Created
    User: admin
    Set-Cookie: AMBARISESSIONID=1luaj6ebadcom1ejqd9k5ul7vd;Path=/;HttpOnly
    Expires: Thu, 01 Jan 1970 00:00:00 GMT
    Content-Type: text/plain
    Content-Length: 0
    Server: Jetty(8.1.17.v20150415)

    HTTP/1.1 100 Continue

    HTTP/1.1 200 OK
    User: admin
    Set-Cookie: AMBARISESSIONID=14wv8tphqrzrxryhijqcfr0d6;Path=/;HttpOnly
    Expires: Thu, 01 Jan 1970 00:00:00 GMT
    Content-Type: text/plain
    Vary: Accept-Encoding, User-Agent
    Content-Length: 1640
    Server: Jetty(8.1.17.v20150415)

    {
    “resources” : [
    {
    “href” : “http://hdp1.argotech.com:8080/api/v1/clusters/argotech/configurations/service_config_versions?service_name=KERBEROS&service_config_version=2″,
    “configurations” : [
    {
    “clusterName” : “argotech”,
    “stackId” : {
    “stackName” : “HDP”,
    “stackVersion” : “2.3”,
    “stackId” : “HDP-2.3″
    },
    “type” : “kerberos-env”,
    “versionTag” : “version1″,
    “version” : 1,
    “serviceConfigVersions” : null,
    “configs” : {
    “kdc_type” : “mit-kdc”,
    “password_min_uppercase_letters” : “1”,
    “password_min_whitespace” : “0”,
    “password_min_punctuation” : “1”,
    “password_min_digits” : “1”,
    “encryption_types” : “aes des3-cbc-sha1 rc4 des-cbc-md5″,
    “admin_server_host” : “hdp1.argotech.com.com”,
    “password_min_lowercase_letters” : “1”,
    “password_length” : “20”,
    “case_insensitive_username_rules” : “false”,
    “manage_identities” : “true”,
    “service_check_principal_name” : “-“,
    “kdc_host” : “hdp1.argotech.com.com”,
    “install_packages” : “true”,
    “realm” : “ARGOTECH.COM”,
    “executable_search_paths” : “/usr/bin, /usr/kerberos/bin, /usr/sbin, /usr/lib/mit/bin, /usr/lib/mit/sbin”
    },
    “configAttributes” : { }
    }
    ],
    “group_id” : -1,
    “group_name” : null,
    “service_config_version” : 2,
    “service_config_version_note” : null,
    “service_name” : “KERBEROS”
    }
    ]
    }
    2016-04-24,21:18:17 Creating the KERBEROS_CLIENT host components for each host
    HTTP/1.1 201 Created
    User: admin
    Set-Cookie: AMBARISESSIONID=1p4wixclhn97fe64vv00o8yjr;Path=/;HttpOnly
    Expires: Thu, 01 Jan 1970 00:00:00 GMT
    Content-Type: text/plain
    Content-Length: 0
    Server: Jetty(8.1.17.v20150415)

    2016-04-24,21:18:18 Installing the KERBEROS service and components
    HTTP/1.1 202 Accepted
    User: admin
    Set-Cookie: AMBARISESSIONID=1ag9cj7bwkkkr9ll6byw7r5f;Path=/;HttpOnly
    Expires: Thu, 01 Jan 1970 00:00:00 GMT
    Content-Type: text/plain
    Vary: Accept-Encoding, User-Agent
    Content-Length: 146
    Server: Jetty(8.1.17.v20150415)

    {
    “href” : “http://hdp1.argotech.com:8080/api/v1/clusters/argotech/requests/61″,
    “Requests” : {
    “id” : 61,
    “status” : “Accepted”
    }
    }
    2016-04-24,21:18:18 Sleeping for 1 minute

    2016-04-24,21:19:18 Stopping all the services
    HTTP/1.1 200 OK
    User: admin
    Set-Cookie: AMBARISESSIONID=g3w5b7n1d8zpr3uiy9y0rvl7;Path=/;HttpOnly
    Expires: Thu, 01 Jan 1970 00:00:00 GMT
    Content-Type: text/plain
    Content-Length: 0
    Server: Jetty(8.1.17.v20150415)

    2016-04-24,21:19:18 Sleeping for 2 minutes

    2016-04-24,21:21:18 Enabling Kerberos
    HTTP/1.1 500 Internal Server Error
    User: admin
    Set-Cookie: AMBARISESSIONID=llojdxrepoyrjxakmmnudsaf;Path=/;HttpOnly
    Expires: Thu, 01 Jan 1970 00:00:00 GMT
    Content-Type: text/plain
    Content-Length: 189
    Server: Jetty(8.1.17.v20150415)

    {
    “status” : 500,
    “message” : “org.apache.ambari.server.controller.spi.SystemException: An internal system exception occurred: Unexpected error condition executing the kadmin command”
    }
    2016-04-24,21:23:26 Starting all services
    HTTP/1.1 202 Accepted
    User: admin
    Set-Cookie: AMBARISESSIONID=fzz2064m9k4j1gdusmy3rjpc2;Path=/;HttpOnly
    Expires: Thu, 01 Jan 1970 00:00:00 GMT
    Content-Type: text/plain
    Vary: Accept-Encoding, User-Agent
    Content-Length: 146
    Server: Jetty(8.1.17.v20150415)

    {
    “href” : “http://hdp1.argotech.com:8080/api/v1/clusters/argotech/requests/63″,
    “Requests” : {
    “id” : 63,
    “status” : “Accepted”
    }
    }
    2016-04-24,21:23:26 Please check Ambari UI
    Thank You! :)

  • Kuldeep Kulkarni

    Hi Avimanyu,

    Looks like below curl call did not work, can you please try it manually and let me know how it goes?

    Error in your log:

    {
    “status” : 500,
    “message” : “org.apache.ambari.server.controller.spi.SystemException: An internal system exception occurred: Unexpected error condition executing the kadmin command”
    }

    This error is because below curl did not complete successfully:

    curl -H “X-Requested-By:ambari” -u $AMBARI_ADMIN_USER:$AMBARI_ADMIN_PASSWORD -i -X PUT -d @$LOC/payload http://$AMBARI_HOST:8080/api/v1/clusters/$CLUSTER_NAME

    Payload for above command is:

    {
    “session_attributes” : {
    “kerberos_admin” : {
    “principal” : “admin/admin”,
    “password” : “hadoop”
    }
    },
    “Clusters”: {
    “security_type” : “KERBEROS”
    }
    }

  • Sathish

    Hi,

    Is there any possibility to configure HDFS Federation with High Availability (Active/Standby). If it is, could you please post the steps.

    Thanks in advance.

    Regards,
    Sathish G.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>