How to deploy HDP cluster with Kerberos authentication using Ambari Blueprint?
You are at correct place! Please follow my below article on HCC to setup single node HDP cluster using Ambari Blueprint with Kerberos Authentication(MIT KDC)
How to deploy HDP cluster with Kerberos authentication using Ambari Blueprint?
You are at correct place! Please follow my below article on HCC to setup single node HDP cluster using Ambari Blueprint with Kerberos Authentication(MIT KDC)
In previous post we have seen how to install multi node HDP cluster using Ambari Blueprints. In this post we will see how to Automate HDP installation using Ambari Blueprints to configure Namenode HA.
Below are simple steps to install HDP multinode cluster with Namenode HA using internal repository via Ambari Blueprints.
Step 1: Install Ambari server using steps mentioned under below link
Install ambari-agent package on all the nodes in the cluster and modify hostname to ambari server host(fqdn) in /etc/ambari-agent/conf/ambari-agent.ini
Step 3: Configure blueprints
Please follow below steps to create Blueprints
3.1 Create hostmapping.json file as shown below:
Note – This file will have information related to all the hosts which are part of your HDP cluster.
Note – I have kept Namenodes on prodnode1 and prodnode3, you can change it according to your requirement. I have added few more services like Hive, Falcon, Oozie etc. You can remove them or add few more according to your requirement.
Step 4: Create an internal repository map
4.1: hdp repository – copy below contents, modify base_url to add hostname/ip-address of your internal repository server and save it in repo.json file.
4.2: hdp-utils repository – copy below contents, modify base_url to add hostname/ip-address of your internal repository server and save it in hdputils-repo.json file.
Automated Kerberos Installation and Configuration – For this post, I have written a shell script which uses Ambari APIs to configure Kerberos on HDP Single or Multinode clusters. You just need to clone our github repository and modify property file according to your cluster environment, execute setup script and phew!! Within 5-10 minutes you should have your cluster completely secured by Kerberos! Cool isn’t it?
Detailed Steps(Demo on HDP Sandbox 2.4):
1. Clone our github repository on your local machine or one of the node in your Hadoop Cluster.
3. Copy setup_kerberos.sh and ambari.props to the host where you want to setup KDC Server
4. Edit and modify ambari.props file according to your cluster environment
Sample output for my Sandbox
[root@sandbox ambari]# cat ambari.props
CLUSTER_NAME=Sandbox
AMBARI_ADMIN_USER=admin
AMBARI_ADMIN_PASSWORD=admin
AMBARI_HOST=sandbox.hortonworks.com
KDC_HOST=sandbox.hortonworks.com
REALM=HWX.COM
KERBEROS_CLIENTS=sandbox.hortonworks.com
##### Notes #####
#1. KERBEROS_CLIENTS - Comma separated list of Kerberos clients in case of multinode cluster
#2. Admin princial is admin/admin and password is hadoop
[root@sandbox ambari]#
5. Start installation by simply executing setup_kerberos.sh
Notes:
1. Please run setup_kerberos.sh from KDC_HOST only, you don’t need to setup or configure KDC, this script will do everything for you.
2. If you are running script on Sandbox then please turn OFF maintenance mode for HDFS and turn ON maintenance mode for Zepplin Notebook before executing the script.
sh setup_kerberos.sh
Screenshots:
1. Before Script Execution
2. Script execution is in progress
3. Script finished
4. Ambari UI shows Kerberos is enabled.
Please comment if you have any feedback/questions/suggestions. Happy Hadooping!!
In previous post we have seen how to install single node HDP cluster using Ambari Blueprints. In this post we will see how to Automate HDP installation using Ambari Blueprints.
Below are simple steps to install HDP multinode cluster using internal repository via Ambari Blueprints.
Step 1: Install Ambari server using steps mentioned under below link
Install ambari-agent package on all the nodes in the cluster and modify hostname to ambari server host(fqdn) in /etc/ambari-agent/conf/ambari-agent.ini
Step 3: Configure blueprints
Please follow below steps to create Blueprints
3.1 Create hostmapping.json file as shown below:
Note – This file will have information related to all the hosts which are part of your HDP cluster.
4.1: hdp repository – copy below contents, modify base_url to add hostname/ip-address of your internal repository server and save it in repo.json file.
4.2: hdp-utils repository – copy below contents, modify base_url to add hostname/ip-address of your internal repository server and save it in hdputils-repo.json file.