Tag : apache-ranger

How to enable Ranger Admin High Availability

It is useful to enable Ranger Admin High availability, since, it would help in having access to Policy Manager even if one of the Ranger Admin is down. This document provides steps to enable Ranger Admin HA [High Availability] using an example. To configure Ranger Admin HA,  it is also required to configure Load Balancing in Linux.

 

Example Setup Details

 
Component Hostname IP Address
Ambari Server hiveenv.hwxblr.com 10.0.1.26
Existing Ranger Admin Node hn1.hwxblr.com 10.0.1.27
New Ranger Admin Node hn2.hwxblr.com 10.0.1.29
Load Balancer Node hn3.hwxblr.com 10.0.1.28

 

Version details for the Example

# hadoop version
Hadoop 2.7.1.2.3.2.0-2950
Subversion git@github.com:hortonworks/hadoop.git -r 5cc60e0003e33aa98205f18bccaeaf36cb193c1c
Compiled by jenkins on 2015-09-30T18:08Z
Compiled with protoc 2.5.0
From source with checksum 69a3bf8c667267c2c252a54fbbf23d
This command was run using /usr/hdp/2.3.2.0-2950/hadoop/lib/hadoop-common-2.7.1.2.3.2.0-2950.jar

# uname -a
Linux hn1.hwxblr.com 3.10.0-327.13.1.el7.x86_64 #1 SMP Thu Mar 31 16:04:38 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

# cat /etc/redhat-release
CentOS release 6.7 (Final)

Pre-requisites

1. If this is a fresh installation of Ranger, follow  Installing Ranger Over Ambari and complete Ranger Service installation with one Ranger Admin, for example, on node hn1.hwxblr.com

2. If this is an existing installation, Cluster should have only one Ranger Admin component installed. If there are more than one installed, they need to be removed using Ambari API’s. 

3. Complete Load balancing configuration for Linux as described in the below article . In the following example, load balancer is installed in hn3.hwxblr.com at port number 6080.

4. The load balancer node should not have any Ranger components installed.

 

Load balancing configuration for Linux

HAProxy is not available by default in CentOS and RedHat.  To enable HAProxy in the Linux system, following steps could be done:

1. HAProxy is installed by EPEL RPM. Add the following RPM’s to repository: 

CentOS/RHEL 5 , 32 bit: 
# rpm -Uvh http://dl.fedoraproject.org/pub/epel/5/i386/epel-release-5-4.noarch.rpm 

CentOS/RHEL 5 , 64 bit: 
# rpm -Uvh http://dl.fedoraproject.org/pub/epel/5/x86_64/epel-release-5-4.noarch.rpm 

CentOS/RHEL 6 , 32 bit: 
# rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm 

CentOS/RHEL 6 , 64 bit: 
# rpm -Uvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm

2. Install HAProxy

# yum install haproxy

3. Starting the HAProxy server: 

# service haproxy start

4. To make the HAProxy service persist through reboots

# chkconfig haproxy on

5. To reload the Service post changes to haproxy.cfg

# service haproxy reload

​6. Configure haproxy.cfg in the Load Balancing Node haproxy.cfg should have the entries to resolve load balancer node to the Ranger Admin nodes. Consider for example the configurations in the Environment table above, the entries in /etc/haproxy/haproxy.cfg would be as follows:

#---------------------------------------------------------------------
# round robin balancing between Ranger HA
#---------------------------------------------------------------------

frontend  haproxy
    bind 10.0.1.28:6080
    reqadd X-Forwarded-Proto:\ http
    default_backend ranger_ha


backend ranger_ha 10.0.1.28:6080
    balance     roundrobin
    mode http
    stats enable
    stats hide-version
    stats uri /stats
    stats realm Haproxy\ Statistics
    stats auth haproxy:redhat
option httpchk
option httpclose
option forwardfor
cookie LB insert
   server  hn1.hwxblr.com 10.0.1.27:6080 cookie A check
    server  hn2.hwxblr.com 10.0.1.29:6080 check

 

Configuring Ranger Admin HA

1. Use Ambari Ranger-configs to update the Policy Manager external URL to point to the load balancer URL in the Ranger Settings. This would update all Ranger Admin clients (Ranger UserSync and Ranger plug-ins). For example,

Screen Shot 2016-06-21 at 11.22.32 AM

2. Save the above and Restart Ranger services as suggested by Ambari.
3. In Ambari Ranger Services, click on Service Actions and choose Enable Ranger Admin HA.
4. In the Wizard provide the Load balancer external URL, for example, http://hn3.hwxblr.com:6080

Screen Shot 2016-06-27 at 12.51.08 PM

 

 

 

 

 

 

 

 

 

5. Select additional Ranger Admin, for example hn2.hwxblr.com.

Screen Shot 2016-06-07 at 3.19.22 PM

6. Proceed with Installation.

Screen Shot 2016-06-07 at 3.28.05 PM

 

7. Once the installation completes, the two Ranger Admin’s would be displayed in Ambari Ranger-Server page and Service Action would have Enable Ranger Admin HA greyed out.

Screen Shot 2016-06-21 at 11.26.16 AM

 

 

 

 

 

 

 

 

 

 

8. To access the Ranger console, invoke the Load Balancer URL:Port, for example, http://hn3.hwxblr.com:6080

Screen Shot 2016-06-22 at 10.37.27 AM

 

Test Ranger Admin HA

As per the configuration above, the load balancer would look up the two Ranger Admin nodes configured in a round robin fashion and resolve to the one which is alive.
To test, bring down one of the two Ranger Admin nodes alternatively and invoke the Load Balancer URL, for example, bring down node hn1.hwxblr.com and attempt to view the Policy Manager Console using the Load balancer URL:  http://hn3.hwxblr.com:6080 

 

REFERENCE
Configuring Ranger Configuration High Availability

 

facebooktwittergoogle_plusredditpinterestlinkedinmailby feather

Apache Ranger installation and Configuration in HDP2.2

Apache Ranger installation and Configuration in HDP2.2

 

In this tutorial I am going to cover how to install and configure Ranger on hortonworks hadoop platform 2.2.

 

What is Ranger?

 

It provides central security policy administration in a Hadoop environment. It covers 3 aspects:

 

Authentication : by the Apache Knox Gateway via the HTTP/REST API

Authorization : Fine-grained access control provides flexibility in defining policies on:

  1. folder and file level, via HDFS
  2. database, table and column level, via Hive
  3. table, column family and column level, via HBase

 

Audit          : Controls access into the system via extensive user access auditing in HDFS, Hive and HBase

 

Installation and Configuration:

 

Let us first see what are the available Ranger packages (optional)

Note – plugins below with orange colour are currently available for ranger.

[root@hdpcm ~]# yum search ranger

Loaded plugins: fastestmirror, priorities, security

Loading mirror speeds from cached hostfile

* base: centos.bytenet.in

* extras: centos.bytenet.in

* updates: centos.bytenet.in

================================================================= N/S Matched: ranger =================================================================

ranger.noarch : ranger HDP virtual package

ranger-admin.noarch : ranger-admin HDP virtual package

ranger-debuginfo.noarch : ranger-debuginfo HDP virtual package

ranger-hbase-plugin.noarch : ranger-hbase-plugin HDP virtual package

ranger-hdfs-plugin.noarch : ranger-hdfs-plugin HDP virtual package

ranger-hive-plugin.noarch : ranger-hive-plugin HDP virtual package

ranger-knox-plugin.noarch : ranger-knox-plugin HDP virtual package

ranger-storm-plugin.noarch : ranger-storm-plugin HDP virtual package

ranger-usersync.noarch : ranger-usersync HDP virtual package

ranger_2_2_0_0_2041-admin.x86_64 : Web Interface for Ranger

ranger_2_2_0_0_2041-debuginfo.x86_64 : Debug information for package ranger_2_2_0_0_2041

ranger_2_2_0_0_2041-hbase-plugin.x86_64 : ranger plugin for hbase

ranger_2_2_0_0_2041-hdfs-plugin.x86_64 : ranger plugin for hdfs

ranger_2_2_0_0_2041-hive-plugin.x86_64 : ranger plugin for hive

ranger_2_2_0_0_2041-knox-plugin.x86_64 : ranger plugin for knox

ranger_2_2_0_0_2041-storm-plugin.x86_64 : ranger plugin for storm

ranger_2_2_0_0_2041-usersync.x86_64 : Synchronize User/Group information from Corporate LD/AD or Unix

 

Name and summary matches only, use “search all” for everything.

 

Now let us start –

Step 1: Go ahead and install Ranger

  1. yum install ranger-admin
  2. yum install ranger-usersync
  3. yum install ranger-hdfs-plugin
  4. yum install ranger-hive-plugin
  5. set JAVA_HOME

 

export JAVA_HOME=/usr/jdk64/jdk1.7.0_67 (substitute this with jdk path on your system)

echo “export JAVA_HOME=/usr/jdk64/jdk1.7.0_67″ >> ~/.bashrc

 

Step2: Set up the ranger admin UI

 

We need to run the setup script present at “/usr/hdp/current/ranger-admin” location. It will –

 

  1. add ranger user and group.
  2. set up ranger DB (Please ensure you know your MySQL root password since it will ask for it while setting up the ranger DB)
  3. create rangeradmin and rangerlogger MySQL users with appropriate grants.

 

Besides MySQL root password, whenever it prompts for password for setting up ranger and audit DB, please enter ‘hortonworks’ or anything else you wish. Just remember it for future use.

 

[root@hdpcm ranger-admin]# pwd

/usr/hdp/current/ranger-admin

 

[root@hdpcm ranger-admin]# ./setup.sh

[2015/03/31 15:58:41]:   ——— Running XASecure PolicyManager Web Application Install Script ———

[2015/03/31 15:58:41]: [I] uname=Linux

[2015/03/31 15:58:41]: [I] hostname=hdpcm.dm.com

[2015/03/31 15:58:41]: [I] DB_FLAVOR=MYSQL

~

~

~

Installation of XASecure PolicyManager Web Application is completed.

 

Step 3: Start ranger-admin service

 

[root@hdpcm ews]# pwd

/usr/hdp/current/ranger-admin/ews

 

[root@hdpcm ews]# sh start-ranger-admin.sh

Apache Ranger Admin has started

[root@hdpcm ews]#

 

Logs available at : /usr/hdp/current/ranger-admin/ews/logs

 

Step 4: Setup up ranger-usersync

By default it will sync UNIX users to the Ranger UI. You can also sync it with LDAP. This article syncs UNIX users.

 

  1. Edit /usr/hdp/current/ranger-usersync/install.properties file.
  2. Update “POLICY_MGR_URL” to point to your ranger host:

POLICY_MGR_URL = http://<IP of your Ranger host>:6080

 

Now run /usr/hdp/current/ranger-usersync/setup.sh

 

Step 5: Start the ranger-usersync service

 

[root@hdpcm ranger-usersync]# pwd

/usr/hdp/current/ranger-usersync

 

[root@hdpcm ranger-usersync]# sh start.sh

Starting UnixAuthenticationService

UnixAuthenticationService has started successfully.

 

Congratulations!! You have installed and configured Ranger successfully :)

 

Now Login to the Ranger Web UI by hitting below URL:

http://<ranger-host>:6080

 

Default password for admin user is “admin”. Once you login you can change this admin password via profile settings

 

1

 

Once you log in successfully, you will see below page:

 

2

 

In next article, I will discuss more about setting up policies for HDFS/Hive etc. via Ranger. Stay tuned for more updates! :-)

 

Please feel free to comment or email me if you have any questions or doubts.

facebooktwittergoogle_plusredditpinterestlinkedinmailby feather