Oozie Tutorials – SSH Action
Oozie Tutorials – SSH Action
Oozie ssh action executes shell script on remote machine in secure shell, workflow will wait until ssh script is complete and then move to next action.
Prerequisites:
1. Shell script must be present on remote host at given path.
2. Shell script will be executed in home directory of specified use on remote host.
3. Password-less ssh must be set from oozie@<oozie-host> to user@<remote-host>
To Configure passwordless ssh:
3.1 Login to Oozie server host via oozie user
3.2 Run below command ( press enter for every question to keep all the options default )
ssh-keygen
3.3 On Oozie host copy contents of ~/.ssh/id_rsa.pub and paste it in <remote-host>’s ~/.ssh/authorized_keys file
3.4 Please make sure to have below permissions in order to get password-less ssh working successfully
700 to ~/.ssh directory on oozie as well as remote-host 600 to ~/.ssh/authorized_keys file on remote-host 600 to ~/.ssh/id_rsa on oozie host 644 on ~/.ssh/id_rsa.pub on oozie host
3.5 Test ssh from oozie@<oozie-host> to <username>@<remote-host>, ssh command should work without password.
Steps:
1. Configure job.properties for your cluster environment
Example:
#*************************************************
# job.properties
#*************************************************
nameNode=hdfs://<namenode-machine-fqdn>:8020
jobTracker=<resource-manager-fqdn>:8050
queueName=default
oozie.libpath=${nameNode}/user/oozie/share/lib
oozie.use.system.libpath=true
oozie.wf.rerun.failnodes=true
oozieProjectRoot=${nameNode}/user/${user.name}
appPath=${oozieProjectRoot}
oozie.wf.application.path=${appPath}
inputDir=${oozieProjectRoot}
remoteNode=<username>@<remote-host-where-you-have-your-shell-script(s)>
shellScriptPath=/tmp/test.sh
emailToAddress=<email-id>
2. Create and configure workflow.xml which contains ssh action
Example:
<!--******************************************-->
<!--workflow.xml -->
<!--******************************************-->
<workflow-app name="DemoSshAction" xmlns="uri:oozie:workflow:0.1">
<start to="sshAction"/>
<action name="sshAction">
<ssh xmlns="uri:oozie:ssh-action:0.1">
<host>${remoteNode}</host>
<command>${shellScriptPath}</command>
<capture-output/>
</ssh>
<ok to="sendEmail"/>
<error to="killAction"/>
</action>
<action name="sendEmail">
<email xmlns="uri:oozie:email-action:0.1">
<to>${emailToAddress}</to>
<subject>Output of workflow ${wf:id()}</subject>
<body>Status of the file move: ${wf:actionData('sshAction')['STATUS']}</body>
</email>
<ok to="end"/>
<error to="end"/>
</action>
<kill name="killAction">
<message>"Killed job due to error"</message>
</kill>
<end name="end"/>
</workflow-app>
3. Write sample /tmp/test.sh shell script
Example:
#!/bin/bash hadoop fs -put ~/test /user/<username>/uploadedbyoozie
4. Upload workflow.xml to the HDFS at ${appPath} defined in job.properties.
hadoop fs -put $local_path/workflow.xml /user/<user-name>/
5. Submit Oozie workflow using below command
oozie job -oozie http://<oozie-host>:11000/oozie -config $local-path/job.properties -run
Screenshots:
Note – Hit below URL to open Oozie WebUI
http://<oozie-server>:11000/oozie
1. Oozie job started.
2. Job completed successfully
Please comment if you have any feedback/questions/suggestions. Happy Hadooping!! 
by 
Loading ...