Oozie Tutorials – SSH Action
Oozie Tutorials – SSH Action
Oozie ssh action executes shell script on remote machine in secure shell, workflow will wait until ssh script is complete and then move to next action.
Prerequisites:
1. Shell script must be present on remote host at given path.
2. Shell script will be executed in home directory of specified use on remote host.
3. Password-less ssh must be set from oozie@<oozie-host> to user@<remote-host>
To Configure passwordless ssh:
3.1 Login to Oozie server host via oozie user
3.2 Run below command ( press enter for every question to keep all the options default )
ssh-keygen
3.3 On Oozie host copy contents of ~/.ssh/id_rsa.pub and paste it in <remote-host>’s ~/.ssh/authorized_keys file
3.4 Please make sure to have below permissions in order to get password-less ssh working successfully
700 to ~/.ssh directory on oozie as well as remote-host 600 to ~/.ssh/authorized_keys file on remote-host 600 to ~/.ssh/id_rsa on oozie host 644 on ~/.ssh/id_rsa.pub on oozie host
3.5 Test ssh from oozie@<oozie-host> to <username>@<remote-host>, ssh command should work without password.
Steps:
1. Configure job.properties for your cluster environment
Example:
#************************************************* # job.properties #************************************************* nameNode=hdfs://<namenode-machine-fqdn>:8020 jobTracker=<resource-manager-fqdn>:8050 queueName=default oozie.libpath=${nameNode}/user/oozie/share/lib oozie.use.system.libpath=true oozie.wf.rerun.failnodes=true oozieProjectRoot=${nameNode}/user/${user.name} appPath=${oozieProjectRoot} oozie.wf.application.path=${appPath} inputDir=${oozieProjectRoot} remoteNode=<username>@<remote-host-where-you-have-your-shell-script(s)> shellScriptPath=/tmp/test.sh emailToAddress=<email-id>
2. Create and configure workflow.xml which contains ssh action
Example:
<!--******************************************--> <!--workflow.xml --> <!--******************************************--> <workflow-app name="DemoSshAction" xmlns="uri:oozie:workflow:0.1"> <start to="sshAction"/> <action name="sshAction"> <ssh xmlns="uri:oozie:ssh-action:0.1"> <host>${remoteNode}</host> <command>${shellScriptPath}</command> <capture-output/> </ssh> <ok to="sendEmail"/> <error to="killAction"/> </action> <action name="sendEmail"> <email xmlns="uri:oozie:email-action:0.1"> <to>${emailToAddress}</to> <subject>Output of workflow ${wf:id()}</subject> <body>Status of the file move: ${wf:actionData('sshAction')['STATUS']}</body> </email> <ok to="end"/> <error to="end"/> </action> <kill name="killAction"> <message>"Killed job due to error"</message> </kill> <end name="end"/> </workflow-app>
3. Write sample /tmp/test.sh shell script
Example:
#!/bin/bash hadoop fs -put ~/test /user/<username>/uploadedbyoozie
4. Upload workflow.xml to the HDFS at ${appPath} defined in job.properties.
hadoop fs -put $local_path/workflow.xml /user/<user-name>/
5. Submit Oozie workflow using below command
oozie job -oozie http://<oozie-host>:11000/oozie -config $local-path/job.properties -run
Screenshots:
Note – Hit below URL to open Oozie WebUI
http://<oozie-server>:11000/oozie
1. Oozie job started.
2. Job completed successfully
Please comment if you have any feedback/questions/suggestions. Happy Hadooping!! 







