Command line execution for Hadoop and ecosystem such as Pig, Sqoop, HBase and Oozie requires Kerberos ticket to work on a secure cluster.
Kerberos is a network authentication protocol. It is designed to provide strong authentication for client/server applications on the basis of 'tickets' to allow nodes communicating over a non-secure network to prove their identity to one another in a secure manner.
To get Kerberos ticket, you need to have Kerberos configuration file krb5.ini to be located in Windows directory like C:\Windows\krb5.ini in the machine where you want to work in command line.
Now update realm and KDC information in krb5.ini file.
Realm is windows server domain name and KDC is windows server hostname. You can add more realm in krb5.ini file but default realm must be one. Below is the template for krb5.ini file.
Where in the above template REALM1 and REALM2 are Windows Server DOMAIN NAME in upper case and KDC1 and KDC2 are respective Windows Server host name in lower case.
Get user ticket and submit MR Job:
Once Kerberos configuration file created in C:\Windows\krb5.ini, follow below steps to get Kerberos ticket to submit jobs.
You can open Big Data Command Prompt shipped with the Syncfusion Big data platform which has the environment variables preset.
hadoop jar %HADOOP_HOME%\share\hadoop\mapreduce\hadoop-mapreduce-examples-*.jar pi 16 100000
|Article ID:||Published Date:||Last Revised Date:||Platform:||Control:|
|7065||08/29/2016||08/29/2016||Big Data Platform||General|