I have made an EC2 key pair and saved it to a location under my home directory on mac. Also I have changed permissions with 'chmod 600 /path/to/saved/keypair/file.pem'. Now I have followed the following instructions to run pig jobs on EC2:
To set up and connect to a pig cluster, perform the following steps:
- Go to http://console.aws.amazon.com/elasticmapreduce/home signing in if necessary.
- Click the "Create Cluster".
- Enter "Data Science Assignment Cluster" or anything you wish as the Cluster Name.
- Uncheck the Logging box so that it is disabled.
- Scroll down to Software Configuration and select AMI Version 2.4.2 (We will be using Hadoop 1.x for compatibility with the version of Pig)
- Scroll down to "Security and Access" and select the Key Pair you created above.
- Scroll to the bottom and select "Create Cluster"
- On the next page, information about your cluster will be displayed. It will begin in the "Starting" state and may take several minutes to startup completely.
- Once the cluster has started, you will see a "Master Public DNS" name of the form ec2-XX-XXX-X-XXX.compute-1.amazonaws.com. Make a note of this; we will refer to it as master.public-dns-name.amazonaws.com.
- Now you are ready to connect to your cluster and run Pig jobs. From a terminal, use the following command: $ ssh -o "ServerAliveInterval 10" -i /path/to/saved/keypair/file.pem [email protected]
- Once you connect successfully, just type
$ pig- Now you should have a pig prompt
grunt>
Having executed the ssh command in step 10 I get a '>' prompt after hitting enter. Further if I write pig at the '>' prompt then I get again a '>' prompt in the next line instead of the expected 'grunt>' prompt. I have checked that ssh is installed on my mac and remote login is enabled for all users in 'sharing' settings under system preferences. What seems to be the problem here?