Hadoop 2.6 (Part 2) – Running the Mapreduce Job
This is the continuation of my previous article on “Installing Hadoop 2.6 on Ubuntu 16.04“. This article will explain how we run one of the examples given with the Hadoop binary.
Once the Hadoop installation is completed, you can run the “wordcount” example provided with the Hadoop examples in order to test a Mapreduce job. This example actually is bundled with the hadoop-examples.jar file in the distribution. (See the below steps for more details)
Step 1: Start the Hadoop Cluster, if not already started.
$ /usr/local/hadoop/sbin/start-dfs.sh $ /usr/local/hadoop/sbin/start-yarn.sh
Step 2: Copy the text files that you are going to consider for a “wordcount” to a local folder (/home/hadoop/textfiles)
Step 3: Copy the text files (in the local folder) to HDFS.
$ echo "Word Count Text File" > textFile.txt $ hdfs dfs -mkdir -p /user/hduser/dfs $ hadoop dfs -copyFromLocal textFile.txt /user/hduser/dfs
$ hadoop dfs -ls /user/hduser/dfs
$ cd /usr/local/hadoop $ hadoop jar ./share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.1.jar wordcount /user/hduser/dfs /user/hduser/dfs-output
You can either choose the command line or the web interface to display the contents of the HDFS directories. If you choose the command line you can try the following command.
$ hadoop dfs -ls /user/hduser/dfs-output
Comments are closed.