Just another WordPress.com site
Trying to Complie Hadoop Sample Code with Eclipse
January 15, 2014Posted by on
I am having trouble building and running the sample code from Hadoop – The Definitive Guide. The problem is that I am not familiar with how to set the CLASSPATH for the Java compiler. Found this webcast from Cloudera:
So, I’ve created a project and add the downloaded sample code. Then I click on the Properties of the project:
Click on “Add External Jars…”
Browse to /user/lib/hadoop/lib and select all the jar files
Add more jar files…
After that, all the jar files show up in my project folder, making it really cluttered.
At this point I gave up and call it a night.
The next night, after being able to compile the WordCount example from Cloudera in command line, it gave me an idea – what if I add the hadoop_mapreduce library in this project as well? Turned out it works. The ~ signs under the import statement initially stayed, then disappeared. Oh, I also need to switch to J2SE-1.5 JRE library. Eclipse asked me if I want to switch so I am guessing the sample code is using the old JRE calls. Now, all I’ve left is the warning about Job() is deprecated.
Job job = new Job();
At this point, I am going to table it and learn from Cloudera or some other places such as Horton Works.
In doing my research, I’ve found out that you can use Eclipse to run Hadoop code in its IDE. This posting: https://github.com/itsamiths/hadoop-eclipse-plugins let you download the Eclipse Hadoop plug-in, or build your own plug in. I think I am going to try it and test it out with WordCount sample code.