softwaremom

Just another WordPress.com site

Trying to Complie Hadoop Sample Code with Eclipse

I am having trouble building and running the sample code from Hadoop – The Definitive Guide.  The problem is that I am not familiar with how to set the CLASSPATH for the Java compiler.  Found this webcast from Cloudera:

http://blog.cloudera.com/blog/2013/08/how-to-use-eclipse-with-mapreduce-in-clouderas-quickstart-vm/

So, I’ve created a project and add the downloaded sample code.  Then I click on the Properties of the project:

classpath1

Click on “Add External Jars…”

Browse to /user/lib/hadoop/lib and select all the jar files

.classpath2

Add more jar files…

classpath3

After that, all the jar files show up in my project folder, making it really cluttered.

classpath4

At this point I gave up and call it a night.

The next night, after being able to compile the WordCount example from Cloudera in command line, it gave me an idea – what if I add the hadoop_mapreduce library in this project as well?  Turned out it works.  The ~ signs under the import statement initially stayed, then disappeared.  Oh, I also need to switch to J2SE-1.5 JRE library.  Eclipse asked me if I want to switch so I am guessing the sample code is using the old JRE calls. Now, all I’ve left is the warning about Job() is deprecated.

Job job = new Job();

At this point, I am going to table it and learn from Cloudera or some other places such as Horton Works.

In doing my research, I’ve found out that you can use Eclipse to run Hadoop code in its IDE.  This posting: https://github.com/itsamiths/hadoop-eclipse-plugins let you download the Eclipse Hadoop plug-in, or build your own plug in.  I think I am going to try it and test it out with WordCount sample code.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: