2014年10月1日 星期三

Eclipse with Hadoop plugin

[Reference]
https://github.com/winghc/hadoop2x-eclipse-plugin  (package download)
http://blog.jourk.com/compile-hadoop-eclipse-plugin-2-4-1-plugin.html  (method)
http://blog.csdn.net/yueritian/article/details/23868175  (Debug)


Software:
eclpise:  eclipse-java-luna-R-linux-gtk-x86_64.tar.gz
Hadoop: 2.4.1
#Because Hadoop2.4.1 doesn't provide the plugin for eclipse, so we need to create one
Plugin create:
Download: https://github.com/winghc/hadoop2x-eclipse-plugin
1.Modify: 
(this tool now(2014/09/15)is set for hadoop2.2.0 version, so it need to be modify for 2.4.1v.)
[Change 1]
  Find ~/hadoop2x-eclipse-plungin-master/src/contrib/eclipse-plugin/build.xml
  Add commons-collections-3.2.1.jar setting :
...
<copy file="${hadoop.home}/share/hadoop/common/lib/commons-collections-${commons-collections.version}.jar" todir="${build.dir}/lib" verbose="true"/>
<!--     Add this Line         -->
<copy file="${hadoop.home}/share/hadoop/common/lib/commons-collections-${commons-collections.version}.jar" todir="${build.dir}/lib" verbose="true"/>
<!--                           -->
<copy file="${hadoop.home}/share/hadoop/common/lib/commons-collections-${commons-collections.version}.jar" todir="${build.dir}/lib" verbose="true"/>
...

    Change
lib/commons-lang-2.5.jar
 to
        lib/commons-lang-2.6.jar

 Add
lib/commons-collections-${commons-collections.version}.jar, 
 
[Change 2]
Find ~/hadoop2x-eclipse-plungin-master/ivy/libraries.properties
Add
commons-collections.version=3.2.1

2.Jar file
$cd src/contrib/eclipse-plugin
$ant jar -Dversion=2.4.1 -Declipse.home=/home/hdp2/eclipse -Dhadoop.home=/home/hdp2/hadoop-2.4.1

Then the jar file will appear in the ~/hadoop2x-eclipse-plungin-master/build/contrib/eclipse-plugin

# -Dversion : the installed Hadoop version
  -Declipse.home : the dir of ECLIPSE_HOME
  -Dhadoop.home : the dir of HADOOP_HOME

3. Move the plugin to eclipse/plugins
$cp build/contrib/eclipse-plugin/hadoop-eclipse-plugin.2.4.1.jar /home/hdp2/eclipse/plugins

4. Start eclipse with debug parameter:
$ eclipse/eclipse -clean -consolelog -debug

[備註] 此指令很重要,若eclipse開啟後點選 map/reduce location 沒反應,可以從terminal去看error info!

5. Eclipse開啟後
- Windows > Open Perspective > Other > Map/Reduce
- Windows > Show View > Other > MapReduce Tools > Map/Reduce Locations
- 點選右下角藍色大象 > 設定Hadoop server Location
[備註] 若點選大象時,沒跳出設定畫面,請查詢terminal上面錯誤訊息為何

- Setting:
Location name : master
MapReduce Master >>  Host: 192.168.0.7  Port: 8032
DFS Master     >>  Host:(use M/R Master host) Port:9000
User name : hduser (hadoop user name)

沒有留言:

張貼留言