注册 登录  
 加关注
   显示下一条  |  关闭
温馨提示!由于新浪微博认证机制调整,您的新浪微博帐号绑定已过期,请重新绑定!立即重新绑定新浪微博》  |  关闭

避尘巷

沧海一粟渡沧海

 
 
 

日志

 
 

Creating and configuring Hadoop eclipse project  

2012-04-22 08:33:58|  分类: Hadoop |  标签: |举报 |字号 订阅

  下载LOFTER 我的照片书  |

Creating and configuring Hadoop eclipse project.

  1. Launch Eclipse.
  2. Right-click on the blank space in the Project Explorer window and select New -> Project.. to create a new project.
  3. Select Map/Reduce Project from the list of project types as shown in the image below.

    Creating and configuring Hadoop eclipse project - 提拉米苏碎了 - 避尘巷
  4. Press the Next button.
  5. You will see the project properties window similar to the one shown below

    Creating and configuring Hadoop eclipse project - 提拉米苏碎了 - 避尘巷
  6. Fill in the project name and click on Configure Hadoop Installation link on the right hand side of the project configuration window. This will bring up the project Preferences window shown in the image below.

    Creating and configuring Hadoop eclipse project - 提拉米苏碎了 - 避尘巷

  7. In the project Preferences window enter the location of the Hadoop directory in the Hadoop installation directory field as shown above.
    If you are not sure of the location of the Hadoop home directory, refer to Step 1 of this section. Hadoop home directory is one level up from the conf directory.

  8. After entering the location close the Preferences window by pressing the OK button. Then close the Project window with the Finish button.
  9. You have now created your first Hadoop Eclipse project. You should see its name in the Project Explorer tab.

Creating Map/Reduce driver class

  1. Right-click on the newly created Hadoop project in the Project Explorer tab and select New -> Other from the context menu.
  2. Go to the Map/Reduce folder, select MapReduceDriver, then press the Next button as shown in the image below.

    Creating and configuring Hadoop eclipse project - 提拉米苏碎了 - 避尘巷
  3. When the MapReduce Driver wizard appears, enter TestDriver in the Name field and press the Finish button. This will create the skeleton code for the MapReduce Driver.

    Creating and configuring Hadoop eclipse project - 提拉米苏碎了 - 避尘巷
  4. Unfortunately the Hadoop plug-in for Eclipse is slightly out of step with the recent Hadoop API, so we need to edit the driver code a bit.

    Find the following two lines in the source code and comment them out:
    conf.setInputPath(new Path("src"));
    conf.setOutputPath(new Path("out"));

    Enter the following code immediatly after the two lines you just commented out (see image below):

    conf.setInputFormat(TextInputFormat.class);
    conf.setOutputFormat(TextOutputFormat.class);

    FileInputFormat.setInputPaths(conf, new Path("In"));
    FileOutputFormat.setOutputPath(conf, new Path("Out"));



    Creating and configuring Hadoop eclipse project - 提拉米苏碎了 - 避尘巷
  5. After you have changed the code, you will see the new lines marked as incorrect by Eclipse. Click on the error icon for each line and select Eclipse's suggestion to import the missing class.

    You need to import the following classes: TextInputFormat, TextOutputFormat, FileInputFormat, FileOutputFormat.
  6. After the missing classes are imported you are ready to run the project.

Running Hadoop Project

 

  1. Right-click on the TestDriver class in the Project Explorer tab and select Run As --> Run on Hadoop. This will bring up a window like the one shown below.

    Creating and configuring Hadoop eclipse project - 提拉米苏碎了 - 避尘巷
  2. In the window shown above select "Choose existing Hadoop location" , then select localhost from the list below. After that click Finish button to start your project.
  3. If you see console output similar to the one shown below, Congratulations! You have started the project successfully!

    Creating and configuring Hadoop eclipse project - 提拉米苏碎了 - 避尘巷
  评论这张
 
阅读(209)| 评论(0)
推荐 转载

历史上的今天

评论

<#--最新日志,群博日志--> <#--推荐日志--> <#--引用记录--> <#--博主推荐--> <#--随机阅读--> <#--首页推荐--> <#--历史上的今天--> <#--被推荐日志--> <#--上一篇,下一篇--> <#-- 热度 --> <#-- 网易新闻广告 --> <#--右边模块结构--> <#--评论模块结构--> <#--引用模块结构--> <#--博主发起的投票-->
 
 
 
 
 
 
 
 
 
 
 
 
 
 

页脚

网易公司版权所有 ©1997-2017