Linux下编译Hadoop2.7.3源码

  • 安装svn软件
yum install -y svn
yum install -y gcc gcc-c++
yum install -y cmake
yum install -y ant
yum install -y openssl-devel
  • 下载hadoop源码
# 新建源码存放目录
mkdir -p /root/github/hadoop
# 下载源码
svn checkout https://github.com/apache/hadoop.git/branches/branch-2.7.3/
  • 编译源码
    不同的机器编译时长不一样,我的用了两个半小时。
cd /root/github/hadoop/branch-2.7.3
mvn package -Pdist,native -DskipTests -Dtar
  • 找到编译好的hadoop安装包
cd /root/github/hadoop/branch-2.7.3/hadoop-dist/target

注意

  • 在使用maven编译时,会下载一些POM文件中指定的jar包,建议使用阿里云的maven repository
  • 如果使用虚拟机编译hadoop源码,如果条件允许建议虚拟机的内存给到8G以上。
  • Hadoop编译时,需要使用Protocolbuffer,参见配置地址

标签: hadoop

相关文章推荐

已有 6 条评论

  1. ERROR] Failed to execute goal on project hadoop-common: Could not resolve dependencies for project org.apache.hadoop:hadoop-common:jar:2.7.4: Could not transfer artifact org.apache.commons:commons-math3:jar

    毛祥溢 回复
  2. Caused by: org.apache.maven.plugin.MojoExecutionException: An Ant BuildException has occured: Execute failed: java.io.IOException: Cannot run program “cmake”

    毛祥溢 回复
  3. [ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (site) on project hadoop-common: An Ant BuildException has occured: stylesheet

    毛祥溢 回复
  4. Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (make) on
    project hadoop-pipes: An Ant BuildException has occured: exec returned: 1
    [ERROR] around Ant part …

    毛祥溢 回复
  5. [ERROR] Failed to execute goal org.apache.maven.plugins:maven-antrun-plugin:1.7:run (dist) on project hadoop-kms: An Ant BuildException has occured: exec returned: 2
    [ERROR] around Ant part …

    毛祥溢 回复
  6. 本文是基于nimi版本的centos,测试了一遍,咩有问题。 不过我开了小飞机。

    毛祥溢 回复

添加新评论,含*的栏目为必填