Hadoop入门系列(7)-Windows10环境下源码编译hadoop-2.9.2

今天药忘吃喽~ 2023-10-10 12:12 12阅读 0赞

## 基础环境准备 ##

**都是常用开发环境，程序员一般都有这个环境，安装方法不赘述，实在不知道的自行百度**  
1.下载源码并解压到D盘根目录下,查看编译说明。  
[官网下载地址][Link 1]  
[Github克隆地址][Github]  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNDM3ODc0_size_16_color_FFFFFF_t_70]

Requirements:
    
    * Windows System
    * JDK 1.7 or 1.8
    * Maven 3.0 or later
    * Findbugs 1.3.9 (if running findbugs)
    * ProtocolBuffer 2.5.0
    * CMake 2.6 or newer
    * Windows SDK 7.1 or Visual Studio 2010 Professional
    * Windows SDK 8.1 (if building CPU rate control for the container executor)
    * zlib headers (if building native code bindings for zlib)
    * Internet connection for first build (to fetch all Maven and Hadoop dependencies)
    * Unix command-line tools from GnuWin32: sh, mkdir, rm, cp, tar, gzip. These
      tools must be present on your PATH.
    * Python ( for generation of docs using 'mvn site')

1.  下载并安装JDK1.8，配置环境变量  
    ![在这里插入图片描述][20201223145532938.png]
2.  下载并安装Maven3.6.2,配置环境变量  
    ![在这里插入图片描述][20201223145629863.png]
3.  下载并安装Git,此处是为了使用Git Bash功能，因为编译时需要执行Shell脚本,配置bash环境变量  
    [下载地址][Link 2]  
    ![在这里插入图片描述][20201223151309468.png]  
    ![在这里插入图片描述][20201224135327635.png]

## 特殊环境安装 ##

### 安装ProtocolBuffer ###

Hadoop从2.x版本开始，底层的RPC远程调用使用ProtocolBuffer格式来传递数据，所以需要安装对应版本的Protocol Buffer服务。

1.  下载安装包  
    [Github下载地址][Github 1]  
    ![在这里插入图片描述][20201223154458142.png]
2.  解压到指定目录D:\\javaDev\\protoc-2.5.0-win32  
    ![在这里插入图片描述][20201223160331210.png]
3.  配置环境变量D:\\javaDev\\protoc-2.5.0-win32  
    ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNDM3ODc0_size_16_color_FFFFFF_t_70 1]
4.  CMD输入protoc --version验证  
    ![在这里插入图片描述][2020122316074611.png]

### 安装cmake ###

1.  下载  
    [下载地址][Link 3]  
    ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNDM3ODc0_size_16_color_FFFFFF_t_70 2]
2.  安装  
    ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNDM3ODc0_size_16_color_FFFFFF_t_70 3]
3.  验证

![在这里插入图片描述][20201224125122872.png]

### 安装zlib ###

1.  下载  
    [下载地址][Link 4]
2.  配置环境变量  
    ![在这里插入图片描述][20201224163015838.png]

### 安装Visual Studio 2010 ###

构建文档中说安装Windows SDK 7.1或者Visual Studio 2010 Professional。  
试着安装SDK，但是各种问题未能解决，所以选择安装VS。

1.  卸载之前安装过得VS code2019版本
2.  下载并安装Visual Studio 2010 Professional,提示Sql Server安装失败，再次双击安装，选择更新安装后安装成功。  
    [下载链接][Link 5]  
    ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNDM3ODc0_size_16_color_FFFFFF_t_70 4]  
    ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNDM3ODc0_size_16_color_FFFFFF_t_70 5]
3.  添加msbuild环境变量  
    ![在这里插入图片描述][20201224141929989.png]  
    ![在这里插入图片描述][20201224141953884.png]
4.  设置VS：进入C:\\Program Files (x86)\\Microsoft Visual Studio 10.0\\VC目录，执行"C:\\Program Files (x86)\\Microsoft Visual Studio 10.0\\VC\\vcvarsall.bat" x64![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNDM3ODc0_size_16_color_FFFFFF_t_70 6]

## 编译 ##

1.  运行VS  
    ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNDM3ODc0_size_16_color_FFFFFF_t_70 7]
2.  输入mvn package -Pdist,native-win -DskipTests -Dta开始编译
3.  经过漫长等待,编译成功  
    ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNDM3ODc0_size_16_color_FFFFFF_t_70 8]

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNDM3ODc0_size_16_color_FFFFFF_t_70 9]

## 总结 ##

大概花费了一天的时间，遇到了各种问题，遇到问题不用怕，一个个解决，功夫不负有心人，需要资料的可以联系我  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNDM3ODc0_size_16_color_FFFFFF_t_70 10]

[Link 1]: https://hadoop.apache.org/release/2.9.2.html
[Github]: https://github.com/apache/hadoop
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNDM3ODc0_size_16_color_FFFFFF_t_70]: https://img-blog.csdnimg.cn/20201223143657673.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNDM3ODc0,size_16,color_FFFFFF,t_70
[20201223145532938.png]: https://img-blog.csdnimg.cn/20201223145532938.png
[20201223145629863.png]: https://img-blog.csdnimg.cn/20201223145629863.png
[Link 2]: https://gitforwindows.org/
[20201223151309468.png]: https://img-blog.csdnimg.cn/20201223151309468.png
[20201224135327635.png]: https://img-blog.csdnimg.cn/20201224135327635.png
[Github 1]: https://github.com/protocolbuffers/protobuf/releases/tag/v2.5.0
[20201223154458142.png]: https://img-blog.csdnimg.cn/20201223154458142.png
[20201223160331210.png]: https://img-blog.csdnimg.cn/20201223160331210.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNDM3ODc0_size_16_color_FFFFFF_t_70 1]: https://img-blog.csdnimg.cn/20201223160657923.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNDM3ODc0,size_16,color_FFFFFF,t_70
[2020122316074611.png]: https://img-blog.csdnimg.cn/2020122316074611.png
[Link 3]: https://cmake.org/download/
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNDM3ODc0_size_16_color_FFFFFF_t_70 2]: https://img-blog.csdnimg.cn/20201224125102934.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNDM3ODc0,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNDM3ODc0_size_16_color_FFFFFF_t_70 3]: https://img-blog.csdnimg.cn/20201224125112257.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNDM3ODc0,size_16,color_FFFFFF,t_70
[20201224125122872.png]: https://img-blog.csdnimg.cn/20201224125122872.png
[Link 4]: http://www.zlib.net/
[20201224163015838.png]: https://img-blog.csdnimg.cn/20201224163015838.png
[Link 5]: https://blog.csdn.net/weixin_30633507/article/details/95715701
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNDM3ODc0_size_16_color_FFFFFF_t_70 4]: https://img-blog.csdnimg.cn/20201224112607565.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNDM3ODc0,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNDM3ODc0_size_16_color_FFFFFF_t_70 5]: https://img-blog.csdnimg.cn/20201224115241489.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNDM3ODc0,size_16,color_FFFFFF,t_70
[20201224141929989.png]: https://img-blog.csdnimg.cn/20201224141929989.png
[20201224141953884.png]: https://img-blog.csdnimg.cn/20201224141953884.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNDM3ODc0_size_16_color_FFFFFF_t_70 6]: https://img-blog.csdnimg.cn/2020122416324396.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNDM3ODc0,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNDM3ODc0_size_16_color_FFFFFF_t_70 7]: https://img-blog.csdnimg.cn/20201224164126782.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNDM3ODc0,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNDM3ODc0_size_16_color_FFFFFF_t_70 8]: https://img-blog.csdnimg.cn/20201224164453617.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNDM3ODc0,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNDM3ODc0_size_16_color_FFFFFF_t_70 9]: https://img-blog.csdnimg.cn/20201224164534767.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNDM3ODc0,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNDM3ODc0_size_16_color_FFFFFF_t_70 10]: https://img-blog.csdnimg.cn/20201224165006206.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQzNDM3ODc0,size_16,color_FFFFFF,t_70