本文共 4116 字，大约阅读时间需要 13 分钟。
说明:
由于之前使用CDH5.4.8,其Hive版本为1.1.0,其业务的脚本和jar也在此版本开发，所以有很多依赖性，兼容性等等。然后当我们计划将CDH5.4.8 Job迁移至AliYun EMR平台上，发现hive-1.1.0-cdh5.4.8与该平台的Apache Hadoop2.7.2 严重不能兼容，于是花了很长时间在做这件事--根据业务脚本和jar包定制我们的hive-1.2.1-emr版本。
其中我们尝试了以下
hive-1.1.0-apache,
hive-2.0.0-apache,
hive-1.1.0-cdh5.8.0,
hive-1.1.0-cdh5.4.8等版本与Apache hadoop2.7.2与脚本内容兼容错误甚多，且尝试无法解决，于是选择hive-1.x.x系列的最新版。
1.Download hive-1.2.1 source code
[root@sht-sgmhadoopnn-01 hadoop]# wget https://archive.apache.org/dist/hive/hive-1.2.1/apache-hive-1.2.1-src.tar.gz[root@sht-sgmhadoopnn-01 hadoop]# tar -xzvf apache-hive-1.2.1-src.tar.gz[root@sht-sgmhadoopnn-01 hadoop]# cd apache-hive-1.2.1-src
2.Download patch
Bug1:ADD JAR failing with URL schemes other than file/ivy/hdfs
[root@sht-sgmhadoopnn-01 apache-hive-1.2.1-src]# wget https://issues.apache.org/jira/secure/attachment/12761724/HIVE-11920.1.patch[root@sht-sgmhadoopnn-01 apache-hive-1.2.1-src]# cat HIVE-11920.1.patchdiff --git a/ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java b/ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.javaindex 9f738df..eece93e 100644--- a/ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java+++ b/ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java....................................
3.Patch HIVE-11920.1.patch
Format: patch -p0|-p1 < xxxx.patch
如果使用参数-p0，就表示从当前目录，找一个叫作b的目录，在它下面找一个叫ql的目录，再在它下面找一个叫src的目录。
如果使用参数-p1, 就表示忽略第一层，从当前目录找一个叫ql的目录，在它下面找一个叫src的目录。这样会忽略掉补丁头提到的b目录。
[root@sht-sgmhadoopnn-01 apache-hive-1.2.1-src]# patch -p1 < HIVE-11920.1.patchpatching file ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.javapatching file ql/src/test/queries/clientpositive/add_jar_pfile.qpatching file ql/src/test/results/clientpositive/add_jar_pfile.q.out
4.Download maven tool and configure parameter
[root@sht-sgmhadoopnn-01 apache-hive-1.2.1-src]# export MAVEN_HOME=/hadoop/maven[root@sht-sgmhadoopnn-01 apache-hive-1.2.1-src]# export PATH=$MAVEN_HOME/bin:$PATH
5.Compile hive-1.2.1
[root@sht-sgmhadoopnn-01 apache-hive-1.2.1-src]# mvn clean install -Phadoop-2,dist -DskipTests -Dhadoop-23.version=2.7.2 -Dhive.version=1.2.1[root@sht-sgmhadoopnn-01 apache-hive-1.2.1-src]# ll ./packaging/target/apache-hive-1.2.1-bin.tar.gz-rw-r--r-- 1 root root 94035955 Oct 4 16:29 ./packaging/target/apache-hive-1.2.1-bin.tar.gz
6.Upload mysql-connector-java-5.1.36-bin.jar
[root@sht-sgmhadoopnn-01 apache-hive-1.2.1-src]# cp ./packaging/target/apache-hive-1.2.1-bin.tar.gz ../[root@sht-sgmhadoopnn-01 apache-hive-1.2.1-src]# cd ../[root@sht-sgmhadoopnn-01 hadoop]# tar xzvf apache-hive-1.2.1-bin.tar.gz[root@sht-sgmhadoopnn-01 hadoop]# cd ./apache-hive-1.2.1-bin/lib[root@sht-sgmhadoopnn-01 lib]# rzrz waiting to receive.Starting zmodem transfer. Press Ctrl+C to cancel.Transferring mysql-connector-java-5.1.36-bin.jar...100% 949 KB 949 KB/sec 00:00:01 0 Errors
7.Modify hive-site.xml
[root@sht-sgmhadoopnn-01 lib]# cd ../conf[root@sht-sgmhadoopnn-01 lib]# vi hive-site.xmlhive.metastore.warehouse.dir /user/hive/warehouse hive.metastore.local true javax.jdo.option.ConnectionURL jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true JDBC connect string for a JDBC metastore javax.jdo.option.ConnectionDriverName com.mysql.jdbc.Driver Driver class name for a JDBC metastore javax.jdo.option.ConnectionUserName root username to use against metastore database javax.jdo.option.ConnectionPassword EMRroot1234 password to use against metastore database
8.Rerun tar hive files
[root@sht-sgmhadoopnn-01 conf]# cd ../[root@sht-sgmhadoopnn-01 apache-hive-1.2.1-bin]# tar -zcvf hive-1.2.1-emr.tar.gz *
转载地址：http://hrqyx.baihongyu.com/