基于Ambari的Hadoop集群快速部署研究

Hadoop作为处理大数据的一个优秀分布式计算框架,在企业应用非常普通。然而Hadoop集群部署需要考虑各组件的兼容性、编译问题及繁琐的组件参数配置,初学者往往耗时几天也不能部署成功。Ambari是一种支持Hadoop集群部署、监控和管理的开源工具。针对Hadoop集群部署的复杂性,提出基于Ambari工具部署Hadoop集群各组件的实践方法并讨论了快速部署的若干要点及重要步骤;通过Ambari工具,完成了Hadoop生态圈最小化集群大部分常用组件的快速部署,如HDFS、HBase、Hive、Pig、Oozie、Zookeeper、Sqoop、Spark、Storm、Kafka、Flume等;项目实践表明:利用Ambari工具能够在8 h内部署完毕Hadoop集群,相比较传统手工部署方式,Ambari工具极大提高了Hadoop集群部署的效率及成功率。

Research on Rapid Deployment of Hadoop Cluster Based on Ambari

As an excellent distributed computing framework to deal with big data, Hadoop is very popular in enterprises. However, the deployment of Hadoop cluster needs to consider the compatibility of each component, compilation problems and tedious component parameter configuration,and beginners often cannot deploy successfully even in several days. Ambari is an open source tool that supports Hadoop cluster deployment, monitoring and management. In view of the complexity of Hadoop cluster deployment, this paper puts forward the practical method of deploying each component of Hadoop cluster based on Ambari tool, and discusses some key points and important steps of rapid deployment. Through Ambari tool, the rapid deployment of most common components of Hadoop ecosphere minimization cluster has been completed, such as HDFS,HBase,Hive,Pig,Oozie,Zookeeper,Sqoop,Spark,Storm,Kafka,Flume and so on. Project practice shows that Hadoop cluster can be deployed within 8 hours by using Ambari tools. Compared with the traditional manual deployment, Ambari tools greatly improve the efficiency and success rate of Hadoop cluster deployment.