基于Dijkstra算法分布式JobTracker节点模型通信方式的优化

针对MapReduce框架下JobTracker单点失效而引发的系统运行崩溃问题,通过建立分布式JobTracker节点模型来改善JobTracker的可靠性,并基于Dijkstra算法的单源最短路径思想对新建立模型中主从节点间的通信方式进行优化与改进,使任务节点优先与其最近的不同机架和不同数据中心的3个控制节点进行通信.实验结果表明,分布式JobTracker节点模型能在一定程度上缓解由单JobTracker节点失效引起的作业失败问题,并且改进后的通信方式能够缓解单JobTracker节点存在的瓶颈,均衡JobTracker节点的负载.

Based on the Dijkstra Algorithm Optimize the Distributed JobTracker Node Model of Communication

For the faliure of single JobTracker which may cause the running crashes of system under MapReduce framework,establishing a model of distributed JobTracker node can improve the reliability of JobTracker,and through optimizing and improving communication based on the idea of single-source shortest path Dijkstra algorithm,task nodes can communicate with a JobTracker priority which is in the nearest rack or in the nearest data center.Experimental results have shown that the model of distributed JobTracker node can alleviate problems caused by the failutre of JobTracker to some extent.In addition,improved communication can also alleviate bottlenecks of single JobTracker and balance the load of JobTracker.