基于MapReduce模型带准备时间的平行机调度优化

研究了一类基于MapReduce模型的平行机调度问题.每个工件包含Map和Reduce两道加工工序,Map工序可以分割为若干个子任务,并且在多台平行机上同时并行加工,Reduce工序只有在该工件的所有Map工序的子任务加工完成后才能进行,而且Reduce只能在一台机器上加工且不可中断.结合工件具有释放时间和加工准备时间等约束,以最小化最大完工时间为目标,构建了混合整数规划模型,并设计了采用差分变异策略和逐维Levy扰动机制的改进正弦余弦算法来求解该模型.最后,利用数值仿真实验与标准正弦余弦算法及遗传算法进行对比,实验结果表明,运用改进正弦余弦算法求解的结果与下界值的平均相对偏差GAP为3.02%,较标准正弦余弦算法以及遗传算法的效果提升显著,显示了该改进算法的有效性.     

MapReduce系统中的两阶段混合流水作业调度算法

以极小化最大完工时间为目标,研究MapReduce系统中的两阶段混合流水作业调度问题.每个工件都包含两个任务集,即map任务集和reduce任务集.所有map任务必须在第一阶段的m1台平行机上加工,而reduce任务则必须在第二阶段的m2台平行机上加工.一个工件的reduce任务只有在该工件的所有map任务完成后才能开始加工.所有reduce任务不允许中断.对map任务不可中断情形,给出了一个最坏情况界为2-1/max{m1,m2}的近似算法.对map任务可任意分割情形,分别给出了基于Johnson规则和LPT规则的近似算法H(2,J)和H(2,L),并证明了这两个算法的最坏情况界分别为2-1/m2和2.通过数值实验发现,一般情况下H(2,J)性能要优于H2,L,但在reduce任务的总加工时间大于map任务且m2较大时则相反.最后,当map任务和reduce任务的总加工时间成比例关系时,给出了算法H(2,J)的参数最坏情况界.     

一种小功率开关稳压电源设计

本文设计了一种以电流型PWM控制器5L0380为控制核心的单端反激式开关稳压电源。重点给出EMI滤波器、整流电路、变压器、控制电路、反馈电路的参数设计。实验结果表明,所制作的电源具有稳压性能优良、纹波小、电压调整率和负载调整率小等优点。     

基于Hadoop建立云计算系统

云计算是2007年底正式提出的一个新概念,本文分析了云计算的特征和Hadoop的作用地位.针对Hadoop这样一个在集群上运行大型数据库处理应用程序的开放式源代码框架进行了云计算平台的实现,并进行了实验验证.     

k-means聚类算法的MapReduce并行化实现

针对k-means聚类算法特点,给出了MapReduce编程模型实现k-means聚类算法的方法,Map函数完成每个记录到聚类中心距离的计算并重新标记其属于的新聚类类别,Reduce函数根据Map函数得到的中间结果计算出新的聚类中心,供下一轮MapReduce Job使用.实验结果表明:k-means算法MapReduce并行化后部署在Hadoop集群上运行,具有较好的加速比和良好的扩展性.     

Network Motif Detection： Algorithms, Parallel and Cloud Computing,and Related Tools

Network motif is defined as a frequent and unique subgraph pattern in a network, and the search involves counting all the possible instances or listing all patterns, testing isomorphism known as NP-hard and large amounts of repeated processes for statistical evaluation. Although many efficient algorithms have been introduced, exhaustive search methods are still infeasible and feasible approximation methods are yet implausible.Additionally, the fast and continual growth of biological networks makes the problem more challenging. As a consequence, parallel algorithms have been developed and distributed computing has been tested in the cloud computing environment as well. In this paper, we survey current algorithms for network motif detection and existing software tools. Then, we show that some methods have been utilized for parallel network motif search algorithms with static or dynamic load balancing techniques. With the advent of cloud computing services, network motif search has been implemented with MapReduce in Hadoop Distributed File System(HDFS), and with Storm, but without statistical testing. In this paper, we survey network motif search algorithms in general, including existing parallel methods as well as cloud computing based search, and show the promising potentials for the cloud computing based motif search methods.     

Towards a Cost-Efcient MapReduce: Mitigating Power Peaks for HadoopClusters简

Distributed data processing system is becoming one of the most important components for data-intensive computational tasks in the enterprise software infrastructure. Deploying and operating such systems require large amount of costs, including hardware costs to build clusters and energy costs to run clusters. To make these systems sustainable and scalable, power management has been an important research problem. In this paper, we take Hadoop as an example to illustrate the power peak problem which causes power inefficiency and provides in-depth analysis to explain issues with existing system designs. We propose a novel power capping module in the Hadoop scheduler to mitigate power peaks. Extensive simulation studies show that our proposed solution can effectively smooth the power consumption curve and mitigate temporary power peaks for Hadoop clusters.     

MapReduce框架下k-支配轮廓查询算法

随着维度和数据量的不断增长,已有的轮廓查询方法已无法为用户提供更好的查询支持.本文采用并行处理框架MapReduce 结合k-支配轮廓查询算法来解决此类查询问题.在对输入的数据集进行预排序的基础上,提出了基于MapReduce 的单轮扫描算法和双轮扫描算法,通过在不同维度、数据分布、节点个数等方面的实验测试及对比,验证了算法的有效性.     

基于MapReduce的Eclat改进算法研究与应用

针对Eclat算法在挖掘海量数据中的频繁项目集时存在的内存和计算资源不足等问题,将Eclat算法与目前流行的大规模数据集并行编程模型MapReduce结合,解决了Eclat算法在数据挖掘过程存在的瓶颈问题,运用于动车组故障诊断系统中,提高了关联规则挖掘的效率.