VirtCO:Joint Coflow Scheduling and Virtual Machine Placement in Cloud Data Centers

Cloud data centers, such as Amazon EC2, host myriad big data applications using Virtual Machines(VMs). As these applications are communication-intensive, optimizing network transfer between VMs is critical to the performance of these applications and network utilization of data centers. Previous studies have addressed this issue by scheduling network flows with coflow semantics or optimizing VM placement with traffic considerations.However, coflow scheduling and VM placement have been conducted orthogonally. In fact, these two mechanisms are mutually dependent, and optimizing these two complementary degrees of freedom independently turns out to be suboptimal. In this paper, we present VirtCO, a practical framework that jointly schedules coflows and places VMs ahead of VM launch to optimize the overall performance of data center applications. We model the joint coflow scheduling and VM placement optimization problem, and propose effective heuristics for solving it. We further implement VirtCO with OpenStack and deploy it in a testbed environment. Extensive evaluation of real-world traces shows that compared with state-of-the-art solutions, VirtCO greatly reduces the average coflow completion time by up to 36.5%. This new framework is also compatible with and readily deployable within existing data center architectures.     

Load Feedback-Based Resource Scheduling and Dynamic Migration-Based Data Locality for Virtual Hadoop Clusters in OpenStack-Based Clouds

With cloud computing technology becoming more mature, it is essential to combine the big data processing tool Hadoop with the Infrastructure as a Service(Iaa S) cloud platform. In this study, we first propose a new Dynamic Hadoop Cluster on Iaa S(DHCI) architecture, which includes four key modules: monitoring,scheduling, Virtual Machine(VM) management, and VM migration modules. The load of both physical hosts and VMs is collected by the monitoring module and can be used to design resource scheduling and data locality solutions. Second, we present a simple load feedback-based resource scheduling scheme. The resource allocation can be avoided on overburdened physical hosts or the strong scalability of virtual cluster can be achieved by fluctuating the number of VMs. To improve the flexibility, we adopt the separated deployment of the computation and storage VMs in the DHCI architecture, which negatively impacts the data locality. Third, we reuse the method of VM migration and propose a dynamic migration-based data locality scheme using parallel computing entropy. We migrate the computation nodes to different host(s) or rack(s) where the corresponding storage nodes are deployed to satisfy the requirement of data locality. We evaluate our solutions in a realistic scenario based on Open Stack.Substantial experimental results demonstrate the effectiveness of our solutions that contribute to balance the workload and performance improvement, even under heavy-loaded cloud system conditions.     

基于Linux的用户公平分享调度

针对Linux系统的调度时机和调度策略 ,提出了基于用户uid的公平分享调度机制 .然后详细给出了这个调度机制的实现     

云计算下安全可控的移动弹性资源动态调度算法

当前云计算下移动弹性资源动态调度算法CPU利用率较低,且运行不稳定。为此,提出一种新的云计算下安全可控的移动弹性资源动态调度算法,把云计算下移动弹性资源的CPU平均利用率、内存负载、带宽负载以及可靠性看作移动弹性资源调度目标的约束,给出移动弹性资源的状态模型。依据遗传算法的优点,设计了一种基于蚁群算法的、安全可控的移动弹性资源动态调度算法。依据云计算下移动弹性资源动态调度特点和约束条件,利用适应度指标,对种群中个体的优良度进行描述,给出适应度函数。通过一维字符串完成对解的编码,利用轮盘赌法设置合理的阈值,获取所有染色体中选择性能较好的染色体。通过选择的实数编码完成对字符串的逆向解码,得到云计算下移动弹性资源调度的最优方案。实验结果表明,所提算法稳定性、效率和资源利用率高,迁移率低,负载均衡性好。     

一种基于动态聚合方法的网格任务调度算法

网格中的资源都是动态的,传统的静态任务调度算法不能适应网格的动态特性。通过对资源在未来一段时间内的状态进行预测,可以提高调度算法的性能。文中提出了一种用动态聚合进行调度的算法。首先对处理器的负载进行取样,然后根据网格任务的执行时间,对处理器的取样值进行动态聚合,再利用AR(p)模型进行预测,最后利用预测到的值作为参数对网格任务进行调度,把网格任务分配给每个处理器,使得每个处理器完成子任务的时间都相同,从而使得整个任务的执行时间最短。实验表明,这种算法能很好地适应处理器负载高度变化的情况。     

多核平台的并行实时调度与内存分配算法

多核处理器凭借着低功耗高性能的优势占据了市场.针对多核平台上并行实时任务,提出局部与全局EDF相结合的调度算法,其中任务的截止期划分、执行预算以及迁移时机由所设计的处理器带宽预留服务器决定.同时,提出了内存分配算法,该算法能够更好地为并行实时任务管理内存资源.实验结果表明新的调度算法具有更高的调度成功率.另外,在内存资源竞争的前提下,内存分配算法可以保证并行任务的实时性与系统稳定性.     

A less resource-consumed security architecture on cloud platform

Traditional security framework in cloud platform usually brings self-vulnerability and considerable additional resource consumption. To solve these problems, we propose an external processes monitoring architecture for current popular cloud platform OpenStack with kernel-based virtual machine (KVM). With this architecture, we can monitor all active processes in online virtual machine (VMs) and scan them for their potential maliciousness in OpenStack with no agent, and can also detect hidden processes in offline VMs’ memory snapshots and notice the user to decide whether to kill them when VMs become active. Analysis and experimental results show that our architecture is able to reduce consumption of CPU, memory and bandwidth in cloud platform and can detect viruses and hidden processes effectively in VMs.     

HowU网格资源管理及调度

讨论了一种基于web的HowU网格系统的实现,以及采用的关键技术．HowU网格系统通过资源请求代理进行网格作业提交,HowU网格系统采用了网格资源的分层管理策略及全局调度思想．根据网格的特性,并针对CPU性能对作业调度的影响,提出了三种全局资源调度算法：资源轮转法;当前最优资源优先调度法;最优资源优先调度法．研究结果表明,当作业数量比较少时,资源轮转法优势最明显,最优资源优先调度法性能最差．但当作业数量增大时,资源轮转法性能明显下降,而最优资源优先调度法比其他两者更有性能优势．     

Parallel scheduling strategy of web-based spatial computing tasks in multi-core environment

In order to improve the concurrent access performance of the web-based spatial computing system in cluster,a parallel scheduling strategy based on the multi-core environment is proposed,which includes two levels of parallel processing mechanisms.One is that it can evenly allocate tasks to each server node in the cluster and the other is that it can implement the load balancing inside a server node.Based on the strategy,a new web-based spatial computing model is designed in this paper,in which,a task response ratio calculation method,a request queue buffer mechanism and a thread scheduling strategy are focused on.Experimental results show that the new model can fully use the multi-core computing advantage of each server node in the concurrent access environment and improve the average hits per second,average I/O Hits,CPU utilization and throughput.Using speed-up ratio to analyze the traditional model and the new one,the result shows that the new model has the best performance.The performance of the multi-core server nodes in the cluster is optimized; the resource utilization and the parallel processing capabilities are enhanced.The more CPU cores you have,the higher parallel processing capabilities will be obtained.     

基于Xen平台的虚拟机交互式性能隔离改进

分析了高整合程度环境下, 虚拟机出现连续性网络延迟峰值的异常现象, 设计并实现 了针对交互式性能隔离的虚拟机调度算法优化方案, 包括协作式抢占、抢占返回和精确核算机制, 这些优化方案不需要对客户操作系统做任何修改。实验表明, 在8台CPU密集型虚拟机同时运行的情况下, 另外8台虚拟机网络延迟的最高5%平均值降低至优化前的0.93%, Firefox浏览邮件延迟的最高5%平均值降低至优化前的56.1%。     

一种多QoS约束的网格资源调度算法

针对网格计算中资源调度问题,提出一种多QoS（Quality of Service）约束的贪婪推广算法.调度时以调度驱动函数为贪婪准则.调度驱动函数与网格用户提供的“deadline”,“budget”,“timeweight”和“costweight”等参数有关.每次根据调度驱动函数选择最经济的资源进行调度.通过在Gridsim环境下的分析与比较,该算法能够在满足用户QoS的同时,能够以最小的调度驱动函数值完成任务,最大化地实现用户的满意度.     

针对不确定执行时间任务的实时调度方案

为了解决在实时调度系统中,任务执行时间不确定性所带来的问题,提出了基于时间预测的调度方案。该方案设计了VSM(vectorspacemodel)模型、Markov模型和MVSM(Markovvectorspacemodel)模型。对这3种模型的比较表明:基于MVSM模型的调度方案可以很好地保证实时系统的效率和稳定性,即使在处理器超载的情况下,也能自动调节,超过99%的作业可以在时间期限之前完成。采用时间预测的方法,可以较好地解决任务执行时间不确定性所带来的影响,为不确定环境下的实时调度系统提供一种很好的参考解决方案。     

A Novel Processor Scheduling Schema in Programmable Router Based on Queue Length Thresholds

Programmable router may execute computing tasks except for forwarding IP packets. How to schedule the CPU in the programmable router is an important issue that needs to be solved. In this paper, the authors establish general software architecture of programmable router firstly. Based on this architecture, authors present a novel CPU scheduling algorithm based on queue length thresholds. We model this algorithm using stochastic Petri nets. The analytic results show that this scheduling algorithm can guarantee the requirements of computing of best-effort flows and QoS flows in prograrmmable router at the same time.     

信息调度中的一个组合问题

研究了多用户信息服务中的实时调度问题,在基于双向扫描调度模型的基础上使用组合分析技术,对使用扫描算法进行调度时造成的服务延迟这一组合问题进行定量分析,推导出了计算延迟的递推公式.数值计算表明平均延迟时间随用户数的增长率近于常数.该公式可用于对双向扫描算法服务性能的评价.     

海量电子图书信息资源利用率均衡调度方法

为解决传统方法调度均衡性差,资源利用率低的弊端,提出一种新的海量电子图书信息资源利用率均衡调度方法。依据海量电子图书信息资源池中物理服务器的工作情况,对资源池运行工作进行判断,针对不饱和运行状态,通过分析资源利用率方差、资源利用率均衡度和整个资源利用率均衡度,建立均衡调度目标函数,采用蚁群算法对其进行求解决,获取最优调度策略。在资源池饱和的情况下,求出全部虚拟机资源的等待时间,选用等待时间最短的虚拟机执行当前海量电子图书网络用户请求资源。结果表明,所提方法调度结果和其他方法相比最均衡,只有少部分均衡性相对较差;对CPU和I/O资源利用率调度均衡性较好,对内存利用率调度均衡性相对较差,但和其他方法相比较好;对不同资源利用率均为95%～99%。可见所提方法对电子图书信息资源利用率调度均衡性好,有很高的竞争力和资源利用率。     

带重入约束的双集束型晶圆制造设备调度算法

为了有效解决集束型晶圆制造设备群加工过程中存在的重入调度问题,构建了带重入约束的双集束型晶圆制造设备调度算法.首先分析了双集束型晶圆制造设备中重入约束特征.在此基础上,建立调度问题域,并以系统加工周期最小为目标,建立调度模型.采用分解策略,将双集束型晶圆制造设备调度问题分解成单个设备调度问题,并提出了一种全新的机械手调度顺序搜索的启发式算法.最后对调度算法进行仿真实验分析,结果表明该算法是有效的.     

基于负载均衡的大规模TTE消息调度表生成方法

在复杂拓扑条件下，机载网络存在大规模数据的高效传输需求.此时，传统的调度表生成方法存在高时间复杂度问题.为了有效解决这个问题，本文提出了一种基于负载均衡的时间触发以太网(Time-Triggered Ethernet, TTE)消息调度表生成方法.该方法首先生成消息传输备选较短路径集，综合地考虑消息长度和链路负载来选择消息路径、得到具有负载均衡特性的路径规划结果，然后以提高时序规划效率为目标，基于静态优先级生成调度表，依据消息长度和周期进行消息排序，并使用“背靠背”规则进行传输调度分配，以保证消息的实时性和确定性.仿真结果表明，当可调度消息帧数超过6000条时，在保证消息实时性的前提下，该方法比传统的满足性模理论(Satisfiability Modulo Theory, SMT)时间规划方法减少90%以上的计算时间，计算效率提升10～100倍，可见该方法适合于求解复杂大规模数据调度表的生成问题.本文的研究为改善机载网络的消息调度性能提供了一种可行方案.     

一种C-RAN架构下载波迁移的调度策略

在云无线电接入网络(cloud-radio access network,RAN)架构的基础上,为提高载波迁移后虚拟多模基站的资源利用率和负载均衡度,把CPU资源适度分离成基带CPU资源和协议CPU资源,并提出一种基于负载特征的载波迁移调度策略.针对物理层主机的触发类型和虚拟多模基站的负载特征,采用双阈值上下触发方式触发迁移,根据9种负载状态并采用最小资源代价策略,完成对需要迁移的虚拟多模基站以及目标主机的选择.实验结果表明,基带资源利用率方差均值降低0.04左右,协议资源利用率方差均值降低0.03左右,实现了虚拟基站迁移的自主管理,并且达到了节约资源的目的.迁移前后,虚拟基站的状态没发生太大的改变,具有良好的一致性.     

多模式资源受限项目调度问题的混合遗传算法

多模式资源受限项目调度问题是一种NP难的组合优化问题.提出了与基于关键链的启发式算法相结合的二层混合遗传算法对该问题进行求解.在由上层算法确定的调度顺序下,下层遗传算法结合基于关键链的启发式算法,对系统资源重新优化配置,使算法加速向最优解区域收敛,并在下层设计了随迭代代数增加的可变变异概率,以避免早熟收敛.利用标准问题库对算法进行测试,分析问题参数与算法参数对算法结果的影响,发现实验结果的绩效随迭代数的增加而提高,算法耗时随任务数和迭代数的增加而增加.数值测试结果验证了算法的可行性和可靠性.