高速网络流实时处理模型研究与实现

针对当前网络流处理存在的实时性不足,难以适应不同网络流量规模的问题,研究并实现了一种多流量场景实时网络流重组架构,旨在进行网络流处理时能够适应不同的网络流量场景,并且有效提升系统资源利用率。该架构基于DPDK实现高速流量实时采集,并且对流重组过程进行阶段划分,构造不同场景的运行拓扑,实现系统资源的动态分配。在单机环境下实现了该系统,并在10 Gbps网络环境下进行测试,结果表明该架构在吞吐量和资源利用率上优于同类其他方法。

Research and implementation of real-time processing model of high-speed network stream

With the increasing scale of network traffic, it becomes more and more difficult to analyze network traffic in real time, efficiently and accurately. Aiming at the problem that the current network flow processing is not real-time enough and it is difficult to adapt to different network traffic scale, this paper studies and implements a real-time network flow restructuring architecture with multiple traffic scenarios,Designed to be able to adapt to different network traffic scenarios during network stream processing, And effectively improve system resource utilization. The architecture is based on DPDK to achieve real-time collection of high-speed traffic, and the process of stream reorganization is divided into stages, to construct operating topologies for different scenarios, and to achieve dynamic allocation of system resources. The system was implemented in a stand-alone environment and tested in a 10Gbps network environment. The results show that the architecture is superior to similar methods in throughput and resource utilization.