一种用于数据挖掘算法的数据生成方法

由于受到保密性、时间和数据多样性等一些原因的限制,测试数据集的获取一直困扰着数据挖掘算法的研究.因此,提出一种基于遗传算法和熵的测试数据集的模拟生成方法,生成方法利用遗传算法具有继承性的特性对采集到的少量的真实数据进行扩充和模拟,用熵衡量生成数据与真实数据的相似程度,最终生成规模大的测试数据集,并给出了描述型数据的生成算法.使用此方法,可以生成同真实数据集具有相同的属性,相同的属性取值区间和属性值分布,类似属性关联关系的测试数据集,加速数据挖掘算法的研究进程.

A Method Generating Data Sets to Test Data Mining Algorithms

Because of security,uncertain time,diversity of data etc,the problem of how to acquire the data set to test data mining algorithms has been confusing the study on data mining.A simulating method is therefore suggested to generate the data set on the basis of the genetic algorithm and entropy.The method extends a few data which were collected from reality by GA, then evaluates the similarity between extended data sets and real one with entropy,and generates the most similar data set of big size among the extended ones as the data set to test the data mining algorithms.A generation algorithm is also given.This method is available to generate the data set for testing,which has the same attributes,scales of attribute value and distributions of attribute value to the data set from reality,as well as the correlations among the attributes.This data set for testing will accelerate the study on data mining algorithms.