民用建筑“四节一环保”数据的清洗与修复方法研究

针对民用建筑"四节一环保"原始数据中存在的数据质量问题,使用多种方法实现数据清洗与数据修复。数据清洗方面,重点关注单栋建筑能耗数据中存在的相似重复记录及异常记录。其中,识别异常记录采用3σ准则、DBSCAN聚类算法及箱线图内限3种方法。数据修复方面,重点关注缺失值的填补及基于模型的数据修正。其中,缺失值的填充使用简单填充、线性回归模型和基于用户的协同过滤推荐算法,并以平均绝对误差为评估指标进行对比。基于多元线性回归、主成分回归、偏最小二乘回归、岭回归及Lasso回归5种模型,拟合建筑运行能耗与各解释变量间的关系,对上海市建筑运行能耗相关数据进行数据修复。结果显示,单栋建筑能耗数据适合采用箱线图内限来识别异常记录,并使用中位数填补缺失数据;上海市建筑运行能耗相关数据中,岭回归模型的拟合情况最好。

Research on Cleaning and Repairing Methods of Civil Building Data on Resources Saving and Environment Protection

Aiming at the data quality issues existing in the original civil building data on resources saving and environment protection, various methods are used to achieve data cleaning and data repairing. In terms of data cleaning, the authors focus on the approximately duplicated records and abnormal records in the energy consumption data of single building. In particular, the methods for identifying abnormal records include the empirical rule, the DBSCAN clustering algorithm, and inner fence of boxplot. In terms of data repairing, the authors focus on completing missing values and using the models to achieve data correction. In particular, the missing values are filled in these ways: existing values in the datasets, the predicted values of the linear regression model, and the output of the user-based collaborative filtering recommendation algorithm. The average absolute error is used as an evaluation index to compare these filling results. While repairing the building energy consumption data from Shanghai, multiple linear regression, principal component regression, partial least squares regression, ridge regression and Lasso regression are used to fit the correlation between building energy consumption and explanatory variables. The results show that for the energy consumption data of single building, it’s suitable to use the inner fence of boxplot to identify abnormal records, and use the median to complete missing values. For the building energy consumption data from Shanghai, the ridge regression model fits best.