基于Thrift的HBase数据存储机制优化

针对Thrift接口服务定义的IDL对HBase数据库按行存储,当数据量大时频繁进行数据请求操作,增加服务调用时间,影响数据通信性能的问题,在详细分析Thrift源码架构基础上,提出了一种新的Thrift IDL设计模型,该模型重新定义了数据传输结构,将多行数据绑定在一起,经过一次RPC调用即可完成多行数据按块存储；采用新的IDL模型,修改了HBase Thrift服务端的接口以及客户端的非阻塞实现。理论分析和实验结果表明,该方法可有效降低IDL向服务端发送数据操作请求频率,使得HBase储存效率提高4~5倍。

Application Optimization of Thrift Communication Mechanism in HBase Stored Data

The IDL defined for the Thrift interface service stores the HBase database in rows. When the amount of data is large, the data request operation is frequently performed, the service call time is increased, and the data communication performance is affected. Based on the detailed analysis of the Thrift source architecture, a new Thrift IDL design model is proposed, which redefined the data transfer structure, bound multiple rows of data together, and completed multi-row data block storage after an RPC call.With the new IDL model, the interface of the HBase Thrift server and the non-blocking implementation of the client are modified.Theoretical analysis and experimental results show that this method can effectively reduce the frequency of IDL sending data operation requests to the server, which makes HBase storage efficiency increase by 4~5 times.