分布式环境中数据库模式设计实践

近年来,数据规模呈爆炸式增长,使得传统集中式数据库难以满足业务需求.而分布式数据库可以将数据存储在多个节点上,具有更好的扩展性,从而可以支撑业务的不断增长.目前,许多企业已经开发出了成功的分布式数据库产品,例如Google Spanner、淘宝的OceanBase等.传统数据库模式设计中,三大范式(1NF、2NF和3NF)及其扩展范式能够减少数据冗余和更新异常,并保证数据的完整性.然而,在分布式架构下,严格遵循范式的模式设计可能带来查询效率较低等问题,而使用反范式模式设计方法通常可以有效提高查询效率.OceanBase是淘宝自主研发的分布式数据库,支持跨行跨表事务,并在OLTP中具有良好的性能,但是对于OLAP业务,其性能并不高.本文将以OceanBase为例,介绍如何利用反范式设计分布式数据库模式,以改善OLAP的查询性能,并通过在OceanBase上部署TPC-H基准评测验证了反范式模式设计的有效性和高效性.     

OceanBase一致性与可用性分析

OceanBase作为一个面向海量数据查询的分布式数据库,支持关系查询和跨行跨表事务,同时保证了一致性和可用性.本文在详细阐述了一致性与可用性的背景基础上,根据传统数据库和分布式数据库保证一致性和可用性的协议和策略,对OceanBase的一致性和可用性架构进行分析.最后,探讨了基于OceanBase架构演变而来的3种架构的实现方案.     

OceanBase中基于布隆过滤器的连接算法

在大数据时代,"去IOE"运动的推进以及"双11"等活动的兴起对分布式数据库系统提出了更高的要求.OceanBase是阿里巴巴集团自主研发的开源分布式数据库,支持海量数据跨行跨表事务,但是对复杂查询的处理性能仍有待提高,其中连接操作带来的网络传输严重影响了数据库的性能.本文提出了一种基于布隆过滤器的连接算法,通过构建布隆过滤器对右表数据进行过滤,减少了不必要的数据传输开销,降低了数据处理带来的内存资源的消耗.本文在OceanBase上实现了该算法,并通过实验证明,该算法极大提高了连接操作的效率.     

一种面向海量分布式数据库的嵌套查询策略

面向大数据分析和处理的NoSQL数据库具有非常好的读写性能和可扩展性,但是无法支持完整的SQL查询和跨行跨表的事务,对于传统的以关系数据库为主的金融业务在应用上有所限制.OceanBase是面向海量数据查询的分布式数据库,结合了关系数据库和非关系数据库的优势,同时支持关系查询和跨行跨表事务,具有可扩展性.但是,目前OceanBase只支持简单的、非嵌套子查询的SQL语句,无法很好地支持金融应用.本文在研究OceanBase架构和查询策略的基础上,提出了一种基于BloomFilter和HashMap的查询策略,实验表明该策略能够提高和改善现有查询策略的不足,在实现嵌套查询的基础上,可提高查询性能.     

基于Spark/Shark的电力用采大数据OLAP分析系统

用电信息大数据上的OLAP查询涉及数据量大,具有多表连接操作频繁、SQL结构复杂等特点,传统关系型数据库面对该类应用,表现出可扩展性弱、数据写入吞吐量低与查询效率低等问题.为此设计了一套基于Spark/Shark的电力大数据OLAP分析系统,该系统采用分布式文件系统HDFS保存电力用电信息采集系统的大数据,通过Shark进行前端SQL解析,Spark进行查询计算;然而,原生Shark只支持粗粒度分区,不支持细粒度的索引技术,难以高效地过滤无关数据,影响了查询性能.为克服这一不足,该系统设计了一种基于前缀树的细粒度索引结构TrieIndex,并通过数据重组技术优化了数据在HDFS的分布,提升了Shark的数据过滤能力以及用电信息大数据OLAP分析的性能.真实用电信息采集系统数据与查询的实验结果表明,该系统比关系型数据库的写入速度提升了12倍,比原生Shark的查询效率提升了10倍以上.     

基于分布式系统OceanBase的并行连接

随着应用数据的飞速增长以及分布式数据库系统的不断涌现,数据存储在物理独立的节点已经成为一种趋势.在这种情况下,当应用需要进行复杂join查询时,就会不可避免地产生非常多的网络传输代价.所以,如何提高分布式系统中join查询的效率成为研究热点.本文在分析分布式数据库系统OceanBase执行nested loop join、Hash join、semi-join等算法的基础上,提出了合理利用硬件资源采用多线程并行执行join操作的优化思想,并在OceanBase数据库中分别对nested loop join、Hash join、semi-join等算法进行了并行改造.实验结果表明,在一定线程数内join算法执行效率与并行度呈正相关.     

基于一致性哈希算法的分布式数据库索引查询优化方法

为了解决分布式数据库索引查询负载不平衡的问题,基于一致性哈希算法设计了分布式数据库索引查询优化方法.根据分布式数据库服务器节点CPU处理能力、带宽和缓存大小,采用动态反馈机制计算各服务器节点的索引查询负载率,通过一致性哈希负载平衡算法将任务分配给各服务器节点,并实时调整负载.以最小总代价为目标,利用多蚁群算法求得最佳查询路径.实验结果显示,基于一致性哈希算法的优化方法整体曲线趋势较平稳,在用户请求数量为4 000条时,最大负载平衡度为1.5%,全程响应时间小于35 ms.这说明该方法能够有效调整分布式数据库的负载分布,且受数据库规模影响较小,具备强大的查询响应能力.     

面向OceanBase的存储过程设计与实现

存储过程是数据库管理系统的一个重要特性,它是标准结构化查询语言(Structured Query Language,SQL)的一个扩展.OceanBase是一个新型的支持海量数据处理的分布式数据库系统,但现有OceanBase的开源版本不支持存储过程功能,影响了该系统在企业和机构中的推广和应用.本文在深度分析存储过程原理以及OceanBase查询处理策略的基础上,设计并实现了支持PL/SQL(Procedural Language/SQL)的存储过程机制.     

中文密文数据库正则查询的研究与实现

为提高中文密文数据库查询的灵活性、便捷性、高效性,设计了一种新的密文索引结构.它通过登记汉字所在记录的记录标识以及汉字在属性值中的位置,来提供对中文正则查询的支持.在这种索引的基础上,提出了一系列正则查询处理算法,这些算法利用索引登记的汉字位置信息来筛选符合查询条件的记录.仿真实验证明,利用这种索引和算法,能够在中文密...     

FOLAPE：基于位向量索引的快速OLAP引擎研究

提高联机分析处理OLAP效率是学术界和工业界共同关注的课题.传统OLAP在处理复杂应用时,运算代价随维度增大而增大,为解决这一问题,做了如下工作:(1)分析现有OLAP模型的不足,设计了一种采用位向量索引技术的快速OLAP模型,FOLAPE;(2)FOLAPE支持索引的增量式更新,提高了数据更新效率;(3)针对FOLAPE特点,设计了新的聚集函数算法;(4)在真实数据上进行了实验验证.实验表明,FOLAPE相对于最新OLAP模型,其平均查询时间最低减少90%,最高减少95%,同时存储空间需求也显著降低.     

Markedness and its Interpretation in Language Use

Language markedness is a common phenomenon in languages, and is reflected from hearing, vision and sense, i.e. the variation in the three aspects such as phonology, morphology and semantics. This paper focuses on the interpretation of markedness in language use following the three perspectives, i.e. pragmatic interpretation, psychological interpretation and cognitive interpretation, with an aim to define the function of markedness.     

Lower Paleozoic oil relationships within Williston Basin, Canada

The Williston Basin is a significant petroleum province, containing oil production zones that include the Middle Cambrian to Lower Ordovician, Upper Ordovician, Middle Devonian, Upper Devonian and Mississippian and within the Jurassic and Cretaceous. The oils of the Williston Basin exhibit a wide range of geochemical characteristics defined as "oil families", although the geochemical signature of the Cambrian Deadwood Formation and Lower Ordovician Winnipeg reservoired oils does not match any "oil family". Despite their close stratigraphic proximity, it is evident that the oils of the Lower Palaeozoic within the Williston Basin are distinct. This suggests the presence of a new "oil family" within the Williston Basin. Diagnostic geochemical signatures occur in the gasoline range chromatograms, within saturate fraction gas chromatograms and biomarker fingerprints. However, some of the established criteria and cross-plots that are currently used to segregate oils into distinct genetic families within the basin do not always meet with success, particularly when applied to the Lower Palaeozoic oils of the Deadwood and Winnipeg Formation.     

An Analysis of Catherine＇s Character——Wuthering Heights

Wuthering Heights, Emily Bronte＇s only novel, was published in December of 1847 under the pseudonym Ellis Bell. The book did not gain immediate success, but it is now thought one of the finest novels in the English language. Catherine is the key character of this masterpiece, because everybody and everything center on her though she had a short life. We can understand this masterpiece better if we know Catherine well.     

Characterization of Ordovician carbonate reservoirs, southeastern Saskatchewan, Canada

The discovery of the prolific Ordovician Red River reservoirs in 1995 in southeastern Saskatchewan was the catalyst for extensive exploration activity which resulted in the discovery of more than 15 new Red River pools. The best yields of Red River production to date have been from dolomite reservoirs. Understanding the processes of dolomitization is, therefore, crucial for the prediction of the connectivity, spatial distribution and heterogeneity of dolomite reservoirs.The Red River reservoirs in the Midale area consist of 3～4 thin dolomitized zones, with a total thickness of about 20 m, which occur at the top of the Yeoman Formation. Two types of replacement dolomite were recognized in the Red River reservoir: dolomitized burrow infills and dolomitized host matrix. The spatial distribution of dolomite suggests that burrowing organisms played an important role in facilitating the fluid flow in the backfilled sediments. This resulted in penecontemporaneous dolomitization of burrow infills by normal seawater. The dolomite in the host matrix is interpreted as having occurred at shallow burial by evaporitic seawater during precipitation of Lake Almar anhydrite that immediately overlies the Yeoman Formation. However, the low δ18O values of dolomited burrow infills (-5.9‰～ -7.8‰, PDB) and matrix dolomites (-6.6‰～ -8.1‰, avg. -7.4‰ PDB) compared to the estimated values for the late Ordovician marine dolomite could be attributed to modification and alteration of dolomite at higher temperatures during deeper burial, which could also be responsible for its 87Sr/86Sr ratios (0.7084～0.7088) that are higher than suggested for the late Ordovician seawaters (0.7078～0.7080). The trace amounts of saddle dolomite cement in the Red River carbonates are probably related to "cannibalization" of earlier replacement dolomite during the chemical compaction.     

Features of Women's language

Language is a means of verbal communication. People use language to communicate with each other. In the society, no two speakers are exactly alike in the way of speaking. Some differences are due to age, gender, statue and personality. Above all, gender is one of the obvious reasons. The writer of this paper tries to describe the features of women＇s language from these perspectives： pronunciation, intonation, diction, subjects, grammar and discourse. From the discussion of the features of women＇s language, more attention should be paid to language use in social context. What＇s more, the linguistic phenomena in a speaking community can be understood more thoroughly.     

Design and analysis of generic LBS application developing platform

Location based services is promising due to its novel working style and contents.A software platform is proposed to provide application programs of typical location based services and support new applications developing efficiently. The analysis shows that this scheme is easy implemented, low cost and adapt to all kinds of mobile nework system.     

橡胶颗粒沥青混合料低温抗裂性试验研究

以AC-13级配为基础,将橡胶颗粒代替部分集料掺入混合料中,以低温弯曲试验为评价方法对不同橡胶颗粒掺量下沥青混合料的低温抗裂性进行研究,并引入应变能密度值对混合料的低温抗裂性进行综合评价.试验结果表明:橡胶颗粒沥青混合料试件的破坏微应变均超过2 300,满足冬寒区的技术指标;无论是否掺加橡胶颗粒,随着温度的下降,沥青混合料破坏时的最大弯拉强度增大,弯拉应变降低,劲度模量增大;弯曲应变能密度在胶粒掺量为1%左右时具有较大的弯曲应变能密度值,此时橡胶颗粒沥青混合料具有较好的低温抗裂性.     

A computer generator for randomly layered structures

AcomputergeneratorforrandomlylayeredstructuresYUJia shun1,2,HEZhen hua2(1.TheInstituteofGeologicalandNuclearSciences,NewZealand;2.StateKeyLaboratoryofOilandGasReservoirGeologyandExploitation,ChengduUniversityofTechnology,China)Abstract:Analgorithmisintrod…     

低渗透非均质砂岩油藏启动压力梯度研究

理论推导与室内实验相结合,建立了低渗透非均质砂岩油藏启动压力梯度确定方法。首先借助油藏流场与电场相似的原理,推导了非均质砂岩油藏启动压力梯度计算公式。其次基于稳定流实验方法,建立了非均质砂岩油藏启动压力梯度测试方法。结果表明:低渗透非均质砂岩油藏的启动压力梯度确定遵循两个等效原则。平面非均质油藏的启动压力梯度等于各级渗透率段的启动压力梯度关于长度的加权平均;纵向非均质油藏的启动压力梯度等于各渗透率层的启动压力梯度关于渗透率与渗流面积乘积的加权平均。研究成果可用于有效指导低渗透非均质砂岩油藏的合理井距确定,促进该类油藏的高效开发。     

On Tragic Consciousness In The Works By Ernest Hemingway

As an American modern novelist who were famous in the literary world, Hemingway was not a person who always followed the trend but a sharp observer. At the same time, he was a tragedy maestro, he paid great attention on existence, fate and end-result. The dramatis personae's tragedy of his works was an extreme limit by all means tragedy on the meaning of fearless challenge that failed. The beauty of tragedy was not produced on the destruction of life, but now this kind of value was in the impact activity. They performed for the reader about the tragedy on challenging for the limit and the death.