基于Hadoop的大数据论坛日志系统的分析与设计

大数据时代的诞生,为解决挖掘海量数据存储的信息带来了技术上的革命,通过网站点击存留的日志数据,分析这些数据,可获知许多对网站运营至关重要的信息。本文采用Hadoop分布式平台,使用HDFS数据存储,利用Hive来分析日志数据,设计一种Web日志分析系统,并阐述了系统的设计过程。     

基于Web日志分析的校园网受众分析

数字化校园发展迅速,同时带来了海量的网络信息.本文针对校园网信息传播及其监控的有效性问题,设计并实现了基于Web日志分析的校园网受众分析系统,通过挖掘Web日志中的相关数据并进行统计、分析、综合,得到了特定信息在特定时期内的传播范围及其用户分布情况,同时基于访问量特征提出了校园网信息的3种基本类型.本文还结合实际校园网的运行数据进行了实验和相应的分析,显示出所建立模型的合理性和有效性.     

基于Hadoop与Storm的日志实时处理系统研究

日志数据记录着丰富的信息,具有较高的实用价值,但在当今大数据时代环境下,数据量的陡增为日志数据的处理带来了挑战.为了有效地解决海量日志数据处理面临的瓶颈问题,本文整合Hadoop和Storm分布式框架,构建一种融合了实时计算与离线计算的分布式日志实时处理系统.系统架构由数据服务层、业务逻辑层和Web展示层组成,数据服务层使用Flume实时采集日志数据,并分别采用Kafka与HBase完成实时日志流数据的缓冲和系统数据的持久化存储;业务逻辑层利用Storm对实时日志流数据进行实时分析,并使用Hadoop的计算引擎MapReduce结合数据挖掘技术完成对海量历史日志数据的离线分析,离线分析的结果为实时分析提供支持、参考;Web展示层负责日志数据及其分析结果的展示.实验结果表明,系统能有效地解决日志数据的采集存储、实时日志流数据的实时分析和历史日志数据的离线分析等问题,并成功地融合了Hadoop与Storm各自的优势,为日志数据的采集和分析系统的构建提供新的技术参考.     

基于Web日志分析的校园网受众分析

数字化校园发展迅速,同时带来了海量的网络信息.本文针对校园信息传播及其监控的有效性问题,设计并实现了基于Web日志分析的校园网受众分析系统,通过挖掘Web日志中的相关数据并进行统计、分析、综合,得到了特定信息在特定时期内的传播范围及其用户分布情况,同时基于访问量特征提出了校园网信息的3种基本类型.本文还结合实际校园网的运行数据进行了实验和相应的分析,显示出所建立模型的合理性和有效性.     

Web日志挖掘中的数据预处理技术

用户访问网站,Web日志中会记录下大量的用户访问信息,通过挖掘这些日志数据可以获得相关页面、相似用户群体和用户访问模式等信息,Web日志挖掘对于优化网站结构、提供个性化服务和构建智能化网站具有重要作用.数据预处理是保证Web日志挖掘质量的重要基础,预处理主要包括数据清洗、用户识别、会话识别、路径补充和结果评价等工作.概述了数据预处理技术,重点介绍了目前常用的会话划分算法和评价标准,并实现了一个数据预处理系统.     

Spark日志整合与FCM-DNN的网络流量分析算法

为了解决网络设备类型划分粒度粗,导致网络流量无法准确分类的问题,提出了一种基于Spark日志集成与FCM-DNN的流量分析算法。首先,该方法使用Spark集成会话日志以获取可分析的结构化数据;然后对同一网站的行为数据进行聚类,提取网站的多类簇特征集合,以解决单个会话连接特征维度较少、特征相似且不平衡的问题;最后,构建DNN网络,将统一化后的聚类特征与原始特征结合进行训练,并从聚类分组长度和损失函数等多个方面进行算法优化。仿真实验结果表明,对于特征较少的会话日志数据,该算法有效提高了网站分类的准确性,同时在保留学生上网特征的前提下将日志压缩了700倍,从而节省了存储开销。     

多主数据库中基于分区的并发控制

大数据时代,存储计算架构分离的单写多读场景已无法满足海量数据的高效读写需求;另一方面,多个计算节点同时提供写服务还会引起计算节点间的缓存不一致.已有的研究采用全局有序的事务日志来进行冲突检测,并通过广播和回放事务日志维护整个系统的数据一致性.但该类方案由于是在每个写节点维护全局写日志,可扩展性较差.针对这些问题,提出了一个基于分区的并发控制方案:通过分区的方式降低每个写节点需要维护的事务日志,以有效提升系统的扩展能力.基于此想法,在MySQL上实现了分区多主插件,并通过实验验证了该解决方案对系统性能的影响.     

基于模型堆叠的上网行为日志用户画像方法

上网行为日志数据中包含着大量的用户个性化信息,如何充分挖掘和分析这些信息至关重要。在分析上网行为日志数据的重要性后,提出了一种基于上网行为日志用户画像方法。在该方法中,首先通过特征选择和特征提取方式构建用户特征集,然后利用模型堆叠的技术组合多种单一分类器,构建用户画像模型。利用校园网行为日志数据对性别、年级、年龄三个维度进行用户画像,实验结果表明了所提方法的有效性。     

基于关联规则的虚拟主机网站日志挖掘

网站日志记录了用户的浏览行为，通过挖掘网站日志，可以分析用户行为，为优化网站结构提供有价值的数据．但是很多虚拟主机网站没有日志记录功能，这给日志挖掘带来了极大的不便．本文介绍一种生成日志的方法，并利用关联规则算法对其进行日志挖掘，发现用户的访问模式，帮助网站改进页面结构．     

基于关联规则的虚拟主机网站日志挖掘

网站日志记录了用户的浏览行为，通过挖掘网站日志，可以分析用户行为，为优化网站结构提供有价值的数据．但是很多虚拟主机网站没有日志记录功能，这给日志挖掘带来了极大的不便．本文介绍一种生成日志的方法，并利用关联规则算法对其进行日志挖掘，发现用户的访问模式，帮助网站改进页面结构．     

Markedness and its Interpretation in Language Use

Language markedness is a common phenomenon in languages, and is reflected from hearing, vision and sense, i.e. the variation in the three aspects such as phonology, morphology and semantics. This paper focuses on the interpretation of markedness in language use following the three perspectives, i.e. pragmatic interpretation, psychological interpretation and cognitive interpretation, with an aim to define the function of markedness.     

Lower Paleozoic oil relationships within Williston Basin, Canada

The Williston Basin is a significant petroleum province, containing oil production zones that include the Middle Cambrian to Lower Ordovician, Upper Ordovician, Middle Devonian, Upper Devonian and Mississippian and within the Jurassic and Cretaceous. The oils of the Williston Basin exhibit a wide range of geochemical characteristics defined as "oil families", although the geochemical signature of the Cambrian Deadwood Formation and Lower Ordovician Winnipeg reservoired oils does not match any "oil family". Despite their close stratigraphic proximity, it is evident that the oils of the Lower Palaeozoic within the Williston Basin are distinct. This suggests the presence of a new "oil family" within the Williston Basin. Diagnostic geochemical signatures occur in the gasoline range chromatograms, within saturate fraction gas chromatograms and biomarker fingerprints. However, some of the established criteria and cross-plots that are currently used to segregate oils into distinct genetic families within the basin do not always meet with success, particularly when applied to the Lower Palaeozoic oils of the Deadwood and Winnipeg Formation.     

An Analysis of Catherine＇s Character——Wuthering Heights

Wuthering Heights, Emily Bronte＇s only novel, was published in December of 1847 under the pseudonym Ellis Bell. The book did not gain immediate success, but it is now thought one of the finest novels in the English language. Catherine is the key character of this masterpiece, because everybody and everything center on her though she had a short life. We can understand this masterpiece better if we know Catherine well.     

Characterization of Ordovician carbonate reservoirs, southeastern Saskatchewan, Canada

The discovery of the prolific Ordovician Red River reservoirs in 1995 in southeastern Saskatchewan was the catalyst for extensive exploration activity which resulted in the discovery of more than 15 new Red River pools. The best yields of Red River production to date have been from dolomite reservoirs. Understanding the processes of dolomitization is, therefore, crucial for the prediction of the connectivity, spatial distribution and heterogeneity of dolomite reservoirs.The Red River reservoirs in the Midale area consist of 3～4 thin dolomitized zones, with a total thickness of about 20 m, which occur at the top of the Yeoman Formation. Two types of replacement dolomite were recognized in the Red River reservoir: dolomitized burrow infills and dolomitized host matrix. The spatial distribution of dolomite suggests that burrowing organisms played an important role in facilitating the fluid flow in the backfilled sediments. This resulted in penecontemporaneous dolomitization of burrow infills by normal seawater. The dolomite in the host matrix is interpreted as having occurred at shallow burial by evaporitic seawater during precipitation of Lake Almar anhydrite that immediately overlies the Yeoman Formation. However, the low δ18O values of dolomited burrow infills (-5.9‰～ -7.8‰, PDB) and matrix dolomites (-6.6‰～ -8.1‰, avg. -7.4‰ PDB) compared to the estimated values for the late Ordovician marine dolomite could be attributed to modification and alteration of dolomite at higher temperatures during deeper burial, which could also be responsible for its 87Sr/86Sr ratios (0.7084～0.7088) that are higher than suggested for the late Ordovician seawaters (0.7078～0.7080). The trace amounts of saddle dolomite cement in the Red River carbonates are probably related to "cannibalization" of earlier replacement dolomite during the chemical compaction.     

Design and analysis of generic LBS application developing platform

Location based services is promising due to its novel working style and contents.A software platform is proposed to provide application programs of typical location based services and support new applications developing efficiently. The analysis shows that this scheme is easy implemented, low cost and adapt to all kinds of mobile nework system.     

橡胶颗粒沥青混合料低温抗裂性试验研究

以AC-13级配为基础,将橡胶颗粒代替部分集料掺入混合料中,以低温弯曲试验为评价方法对不同橡胶颗粒掺量下沥青混合料的低温抗裂性进行研究,并引入应变能密度值对混合料的低温抗裂性进行综合评价.试验结果表明:橡胶颗粒沥青混合料试件的破坏微应变均超过2 300,满足冬寒区的技术指标;无论是否掺加橡胶颗粒,随着温度的下降,沥青混合料破坏时的最大弯拉强度增大,弯拉应变降低,劲度模量增大;弯曲应变能密度在胶粒掺量为1%左右时具有较大的弯曲应变能密度值,此时橡胶颗粒沥青混合料具有较好的低温抗裂性.     

A computer generator for randomly layered structures

AcomputergeneratorforrandomlylayeredstructuresYUJia shun1,2,HEZhen hua2(1.TheInstituteofGeologicalandNuclearSciences,NewZealand;2.StateKeyLaboratoryofOilandGasReservoirGeologyandExploitation,ChengduUniversityofTechnology,China)Abstract:Analgorithmisintrod…     

低渗透非均质砂岩油藏启动压力梯度研究

理论推导与室内实验相结合,建立了低渗透非均质砂岩油藏启动压力梯度确定方法。首先借助油藏流场与电场相似的原理,推导了非均质砂岩油藏启动压力梯度计算公式。其次基于稳定流实验方法,建立了非均质砂岩油藏启动压力梯度测试方法。结果表明:低渗透非均质砂岩油藏的启动压力梯度确定遵循两个等效原则。平面非均质油藏的启动压力梯度等于各级渗透率段的启动压力梯度关于长度的加权平均;纵向非均质油藏的启动压力梯度等于各渗透率层的启动压力梯度关于渗透率与渗流面积乘积的加权平均。研究成果可用于有效指导低渗透非均质砂岩油藏的合理井距确定,促进该类油藏的高效开发。     

Quantitative trait loci （QTLs） for quality traits related to protein and starch in wheat

Quality traits in wheat （Triticum aestirum L.） were studied by quantitative trait locus （QTL） analysis in a recombinant inbred line （RIL） population, a set of 131 lines derived from Chuan 35050 × Shannong 483 cross （ChSh）. Grains from RILs were assayed for 21 quality traits related to protein and starch. A total of 35 putative QTLs for 19 traits with a single QTL explaining 7.99-40.52% of phenotypic variations were detected on 10 chromosomes, 1D, 2A, 2D, 3B, 3D, 5A, 6A, 6B, 6D, and 7B. The additive effects of 30 QTLs were positive, contributed by Chuan 35050, the remaining 5 QTLs were negative with the additive effect contributed by Shannong 483. For protein traits, 15 QTLs were obtained and most of them were located on chromosomes 1 D, 3B and 6D, while 20 QTLs for starch traits were detected and most of them were located on chromosomes 3D, 6B and 7B. Only 7 QTLs for protein and starch traits were co-located in three regions on chromosomes 1D, 2A and 2D. These protein and starch trait QTLs showed a distinct distribution pattern in certain regions and chromosomes. Twenty-two QTLs were clustered in 6 regions of 5 chromosomes. Two QTL clusters for protein traits were located on chromosomes 1D and 3B, respectively, three clusters for starch traits on chromosomes 3D, 6B and 7B, and one cluster including protein and starch traits on chromosome 1D.     

On Tragic Consciousness In The Works By Ernest Hemingway

As an American modern novelist who were famous in the literary world, Hemingway was not a person who always followed the trend but a sharp observer. At the same time, he was a tragedy maestro, he paid great attention on existence, fate and end-result. The dramatis personae's tragedy of his works was an extreme limit by all means tragedy on the meaning of fearless challenge that failed. The beauty of tragedy was not produced on the destruction of life, but now this kind of value was in the impact activity. They performed for the reader about the tragedy on challenging for the limit and the death.