基于Hadoop的大数据论坛日志系统的分析与设计

大数据时代的诞生,为解决挖掘海量数据存储的信息带来了技术上的革命,通过网站点击存留的日志数据,分析这些数据,可获知许多对网站运营至关重要的信息。本文采用Hadoop分布式平台,使用HDFS数据存储,利用Hive来分析日志数据,设计一种Web日志分析系统,并阐述了系统的设计过程。     

基于Web日志分析的校园网受众分析

数字化校园发展迅速,同时带来了海量的网络信息.本文针对校园网信息传播及其监控的有效性问题,设计并实现了基于Web日志分析的校园网受众分析系统,通过挖掘Web日志中的相关数据并进行统计、分析、综合,得到了特定信息在特定时期内的传播范围及其用户分布情况,同时基于访问量特征提出了校园网信息的3种基本类型.本文还结合实际校园网的运行数据进行了实验和相应的分析,显示出所建立模型的合理性和有效性.     

基于Hadoop与Storm的日志实时处理系统研究

日志数据记录着丰富的信息,具有较高的实用价值,但在当今大数据时代环境下,数据量的陡增为日志数据的处理带来了挑战.为了有效地解决海量日志数据处理面临的瓶颈问题,本文整合Hadoop和Storm分布式框架,构建一种融合了实时计算与离线计算的分布式日志实时处理系统.系统架构由数据服务层、业务逻辑层和Web展示层组成,数据服务层使用Flume实时采集日志数据,并分别采用Kafka与HBase完成实时日志流数据的缓冲和系统数据的持久化存储;业务逻辑层利用Storm对实时日志流数据进行实时分析,并使用Hadoop的计算引擎MapReduce结合数据挖掘技术完成对海量历史日志数据的离线分析,离线分析的结果为实时分析提供支持、参考;Web展示层负责日志数据及其分析结果的展示.实验结果表明,系统能有效地解决日志数据的采集存储、实时日志流数据的实时分析和历史日志数据的离线分析等问题,并成功地融合了Hadoop与Storm各自的优势,为日志数据的采集和分析系统的构建提供新的技术参考.     

基于Web日志分析的校园网受众分析

数字化校园发展迅速,同时带来了海量的网络信息.本文针对校园信息传播及其监控的有效性问题,设计并实现了基于Web日志分析的校园网受众分析系统,通过挖掘Web日志中的相关数据并进行统计、分析、综合,得到了特定信息在特定时期内的传播范围及其用户分布情况,同时基于访问量特征提出了校园网信息的3种基本类型.本文还结合实际校园网的运行数据进行了实验和相应的分析,显示出所建立模型的合理性和有效性.     

Web日志挖掘中的数据预处理技术

用户访问网站,Web日志中会记录下大量的用户访问信息,通过挖掘这些日志数据可以获得相关页面、相似用户群体和用户访问模式等信息,Web日志挖掘对于优化网站结构、提供个性化服务和构建智能化网站具有重要作用.数据预处理是保证Web日志挖掘质量的重要基础,预处理主要包括数据清洗、用户识别、会话识别、路径补充和结果评价等工作.概述了数据预处理技术,重点介绍了目前常用的会话划分算法和评价标准,并实现了一个数据预处理系统.     

Spark日志整合与FCM-DNN的网络流量分析算法

为了解决网络设备类型划分粒度粗,导致网络流量无法准确分类的问题,提出了一种基于Spark日志集成与FCM-DNN的流量分析算法。首先,该方法使用Spark集成会话日志以获取可分析的结构化数据;然后对同一网站的行为数据进行聚类,提取网站的多类簇特征集合,以解决单个会话连接特征维度较少、特征相似且不平衡的问题;最后,构建DNN网络,将统一化后的聚类特征与原始特征结合进行训练,并从聚类分组长度和损失函数等多个方面进行算法优化。仿真实验结果表明,对于特征较少的会话日志数据,该算法有效提高了网站分类的准确性,同时在保留学生上网特征的前提下将日志压缩了700倍,从而节省了存储开销。     

多主数据库中基于分区的并发控制

大数据时代,存储计算架构分离的单写多读场景已无法满足海量数据的高效读写需求;另一方面,多个计算节点同时提供写服务还会引起计算节点间的缓存不一致.已有的研究采用全局有序的事务日志来进行冲突检测,并通过广播和回放事务日志维护整个系统的数据一致性.但该类方案由于是在每个写节点维护全局写日志,可扩展性较差.针对这些问题,提出了一个基于分区的并发控制方案:通过分区的方式降低每个写节点需要维护的事务日志,以有效提升系统的扩展能力.基于此想法,在MySQL上实现了分区多主插件,并通过实验验证了该解决方案对系统性能的影响.     

基于模型堆叠的上网行为日志用户画像方法

上网行为日志数据中包含着大量的用户个性化信息,如何充分挖掘和分析这些信息至关重要。在分析上网行为日志数据的重要性后,提出了一种基于上网行为日志用户画像方法。在该方法中,首先通过特征选择和特征提取方式构建用户特征集,然后利用模型堆叠的技术组合多种单一分类器,构建用户画像模型。利用校园网行为日志数据对性别、年级、年龄三个维度进行用户画像,实验结果表明了所提方法的有效性。     

基于关联规则的虚拟主机网站日志挖掘

网站日志记录了用户的浏览行为，通过挖掘网站日志，可以分析用户行为，为优化网站结构提供有价值的数据．但是很多虚拟主机网站没有日志记录功能，这给日志挖掘带来了极大的不便．本文介绍一种生成日志的方法，并利用关联规则算法对其进行日志挖掘，发现用户的访问模式，帮助网站改进页面结构．     

基于关联规则的虚拟主机网站日志挖掘

网站日志记录了用户的浏览行为，通过挖掘网站日志，可以分析用户行为，为优化网站结构提供有价值的数据．但是很多虚拟主机网站没有日志记录功能，这给日志挖掘带来了极大的不便．本文介绍一种生成日志的方法，并利用关联规则算法对其进行日志挖掘，发现用户的访问模式，帮助网站改进页面结构．     

Markedness and its Interpretation in Language Use

Language markedness is a common phenomenon in languages, and is reflected from hearing, vision and sense, i.e. the variation in the three aspects such as phonology, morphology and semantics. This paper focuses on the interpretation of markedness in language use following the three perspectives, i.e. pragmatic interpretation, psychological interpretation and cognitive interpretation, with an aim to define the function of markedness.     

Characterization of Ordovician carbonate reservoirs, southeastern Saskatchewan, Canada

The discovery of the prolific Ordovician Red River reservoirs in 1995 in southeastern Saskatchewan was the catalyst for extensive exploration activity which resulted in the discovery of more than 15 new Red River pools. The best yields of Red River production to date have been from dolomite reservoirs. Understanding the processes of dolomitization is, therefore, crucial for the prediction of the connectivity, spatial distribution and heterogeneity of dolomite reservoirs.The Red River reservoirs in the Midale area consist of 3～4 thin dolomitized zones, with a total thickness of about 20 m, which occur at the top of the Yeoman Formation. Two types of replacement dolomite were recognized in the Red River reservoir: dolomitized burrow infills and dolomitized host matrix. The spatial distribution of dolomite suggests that burrowing organisms played an important role in facilitating the fluid flow in the backfilled sediments. This resulted in penecontemporaneous dolomitization of burrow infills by normal seawater. The dolomite in the host matrix is interpreted as having occurred at shallow burial by evaporitic seawater during precipitation of Lake Almar anhydrite that immediately overlies the Yeoman Formation. However, the low δ18O values of dolomited burrow infills (-5.9‰～ -7.8‰, PDB) and matrix dolomites (-6.6‰～ -8.1‰, avg. -7.4‰ PDB) compared to the estimated values for the late Ordovician marine dolomite could be attributed to modification and alteration of dolomite at higher temperatures during deeper burial, which could also be responsible for its 87Sr/86Sr ratios (0.7084～0.7088) that are higher than suggested for the late Ordovician seawaters (0.7078～0.7080). The trace amounts of saddle dolomite cement in the Red River carbonates are probably related to "cannibalization" of earlier replacement dolomite during the chemical compaction.     

Features of Women's language

Language is a means of verbal communication. People use language to communicate with each other. In the society, no two speakers are exactly alike in the way of speaking. Some differences are due to age, gender, statue and personality. Above all, gender is one of the obvious reasons. The writer of this paper tries to describe the features of women＇s language from these perspectives： pronunciation, intonation, diction, subjects, grammar and discourse. From the discussion of the features of women＇s language, more attention should be paid to language use in social context. What＇s more, the linguistic phenomena in a speaking community can be understood more thoroughly.     

A computer generator for randomly layered structures

AcomputergeneratorforrandomlylayeredstructuresYUJia shun1,2,HEZhen hua2(1.TheInstituteofGeologicalandNuclearSciences,NewZealand;2.StateKeyLaboratoryofOilandGasReservoirGeologyandExploitation,ChengduUniversityofTechnology,China)Abstract:Analgorithmisintrod…     

低渗透非均质砂岩油藏启动压力梯度研究

理论推导与室内实验相结合,建立了低渗透非均质砂岩油藏启动压力梯度确定方法。首先借助油藏流场与电场相似的原理,推导了非均质砂岩油藏启动压力梯度计算公式。其次基于稳定流实验方法,建立了非均质砂岩油藏启动压力梯度测试方法。结果表明:低渗透非均质砂岩油藏的启动压力梯度确定遵循两个等效原则。平面非均质油藏的启动压力梯度等于各级渗透率段的启动压力梯度关于长度的加权平均;纵向非均质油藏的启动压力梯度等于各渗透率层的启动压力梯度关于渗透率与渗流面积乘积的加权平均。研究成果可用于有效指导低渗透非均质砂岩油藏的合理井距确定,促进该类油藏的高效开发。     

On Tragic Consciousness In The Works By Ernest Hemingway

As an American modern novelist who were famous in the literary world, Hemingway was not a person who always followed the trend but a sharp observer. At the same time, he was a tragedy maestro, he paid great attention on existence, fate and end-result. The dramatis personae's tragedy of his works was an extreme limit by all means tragedy on the meaning of fearless challenge that failed. The beauty of tragedy was not produced on the destruction of life, but now this kind of value was in the impact activity. They performed for the reader about the tragedy on challenging for the limit and the death.     

海南岛地体及其毗邻陆缘晚中生代—新生代古地磁研究和构造演化

本文叙述了对海南岛及其毗邻大陆边缘白垩纪到第四纪地层岩石进行古地磁研究的全部工作过程。通过分析岩石中剩余磁矢量的磁偏角及磁倾角的变化,提出海南岛白垩纪以来经历的构造演化模式如下:早期伴随顺时针旋转而向南迁移,后期伴随逆时针转动并向北运移。联系该地区及邻区的地质、地球物理资料,对海南岛上述的构造地体运动提出以下认识:北部湾内早期有一拉张作用,主要是该作用使湾内地壳显著伸长减薄,形成北部湾盆地。从而导致了海南岛的早期构造运动,而海南岛后期的构造运动则主要是受南海海底扩张的影响。海南地体运动规律的阐明对于了解北部湾油气盆地的形成演化有重要的理论和实际意义。     

An Index mechanism for multi-scale view in spatial databases

There are numerous geometric objects stored in the spatial databases. An importance function in a spatial database is that users can browse the geometric objects as a map efficiently. Thus the spatial database should display the geometric objects users concern about swiftly onto the display window. This process includes two operations:retrieve data from database and then draw them onto screen. Accordingly, to improve the efficiency, we should try to reduce time of both retrieving object and displaying them. The former can be achieved with the aid of spatial index such as R-tree, the latter require to simplify the objects. Simplification means that objects are shown with sufficient but not with unnecessary detail which depend on the scale of browse. So the major problem is how to retrieve data at different detail level efficiently. This paper introduces the implementation of a multi-scale index in the spatial database SISP (Spatial Information Shared Platform) which is generalized from R-tree. The difference between the generalization and the R-tree lies on two facets: One is that every node and geometric object in the generalization is assigned with a importance value which denote the importance of them, and every vertex in the objects are assigned with a importance value,too. The importance value can be use to decide which data should be retrieve from disk in a query. The other difference is that geometric objects in the generalization are divided into one or more sub-blocks, and vertexes are total ordered by their importance value. With the help of the generalized R-tree, one can easily retrieve data at different detail levels.Some experiments are performed on real-life data to evaluate the performance of solutions that separately use normal spatial index and multi-scale spatial index. The results show that the solution using multi-scale index in SISP is satisfying.     

Jane Eyre——A Feminist In The 19th Century

In the 19th century the society was controlled by men, and women were just appendants of them, they had not any rights and freedom. But Jane was an exception, she showed some characteristics of early feminist. Jane showed her characteristics of feminism in three aspects: rebellion, equality, and independence. These characteristics were helpful to her success, and feminism is the only way out for women of that time.     

《Chinese Science Bulletin》SUBJECT INDEX TO VOLUME 59(2014)

正~~