大数据智能分析

该专著致力于研究现代分析系统 (AS) 中知识的概念、处理和有效利用问题。研究了此类系统在数据-信息-知识范式中的功能特征，涵盖了所有阶段——从处理原始数据、提取和分析知识，及其在决策过程中的使用。基于对已知的知识表示模型及其处理方法的综合分析，表明构建现代分析系统最可接受的方法是语义建模，它侧重于数据的语义方面、它们的关联关系和数据的公开、隐藏的依赖。不仅研究了构建现代分析系统的理论方面，还介绍了广泛的流程和知识管理技术，从确定知识在现代分析系统中的作用和位置，到现代技术使用的实际建议和示例为结束，包括用于不同学科领域的网络。

山东省科学院情报研究所始建于1983年9月，是山东省重要的情报研究服务机构之一。经过20年的发展，研究所已经形成了构筑在科技网络及丰富的科技信息资源基础上的，面向社会提供科技信息分析研究、科技查新评估、科技人才培训、科研开发、科技宣传报导等各类科技咨询服务的情报研究服务平台。建所以来，共完成院级以上科研、开发课题40余项，有多项成果获得重大奖励。

\"目　录 1　大数据技术 ············································································1 大数据技术人员 ·······································································1 大数据技术 ·············································································6 2　互联网资源作为大数据 ··························································15 大数据来源 ············································································15 数据收集 ···············································································20 生成JSON文件 ·······································································23 RSS转换为JSON程序示例 ························································27 3　技术栈Elastic ·······································································30 Logstash ·················································································30 Kibana ···················································································30 Elastic Cloud ···········································································31 4　Elasticsearch系统 ································································32 安装Elasticsearch ·····································································34 加载Elasticsearch数据库 ···························································35 CRUD操作 ············································································39 对Elasticsearch的查询 ······························································39 5　 Kibana系统与Elasticsearch系统的连接 ·······························43 安装Kibana系统 ·····································································43 使用Console工具工作 ······························································44 使用Discover工具工作 ·····························································46 使用Visualizations工具工作 ·······················································48 6　数据集 ··················································································50 如何查询集 ············································································50 按需发布的趋势 ······································································51 将数据转换为CSV格式 ····························································53 用Excel表格处理数据 ······························································55 在WinPython环境中运行 ··························································56 时间序列校平 ·········································································57 小波分析项 ············································································62 7　网络数据的分析与可视化 ······················································66 介质 ·····················································································66 网络参数 ···············································································66 全局效率系数 ·········································································68 聚类系数 ···············································································68 模块化 ··················································································69 网络现象 ···············································································69 随机埃尔多斯-任易网络模型 ····················································72 Barabashi-Albert随机网络模型 ···················································72 网络节点排名 ·········································································73 检索文件数组 ·········································································74 形成词典 ···············································································75 选择加权词 ············································································77 概念邻接矩阵的形成 ································································79 8　网络数据可视化：GEPHI系统 ··············································83 辅助选项卡窗口 ······································································85 筛选选项卡 ············································································85 “统计”选项卡 ········································································86 “堆叠”选项卡 ········································································86 通过“图”界面创建图 ·····························································87 测试图布局 ············································································96 LES Miserables图的堆叠 ···························································97 排序和统计 ··········································································100 网络过滤 ·············································································102 9　图形数据库管理系统Neo4j ·················································104 安装Neo4j ···········································································106 启动系统 ·············································································108 Cypher语言 ··········································································110 将数据从CSV导入Neo4j图 ····················································114 10　 NoSQL类型的数据库管理系统 ——MongoDB 数据库管理系统 ··································116 服务器和普通客户端安装 ························································118 一些普通客户端命令 ······························································118 以外部格式从表导入数据库 ·····················································121 MongoDB数据库Compass客户端 ·············································122 以外部格式将数据库导出到表 ··················································123 运行GUI MongoDB Compass ····················································124 执行CRUD命令 ····································································125 内容索引 ··················································································129 参考文