[关键词]
[摘要]
为了探寻城市轨道交通行业大数据平台建设与升级改造的最优方案,本文以城轨大数据平台为研究对象,从城轨大数据平台发展历程出发,梳理了城轨大数据平台发展的三个阶段,分析了各阶段大数据平台所采用的技术与优缺点,重点总结了当前阶段“湖仓一体”大数据技术所具备的“湖仓一体、流批一体、OLTP+OLAP、多重负载”等优点,研究了基于该技术的大数据平台架构升级改造设计要点,并将该技术在北京地铁数据中心的大数据平台升级改造中进行了应用验证。应用表明,“湖仓一体”大数据平台技术兼具数据湖的低成本、数据仓库的高性能等优点,解决了原大数据平台在性能、容量与多用途支持上的不足,为城轨行业大数据平台建设与升级改造提供新的解决思路。
[Key word]
[Abstract]
In order to explore the optimal scheme for the construction and upgrading of the big data platform in the urban rail transit industry, this paper takes the urban rail big data platform as the research object, starts from the development process of the urban rail big data platform, sorts out the three stages of the development of the urban rail big data platform, analyzes the technologies and advantages and disadvantages of the big data platform at each stage, focuses on summarizing the advantages of "Data lake and Warehouse integration, stream processing and batch processing integration, OLTP+OLAP, multiple loads" and other advantages of the "Data lakehouse" big data technology in the current stage, and studies the key points of the architecture upgrade and transformation design of the big data platform based on this technology. The technology is verified in the upgrading and transformation of the big data platform of the Beijing Metro data center. The application shows that the "Data lakehouse" big data platform technology combines the advantages of low cost of data lake and high performance of data warehouse, solves the shortcomings of the original big data platform in performance, capacity and multi-purpose support, and provides new solutions for the construction and upgrading of big data platform in the urban rail industry.
[中图分类号]
U231
[基金项目]