[关键词]
[摘要]
为了探寻城市轨道交通行业大数据平台建设与升级改造的最优方案,本文以城轨大数据平台为研究对象, 从城轨大数据平台发展历程出发,梳理城轨大数据平台发展的 3 个阶段,分析各阶段大数据平台所采用的技术与优 缺点,重点总结当前阶段“湖仓一体”大数据技术所具备的湖仓一体、流批一体、OLTP+OLAP、多重负载等优点, 研究了基于该技术的大数据平台架构升级改造设计要点,并将该技术在北京地铁数据中心的大数据平台升级改造中 进行应用验证。结果表明:“湖仓一体”大数据平台技术兼具数据湖的低成本、数据仓库的高性能等优点,解决了 原大数据平台在性能、容量与多用途支持上的不足,为城轨行业大数据平台建设与升级改造提供了新的解决思路。
[Key word]
[Abstract]
To explore the optimal scheme for the construction and upgrading of the big data platform in the urban rail transitindustry, this study takes the urban rail big data platform as the research object. Our study starts from the development processof the urban rail big data platform, sorts out the three stages of the development of the urban rail big data platform and analyzesthe technology and advantages and disadvantages of the big data platform at each stage. Then it focuses on summarizing theadvantages of “Data lake and Warehouse integration, stream processing and batch processing integration, OLTP+OLAP,multiple loads” and other advantages of the “Data Lakehouse” big data technology in the current stage, and studies the key pointsof the architecture upgrade and transformation design of the big data platform based on this technology. The technology wasverified in the upgradation and transformation of the big data platform of the Beijing Metro Data Center. The application showsthat the “Data lakehouse” big data platform technology combines the advantages of low cost of data lake and high performanceof data warehouse, solves the shortcomings of the original big data platform in performance, capacity and multi-purposesupport, and provides new solutions for the construction and upgradation of big data platforms in the urban rail industry.
[中图分类号]
U231
[基金项目]