中圖分類號：TP393.08文獻標識碼：ADOI:10.19358/j.issn.2097-1788.2023.12.010
引用格式：趙勛,，周成勝，靳文京,，等.基于動態(tài)均衡技術的海量異構數據高并發(fā)可靠接入方法［J］.網絡安全與數據治理,，2023，42（12）：60-66.

High concurrency and reliable access method for massive heterogeneous data based on dynamic balancing technology

Abstract： With the arrival of the era of big data, the highly concurrent and reliable access of massive heterogeneous data has become an urgent problem. This paper proposes a high concurrent and reliable access method for massive heterogeneous data based on dynamic balance technology. The method adopts decentralized task allocation mechanism to access massive data sources. For various heterogeneous data sources, a variety of collection methods based on HTTPS, SFTP, Kafka, and corresponding node allocation and recovery mechanisms are designed. The dynamic load balancing strategy is used to adjust the collection resources in real time to adapt to the changing data load and achieve high concurrency processing. This research provides an effective solution for the efficient and reliable access of massive heterogeneous data.

Key words : massive heterogeneous data; high concurrency; dynamic load balancing strategy