《電子技術(shù)應(yīng)用》
您所在的位置:首頁 > 模擬設(shè)計(jì) > 設(shè)計(jì)應(yīng)用 > 高性能高可用Redis客戶端的設(shè)計(jì)與實(shí)現(xiàn)
高性能高可用Redis客戶端的設(shè)計(jì)與實(shí)現(xiàn)
2022年電子技術(shù)應(yīng)用第1期
劉世超1,,2,,楊 斌1,2,,劉衛(wèi)國1,,2
1.山東大學(xué) 軟件學(xué)院,,山東 濟(jì)南250101;2.國家超級(jí)計(jì)算無錫中心,,江蘇 無錫214072
摘要: Redis是一個(gè)基于內(nèi)存存儲(chǔ)的非結(jié)構(gòu)化數(shù)據(jù)庫,,以高I/O(Input/Output)性能和高響應(yīng)速度著稱,在數(shù)據(jù)緩沖,、消息隊(duì)列,、Key-Value存儲(chǔ)等場景都發(fā)揮著重要的作用。在其支持的眾多客戶端中,,C/C++客戶端Hiredis的應(yīng)用尤為廣泛,。對Hiredis庫做了深入分析,發(fā)現(xiàn)了其管道功能存在高開銷,、指令存儲(chǔ)不當(dāng)以及內(nèi)存混淆問題,。基于此,,在32邏輯核的X86架構(gòu)處理器以及64 GB內(nèi)存的Linux服務(wù)器上,,設(shè)計(jì)并實(shí)現(xiàn)了一個(gè)面向C/C++的高性能高可用Redis客戶端,通過內(nèi)存預(yù)分配以及內(nèi)存隔離的方法提高了大量指令批處理時(shí)的性能并解決了復(fù)雜場景下的內(nèi)存混淆問題,。經(jīng)測試,,新客戶端提高了3~7倍的指令執(zhí)行效率,同時(shí)也保證了復(fù)雜場景下的內(nèi)存安全以及數(shù)據(jù)準(zhǔn)確性,。
中圖分類號(hào): TP391
文獻(xiàn)標(biāo)識(shí)碼: A
DOI:10.16157/j.issn.0258-7998.212432
中文引用格式: 劉世超,,楊斌,劉衛(wèi)國. 高性能高可用Redis客戶端的設(shè)計(jì)與實(shí)現(xiàn)[J].電子技術(shù)應(yīng)用,,2022,,48(1):46-52,,58.
英文引用格式: Liu Shichao,Yang Bin,,Liu Weiguo. Design and implementation of high-performance and high-availability redis client[J]. Application of Electronic Technique,,2022,48(1):46-52,,58.
Design and implementation of high-performance and high-availability redis client
Liu Shichao1,,2,Yang Bin1,,2,,Liu Weiguo1,2
1.School of Software,,Shandong University,Jinan 250101,,China,;2.National Supercomputing Center in Wuxi,Wuxi 214072,,China
Abstract: Redis is an unstructured database based on memory storage. It is known for high I/O(Input/Output) performance and high response speed. It plays an important role in data buffering, message queues, key-value storage and other scenarios. Among the many clients it supports, the C/C++ client Hiredis is particularly widely used. This article did an in-depth analysis of the Hiredis library and found that its pipeline function has high overhead, improper instruction storage, and memory confusion problems. Based on this, this article designs and implements a C/C++-oriented high-performance and high-availability Redis client on a 32-core X86 architecture processor and a 64 GB memory Linux server. It improves the performance of processing a large number of instructions and solves the problem of memory confusion in complex scenarios through memory pre-allocation and memory isolation. After testing, the new client has improved instruction execution efficiency by 3~7 times, while also ensuring memory safety and accuracy in complex scenarios.
Key words : Redis,;pipeline;Hiredis,;memory confusion,;performance optimization

0 引言

    隨著互聯(lián)網(wǎng)飛速發(fā)展以及大規(guī)模應(yīng)用的不斷涌現(xiàn),目前已經(jīng)步入了大數(shù)據(jù)時(shí)代,。非結(jié)構(gòu)化數(shù)據(jù)逐漸替代了傳統(tǒng)結(jié)構(gòu)化數(shù)據(jù)并迅速占據(jù)了主導(dǎo)地位,,為了管理形式多樣的非結(jié)構(gòu)化數(shù)據(jù),涌現(xiàn)了諸如MongoDB[1],、InfluxDB[2],、Elasticsearch[3]等十分有代表性的數(shù)據(jù)庫。這些數(shù)據(jù)庫雖然針對非結(jié)構(gòu)化數(shù)據(jù)的存取做了很多優(yōu)化,,但是受限于硬盤(Hard Disk Drive,,HDD)等底層存儲(chǔ)介質(zhì),往往無法滿足高性能場景的需求,。

    為了提高性能,,以Redis為代表的內(nèi)存數(shù)據(jù)庫應(yīng)運(yùn)而生。Redis是一個(gè)非結(jié)構(gòu)化數(shù)據(jù)庫,,支持使用非結(jié)構(gòu)化語言(Not-only Structured Query Language,,NoSQL)查詢。同時(shí),,Redis通過I/O(Input/Output)多路復(fù)用和DRAM(Dynamic Random Access Memory)提供了高吞吐,、高并發(fā)和低時(shí)延的服務(wù),,在數(shù)據(jù)緩沖、消息隊(duì)列,、Key-Value存儲(chǔ)等場景都發(fā)揮了重要的作用,。

    但是隨著大規(guī)模計(jì)算集群的算力逐漸增大,應(yīng)用的數(shù)據(jù)規(guī)模也隨之變大,,計(jì)算和I/O之間的“存儲(chǔ)墻”也變得愈發(fā)明顯?,F(xiàn)有的Redis也遇到一些網(wǎng)絡(luò)和存儲(chǔ)方面的問題。因此如何改進(jìn)Redis也受到了廣泛的重視,,隨之出現(xiàn)了很多Redis優(yōu)化的相關(guān)工作,,它們從各種角度對Redis服務(wù)端或客戶端做了改進(jìn)。




本文詳細(xì)內(nèi)容請下載:http://forexkbc.com/resource/share/2000003907,。




作者信息:

劉世超1,,2,楊  斌1,,2,,劉衛(wèi)國1,2

(1.山東大學(xué) 軟件學(xué)院,,山東 濟(jì)南250101,;2.國家超級(jí)計(jì)算無錫中心,江蘇 無錫214072)




wd.jpg

此內(nèi)容為AET網(wǎng)站原創(chuàng),,未經(jīng)授權(quán)禁止轉(zhuǎn)載,。