redis源碼筆記 - redis對過期值的處理(in redis.c)


redis允許對key設置超時時間,實現過期key的自動淘汰。這篇blog分析下,其自適應(adaptive)的淘汰機制。

redis每隔100ms定時執行的循環(serverCron function)里有如下語句:

 655     /* Expire a few keys per cycle, only if this is a master.
 656      * On slaves we wait for DEL operations synthesized by the master
 657      * in order to guarantee a strict consistency. */
 658     if (server.masterhost == NULL) activeExpireCycle();

正如文中注釋所示,只有master執行expire cycle,slave會等候由master傳遞的DEL消息,保證master-slave在過期值處理上的一致性。(后邊代碼會看到,redis對過期值的選擇是隨機抽取的,master-slave完全可能抽取不同的值,因此要求master通過DEL消息實現同步,同時這種expire機制也是不可靠的expire,即key超時后有可能不會被刪除)。

activeExpireCycle函數如下:

477 /* Try to expire a few timed out keys. The algorithm used is adaptive and
 478  * will use few CPU cycles if there are few expiring keys, otherwise
 479  * it will get more aggressive to avoid that too much memory is used by
 480  * keys that can be removed from the keyspace. */
 481 void activeExpireCycle(void) {
 482     int j;
 483 
 484     for (j = 0; j < server.dbnum; j++) {
 485         int expired;
 486         redisDb *db = server.db+j;
 487 
 488         /* Continue to expire if at the end of the cycle more than 25%
 489          * of the keys were expired. */
 490         do {
 491             long num = dictSize(db->expires);
 492             time_t now = time(NULL);
 493 
 494             expired = 0;
 495             if (num > REDIS_EXPIRELOOKUPS_PER_CRON)
 496                 num = REDIS_EXPIRELOOKUPS_PER_CRON;
 497             while (num--) {
 498                 dictEntry *de;
 499                 time_t t;
 500 
 501                 if ((de = dictGetRandomKey(db->expires)) == NULL) break;
 502                 t = (time_t) dictGetEntryVal(de);
 503                 if (now > t) {
 504                     sds key = dictGetEntryKey(de);
 505                     robj *keyobj = createStringObject(key,sdslen(key));
 506 
 507                     propagateExpire(db,keyobj);     //將刪除操作傳播給各個slaves,在此之前,還將del操作記錄aof
 508                     dbDelete(db,keyobj);   //這個函數先從db->expires中刪除,然后刪除db->dict
 509                     decrRefCount(keyobj);
 510                     expired++;
 511                     server.stat_expiredkeys++;
 512                 }
 513             }
 514         } while (expired > REDIS_EXPIRELOOKUPS_PER_CRON/4);
 515     }
 516 }

ExpireCycle每次嘗試處理10個key,如果10個key中有>2.5個超時,則繼續處理10個key。其用意在於,如果超時的key比例很高,則一次迭代處理很多個,否則等待下次serverCron循環再隨機抽取。


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM