redis key的删除淘汰策略

redis key的删除淘汰策略

删除淘汰策略结论

1. 被动删除
2. 主动删除
3. 当使用内存达到maxMemory时根据配置淘汰规则
   3.1 TTL
   3.2 LRU
   3.2.1 只对设置了过期时间的key进行lru算法删除
   3.2.2 对所有key进行lru算法删除
   3.3 LFU
   3.3.1 只对设置了过期时间的key进行lfu算法删除
   3.3.2 对所有key进行lfu算法删除
   3.4 random
   3.4.1 随机删除设置了过期时间的key
   3.4.2 随机删除key
   3.5 noeviction 永不过期，返回异常

被动删除： 当读写key的时候检查是否过期，过期删除，无法处理冷数据

主动删除： redis有1s10次的定时任务，检查key是否过期，过期则删除

1. 有ttl时间的key
2. 删除已过期的
3. 上述中100个key中有25个被删除，则重复步骤1

LRU: least recently use LRU淘汰最长时间没有使用的

LFU: least frequently use LFU淘汰使用次数最少的

被动删除

每次当进行键值查询是会执行*expireIfNeeded()*判断键值是否过期，过期删除。

int expireIFNeeded(redisDb *db, robj *key) {
 mstime_t when = getExpire(db, key);
 mstime_t now;
 
 if (when < 0) return 0; // No exipre for this key

 /* Dont't expire anyhing while loading, It will be done later. */
  if (server.loading)return 0;
  
  /*
  * If we are in the context of a lua script, we claim that time is
  * blocked to when the lua script started. This way a key can expire
  * only the first time it is accessed and not in the middle of the 
  * script execution, making propagetion to slaves / AOF consistent.
  * see issue #1525 on Github for more information
  */
   now = server.lua_caller ? server.lua_time_start : mstime();
   /*
   * If we are runing in thr context of a slave, return ASAP:
   * the slave key expiration is controlled by the master that will
   * send us synthesize DEL. operation for expired keys.
   *
   * still we try to return the right information to the caller,
   *  that is , 0 if we think the key should be still valid, 1 if
   *  we think the key is expired at this time. 
   */ 
   if (server.masterhost != NULL) return now > when;
  
  /*Return when this key has not expired */
  if (now <= when) reutrn 0; 
  /*delete the key */
  server.stat_expiredkeys++;
  propagateExpire(db, key, server.lazyfree_lazy_expire);
  notifyKeyspaceEvent(NOTIFY_EXPIRED, "expired", key, db->id);
  return server.lazyfree_lazy_expire ? dbAsyncDelete(db,key): dbSyncDelete(db, key);
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

主动删除

主动删除： redis有1s10次的定时任务，检查key是否过期，过期则删除

1. 有ttl时间的key
2. 删除已过期的
3. 上述中100个key中有25个被删除了，则重复步骤1

/*
* file: redis.c
* version: 2.2
*/
int serverCron(struct aeEventLoop *eventLoop, long long id, void *clientData){
 // 主动删除只在master上执行，slave机器根据同步mater的del操作来做过期处理
 if (server.masterhost == null) activeExpireCycle ; 
}

/*
* 每100ms执行一次，每次取出10个，如果有每次有超过2.5个键过期
* 则继续取出10个键，循环
*/
void activeExpireCycle(void){
 int j;
  for (j =0; j < server.dbnum; j++) {
   int expired;
   redisDb *db = server.db+j;
	do {
	 long num = dictSzie(db->expires);
	 time_t now = time(NULL);
     expired = 0;
     if (num > REDIS_EXPIRELOOKUPS_PRE_CRON) {
		// 每次最大取10个键
		num = REDIS_EXPIRELOOKUPS_PRE_CRON;
		while(num--) {
          dictEntry *de;
           time_t t;
           if (de = dictGetRandomKet(db->expires) == NULL)break;
           t = (time_t) dictGetEntryVal(de);
           //判断是否过期
            if (now > t) {
              sds key = dictGetEntryKey(de);
              robj *keyobj = createStringObject(key, sdslen(key));
			  propagareExpire(db, keyobj);
			  // 过期删除键	
		      dbDelete(db, keyobj);
		      decrRefcount(keyobj);
		     // 计数本轮循环，主动过期的数量
		     expired++;
		     server.stat_expiredkeys++;		 
            }
        }
	 }
	} while(expired > REDIS_EXPIRELOOKUPS_PRE_CRON/4);
 }
}
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48

当使用内存达到maxMemory时根据配置淘汰规则

当使用内存达到maxMemory时根据配置淘汰规则
3.1 TTL
3.2 LRU
3.2.1 只对设置了过期时间的key进行lru算法删除
3.2.2 对所有key进行lru算法删除
3.3 LFU
3.3.1 只对设置了过期时间的key进行lfu算法删除
3.3.2 对所有key进行lfu算法删除
3.4 random
3.4.1 随机删除设置了过期时间的key
3.4.2 随机删除key
3.5 noeviction 永不过期，返回异常

/*
* file: redis.conf
* 设置触发淘汰规则的内存大小，当开启这个，则认为redis使用为缓存，而不是持久化的db
* WARNING: maxmemory can be a good idea mainly if you want to use redis as a 'state' server or cache. 
* not as a real db ,when redis is used as a real database the memory useage will grow over the weeks, it will be obvius if it is going to use too much memory in the long run
*/
maxmemory <bytes>

/*
* redis 2.2的淘汰策略
* volatile-lru
* allkey-lru
* volatile-random
* allkeys-random
* volatile-ttl
* noeviction
* redis 4.0.6有新增淘汰策略
* volatile-lfu
* allkeys-lfu
*/

maxmemory-policy volatile-lru
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22