From 224891d6baf655950853343bb2da11bf1cadf520 Mon Sep 17 00:00:00 2001 From: doujiang24 Date: Wed, 21 Oct 2015 19:36:29 +0800 Subject: [PATCH 01/11] feature: add api in shdict: lpush, lpop, rpush, rpop, llen, expire --- README.markdown | 108 ++++ doc/HttpLuaModule.wiki | 84 +++ src/ngx_http_lua_shdict.c | 1052 ++++++++++++++++++++++++++++++++----- src/ngx_http_lua_shdict.h | 8 + t/133-shdict-list.t | 680 ++++++++++++++++++++++++ 5 files changed, 1789 insertions(+), 143 deletions(-) create mode 100644 t/133-shdict-list.t diff --git a/README.markdown b/README.markdown index 21cafe689d..fea2963b7d 100644 --- a/README.markdown +++ b/README.markdown @@ -3041,6 +3041,12 @@ Nginx API for Lua * [ngx.shared.DICT.replace](#ngxshareddictreplace) * [ngx.shared.DICT.delete](#ngxshareddictdelete) * [ngx.shared.DICT.incr](#ngxshareddictincr) +* [ngx.shared.DICT.lpush](#ngxshareddictlpush) +* [ngx.shared.DICT.rpush](#ngxshareddictrpush) +* [ngx.shared.DICT.lpop](#ngxshareddictlpop) +* [ngx.shared.DICT.rpop](#ngxshareddictrpop) +* [ngx.shared.DICT.llen](#ngxshareddictllen) +* [ngx.shared.DICT.expire](#ngxshareddictexpire) * [ngx.shared.DICT.flush_all](#ngxshareddictflush_all) * [ngx.shared.DICT.flush_expired](#ngxshareddictflush_expired) * [ngx.shared.DICT.get_keys](#ngxshareddictget_keys) @@ -6035,6 +6041,12 @@ The resulting object `dict` has the following methods: * [replace](#ngxshareddictreplace) * [delete](#ngxshareddictdelete) * [incr](#ngxshareddictincr) +* [lpush](#ngxshareddictlpush) +* [rpush](#ngxshareddictrpush) +* [lpop](#ngxshareddictlpop) +* [rpop](#ngxshareddictrpop) +* [llen](#ngxshareddictllen) +* [expire](#ngxshareddictexpire) * [flush_all](#ngxshareddictflush_all) * [flush_expired](#ngxshareddictflush_expired) * [get_keys](#ngxshareddictget_keys) @@ -6298,6 +6310,102 @@ See also [ngx.shared.DICT](#ngxshareddict). [Back to TOC](#nginx-api-for-lua) +ngx.shared.DICT.lpush +--------------------- +**syntax:** *length, err = ngx.shared.DICT:lpush(key, value)* + +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.** + +Insert the specified (numerical or string) `value` at the head of the list named `key` in the shm-based dictionary [ngx.shared.DICT](#ngxshareddict). Returns the length of the list after the push operations. + +If `key` does not exist, it is created as empty list before performing the push operations. When key holds a value that is not a list, it will return `nil` and `"wrongtype operation"`. + +It never overrides the (least recently used) unexpired items in the store when running out of storage in the shared memory zone. In this case, it will immediately return `nil` and the string "no memory". + +This feature was first introduced in the `v0.*.*` release. + +See also [ngx.shared.DICT](#ngxshareddict). + +[Back to TOC](#nginx-api-for-lua) + +ngx.shared.DICT.rpush +--------------------- +**syntax:** *length, err = ngx.shared.DICT:rpush(key, value)* + +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.** + +Similar to the [lpush](#ngxshareddictlpush) method, but insert the specified (numerical or string) `value` at the tail of the list named `key`. + +This feature was first introduced in the `v0.*.*` release. + +See also [ngx.shared.DICT](#ngxshareddict). + +[Back to TOC](#nginx-api-for-lua) + +ngx.shared.DICT.lpop +-------------------- +**syntax:** *val, err = ngx.shared.DICT:lpop(key)* + +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.** + +Removes and returns the first element of the list named `key` in the shm-based dictionary [ngx.shared.DICT](#ngxshareddict). + +If `key` does not exist, it will return `nil`. When key holds a value that is not a list, it will return `nil` and `"wrongtype operation"`. + +This feature was first introduced in the `v0.*.*` release. + +See also [ngx.shared.DICT](#ngxshareddict). + +[Back to TOC](#nginx-api-for-lua) + +ngx.shared.DICT.rpop +-------------------- +**syntax:** *val, err = ngx.shared.DICT:rpop(key)* + +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.** + +Removes and returns the last element of the list named `key` in the shm-based dictionary [ngx.shared.DICT](#ngxshareddict). + +If `key` does not exist, it will return `nil`. When key holds a value that is not a list, it will return `nil` and `"wrongtype operation"`. + +This feature was first introduced in the `v0.*.*` release. + +See also [ngx.shared.DICT](#ngxshareddict). + +[Back to TOC](#nginx-api-for-lua) + +ngx.shared.DICT.llen +-------------------- +**syntax:** *len, err = ngx.shared.DICT:llen(key)* + +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.** + +Returns the length of the list named `key` in the shm-based dictionary [ngx.shared.DICT](#ngxshareddict). + +If key does not exist, it is interpreted as an empty list and 0 is returned. When key holds a value that is not a list, it will return `nil` and `"wrongtype operation"`. + +This feature was first introduced in the `v0.*.*` release. + +See also [ngx.shared.DICT](#ngxshareddict). + +[Back to TOC](#nginx-api-for-lua) + +ngx.shared.DICT.expire +---------------------- +**syntax:** *ok, err = ngx.shared.DICT:expire(key, exptime)* + +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.** + +The `exptime` argument specifies expiration time (in seconds) for the inserted key-value pair. The time resolution is 0.001 seconds. If the exptime takes the value 0, then the item will never be expired. + +If key does not exist or has been expired, it will return `false` and `"not found"`. Otherwise it will return `true`. + +This feature was first introduced in the `v0.*.*` release. + +See also [ngx.shared.DICT](#ngxshareddict). + +[Back to TOC](#nginx-api-for-lua) + ngx.shared.DICT.flush_all ------------------------- **syntax:** *ngx.shared.DICT:flush_all()* diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index b6438eca3a..2a583e3d8e 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -5053,6 +5053,12 @@ The resulting object dict has the following methods: * [[#ngx.shared.DICT.replace|replace]] * [[#ngx.shared.DICT.delete|delete]] * [[#ngx.shared.DICT.incr|incr]] +* [[#ngx.shared.DICT.lpush|lpush]] +* [[#ngx.shared.DICT.rpush|rpush]] +* [[#ngx.shared.DICT.lpop|lpop]] +* [[#ngx.shared.DICT.rpop|rpop]] +* [[#ngx.shared.DICT.llen|llen]] +* [[#ngx.shared.DICT.expire|expire]] * [[#ngx.shared.DICT.flush_all|flush_all]] * [[#ngx.shared.DICT.flush_expired|flush_expired]] * [[#ngx.shared.DICT.get_keys|get_keys]] @@ -5281,6 +5287,84 @@ The optional `init` parameter was first added in the v0.10.6 releas See also [[#ngx.shared.DICT|ngx.shared.DICT]]. +== ngx.shared.DICT.lpush == +'''syntax:''' ''length, err = ngx.shared.DICT:lpush(key, value)'' + +'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*'' + +Insert the specified (numerical or string) value at the head of the list named key in the shm-based dictionary [[#ngx.shared.DICT|ngx.shared.DICT]]. Returns the length of the list after the push operations. + +If key does not exist, it is created as empty list before performing the push operations. When key holds a value that is not a list, it will return nil and "wrongtype operation". + +It never overrides the (least recently used) unexpired items in the store when running out of storage in the shared memory zone. In this case, it will immediately return nil and the string "no memory". + +This feature was first introduced in the v0.*.* release. + +See also [[#ngx.shared.DICT|ngx.shared.DICT]]. + +== ngx.shared.DICT.rpush == +'''syntax:''' ''length, err = ngx.shared.DICT:rpush(key, value)'' + +'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*'' + +Similar to the [[#ngx.shared.DICT.lpush|lpush]] method, but insert the specified (numerical or string) value at the tail of the list named key. + +This feature was first introduced in the v0.*.* release. + +See also [[#ngx.shared.DICT|ngx.shared.DICT]]. + +== ngx.shared.DICT.lpop == +'''syntax:''' ''val, err = ngx.shared.DICT:lpop(key)'' + +'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*'' + +Removes and returns the first element of the list named key in the shm-based dictionary [[#ngx.shared.DICT|ngx.shared.DICT]]. + +If key does not exist, it will return nil. When key holds a value that is not a list, it will return nil and "wrongtype operation". + +This feature was first introduced in the v0.*.* release. + +See also [[#ngx.shared.DICT|ngx.shared.DICT]]. + +== ngx.shared.DICT.rpop == +'''syntax:''' ''val, err = ngx.shared.DICT:rpop(key)'' + +'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*'' + +Removes and returns the last element of the list named key in the shm-based dictionary [[#ngx.shared.DICT|ngx.shared.DICT]]. + +If key does not exist, it will return nil. When key holds a value that is not a list, it will return nil and "wrongtype operation". + +This feature was first introduced in the v0.*.* release. + +See also [[#ngx.shared.DICT|ngx.shared.DICT]]. + +== ngx.shared.DICT.llen == +'''syntax:''' ''len, err = ngx.shared.DICT:llen(key)'' + +'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*'' + +Returns the length of the list named key in the shm-based dictionary [[#ngx.shared.DICT|ngx.shared.DICT]]. + +If key does not exist, it is interpreted as an empty list and 0 is returned. When key holds a value that is not a list, it will return nil and "wrongtype operation". + +This feature was first introduced in the v0.*.* release. + +See also [[#ngx.shared.DICT|ngx.shared.DICT]]. + +== ngx.shared.DICT.expire == +'''syntax:''' ''ok, err = ngx.shared.DICT:expire(key, exptime)'' + +'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*'' + +The exptime argument specifies expiration time (in seconds) for the inserted key-value pair. The time resolution is 0.001 seconds. If the exptime takes the value 0, then the item will never be expired. + +If key does not exist or has been expired, it will return false and "not found". Otherwise it will return true. + +This feature was first introduced in the v0.*.* release. + +See also [[#ngx.shared.DICT|ngx.shared.DICT]]. + == ngx.shared.DICT.flush_all == '''syntax:''' ''ngx.shared.DICT:flush_all()'' diff --git a/src/ngx_http_lua_shdict.c b/src/ngx_http_lua_shdict.c index e9974b4a49..ae26c603cc 100644 --- a/src/ngx_http_lua_shdict.c +++ b/src/ngx_http_lua_shdict.c @@ -34,6 +34,14 @@ static int ngx_http_lua_shdict_delete(lua_State *L); static int ngx_http_lua_shdict_flush_all(lua_State *L); static int ngx_http_lua_shdict_flush_expired(lua_State *L); static int ngx_http_lua_shdict_get_keys(lua_State *L); +static int ngx_http_lua_shdict_lpush(lua_State *L); +static int ngx_http_lua_shdict_rpush(lua_State *L); +static int ngx_http_lua_shdict_push_helper(lua_State *L, int flags); +static int ngx_http_lua_shdict_lpop(lua_State *L); +static int ngx_http_lua_shdict_rpop(lua_State *L); +static int ngx_http_lua_shdict_pop_helper(lua_State *L, int flags); +static int ngx_http_lua_shdict_llen(lua_State *L); +static int ngx_http_lua_shdict_set_expire(lua_State *L); static ngx_inline ngx_shm_zone_t *ngx_http_lua_shdict_get_zone(lua_State *L, @@ -43,6 +51,8 @@ static ngx_inline ngx_shm_zone_t *ngx_http_lua_shdict_get_zone(lua_State *L, #define NGX_HTTP_LUA_SHDICT_ADD 0x0001 #define NGX_HTTP_LUA_SHDICT_REPLACE 0x0002 #define NGX_HTTP_LUA_SHDICT_SAFE_STORE 0x0004 +#define NGX_HTTP_LUA_SHDICT_LEFT 0x0001 +#define NGX_HTTP_LUA_SHDICT_RIGHT 0x0002 enum { @@ -50,6 +60,11 @@ enum { }; +#define ngx_http_lua_shdict_get_list_head(sd, key_len) \ + (ngx_queue_t *) ngx_align_ptr(((u_char *) &sd->data + key_len), \ + NGX_ALIGNMENT) + + ngx_int_t ngx_http_lua_shdict_init_zone(ngx_shm_zone_t *shm_zone, void *data) { @@ -249,13 +264,14 @@ ngx_http_lua_shdict_lookup(ngx_shm_zone_t *shm_zone, ngx_uint_t hash, static int ngx_http_lua_shdict_expire(ngx_http_lua_shdict_ctx_t *ctx, ngx_uint_t n) { - ngx_time_t *tp; - uint64_t now; - ngx_queue_t *q; - int64_t ms; - ngx_rbtree_node_t *node; - ngx_http_lua_shdict_node_t *sd; - int freed = 0; + ngx_time_t *tp; + uint64_t now; + ngx_queue_t *q, *list_queue, *lq; + int64_t ms; + ngx_rbtree_node_t *node; + ngx_http_lua_shdict_node_t *sd; + int freed = 0; + ngx_http_lua_shdict_list_node_t *lnode; tp = ngx_timeofday(); @@ -289,6 +305,21 @@ ngx_http_lua_shdict_expire(ngx_http_lua_shdict_ctx_t *ctx, ngx_uint_t n) } } + if (sd->value_type == LUA_TTABLE) { + + list_queue = ngx_http_lua_shdict_get_list_head(sd, sd->key_len); + + for (lq = ngx_queue_head(list_queue); + lq != ngx_queue_sentinel(list_queue); + lq = ngx_queue_next(lq)) + { + lnode = ngx_queue_data(lq, ngx_http_lua_shdict_list_node_t, + queue); + + ngx_slab_free_locked(ctx->shpool, lnode); + } + } + ngx_queue_remove(q); node = (ngx_rbtree_node_t *) @@ -316,7 +347,7 @@ ngx_http_lua_inject_shdict_api(ngx_http_lua_main_conf_t *lmcf, lua_State *L) lua_createtable(L, 0, lmcf->shm_zones->nelts /* nrec */); /* ngx.shared */ - lua_createtable(L, 0 /* narr */, 13 /* nrec */); /* shared mt */ + lua_createtable(L, 0 /* narr */, 19 /* nrec */); /* shared mt */ lua_pushcfunction(L, ngx_http_lua_shdict_get); lua_setfield(L, -2, "get"); @@ -345,6 +376,24 @@ ngx_http_lua_inject_shdict_api(ngx_http_lua_main_conf_t *lmcf, lua_State *L) lua_pushcfunction(L, ngx_http_lua_shdict_delete); lua_setfield(L, -2, "delete"); + lua_pushcfunction(L, ngx_http_lua_shdict_lpush); + lua_setfield(L, -2, "lpush"); + + lua_pushcfunction(L, ngx_http_lua_shdict_rpush); + lua_setfield(L, -2, "rpush"); + + lua_pushcfunction(L, ngx_http_lua_shdict_lpop); + lua_setfield(L, -2, "lpop"); + + lua_pushcfunction(L, ngx_http_lua_shdict_rpop); + lua_setfield(L, -2, "rpop"); + + lua_pushcfunction(L, ngx_http_lua_shdict_llen); + lua_setfield(L, -2, "llen"); + + lua_pushcfunction(L, ngx_http_lua_shdict_set_expire); + lua_setfield(L, -2, "expire"); + lua_pushcfunction(L, ngx_http_lua_shdict_flush_all); lua_setfield(L, -2, "flush_all"); @@ -540,6 +589,14 @@ ngx_http_lua_shdict_get_helper(lua_State *L, int get_stale) lua_pushboolean(L, c ? 1 : 0); break; + case LUA_TTABLE: + + ngx_shmtx_unlock(&ctx->shpool->mutex); + + lua_pushnil(L); + lua_pushliteral(L, "wrongtype operation"); + return 2; + default: ngx_shmtx_unlock(&ctx->shpool->mutex); @@ -640,16 +697,17 @@ ngx_http_lua_shdict_flush_all(lua_State *L) static int ngx_http_lua_shdict_flush_expired(lua_State *L) { - ngx_queue_t *q, *prev; - ngx_http_lua_shdict_node_t *sd; - ngx_http_lua_shdict_ctx_t *ctx; - ngx_shm_zone_t *zone; - ngx_time_t *tp; - int freed = 0; - int attempts = 0; - ngx_rbtree_node_t *node; - uint64_t now; - int n; + ngx_queue_t *q, *prev, *list_queue, *lq; + ngx_http_lua_shdict_node_t *sd; + ngx_http_lua_shdict_ctx_t *ctx; + ngx_shm_zone_t *zone; + ngx_time_t *tp; + int freed = 0; + int attempts = 0; + ngx_rbtree_node_t *node; + uint64_t now; + int n; + ngx_http_lua_shdict_list_node_t *lnode; n = lua_gettop(L); @@ -690,6 +748,22 @@ ngx_http_lua_shdict_flush_expired(lua_State *L) sd = ngx_queue_data(q, ngx_http_lua_shdict_node_t, queue); if (sd->expires != 0 && sd->expires <= now) { + + if (sd->value_type == LUA_TTABLE) { + + list_queue = ngx_http_lua_shdict_get_list_head(sd, sd->key_len); + + for (lq = ngx_queue_head(list_queue); + lq != ngx_queue_sentinel(list_queue); + lq = ngx_queue_next(lq)) + { + lnode = ngx_queue_data(lq, ngx_http_lua_shdict_list_node_t, + queue); + + ngx_slab_free_locked(ctx->shpool, lnode); + } + } + ngx_queue_remove(q); node = (ngx_rbtree_node_t *) @@ -870,6 +944,7 @@ ngx_http_lua_shdict_set_helper(lua_State *L, int flags) /* indicates whether to foricibly override other * valid entries */ int32_t user_flags = 0; + ngx_queue_t *queue, *q; n = lua_gettop(L); @@ -1015,7 +1090,9 @@ ngx_http_lua_shdict_set_helper(lua_State *L, int flags) replace: - if (value.data && value.len == (size_t) sd->value_len) { + if (value.data && value.len == (size_t) sd->value_len + && sd->value_type != LUA_TTABLE) + { ngx_log_debug0(NGX_LOG_DEBUG_HTTP, ctx->log, 0, "lua shared dict set: found old entry and value " @@ -1060,6 +1137,21 @@ ngx_http_lua_shdict_set_helper(lua_State *L, int flags) remove: + if (sd->value_type == LUA_TTABLE) { + queue = ngx_http_lua_shdict_get_list_head(sd, key.len); + + for (q = ngx_queue_head(queue); + q != ngx_queue_sentinel(queue); + q = ngx_queue_next(q)) + { + p = (u_char *) ngx_queue_data(q, + ngx_http_lua_shdict_list_node_t, + queue); + + ngx_slab_free_locked(ctx->shpool, p); + } + } + ngx_queue_remove(&sd->queue); node = (ngx_rbtree_node_t *) @@ -1503,103 +1595,94 @@ ngx_http_lua_shared_dict_get(ngx_shm_zone_t *zone, u_char *key_data, } -ngx_shm_zone_t * -ngx_http_lua_find_zone(u_char *name_data, size_t name_len) +static int +ngx_http_lua_shdict_lpush(lua_State *L) { - ngx_str_t *name; - ngx_uint_t i; - ngx_shm_zone_t *zone; - volatile ngx_list_part_t *part; - - part = &ngx_cycle->shared_memory.part; - zone = part->elts; + return ngx_http_lua_shdict_push_helper(L, NGX_HTTP_LUA_SHDICT_LEFT); +} - for (i = 0; /* void */ ; i++) { - if (i >= part->nelts) { - if (part->next == NULL) { - break; - } +static int +ngx_http_lua_shdict_rpush(lua_State *L) +{ + return ngx_http_lua_shdict_push_helper(L, NGX_HTTP_LUA_SHDICT_RIGHT); +} - part = part->next; - zone = part->elts; - i = 0; - } - name = &zone[i].shm.name; +static int +ngx_http_lua_shdict_push_helper(lua_State *L, int flags) +{ + int n; + ngx_str_t key; + uint32_t hash; + ngx_int_t rc; + ngx_http_lua_shdict_ctx_t *ctx; + ngx_http_lua_shdict_node_t *sd; + ngx_str_t value; + int value_type; + double num; + ngx_rbtree_node_t *node; + ngx_shm_zone_t *zone; + ngx_queue_t *queue, *q; + ngx_http_lua_shdict_list_node_t *lnode; - dd("name: [%.*s] %d", (int) name->len, name->data, (int) name->len); - dd("name2: [%.*s] %d", (int) name_len, name_data, (int) name_len); + n = lua_gettop(L); - if (name->len == name_len - && ngx_strncmp(name->data, name_data, name_len) == 0) - { - return &zone[i]; - } + if (n != 3) { + return luaL_error(L, "expecting 3 arguments, " + "but only seen %d", n); } - return NULL; -} + if (lua_type(L, 1) != LUA_TTABLE) { + return luaL_error(L, "bad \"zone\" argument"); + } + zone = ngx_http_lua_shdict_get_zone(L, 1); + if (zone == NULL) { + return luaL_error(L, "bad \"zone\" argument"); + } -#ifndef NGX_LUA_NO_FFI_API -int -ngx_http_lua_ffi_shdict_store(ngx_shm_zone_t *zone, int op, u_char *key, - size_t key_len, int value_type, u_char *str_value_buf, - size_t str_value_len, double num_value, int exptime, int user_flags, - char **errmsg, int *forcible) -{ - int i, n; - u_char c, *p; - uint32_t hash; - ngx_int_t rc; - ngx_time_t *tp; - ngx_rbtree_node_t *node; - ngx_http_lua_shdict_ctx_t *ctx; - ngx_http_lua_shdict_node_t *sd; + ctx = zone->data; - if (zone == NULL) { - return NGX_ERROR; + if (lua_isnil(L, 2)) { + lua_pushnil(L); + lua_pushliteral(L, "nil key"); + return 2; } - dd("exptime: %d", exptime); + key.data = (u_char *) luaL_checklstring(L, 2, &key.len); - ctx = zone->data; + if (key.len == 0) { + lua_pushnil(L); + lua_pushliteral(L, "empty key"); + return 2; + } - *forcible = 0; + if (key.len > 65535) { + lua_pushnil(L); + lua_pushliteral(L, "key too long"); + return 2; + } - hash = ngx_crc32_short(key, key_len); + hash = ngx_crc32_short(key.data, key.len); + + value_type = lua_type(L, 3); switch (value_type) { case LUA_TSTRING: - /* do nothing */ + value.data = (u_char *) lua_tolstring(L, 3, &value.len); break; case LUA_TNUMBER: - dd("num value: %lf", num_value); - str_value_buf = (u_char *) &num_value; - str_value_len = sizeof(double); - break; - - case LUA_TBOOLEAN: - c = num_value ? 1 : 0; - str_value_buf = &c; - str_value_len = sizeof(u_char); - break; - - case LUA_TNIL: - if (op & (NGX_HTTP_LUA_SHDICT_ADD|NGX_HTTP_LUA_SHDICT_REPLACE)) { - *errmsg = "attempt to add or replace nil values"; - return NGX_ERROR; - } - - str_value_buf = NULL; - str_value_len = 0; + value.len = sizeof(double); + num = lua_tonumber(L, 3); + value.data = (u_char *) # break; default: - *errmsg = "unsupported value type"; - return NGX_ERROR; + lua_pushnil(L); + lua_pushliteral(L, "bad value type"); + return 2; } ngx_shmtx_lock(&ctx->shpool->mutex); @@ -1608,93 +1691,776 @@ ngx_http_lua_ffi_shdict_store(ngx_shm_zone_t *zone, int op, u_char *key, ngx_http_lua_shdict_expire(ctx, 1); #endif - rc = ngx_http_lua_shdict_lookup(zone, hash, key, key_len, &sd); + rc = ngx_http_lua_shdict_lookup(zone, hash, key.data, key.len, &sd); - dd("lookup returns %d", (int) rc); + dd("shdict lookup returned %d", (int) rc); - if (op & NGX_HTTP_LUA_SHDICT_REPLACE) { + if (rc == NGX_DONE) { + /* exists but expired */ - if (rc == NGX_DECLINED || rc == NGX_DONE) { - ngx_shmtx_unlock(&ctx->shpool->mutex); - *errmsg = "not found"; - return NGX_DECLINED; + if (sd->value_type != LUA_TTABLE) { + /* TODO: reuse when length matched */ + + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, ctx->log, 0, + "lua shared dict push: found old entry and value " + "type not matched, remove it first"); + + ngx_queue_remove(&sd->queue); + + node = (ngx_rbtree_node_t *) + ((u_char *) sd - offsetof(ngx_rbtree_node_t, color)); + + ngx_rbtree_delete(&ctx->sh->rbtree, node); + + ngx_slab_free_locked(ctx->shpool, node); + + dd("go to init_list"); + goto init_list; } - /* rc == NGX_OK */ - goto replace; - } + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, ctx->log, 0, + "lua shared dict push: found old entry and value " + "type matched, reusing it"); - if (op & NGX_HTTP_LUA_SHDICT_ADD) { + sd->expires = 0; - if (rc == NGX_OK) { - ngx_shmtx_unlock(&ctx->shpool->mutex); - *errmsg = "exists"; - return NGX_DECLINED; + /* free list nodes */ + + queue = ngx_http_lua_shdict_get_list_head(sd, key.len); + + for (q = ngx_queue_head(queue); + q != ngx_queue_sentinel(queue); + q = ngx_queue_next(q)) + { + /* TODO: reuse matched size list node */ + lnode = ngx_queue_data(q, ngx_http_lua_shdict_list_node_t, queue); + ngx_slab_free_locked(ctx->shpool, lnode); } - if (rc == NGX_DONE) { - /* exists but expired */ + ngx_queue_init(queue); - dd("go to replace"); - goto replace; + ngx_queue_remove(&sd->queue); + ngx_queue_insert_head(&ctx->sh->queue, &sd->queue); + + dd("go to push_node"); + goto push_node; + + } else if (rc == NGX_OK) { + + if (sd->value_type != LUA_TTABLE) { + ngx_shmtx_unlock(&ctx->shpool->mutex); + + lua_pushnil(L); + lua_pushliteral(L, "wrongtype operation"); + return 2; } - /* rc == NGX_DECLINED */ + queue = ngx_http_lua_shdict_get_list_head(sd, key.len); - dd("go to insert"); - goto insert; + ngx_queue_remove(&sd->queue); + ngx_queue_insert_head(&ctx->sh->queue, &sd->queue); + + dd("go to push_node"); + goto push_node; } - if (rc == NGX_OK || rc == NGX_DONE) { + /* rc == NGX_DECLINED, not found */ - if (value_type == LUA_TNIL) { - goto remove; - } +init_list: -replace: + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, ctx->log, 0, + "lua shared dict list: creating a new entry"); - if (str_value_buf && str_value_len == (size_t) sd->value_len) { + /* NOTICE: we assume the begin point aligned in slab, be careful */ + n = offsetof(ngx_rbtree_node_t, color) + + offsetof(ngx_http_lua_shdict_node_t, data) + + key.len + + sizeof(ngx_queue_t); - ngx_log_debug0(NGX_LOG_DEBUG_HTTP, ctx->log, 0, - "lua shared dict set: found old entry and value " - "size matched, reusing it"); + dd("length before aligned: %d", n); - ngx_queue_remove(&sd->queue); - ngx_queue_insert_head(&ctx->sh->queue, &sd->queue); + n = (int) (uintptr_t) ngx_align_ptr(n, NGX_ALIGNMENT); - sd->key_len = (u_short) key_len; + dd("length after aligned: %d", n); - if (exptime > 0) { - tp = ngx_timeofday(); - sd->expires = (uint64_t) tp->sec * 1000 + tp->msec - + (uint64_t) exptime; + node = ngx_slab_alloc_locked(ctx->shpool, n); - } else { - sd->expires = 0; - } + if (node == NULL) { + ngx_shmtx_unlock(&ctx->shpool->mutex); - sd->user_flags = user_flags; + lua_pushboolean(L, 0); + lua_pushliteral(L, "no memory"); + return 2; + } - sd->value_len = (uint32_t) str_value_len; + sd = (ngx_http_lua_shdict_node_t *) &node->color; - dd("setting value type to %d", value_type); + queue = ngx_http_lua_shdict_get_list_head(sd, key.len); - sd->value_type = (uint8_t) value_type; + node->key = hash; + sd->key_len = (u_short) key.len; - p = ngx_copy(sd->data, key, key_len); - ngx_memcpy(p, str_value_buf, str_value_len); + sd->expires = 0; - ngx_shmtx_unlock(&ctx->shpool->mutex); + sd->value_len = 0; - return NGX_OK; - } + dd("setting value type to %d", (int) LUA_TTABLE); - ngx_log_debug0(NGX_LOG_DEBUG_HTTP, ctx->log, 0, - "lua shared dict set: found old entry but value size " - "NOT matched, removing it first"); + sd->value_type = (uint8_t) LUA_TTABLE; + + ngx_memcpy(sd->data, key.data, key.len); + + ngx_queue_init(queue); + + ngx_rbtree_insert(&ctx->sh->rbtree, node); + + ngx_queue_insert_head(&ctx->sh->queue, &sd->queue); + +push_node: + + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, ctx->log, 0, + "lua shared dict list: creating a new list node"); + + n = sizeof(ngx_http_lua_shdict_list_node_t) + + value.len; + + lnode = ngx_slab_alloc_locked(ctx->shpool, + sizeof(ngx_http_lua_shdict_list_node_t)); + + if (lnode == NULL) { + + if (sd->value_len == 0) { + + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, ctx->log, 0, + "lua shared dict list: no memory for create" + " list node and list empty, remove it"); + + ngx_queue_remove(&sd->queue); + + node = (ngx_rbtree_node_t *) + ((u_char *) sd - offsetof(ngx_rbtree_node_t, color)); + + ngx_rbtree_delete(&ctx->sh->rbtree, node); + + ngx_slab_free_locked(ctx->shpool, node); + } + + ngx_shmtx_unlock(&ctx->shpool->mutex); + + lua_pushboolean(L, 0); + lua_pushliteral(L, "no memory"); + return 2; + } + + dd("setting list length to %d", sd->value_len + 1); + + sd->value_len = sd->value_len + 1; + + dd("setting list node value length to %d", (int) value.len); + + lnode->value_len = (uint32_t) value.len; + + dd("setting list node value type to %d", value_type); + + lnode->value_type = (uint8_t) value_type; + + ngx_memcpy(lnode->data, value.data, value.len); + + if (flags == NGX_HTTP_LUA_SHDICT_LEFT) { + ngx_queue_insert_head(queue, &lnode->queue); + + } else { + ngx_queue_insert_tail(queue, &lnode->queue); + } + + ngx_shmtx_unlock(&ctx->shpool->mutex); + + lua_pushnumber(L, sd->value_len); + return 1; +} + + +static int +ngx_http_lua_shdict_lpop(lua_State *L) +{ + return ngx_http_lua_shdict_pop_helper(L, NGX_HTTP_LUA_SHDICT_LEFT); +} + + +static int +ngx_http_lua_shdict_rpop(lua_State *L) +{ + return ngx_http_lua_shdict_pop_helper(L, NGX_HTTP_LUA_SHDICT_RIGHT); +} + + + +static int +ngx_http_lua_shdict_pop_helper(lua_State *L, int flags) +{ + int n; + ngx_str_t name; + ngx_str_t key; + uint32_t hash; + ngx_int_t rc; + ngx_http_lua_shdict_ctx_t *ctx; + ngx_http_lua_shdict_node_t *sd; + ngx_str_t value; + int value_type; + double num; + ngx_rbtree_node_t *node; + ngx_shm_zone_t *zone; + ngx_queue_t *queue; + ngx_http_lua_shdict_list_node_t *lnode; + + n = lua_gettop(L); + + if (n != 2) { + return luaL_error(L, "expecting 2 arguments, " + "but only seen %d", n); + } + + if (lua_type(L, 1) != LUA_TTABLE) { + return luaL_error(L, "bad \"zone\" argument"); + } + + zone = ngx_http_lua_shdict_get_zone(L, 1); + if (zone == NULL) { + return luaL_error(L, "bad \"zone\" argument"); + } + + ctx = zone->data; + name = ctx->name; + + if (lua_isnil(L, 2)) { + lua_pushnil(L); + lua_pushliteral(L, "nil key"); + return 2; + } + + key.data = (u_char *) luaL_checklstring(L, 2, &key.len); + + if (key.len == 0) { + lua_pushnil(L); + lua_pushliteral(L, "empty key"); + return 2; + } + + if (key.len > 65535) { + lua_pushnil(L); + lua_pushliteral(L, "key too long"); + return 2; + } + + hash = ngx_crc32_short(key.data, key.len); + + ngx_shmtx_lock(&ctx->shpool->mutex); + +#if 1 + ngx_http_lua_shdict_expire(ctx, 1); +#endif + + rc = ngx_http_lua_shdict_lookup(zone, hash, key.data, key.len, &sd); + + dd("shdict lookup returned %d", (int) rc); + + if (rc == NGX_DECLINED || rc == NGX_DONE) { + ngx_shmtx_unlock(&ctx->shpool->mutex); + lua_pushnil(L); + return 1; + } + + /* rc == NGX_OK */ + + if (sd->value_type != LUA_TTABLE) { + ngx_shmtx_unlock(&ctx->shpool->mutex); + + lua_pushnil(L); + lua_pushliteral(L, "wrongtype operation"); + return 2; + } + + if (sd->value_len <= 0) { + ngx_shmtx_unlock(&ctx->shpool->mutex); + + return luaL_error(L, "bad lua list length found for key %s " + "in shared_dict %s: %lu", key.data, name.data, + (unsigned long) sd->value_len); + } + + queue = ngx_http_lua_shdict_get_list_head(sd, key.len); + + if (flags == NGX_HTTP_LUA_SHDICT_LEFT) { + queue = ngx_queue_head(queue); + + } else { + queue = ngx_queue_last(queue); + } + + lnode = ngx_queue_data(queue, ngx_http_lua_shdict_list_node_t, queue); + + value_type = lnode->value_type; + + dd("data: %p", lnode->data); + dd("value len: %d", (int) sd->value_len); + + value.data = lnode->data; + value.len = (size_t) lnode->value_len; + + switch (value_type) { + case LUA_TSTRING: + + lua_pushlstring(L, (char *) value.data, value.len); + break; + + case LUA_TNUMBER: + + if (value.len != sizeof(double)) { + + ngx_shmtx_unlock(&ctx->shpool->mutex); + + return luaL_error(L, "bad lua list node number value size found " + "for key %s in shared_dict %s: %lu", key.data, + name.data, (unsigned long) value.len); + } + + ngx_memcpy(&num, value.data, sizeof(double)); + + lua_pushnumber(L, num); + break; + + default: + + ngx_shmtx_unlock(&ctx->shpool->mutex); + + return luaL_error(L, "bad list node value type found for key %s in " + "shared_dict %s: %d", key.data, name.data, + value_type); + } + + ngx_queue_remove(queue); + + ngx_slab_free_locked(ctx->shpool, lnode); + + if (sd->value_len == 1) { + + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, ctx->log, 0, + "lua shared dict list: empty node after pop, " + "remove it"); + + ngx_queue_remove(&sd->queue); + + node = (ngx_rbtree_node_t *) + ((u_char *) sd - offsetof(ngx_rbtree_node_t, color)); + + ngx_rbtree_delete(&ctx->sh->rbtree, node); + + ngx_slab_free_locked(ctx->shpool, node); + + } else { + + sd->value_len = sd->value_len - 1; + + ngx_queue_remove(&sd->queue); + ngx_queue_insert_head(&ctx->sh->queue, &sd->queue); + } + + ngx_shmtx_unlock(&ctx->shpool->mutex); + + return 1; +} + + +static int +ngx_http_lua_shdict_llen(lua_State *L) +{ + int n; + ngx_str_t key; + uint32_t hash; + ngx_int_t rc; + ngx_http_lua_shdict_ctx_t *ctx; + ngx_http_lua_shdict_node_t *sd; + ngx_shm_zone_t *zone; + + n = lua_gettop(L); + + if (n != 2) { + return luaL_error(L, "expecting 2 arguments, " + "but only seen %d", n); + } + + if (lua_type(L, 1) != LUA_TTABLE) { + return luaL_error(L, "bad \"zone\" argument"); + } + + zone = ngx_http_lua_shdict_get_zone(L, 1); + if (zone == NULL) { + return luaL_error(L, "bad \"zone\" argument"); + } + + ctx = zone->data; + + if (lua_isnil(L, 2)) { + lua_pushnil(L); + lua_pushliteral(L, "nil key"); + return 2; + } + + key.data = (u_char *) luaL_checklstring(L, 2, &key.len); + + if (key.len == 0) { + lua_pushnil(L); + lua_pushliteral(L, "empty key"); + return 2; + } + + if (key.len > 65535) { + lua_pushnil(L); + lua_pushliteral(L, "key too long"); + return 2; + } + + hash = ngx_crc32_short(key.data, key.len); + + ngx_shmtx_lock(&ctx->shpool->mutex); + +#if 1 + ngx_http_lua_shdict_expire(ctx, 1); +#endif + + rc = ngx_http_lua_shdict_lookup(zone, hash, key.data, key.len, &sd); + + dd("shdict lookup returned %d", (int) rc); + + if (rc == NGX_OK) { + + if (sd->value_type != LUA_TTABLE) { + ngx_shmtx_unlock(&ctx->shpool->mutex); + + lua_pushnil(L); + lua_pushliteral(L, "wrongtype operation"); + return 2; + } + + ngx_queue_remove(&sd->queue); + ngx_queue_insert_head(&ctx->sh->queue, &sd->queue); + + lua_pushnumber(L, (lua_Number) sd->value_len); + + ngx_shmtx_unlock(&ctx->shpool->mutex); + return 1; + } + + ngx_shmtx_unlock(&ctx->shpool->mutex); + + lua_pushnumber(L, 0); + return 1; +} + + +static int +ngx_http_lua_shdict_set_expire(lua_State *L) +{ + int n; + ngx_str_t key; + uint32_t hash; + ngx_int_t rc; + ngx_http_lua_shdict_ctx_t *ctx; + ngx_http_lua_shdict_node_t *sd; + ngx_shm_zone_t *zone; + lua_Number exptime = 0; + ngx_time_t *tp; + + n = lua_gettop(L); + + if (n != 3) { + return luaL_error(L, "expecting 3 arguments, " + "but only seen %d", n); + } + + if (lua_type(L, 1) != LUA_TTABLE) { + return luaL_error(L, "bad \"zone\" argument"); + } + + zone = ngx_http_lua_shdict_get_zone(L, 1); + if (zone == NULL) { + return luaL_error(L, "bad \"zone\" argument"); + } + + ctx = zone->data; + + if (lua_isnil(L, 2)) { + lua_pushnil(L); + lua_pushliteral(L, "nil key"); + return 2; + } + + key.data = (u_char *) luaL_checklstring(L, 2, &key.len); + + if (key.len == 0) { + lua_pushnil(L); + lua_pushliteral(L, "empty key"); + return 2; + } + + if (key.len > 65535) { + lua_pushnil(L); + lua_pushliteral(L, "key too long"); + return 2; + } + + hash = ngx_crc32_short(key.data, key.len); + + exptime = luaL_checknumber(L, 3); + if (exptime < 0) { + exptime = 0; + } + + ngx_shmtx_lock(&ctx->shpool->mutex); + + rc = ngx_http_lua_shdict_lookup(zone, hash, key.data, key.len, &sd); + + dd("shdict lookup returned %d", (int) rc); + + if (rc == NGX_OK) { + + if (exptime > 0) { + tp = ngx_timeofday(); + sd->expires = (uint64_t) tp->sec * 1000 + tp->msec + + (uint64_t) (exptime * 1000); + + } else { + sd->expires = 0; + } + + ngx_queue_remove(&sd->queue); + ngx_queue_insert_head(&ctx->sh->queue, &sd->queue); + + ngx_shmtx_unlock(&ctx->shpool->mutex); + + lua_pushboolean(L, 1); + return 1; + } + + ngx_shmtx_unlock(&ctx->shpool->mutex); + + lua_pushboolean(L, 0); + lua_pushliteral(L, "not found"); + return 2; +} + + +ngx_shm_zone_t * +ngx_http_lua_find_zone(u_char *name_data, size_t name_len) +{ + ngx_str_t *name; + ngx_uint_t i; + ngx_shm_zone_t *zone; + volatile ngx_list_part_t *part; + + part = &ngx_cycle->shared_memory.part; + zone = part->elts; + + for (i = 0; /* void */ ; i++) { + + if (i >= part->nelts) { + if (part->next == NULL) { + break; + } + + part = part->next; + zone = part->elts; + i = 0; + } + + name = &zone[i].shm.name; + + dd("name: [%.*s] %d", (int) name->len, name->data, (int) name->len); + dd("name2: [%.*s] %d", (int) name_len, name_data, (int) name_len); + + if (name->len == name_len + && ngx_strncmp(name->data, name_data, name_len) == 0) + { + return zone; + } + } + + return NULL; +} + + +#ifndef NGX_LUA_NO_FFI_API +int +ngx_http_lua_ffi_shdict_store(ngx_shm_zone_t *zone, int op, u_char *key, + size_t key_len, int value_type, u_char *str_value_buf, + size_t str_value_len, double num_value, int exptime, int user_flags, + char **errmsg, int *forcible) +{ + int i, n; + u_char c, *p; + uint32_t hash; + ngx_int_t rc; + ngx_time_t *tp; + ngx_queue_t *queue, *q; + ngx_rbtree_node_t *node; + ngx_http_lua_shdict_ctx_t *ctx; + ngx_http_lua_shdict_node_t *sd; + + if (zone == NULL) { + return NGX_ERROR; + } + + dd("exptime: %d", exptime); + + ctx = zone->data; + + *forcible = 0; + + hash = ngx_crc32_short(key, key_len); + + switch (value_type) { + case LUA_TSTRING: + /* do nothing */ + break; + + case LUA_TNUMBER: + dd("num value: %lf", num_value); + str_value_buf = (u_char *) &num_value; + str_value_len = sizeof(double); + break; + + case LUA_TBOOLEAN: + c = num_value ? 1 : 0; + str_value_buf = &c; + str_value_len = sizeof(u_char); + break; + + case LUA_TNIL: + if (op & (NGX_HTTP_LUA_SHDICT_ADD|NGX_HTTP_LUA_SHDICT_REPLACE)) { + *errmsg = "attempt to add or replace nil values"; + return NGX_ERROR; + } + + str_value_buf = NULL; + str_value_len = 0; + break; + + default: + *errmsg = "unsupported value type"; + return NGX_ERROR; + } + + ngx_shmtx_lock(&ctx->shpool->mutex); + +#if 1 + ngx_http_lua_shdict_expire(ctx, 1); +#endif + + rc = ngx_http_lua_shdict_lookup(zone, hash, key, key_len, &sd); + + dd("lookup returns %d", (int) rc); + + if (op & NGX_HTTP_LUA_SHDICT_REPLACE) { + + if (rc == NGX_DECLINED || rc == NGX_DONE) { + ngx_shmtx_unlock(&ctx->shpool->mutex); + *errmsg = "not found"; + return NGX_DECLINED; + } + /* rc == NGX_OK */ + + goto replace; + } + + if (op & NGX_HTTP_LUA_SHDICT_ADD) { + + if (rc == NGX_OK) { + ngx_shmtx_unlock(&ctx->shpool->mutex); + *errmsg = "exists"; + return NGX_DECLINED; + } + + if (rc == NGX_DONE) { + /* exists but expired */ + + dd("go to replace"); + goto replace; + } + + /* rc == NGX_DECLINED */ + + dd("go to insert"); + goto insert; + } + + if (rc == NGX_OK || rc == NGX_DONE) { + + if (value_type == LUA_TNIL) { + goto remove; + } + +replace: + + if (str_value_buf && str_value_len == (size_t) sd->value_len + && sd->value_type != LUA_TTABLE) + { + + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, ctx->log, 0, + "lua shared dict set: found old entry and value " + "size matched, reusing it"); + + ngx_queue_remove(&sd->queue); + ngx_queue_insert_head(&ctx->sh->queue, &sd->queue); + + sd->key_len = (u_short) key_len; + + if (exptime > 0) { + tp = ngx_timeofday(); + sd->expires = (uint64_t) tp->sec * 1000 + tp->msec + + (uint64_t) exptime; + + } else { + sd->expires = 0; + } + + sd->user_flags = user_flags; + + sd->value_len = (uint32_t) str_value_len; + + dd("setting value type to %d", value_type); + + sd->value_type = (uint8_t) value_type; + + p = ngx_copy(sd->data, key, key_len); + ngx_memcpy(p, str_value_buf, str_value_len); + + ngx_shmtx_unlock(&ctx->shpool->mutex); + + return NGX_OK; + } + + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, ctx->log, 0, + "lua shared dict set: found old entry but value size " + "NOT matched, removing it first"); remove: + if (sd->value_type == LUA_TTABLE) { + + queue = ngx_http_lua_shdict_get_list_head(sd, key_len); + + for (q = ngx_queue_head(queue); + q != ngx_queue_sentinel(queue); + q = ngx_queue_next(q)) + { + p = (u_char *) ngx_queue_data(q, + ngx_http_lua_shdict_list_node_t, + queue); + + ngx_slab_free_locked(ctx->shpool, p); + } + } + ngx_queue_remove(&sd->queue); node = (ngx_rbtree_node_t *) diff --git a/src/ngx_http_lua_shdict.h b/src/ngx_http_lua_shdict.h index aa1bb5800a..77954ab0e3 100644 --- a/src/ngx_http_lua_shdict.h +++ b/src/ngx_http_lua_shdict.h @@ -24,6 +24,14 @@ typedef struct { } ngx_http_lua_shdict_node_t; +typedef struct { + uint8_t value_type; + uint32_t value_len; + ngx_queue_t queue; + u_char data[1]; +} ngx_http_lua_shdict_list_node_t; + + typedef struct { ngx_rbtree_t rbtree; ngx_rbtree_node_t sentinel; diff --git a/t/133-shdict-list.t b/t/133-shdict-list.t new file mode 100644 index 0000000000..649fa377b4 --- /dev/null +++ b/t/133-shdict-list.t @@ -0,0 +1,680 @@ +# vim:set ft= ts=4 sw=4 et fdm=marker: +use Test::Nginx::Socket::Lua; + +#worker_connections(1014); +#master_process_enabled(1); +#log_level('warn'); + +#repeat_each(2); + +plan tests => repeat_each() * (blocks() * 3 + 0); + +#no_diff(); +no_long_string(); +#master_on(); +#workers(2); + +run_tests(); + +__DATA__ + +=== TEST 1: lpush & lpop +--- http_config + lua_shared_dict dogs 1m; +--- config + location = /test { + content_by_lua_block { + local dogs = ngx.shared.dogs + + local len, err = dogs:lpush("foo", "bar") + if len then + ngx.say("push success") + else + ngx.say("push err: ", err) + end + + local val, err = dogs:llen("foo") + ngx.say(val, " ", err) + + local val, err = dogs:lpop("foo") + ngx.say(val, " ", err) + + local val, err = dogs:llen("foo") + ngx.say(val, " ", err) + + local val, err = dogs:lpop("foo") + ngx.say(val, " ", err) + } + } +--- request +GET /test +--- response_body +push success +1 nil +bar nil +0 nil +nil nil +--- no_error_log +[error] + + + +=== TEST 2: get operation on list type +--- http_config + lua_shared_dict dogs 1m; +--- config + location = /test { + content_by_lua_block { + local dogs = ngx.shared.dogs + + local len, err = dogs:lpush("foo", "bar") + if len then + ngx.say("push success") + else + ngx.say("push err: ", err) + end + + local val, err = dogs:get("foo") + ngx.say(val, " ", err) + } + } +--- request +GET /test +--- response_body +push success +nil wrongtype operation +--- no_error_log +[error] + + + +=== TEST 3: set operation on list type +--- http_config + lua_shared_dict dogs 1m; +--- config + location = /test { + content_by_lua_block { + local dogs = ngx.shared.dogs + + local len, err = dogs:lpush("foo", "bar") + if len then + ngx.say("push success") + else + ngx.say("push err: ", err) + end + + local ok, err = dogs:set("foo", "bar") + ngx.say(ok, " ", err) + + local val, err = dogs:get("foo") + ngx.say(val, " ", err) + } + } +--- request +GET /test +--- response_body +push success +true nil +bar nil +--- no_error_log +[error] + + + +=== TEST 4: replace operation on list type +--- http_config + lua_shared_dict dogs 1m; +--- config + location = /test { + content_by_lua_block { + local dogs = ngx.shared.dogs + + local len, err = dogs:lpush("foo", "bar") + if len then + ngx.say("push success") + else + ngx.say("push err: ", err) + end + + local ok, err = dogs:replace("foo", "bar") + ngx.say(ok, " ", err) + + local val, err = dogs:get("foo") + ngx.say(val, " ", err) + } + } +--- request +GET /test +--- response_body +push success +true nil +bar nil +--- no_error_log +[error] + + + +=== TEST 5: add operation on list type +--- http_config + lua_shared_dict dogs 1m; +--- config + location = /test { + content_by_lua_block { + local dogs = ngx.shared.dogs + + local len, err = dogs:lpush("foo", "bar") + if len then + ngx.say("push success") + else + ngx.say("push err: ", err) + end + + local ok, err = dogs:add("foo", "bar") + ngx.say(ok, " ", err) + + local val, err = dogs:get("foo") + ngx.say(val, " ", err) + } + } +--- request +GET /test +--- response_body +push success +false exists +nil wrongtype operation +--- no_error_log +[error] + + + +=== TEST 6: delete operation on list type +--- http_config + lua_shared_dict dogs 1m; +--- config + location = /test { + content_by_lua_block { + local dogs = ngx.shared.dogs + + local len, err = dogs:lpush("foo", "bar") + if len then + ngx.say("push success") + else + ngx.say("push err: ", err) + end + + local ok, err = dogs:delete("foo") + ngx.say(ok, " ", err) + + local val, err = dogs:get("foo") + ngx.say(val, " ", err) + } + } +--- request +GET /test +--- response_body +push success +true nil +nil nil +--- no_error_log +[error] + + + +=== TEST 7: incr operation on list type +--- http_config + lua_shared_dict dogs 1m; +--- config + location = /test { + content_by_lua_block { + local dogs = ngx.shared.dogs + + local len, err = dogs:lpush("foo", "bar") + if len then + ngx.say("push success") + else + ngx.say("push err: ", err) + end + + local ok, err = dogs:incr("foo", 1) + ngx.say(ok, " ", err) + + local val, err = dogs:get("foo") + ngx.say(val, " ", err) + } + } +--- request +GET /test +--- response_body +push success +nil not a number +nil wrongtype operation +--- no_error_log +[error] + + + +=== TEST 8: get_keys operation on list type +--- http_config + lua_shared_dict dogs 1m; +--- config + location = /test { + content_by_lua_block { + local dogs = ngx.shared.dogs + + local len, err = dogs:lpush("foo", "bar") + if len then + ngx.say("push success") + else + ngx.say("push err: ", err) + end + + local keys, err = dogs:get_keys() + ngx.say("key: ", keys[1]) + } + } +--- request +GET /test +--- response_body +push success +key: foo +--- no_error_log +[error] + + + +=== TEST 9: push operation on key-value type +--- http_config + lua_shared_dict dogs 1m; +--- config + location = /test { + content_by_lua_block { + local dogs = ngx.shared.dogs + + local ok, err = dogs:set("foo", "bar") + if ok then + ngx.say("set success") + else + ngx.say("set err: ", err) + end + + local len, err = dogs:lpush("foo", "bar") + ngx.say(len, " ", err) + + local val, err = dogs:get("foo") + ngx.say(val, " ", err) + } + } +--- request +GET /test +--- response_body +set success +nil wrongtype operation +bar nil +--- no_error_log +[error] + + + +=== TEST 10: pop operation on key-value type +--- http_config + lua_shared_dict dogs 1m; +--- config + location = /test { + content_by_lua_block { + local dogs = ngx.shared.dogs + + local ok, err = dogs:set("foo", "bar") + if ok then + ngx.say("set success") + else + ngx.say("set err: ", err) + end + + local val, err = dogs:lpop("foo") + ngx.say(val, " ", err) + + local val, err = dogs:get("foo") + ngx.say(val, " ", err) + } + } +--- request +GET /test +--- response_body +set success +nil wrongtype operation +bar nil +--- no_error_log +[error] + + + +=== TEST 11: llen operation on key-value type +--- http_config + lua_shared_dict dogs 1m; +--- config + location = /test { + content_by_lua_block { + local dogs = ngx.shared.dogs + + local ok, err = dogs:set("foo", "bar") + if ok then + ngx.say("set success") + else + ngx.say("set err: ", err) + end + + local val, err = dogs:llen("foo") + ngx.say(val, " ", err) + + local val, err = dogs:get("foo") + ngx.say(val, " ", err) + } + } +--- request +GET /test +--- response_body +set success +nil wrongtype operation +bar nil +--- no_error_log +[error] + + + +=== TEST 12: lpush and lpop +--- http_config + lua_shared_dict dogs 1m; +--- config + location = /test { + content_by_lua_block { + local dogs = ngx.shared.dogs + + for i = 1, 3 do + local len, err = dogs:lpush("foo", i) + if len ~= i then + ngx.say("push err: ", err) + break + end + end + + for i = 1, 3 do + local val, err = dogs:lpop("foo") + if not val then + ngx.say("pop err: ", err) + break + else + ngx.say(val) + end + end + } + } +--- request +GET /test +--- response_body +3 +2 +1 +--- no_error_log +[error] + + + +=== TEST 13: lpush and rpop +--- http_config + lua_shared_dict dogs 1m; +--- config + location = /test { + content_by_lua_block { + local dogs = ngx.shared.dogs + + for i = 1, 3 do + local len, err = dogs:lpush("foo", i) + if len ~= i then + ngx.say("push err: ", err) + break + end + end + + for i = 1, 3 do + local val, err = dogs:rpop("foo") + if not val then + ngx.say("pop err: ", err) + break + else + ngx.say(val) + end + end + } + } +--- request +GET /test +--- response_body +1 +2 +3 +--- no_error_log +[error] + + + +=== TEST 14: rpush and lpop +--- http_config + lua_shared_dict dogs 1m; +--- config + location = /test { + content_by_lua_block { + local dogs = ngx.shared.dogs + + for i = 1, 3 do + local len, err = dogs:rpush("foo", i) + if len ~= i then + ngx.say("push err: ", err) + break + end + end + + for i = 1, 3 do + local val, err = dogs:lpop("foo") + if not val then + ngx.say("pop err: ", err) + break + else + ngx.say(val) + end + end + } + } +--- request +GET /test +--- response_body +1 +2 +3 +--- no_error_log +[error] + + + +=== TEST 15: list removed: expired +--- http_config + lua_shared_dict dogs 1m; +--- config + location = /test { + content_by_lua_block { + local dogs = ngx.shared.dogs + + local max + for i = 1, 10000 do + local key = string.format("%05d", i) + + local len , err = dogs:lpush(key, i) + if not len then + max = i + break + end + + local ok = dogs:expire(key, 0.01) + if not ok then + ngx.say("expire error") + end + end + + local keys = dogs:get_keys(0) + + ngx.say("max-1 matched keys length: ", max-1 == #keys) + + ngx.sleep(0.01) + + local keys = dogs:get_keys(0) + + ngx.say("keys all expired, left number: ", #keys) + + -- some reused, some removed + for i = 10000, 1, -1 do + local key = string.format("%05d", i) + + local len, err = dogs:lpush(key, i) + if not len then + ngx.say("loop again, max matched: ", 10001-i == max) + break + end + end + + dogs:flush_all() + + dogs:flush_expired() + + for i = 1, 10000 do + local key = string.format("%05d", i) + + local len, err = dogs:lpush(key, i) + if not len then + ngx.say("loop again, max matched: ", i == max) + break + end + end + } + } +--- request +GET /test +--- response_body +max-1 matched keys length: true +keys all expired, left number: 0 +loop again, max matched: true +loop again, max matched: true +--- no_error_log +[error] + + + +=== TEST 16: list removed: forcibly +--- http_config + lua_shared_dict dogs 1m; +--- config + location = /test { + content_by_lua_block { + local dogs = ngx.shared.dogs + + local max + for i = 1, 20000 do + local ok, err, forcible = dogs:set(i, i) + if not ok or forcible then + max = i + break + end + end + + local two = dogs:get(2) + + ngx.say("two == number 2: ", two == 2) + + dogs:flush_all() + dogs:flush_expired() + + local keys = dogs:get_keys(0) + + ngx.say("no one left: ", #keys) + + for i = 1, 20000 do + local key = string.format("%05d", i) + + local len, err = dogs:lpush(key, i) + if not len then + break + end + end + + for i = 1, max do + local ok, err = dogs:set(i, i) + if not ok then + ngx.say("set err: ", err) + break + end + end + + local two = dogs:get(2) + + ngx.say("two == number 2: ", two == 2) + } + } +--- request +GET /test +--- response_body +two == number 2: true +no one left: 0 +two == number 2: true +--- no_error_log +[error] + + + +=== TEST 16: expire on all types +--- http_config + lua_shared_dict dogs 1m; +--- config + location = /test { + content_by_lua_block { + local dogs = ngx.shared.dogs + + local len, err = dogs:lpush("list", "foo") + if not len then + ngx.say("push err: ", err) + end + + local ok, err = dogs:set("key", "bar") + if not ok then + ngx.say("set err: ", err) + end + + local keys = dogs:get_keys(0) + + ngx.say("keys number: ", #keys) + + for i = 1, #keys do + local ok, err = dogs:expire(keys[i], 0.01) + if not ok then + ngx.say("expire err: ", err) + end + end + + local ok, err = dogs:expire("not-exits", 1) + if not ok then + ngx.say("expire on not-exists, err: ", err) + end + + ngx.sleep(0.01) + + local keys = dogs:get_keys(0) + + ngx.say("keys number: ", #keys) + } + } +--- request +GET /test +--- response_body +keys number: 2 +expire on not-exists, err: not found +keys number: 0 +--- no_error_log +[error] + From 4cdb80d9374936d6c18ce085478469086b68dfe3 Mon Sep 17 00:00:00 2001 From: doujiang24 Date: Thu, 29 Oct 2015 10:44:15 +0800 Subject: [PATCH 02/11] code style fix & bugfix: use the wrong length for create list node --- README.markdown | 4 ++-- doc/HttpLuaModule.wiki | 4 ++-- src/ngx_http_lua_shdict.c | 42 ++++++++++++++++++++++++--------------- src/ngx_http_lua_shdict.h | 4 ++-- t/133-shdict-list.t | 36 +++++++++++++++++++++++++++++++++ 5 files changed, 68 insertions(+), 22 deletions(-) diff --git a/README.markdown b/README.markdown index fea2963b7d..40e1055147 100644 --- a/README.markdown +++ b/README.markdown @@ -6316,7 +6316,7 @@ ngx.shared.DICT.lpush **context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.** -Insert the specified (numerical or string) `value` at the head of the list named `key` in the shm-based dictionary [ngx.shared.DICT](#ngxshareddict). Returns the length of the list after the push operations. +Inserts the specified (numerical or string) `value` at the head of the list named `key` in the shm-based dictionary [ngx.shared.DICT](#ngxshareddict). Returns the length of the list after the push operations. If `key` does not exist, it is created as empty list before performing the push operations. When key holds a value that is not a list, it will return `nil` and `"wrongtype operation"`. @@ -6334,7 +6334,7 @@ ngx.shared.DICT.rpush **context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.** -Similar to the [lpush](#ngxshareddictlpush) method, but insert the specified (numerical or string) `value` at the tail of the list named `key`. +Similar to the [lpush](#ngxshareddictlpush) method, but inserts the specified (numerical or string) `value` at the tail of the list named `key`. This feature was first introduced in the `v0.*.*` release. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 2a583e3d8e..4e2682b555 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -5292,7 +5292,7 @@ See also [[#ngx.shared.DICT|ngx.shared.DICT]]. '''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*'' -Insert the specified (numerical or string) value at the head of the list named key in the shm-based dictionary [[#ngx.shared.DICT|ngx.shared.DICT]]. Returns the length of the list after the push operations. +Inserts the specified (numerical or string) value at the head of the list named key in the shm-based dictionary [[#ngx.shared.DICT|ngx.shared.DICT]]. Returns the length of the list after the push operations. If key does not exist, it is created as empty list before performing the push operations. When key holds a value that is not a list, it will return nil and "wrongtype operation". @@ -5307,7 +5307,7 @@ See also [[#ngx.shared.DICT|ngx.shared.DICT]]. '''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*'' -Similar to the [[#ngx.shared.DICT.lpush|lpush]] method, but insert the specified (numerical or string) value at the tail of the list named key. +Similar to the [[#ngx.shared.DICT.lpush|lpush]] method, but inserts the specified (numerical or string) value at the tail of the list named key. This feature was first introduced in the v0.*.* release. diff --git a/src/ngx_http_lua_shdict.c b/src/ngx_http_lua_shdict.c index ae26c603cc..6cb2e52e88 100644 --- a/src/ngx_http_lua_shdict.c +++ b/src/ngx_http_lua_shdict.c @@ -310,8 +310,8 @@ ngx_http_lua_shdict_expire(ngx_http_lua_shdict_ctx_t *ctx, ngx_uint_t n) list_queue = ngx_http_lua_shdict_get_list_head(sd, sd->key_len); for (lq = ngx_queue_head(list_queue); - lq != ngx_queue_sentinel(list_queue); - lq = ngx_queue_next(lq)) + lq != ngx_queue_sentinel(list_queue); + lq = ngx_queue_next(lq)) { lnode = ngx_queue_data(lq, ngx_http_lua_shdict_list_node_t, queue); @@ -552,6 +552,7 @@ ngx_http_lua_shdict_get_helper(lua_State *L, int get_stale) value.len = (size_t) sd->value_len; switch (value_type) { + case LUA_TSTRING: lua_pushlstring(L, (char *) value.data, value.len); @@ -754,8 +755,8 @@ ngx_http_lua_shdict_flush_expired(lua_State *L) list_queue = ngx_http_lua_shdict_get_list_head(sd, sd->key_len); for (lq = ngx_queue_head(list_queue); - lq != ngx_queue_sentinel(list_queue); - lq = ngx_queue_next(lq)) + lq != ngx_queue_sentinel(list_queue); + lq = ngx_queue_next(lq)) { lnode = ngx_queue_data(lq, ngx_http_lua_shdict_list_node_t, queue); @@ -989,6 +990,7 @@ ngx_http_lua_shdict_set_helper(lua_State *L, int flags) value_type = lua_type(L, 3); switch (value_type) { + case LUA_TSTRING: value.data = (u_char *) lua_tolstring(L, 3, &value.len); break; @@ -1090,7 +1092,8 @@ ngx_http_lua_shdict_set_helper(lua_State *L, int flags) replace: - if (value.data && value.len == (size_t) sd->value_len + if (value.data + && value.len == (size_t) sd->value_len && sd->value_type != LUA_TTABLE) { @@ -1141,8 +1144,8 @@ ngx_http_lua_shdict_set_helper(lua_State *L, int flags) queue = ngx_http_lua_shdict_get_list_head(sd, key.len); for (q = ngx_queue_head(queue); - q != ngx_queue_sentinel(queue); - q = ngx_queue_next(q)) + q != ngx_queue_sentinel(queue); + q = ngx_queue_next(q)) { p = (u_char *) ngx_queue_data(q, ngx_http_lua_shdict_list_node_t, @@ -1534,6 +1537,7 @@ ngx_http_lua_shared_dict_get(ngx_shm_zone_t *zone, u_char *key_data, len = (size_t) sd->value_len; switch (value->type) { + case LUA_TSTRING: if (value->value.s.data == NULL || value->value.s.len == 0) { @@ -1669,6 +1673,7 @@ ngx_http_lua_shdict_push_helper(lua_State *L, int flags) value_type = lua_type(L, 3); switch (value_type) { + case LUA_TSTRING: value.data = (u_char *) lua_tolstring(L, 3, &value.len); break; @@ -1729,8 +1734,8 @@ ngx_http_lua_shdict_push_helper(lua_State *L, int flags) queue = ngx_http_lua_shdict_get_list_head(sd, key.len); for (q = ngx_queue_head(queue); - q != ngx_queue_sentinel(queue); - q = ngx_queue_next(q)) + q != ngx_queue_sentinel(queue); + q = ngx_queue_next(q)) { /* TODO: reuse matched size list node */ lnode = ngx_queue_data(q, ngx_http_lua_shdict_list_node_t, queue); @@ -1821,11 +1826,12 @@ ngx_http_lua_shdict_push_helper(lua_State *L, int flags) ngx_log_debug0(NGX_LOG_DEBUG_HTTP, ctx->log, 0, "lua shared dict list: creating a new list node"); - n = sizeof(ngx_http_lua_shdict_list_node_t) + n = offsetof(ngx_http_lua_shdict_list_node_t, data) + value.len; - lnode = ngx_slab_alloc_locked(ctx->shpool, - sizeof(ngx_http_lua_shdict_list_node_t)); + dd("list node length: %d", n); + + lnode = ngx_slab_alloc_locked(ctx->shpool, n); if (lnode == NULL) { @@ -1894,7 +1900,6 @@ ngx_http_lua_shdict_rpop(lua_State *L) } - static int ngx_http_lua_shdict_pop_helper(lua_State *L, int flags) { @@ -2008,6 +2013,7 @@ ngx_http_lua_shdict_pop_helper(lua_State *L, int flags) value.len = (size_t) lnode->value_len; switch (value_type) { + case LUA_TSTRING: lua_pushlstring(L, (char *) value.data, value.len); @@ -2319,6 +2325,7 @@ ngx_http_lua_ffi_shdict_store(ngx_shm_zone_t *zone, int op, u_char *key, hash = ngx_crc32_short(key, key_len); switch (value_type) { + case LUA_TSTRING: /* do nothing */ break; @@ -2367,6 +2374,7 @@ ngx_http_lua_ffi_shdict_store(ngx_shm_zone_t *zone, int op, u_char *key, *errmsg = "not found"; return NGX_DECLINED; } + /* rc == NGX_OK */ goto replace; @@ -2401,7 +2409,8 @@ ngx_http_lua_ffi_shdict_store(ngx_shm_zone_t *zone, int op, u_char *key, replace: - if (str_value_buf && str_value_len == (size_t) sd->value_len + if (str_value_buf + && str_value_len == (size_t) sd->value_len && sd->value_type != LUA_TTABLE) { @@ -2450,8 +2459,8 @@ ngx_http_lua_ffi_shdict_store(ngx_shm_zone_t *zone, int op, u_char *key, queue = ngx_http_lua_shdict_get_list_head(sd, key_len); for (q = ngx_queue_head(queue); - q != ngx_queue_sentinel(queue); - q = ngx_queue_next(q)) + q != ngx_queue_sentinel(queue); + q = ngx_queue_next(q)) { p = (u_char *) ngx_queue_data(q, ngx_http_lua_shdict_list_node_t, @@ -2628,6 +2637,7 @@ ngx_http_lua_ffi_shdict_get(ngx_shm_zone_t *zone, u_char *key, } switch (*value_type) { + case LUA_TSTRING: *str_value_len = value.len; ngx_memcpy(*str_value_buf, value.data, value.len); diff --git a/src/ngx_http_lua_shdict.h b/src/ngx_http_lua_shdict.h index 77954ab0e3..2d1178bffc 100644 --- a/src/ngx_http_lua_shdict.h +++ b/src/ngx_http_lua_shdict.h @@ -25,9 +25,9 @@ typedef struct { typedef struct { - uint8_t value_type; - uint32_t value_len; ngx_queue_t queue; + uint32_t value_len; + uint8_t value_type; u_char data[1]; } ngx_http_lua_shdict_list_node_t; diff --git a/t/133-shdict-list.t b/t/133-shdict-list.t index 649fa377b4..5c104ece03 100644 --- a/t/133-shdict-list.t +++ b/t/133-shdict-list.t @@ -678,3 +678,39 @@ keys number: 0 --- no_error_log [error] + + +=== TEST 17: long list node +--- http_config + lua_shared_dict dogs 1m; +--- config + location = /test { + content_by_lua_block { + local dogs = ngx.shared.dogs + + local long_str = string.rep("foo", 10) + + for i = 1, 3 do + local len, err = dogs:lpush("list", long_str) + if not len then + ngx.say("push err: ", err) + end + end + + for i = 1, 3 do + local val, err = dogs:lpop("list") + if val then + ngx.say(val) + end + end + } + } +--- request +GET /test +--- response_body +foofoofoofoofoofoofoofoofoofoo +foofoofoofoofoofoofoofoofoofoo +foofoofoofoofoofoofoofoofoofoo +--- no_error_log +[error] + From ffbfe17b07f9aa41f2be737a5bdf754bd3edf760 Mon Sep 17 00:00:00 2001 From: doujiang24 Date: Fri, 29 Jul 2016 11:02:25 +0800 Subject: [PATCH 03/11] some code style fix & use selfdefine type constants instead of abusing Lua's types --- src/ngx_http_lua_shdict.c | 81 ++++++++++++++++++++++----------------- 1 file changed, 45 insertions(+), 36 deletions(-) diff --git a/src/ngx_http_lua_shdict.c b/src/ngx_http_lua_shdict.c index 6cb2e52e88..23656e39e6 100644 --- a/src/ngx_http_lua_shdict.c +++ b/src/ngx_http_lua_shdict.c @@ -51,6 +51,7 @@ static ngx_inline ngx_shm_zone_t *ngx_http_lua_shdict_get_zone(lua_State *L, #define NGX_HTTP_LUA_SHDICT_ADD 0x0001 #define NGX_HTTP_LUA_SHDICT_REPLACE 0x0002 #define NGX_HTTP_LUA_SHDICT_SAFE_STORE 0x0004 + #define NGX_HTTP_LUA_SHDICT_LEFT 0x0001 #define NGX_HTTP_LUA_SHDICT_RIGHT 0x0002 @@ -59,6 +60,14 @@ enum { SHDICT_USERDATA_INDEX = 1, }; +enum { + SHDICT_TNIL = 0, // same as LUA_TNIL + SHDICT_TBOOLEAN = 1, // same as LUA_TBOOLEAN + SHDICT_TNUMBER = 3, // same as LUA_TNUMBER + SHDICT_TSTRING = 4, // same as LUA_TSTRING + SHDICT_TLIST = 5, +}; + #define ngx_http_lua_shdict_get_list_head(sd, key_len) \ (ngx_queue_t *) ngx_align_ptr(((u_char *) &sd->data + key_len), \ @@ -305,7 +314,7 @@ ngx_http_lua_shdict_expire(ngx_http_lua_shdict_ctx_t *ctx, ngx_uint_t n) } } - if (sd->value_type == LUA_TTABLE) { + if (sd->value_type == SHDICT_TLIST) { list_queue = ngx_http_lua_shdict_get_list_head(sd, sd->key_len); @@ -553,12 +562,12 @@ ngx_http_lua_shdict_get_helper(lua_State *L, int get_stale) switch (value_type) { - case LUA_TSTRING: + case SHDICT_TSTRING: lua_pushlstring(L, (char *) value.data, value.len); break; - case LUA_TNUMBER: + case SHDICT_TNUMBER: if (value.len != sizeof(double)) { @@ -574,7 +583,7 @@ ngx_http_lua_shdict_get_helper(lua_State *L, int get_stale) lua_pushnumber(L, num); break; - case LUA_TBOOLEAN: + case SHDICT_TBOOLEAN: if (value.len != sizeof(u_char)) { @@ -590,7 +599,7 @@ ngx_http_lua_shdict_get_helper(lua_State *L, int get_stale) lua_pushboolean(L, c ? 1 : 0); break; - case LUA_TTABLE: + case SHDICT_TLIST: ngx_shmtx_unlock(&ctx->shpool->mutex); @@ -750,7 +759,7 @@ ngx_http_lua_shdict_flush_expired(lua_State *L) if (sd->expires != 0 && sd->expires <= now) { - if (sd->value_type == LUA_TTABLE) { + if (sd->value_type == SHDICT_TLIST) { list_queue = ngx_http_lua_shdict_get_list_head(sd, sd->key_len); @@ -991,17 +1000,17 @@ ngx_http_lua_shdict_set_helper(lua_State *L, int flags) switch (value_type) { - case LUA_TSTRING: + case SHDICT_TSTRING: value.data = (u_char *) lua_tolstring(L, 3, &value.len); break; - case LUA_TNUMBER: + case SHDICT_TNUMBER: value.len = sizeof(double); num = lua_tonumber(L, 3); value.data = (u_char *) # break; - case LUA_TBOOLEAN: + case SHDICT_TBOOLEAN: value.len = sizeof(u_char); c = lua_toboolean(L, 3) ? 1 : 0; value.data = &c; @@ -1094,7 +1103,7 @@ ngx_http_lua_shdict_set_helper(lua_State *L, int flags) if (value.data && value.len == (size_t) sd->value_len - && sd->value_type != LUA_TTABLE) + && sd->value_type != SHDICT_TLIST) { ngx_log_debug0(NGX_LOG_DEBUG_HTTP, ctx->log, 0, @@ -1140,7 +1149,7 @@ ngx_http_lua_shdict_set_helper(lua_State *L, int flags) remove: - if (sd->value_type == LUA_TTABLE) { + if (sd->value_type == SHDICT_TLIST) { queue = ngx_http_lua_shdict_get_list_head(sd, key.len); for (q = ngx_queue_head(queue); @@ -1380,7 +1389,7 @@ ngx_http_lua_shdict_incr(lua_State *L) /* rc == NGX_OK */ - if (sd->value_type != LUA_TNUMBER || sd->value_len != sizeof(double)) { + if (sd->value_type != SHDICT_TNUMBER || sd->value_len != sizeof(double)) { ngx_shmtx_unlock(&ctx->shpool->mutex); lua_pushnil(L); @@ -1538,7 +1547,7 @@ ngx_http_lua_shared_dict_get(ngx_shm_zone_t *zone, u_char *key_data, switch (value->type) { - case LUA_TSTRING: + case SHDICT_TSTRING: if (value->value.s.data == NULL || value->value.s.len == 0) { ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0, "no string buffer " @@ -1557,7 +1566,7 @@ ngx_http_lua_shared_dict_get(ngx_shm_zone_t *zone, u_char *key_data, ngx_memcpy(value->value.s.data, data, len); break; - case LUA_TNUMBER: + case SHDICT_TNUMBER: if (len != sizeof(double)) { ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0, "bad lua number " @@ -1571,7 +1580,7 @@ ngx_http_lua_shared_dict_get(ngx_shm_zone_t *zone, u_char *key_data, ngx_memcpy(&value->value.b, data, len); break; - case LUA_TBOOLEAN: + case SHDICT_TBOOLEAN: if (len != sizeof(u_char)) { ngx_log_error(NGX_LOG_ERR, ngx_cycle->log, 0, "bad lua boolean " @@ -1674,11 +1683,11 @@ ngx_http_lua_shdict_push_helper(lua_State *L, int flags) switch (value_type) { - case LUA_TSTRING: + case SHDICT_TSTRING: value.data = (u_char *) lua_tolstring(L, 3, &value.len); break; - case LUA_TNUMBER: + case SHDICT_TNUMBER: value.len = sizeof(double); num = lua_tonumber(L, 3); value.data = (u_char *) # @@ -1703,7 +1712,7 @@ ngx_http_lua_shdict_push_helper(lua_State *L, int flags) if (rc == NGX_DONE) { /* exists but expired */ - if (sd->value_type != LUA_TTABLE) { + if (sd->value_type != SHDICT_TLIST) { /* TODO: reuse when length matched */ ngx_log_debug0(NGX_LOG_DEBUG_HTTP, ctx->log, 0, @@ -1752,7 +1761,7 @@ ngx_http_lua_shdict_push_helper(lua_State *L, int flags) } else if (rc == NGX_OK) { - if (sd->value_type != LUA_TTABLE) { + if (sd->value_type != SHDICT_TLIST) { ngx_shmtx_unlock(&ctx->shpool->mutex); lua_pushnil(L); @@ -1809,9 +1818,9 @@ ngx_http_lua_shdict_push_helper(lua_State *L, int flags) sd->value_len = 0; - dd("setting value type to %d", (int) LUA_TTABLE); + dd("setting value type to %d", (int) SHDICT_TLIST); - sd->value_type = (uint8_t) LUA_TTABLE; + sd->value_type = (uint8_t) SHDICT_TLIST; ngx_memcpy(sd->data, key.data, key.len); @@ -1977,7 +1986,7 @@ ngx_http_lua_shdict_pop_helper(lua_State *L, int flags) /* rc == NGX_OK */ - if (sd->value_type != LUA_TTABLE) { + if (sd->value_type != SHDICT_TLIST) { ngx_shmtx_unlock(&ctx->shpool->mutex); lua_pushnil(L); @@ -2014,12 +2023,12 @@ ngx_http_lua_shdict_pop_helper(lua_State *L, int flags) switch (value_type) { - case LUA_TSTRING: + case SHDICT_TSTRING: lua_pushlstring(L, (char *) value.data, value.len); break; - case LUA_TNUMBER: + case SHDICT_TNUMBER: if (value.len != sizeof(double)) { @@ -2140,7 +2149,7 @@ ngx_http_lua_shdict_llen(lua_State *L) if (rc == NGX_OK) { - if (sd->value_type != LUA_TTABLE) { + if (sd->value_type != SHDICT_TLIST) { ngx_shmtx_unlock(&ctx->shpool->mutex); lua_pushnil(L); @@ -2326,17 +2335,17 @@ ngx_http_lua_ffi_shdict_store(ngx_shm_zone_t *zone, int op, u_char *key, switch (value_type) { - case LUA_TSTRING: + case SHDICT_TSTRING: /* do nothing */ break; - case LUA_TNUMBER: + case SHDICT_TNUMBER: dd("num value: %lf", num_value); str_value_buf = (u_char *) &num_value; str_value_len = sizeof(double); break; - case LUA_TBOOLEAN: + case SHDICT_TBOOLEAN: c = num_value ? 1 : 0; str_value_buf = &c; str_value_len = sizeof(u_char); @@ -2411,7 +2420,7 @@ ngx_http_lua_ffi_shdict_store(ngx_shm_zone_t *zone, int op, u_char *key, if (str_value_buf && str_value_len == (size_t) sd->value_len - && sd->value_type != LUA_TTABLE) + && sd->value_type != SHDICT_TLIST) { ngx_log_debug0(NGX_LOG_DEBUG_HTTP, ctx->log, 0, @@ -2454,7 +2463,7 @@ ngx_http_lua_ffi_shdict_store(ngx_shm_zone_t *zone, int op, u_char *key, remove: - if (sd->value_type == LUA_TTABLE) { + if (sd->value_type == SHDICT_TLIST) { queue = ngx_http_lua_shdict_get_list_head(sd, key_len); @@ -2622,12 +2631,12 @@ ngx_http_lua_ffi_shdict_get(ngx_shm_zone_t *zone, u_char *key, value.len = (size_t) sd->value_len; if (*str_value_len < (size_t) value.len) { - if (*value_type == LUA_TBOOLEAN) { + if (*value_type == SHDICT_TBOOLEAN) { ngx_shmtx_unlock(&ctx->shpool->mutex); return NGX_ERROR; } - if (*value_type == LUA_TSTRING) { + if (*value_type == SHDICT_TSTRING) { *str_value_buf = malloc(value.len); if (*str_value_buf == NULL) { ngx_shmtx_unlock(&ctx->shpool->mutex); @@ -2638,12 +2647,12 @@ ngx_http_lua_ffi_shdict_get(ngx_shm_zone_t *zone, u_char *key, switch (*value_type) { - case LUA_TSTRING: + case SHDICT_TSTRING: *str_value_len = value.len; ngx_memcpy(*str_value_buf, value.data, value.len); break; - case LUA_TNUMBER: + case SHDICT_TNUMBER: if (value.len != sizeof(double)) { ngx_shmtx_unlock(&ctx->shpool->mutex); @@ -2658,7 +2667,7 @@ ngx_http_lua_ffi_shdict_get(ngx_shm_zone_t *zone, u_char *key, ngx_memcpy(num_value, value.data, sizeof(double)); break; - case LUA_TBOOLEAN: + case SHDICT_TBOOLEAN: if (value.len != sizeof(u_char)) { ngx_shmtx_unlock(&ctx->shpool->mutex); @@ -2770,7 +2779,7 @@ ngx_http_lua_ffi_shdict_incr(ngx_shm_zone_t *zone, u_char *key, /* rc == NGX_OK */ - if (sd->value_type != LUA_TNUMBER || sd->value_len != sizeof(double)) { + if (sd->value_type != SHDICT_TNUMBER || sd->value_len != sizeof(double)) { ngx_shmtx_unlock(&ctx->shpool->mutex); *err = "not a number"; return NGX_ERROR; From 7345b93853c4fe289dd958c41e148860252c185a Mon Sep 17 00:00:00 2001 From: doujiang24 Date: Fri, 29 Jul 2016 11:15:13 +0800 Subject: [PATCH 04/11] use an inline function here instead of a macro for ngx_http_lua_shdict_get_list_head --- src/ngx_http_lua_shdict.c | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/src/ngx_http_lua_shdict.c b/src/ngx_http_lua_shdict.c index 23656e39e6..af95c26747 100644 --- a/src/ngx_http_lua_shdict.c +++ b/src/ngx_http_lua_shdict.c @@ -61,17 +61,20 @@ enum { }; enum { - SHDICT_TNIL = 0, // same as LUA_TNIL - SHDICT_TBOOLEAN = 1, // same as LUA_TBOOLEAN - SHDICT_TNUMBER = 3, // same as LUA_TNUMBER - SHDICT_TSTRING = 4, // same as LUA_TSTRING + SHDICT_TNIL = 0, /* same as LUA_TNIL */ + SHDICT_TBOOLEAN = 1, /* same as LUA_TBOOLEAN */ + SHDICT_TNUMBER = 3, /* same as LUA_TNUMBER */ + SHDICT_TSTRING = 4, /* same as LUA_TSTRING */ SHDICT_TLIST = 5, }; -#define ngx_http_lua_shdict_get_list_head(sd, key_len) \ - (ngx_queue_t *) ngx_align_ptr(((u_char *) &sd->data + key_len), \ - NGX_ALIGNMENT) +static ngx_queue_t * +ngx_http_lua_shdict_get_list_head(ngx_http_lua_shdict_node_t *sd, u_short len) +{ + return (ngx_queue_t *) ngx_align_ptr(((u_char *) &sd->data + len), + NGX_ALIGNMENT); +} ngx_int_t From 2683c4d39888c86492cddcc95618929c77cd4515 Mon Sep 17 00:00:00 2001 From: doujiang24 Date: Fri, 29 Jul 2016 11:24:13 +0800 Subject: [PATCH 05/11] fix test case: shdict count --- t/062-count.t | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/t/062-count.t b/t/062-count.t index a24611cfb6..1e36bd18e0 100644 --- a/t/062-count.t +++ b/t/062-count.t @@ -283,7 +283,7 @@ n = 4 --- request GET /test --- response_body -n = 13 +n = 19 --- no_error_log [error] From 3eb0be9e15f75a1021c94afbe6fec5a2921428a5 Mon Sep 17 00:00:00 2001 From: doujiang24 Date: Tue, 2 Aug 2016 18:40:14 +0800 Subject: [PATCH 06/11] some doc fix and bugfix --- README.markdown | 28 ++++++++++++++-------------- doc/HttpLuaModule.wiki | 16 ++++++++-------- src/ngx_http_lua_shdict.c | 26 +++++++++++++++----------- t/133-shdict-list.t | 12 ++++++------ 4 files changed, 43 insertions(+), 39 deletions(-) diff --git a/README.markdown b/README.markdown index 40e1055147..79d23f2d4e 100644 --- a/README.markdown +++ b/README.markdown @@ -6105,7 +6105,7 @@ ngx.shared.DICT.get **context:** *set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua** -Retrieving the value in the dictionary [ngx.shared.DICT](#ngxshareddict) for the key `key`. If the key does not exist or has been expired, then `nil` will be returned. +Retrieving the value in the dictionary [ngx.shared.DICT](#ngxshareddict) for the key `key`. If the key does not exist or has expired, then `nil` will be returned. In case of errors, `nil` and a string describing the error will be returned. @@ -6314,11 +6314,11 @@ ngx.shared.DICT.lpush --------------------- **syntax:** *length, err = ngx.shared.DICT:lpush(key, value)* -**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.** +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.** -Inserts the specified (numerical or string) `value` at the head of the list named `key` in the shm-based dictionary [ngx.shared.DICT](#ngxshareddict). Returns the length of the list after the push operations. +Inserts the specified (numerical or string) `value` at the head of the list named `key` in the shm-based dictionary [ngx.shared.DICT](#ngxshareddict). Returns the number of elements in the list after the push operation. -If `key` does not exist, it is created as empty list before performing the push operations. When key holds a value that is not a list, it will return `nil` and `"wrongtype operation"`. +If `key` does not exist, it is created as an empty list before performing the push operations. When the `key` already takes a value that is not a list, it will return `nil` and `"value not a list"`. It never overrides the (least recently used) unexpired items in the store when running out of storage in the shared memory zone. In this case, it will immediately return `nil` and the string "no memory". @@ -6332,7 +6332,7 @@ ngx.shared.DICT.rpush --------------------- **syntax:** *length, err = ngx.shared.DICT:rpush(key, value)* -**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.** +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.** Similar to the [lpush](#ngxshareddictlpush) method, but inserts the specified (numerical or string) `value` at the tail of the list named `key`. @@ -6346,11 +6346,11 @@ ngx.shared.DICT.lpop -------------------- **syntax:** *val, err = ngx.shared.DICT:lpop(key)* -**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.** +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.** Removes and returns the first element of the list named `key` in the shm-based dictionary [ngx.shared.DICT](#ngxshareddict). -If `key` does not exist, it will return `nil`. When key holds a value that is not a list, it will return `nil` and `"wrongtype operation"`. +If `key` does not exist, it will return `nil`. When the `key` already takes a value that is not a list, it will return `nil` and `"value not a list"`. This feature was first introduced in the `v0.*.*` release. @@ -6362,11 +6362,11 @@ ngx.shared.DICT.rpop -------------------- **syntax:** *val, err = ngx.shared.DICT:rpop(key)* -**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.** +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.** Removes and returns the last element of the list named `key` in the shm-based dictionary [ngx.shared.DICT](#ngxshareddict). -If `key` does not exist, it will return `nil`. When key holds a value that is not a list, it will return `nil` and `"wrongtype operation"`. +If `key` does not exist, it will return `nil`. When the `key` already takes a value that is not a list, it will return `nil` and `"value not a list"`. This feature was first introduced in the `v0.*.*` release. @@ -6378,11 +6378,11 @@ ngx.shared.DICT.llen -------------------- **syntax:** *len, err = ngx.shared.DICT:llen(key)* -**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.** +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.** Returns the length of the list named `key` in the shm-based dictionary [ngx.shared.DICT](#ngxshareddict). -If key does not exist, it is interpreted as an empty list and 0 is returned. When key holds a value that is not a list, it will return `nil` and `"wrongtype operation"`. +If key does not exist, it is interpreted as an empty list and 0 is returned. When the `key` already takes a value that is not a list, it will return `nil` and `"value not a list"`. This feature was first introduced in the `v0.*.*` release. @@ -6394,11 +6394,11 @@ ngx.shared.DICT.expire ---------------------- **syntax:** *ok, err = ngx.shared.DICT:expire(key, exptime)* -**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.** +**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.** -The `exptime` argument specifies expiration time (in seconds) for the inserted key-value pair. The time resolution is 0.001 seconds. If the exptime takes the value 0, then the item will never be expired. +The `exptime` argument specifies a new expiration time (in seconds) for an existing key-value pair in the shm-based dictionary [ngx.shared.DICT](#ngxshareddict) specified by the `key` argument. The time resolution is 0.001 seconds. If the exptime takes the value 0, then the item will never expire. -If key does not exist or has been expired, it will return `false` and `"not found"`. Otherwise it will return `true`. +If key does not exist or has expired, it will return `false` and `"not found"`. Otherwise it will return `true`. This feature was first introduced in the `v0.*.*` release. diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 4e2682b555..3d0c11da77 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -5112,7 +5112,7 @@ This feature was first introduced in the v0.3.1rc22 release. '''context:''' ''set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*, balancer_by_lua*, ssl_certificate_by_lua*, ssl_session_fetch_by_lua*, ssl_session_store_by_lua*'' -Retrieving the value in the dictionary [[#ngx.shared.DICT|ngx.shared.DICT]] for the key key. If the key does not exist or has been expired, then nil will be returned. +Retrieving the value in the dictionary [[#ngx.shared.DICT|ngx.shared.DICT]] for the key key. If the key does not exist or has expired, then nil will be returned. In case of errors, nil and a string describing the error will be returned. @@ -5292,9 +5292,9 @@ See also [[#ngx.shared.DICT|ngx.shared.DICT]]. '''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*'' -Inserts the specified (numerical or string) value at the head of the list named key in the shm-based dictionary [[#ngx.shared.DICT|ngx.shared.DICT]]. Returns the length of the list after the push operations. +Inserts the specified (numerical or string) value at the head of the list named key in the shm-based dictionary [[#ngx.shared.DICT|ngx.shared.DICT]]. Returns the number of elements in the list after the push operation. -If key does not exist, it is created as empty list before performing the push operations. When key holds a value that is not a list, it will return nil and "wrongtype operation". +If key does not exist, it is created as an empty list before performing the push operations. When the key already takes a value that is not a list, it will return nil and "value not a list". It never overrides the (least recently used) unexpired items in the store when running out of storage in the shared memory zone. In this case, it will immediately return nil and the string "no memory". @@ -5320,7 +5320,7 @@ See also [[#ngx.shared.DICT|ngx.shared.DICT]]. Removes and returns the first element of the list named key in the shm-based dictionary [[#ngx.shared.DICT|ngx.shared.DICT]]. -If key does not exist, it will return nil. When key holds a value that is not a list, it will return nil and "wrongtype operation". +If key does not exist, it will return nil. When the key already takes a value that is not a list, it will return nil and "value not a list". This feature was first introduced in the v0.*.* release. @@ -5333,7 +5333,7 @@ See also [[#ngx.shared.DICT|ngx.shared.DICT]]. Removes and returns the last element of the list named key in the shm-based dictionary [[#ngx.shared.DICT|ngx.shared.DICT]]. -If key does not exist, it will return nil. When key holds a value that is not a list, it will return nil and "wrongtype operation". +If key does not exist, it will return nil. When the key already takes a value that is not a list, it will return nil and "value not a list". This feature was first introduced in the v0.*.* release. @@ -5346,7 +5346,7 @@ See also [[#ngx.shared.DICT|ngx.shared.DICT]]. Returns the length of the list named key in the shm-based dictionary [[#ngx.shared.DICT|ngx.shared.DICT]]. -If key does not exist, it is interpreted as an empty list and 0 is returned. When key holds a value that is not a list, it will return nil and "wrongtype operation". +If key does not exist, it is interpreted as an empty list and 0 is returned. When the key already takes a value that is not a list, it will return nil and "value not a list". This feature was first introduced in the v0.*.* release. @@ -5357,9 +5357,9 @@ See also [[#ngx.shared.DICT|ngx.shared.DICT]]. '''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*'' -The exptime argument specifies expiration time (in seconds) for the inserted key-value pair. The time resolution is 0.001 seconds. If the exptime takes the value 0, then the item will never be expired. +The exptime argument specifies a new expiration time (in seconds) for an existing key-value pair in the shm-based dictionary [[#ngx.shared.DICT|ngx.shared.DICT]] specified by the key argument. The time resolution is 0.001 seconds. If the exptime takes the value 0, then the item will never expire. -If key does not exist or has been expired, it will return false and "not found". Otherwise it will return true. +If key does not exist or has expired, it will return false and "not found". Otherwise it will return true. This feature was first introduced in the v0.*.* release. diff --git a/src/ngx_http_lua_shdict.c b/src/ngx_http_lua_shdict.c index af95c26747..f6156a5241 100644 --- a/src/ngx_http_lua_shdict.c +++ b/src/ngx_http_lua_shdict.c @@ -52,6 +52,7 @@ static ngx_inline ngx_shm_zone_t *ngx_http_lua_shdict_get_zone(lua_State *L, #define NGX_HTTP_LUA_SHDICT_REPLACE 0x0002 #define NGX_HTTP_LUA_SHDICT_SAFE_STORE 0x0004 + #define NGX_HTTP_LUA_SHDICT_LEFT 0x0001 #define NGX_HTTP_LUA_SHDICT_RIGHT 0x0002 @@ -60,6 +61,7 @@ enum { SHDICT_USERDATA_INDEX = 1, }; + enum { SHDICT_TNIL = 0, /* same as LUA_TNIL */ SHDICT_TBOOLEAN = 1, /* same as LUA_TBOOLEAN */ @@ -69,7 +71,7 @@ enum { }; -static ngx_queue_t * +static ngx_inline ngx_queue_t * ngx_http_lua_shdict_get_list_head(ngx_http_lua_shdict_node_t *sd, u_short len) { return (ngx_queue_t *) ngx_align_ptr(((u_char *) &sd->data + len), @@ -607,7 +609,7 @@ ngx_http_lua_shdict_get_helper(lua_State *L, int get_stale) ngx_shmtx_unlock(&ctx->shpool->mutex); lua_pushnil(L); - lua_pushliteral(L, "wrongtype operation"); + lua_pushliteral(L, "value is a list"); return 2; default: @@ -1712,8 +1714,9 @@ ngx_http_lua_shdict_push_helper(lua_State *L, int flags) dd("shdict lookup returned %d", (int) rc); + /* exists but expired */ + if (rc == NGX_DONE) { - /* exists but expired */ if (sd->value_type != SHDICT_TLIST) { /* TODO: reuse when length matched */ @@ -1761,14 +1764,17 @@ ngx_http_lua_shdict_push_helper(lua_State *L, int flags) dd("go to push_node"); goto push_node; + } + + /* exists and not expired */ - } else if (rc == NGX_OK) { + if (rc == NGX_OK) { if (sd->value_type != SHDICT_TLIST) { ngx_shmtx_unlock(&ctx->shpool->mutex); lua_pushnil(L); - lua_pushliteral(L, "wrongtype operation"); + lua_pushliteral(L, "value not a list"); return 2; } @@ -1993,7 +1999,7 @@ ngx_http_lua_shdict_pop_helper(lua_State *L, int flags) ngx_shmtx_unlock(&ctx->shpool->mutex); lua_pushnil(L); - lua_pushliteral(L, "wrongtype operation"); + lua_pushliteral(L, "value not a list"); return 2; } @@ -2076,7 +2082,6 @@ ngx_http_lua_shdict_pop_helper(lua_State *L, int flags) ngx_slab_free_locked(ctx->shpool, node); } else { - sd->value_len = sd->value_len - 1; ngx_queue_remove(&sd->queue); @@ -2151,21 +2156,20 @@ ngx_http_lua_shdict_llen(lua_State *L) dd("shdict lookup returned %d", (int) rc); if (rc == NGX_OK) { - if (sd->value_type != SHDICT_TLIST) { ngx_shmtx_unlock(&ctx->shpool->mutex); lua_pushnil(L); - lua_pushliteral(L, "wrongtype operation"); + lua_pushliteral(L, "value not a list"); return 2; } ngx_queue_remove(&sd->queue); ngx_queue_insert_head(&ctx->sh->queue, &sd->queue); - lua_pushnumber(L, (lua_Number) sd->value_len); - ngx_shmtx_unlock(&ctx->shpool->mutex); + + lua_pushnumber(L, (lua_Number) sd->value_len); return 1; } diff --git a/t/133-shdict-list.t b/t/133-shdict-list.t index 5c104ece03..24c4408d2d 100644 --- a/t/133-shdict-list.t +++ b/t/133-shdict-list.t @@ -82,7 +82,7 @@ nil nil GET /test --- response_body push success -nil wrongtype operation +nil value is a list --- no_error_log [error] @@ -181,7 +181,7 @@ GET /test --- response_body push success false exists -nil wrongtype operation +nil value is a list --- no_error_log [error] @@ -247,7 +247,7 @@ GET /test --- response_body push success nil not a number -nil wrongtype operation +nil value is a list --- no_error_log [error] @@ -308,7 +308,7 @@ key: foo GET /test --- response_body set success -nil wrongtype operation +nil value not a list bar nil --- no_error_log [error] @@ -341,7 +341,7 @@ bar nil GET /test --- response_body set success -nil wrongtype operation +nil value not a list bar nil --- no_error_log [error] @@ -374,7 +374,7 @@ bar nil GET /test --- response_body set success -nil wrongtype operation +nil value not a list bar nil --- no_error_log [error] From 21a2287dc984f4164759791460dac0136794258e Mon Sep 17 00:00:00 2001 From: doujiang24 Date: Tue, 2 Aug 2016 18:53:47 +0800 Subject: [PATCH 07/11] some code style fix --- src/ngx_http_lua_shdict.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/src/ngx_http_lua_shdict.c b/src/ngx_http_lua_shdict.c index f6156a5241..007512cd30 100644 --- a/src/ngx_http_lua_shdict.c +++ b/src/ngx_http_lua_shdict.c @@ -320,7 +320,6 @@ ngx_http_lua_shdict_expire(ngx_http_lua_shdict_ctx_t *ctx, ngx_uint_t n) } if (sd->value_type == SHDICT_TLIST) { - list_queue = ngx_http_lua_shdict_get_list_head(sd, sd->key_len); for (lq = ngx_queue_head(list_queue); @@ -765,7 +764,6 @@ ngx_http_lua_shdict_flush_expired(lua_State *L) if (sd->expires != 0 && sd->expires <= now) { if (sd->value_type == SHDICT_TLIST) { - list_queue = ngx_http_lua_shdict_get_list_head(sd, sd->key_len); for (lq = ngx_queue_head(list_queue); @@ -2156,6 +2154,7 @@ ngx_http_lua_shdict_llen(lua_State *L) dd("shdict lookup returned %d", (int) rc); if (rc == NGX_OK) { + if (sd->value_type != SHDICT_TLIST) { ngx_shmtx_unlock(&ctx->shpool->mutex); @@ -2471,7 +2470,6 @@ ngx_http_lua_ffi_shdict_store(ngx_shm_zone_t *zone, int op, u_char *key, remove: if (sd->value_type == SHDICT_TLIST) { - queue = ngx_http_lua_shdict_get_list_head(sd, key_len); for (q = ngx_queue_head(queue); From 04d42ba62d61d65824f3ee94078f31386e042525 Mon Sep 17 00:00:00 2001 From: doujiang24 Date: Tue, 2 Aug 2016 19:12:27 +0800 Subject: [PATCH 08/11] rename queue to lru_queue in ngx_http_lua_shdict_shctx_t --- src/ngx_http_lua_shdict.c | 64 +++++++++++++++++++-------------------- src/ngx_http_lua_shdict.h | 2 +- 2 files changed, 33 insertions(+), 33 deletions(-) diff --git a/src/ngx_http_lua_shdict.c b/src/ngx_http_lua_shdict.c index 007512cd30..b301fa30d0 100644 --- a/src/ngx_http_lua_shdict.c +++ b/src/ngx_http_lua_shdict.c @@ -119,7 +119,7 @@ ngx_http_lua_shdict_init_zone(ngx_shm_zone_t *shm_zone, void *data) ngx_rbtree_init(&ctx->sh->rbtree, &ctx->sh->sentinel, ngx_http_lua_shdict_rbtree_insert_value); - ngx_queue_init(&ctx->sh->queue); + ngx_queue_init(&ctx->sh->lru_queue); len = sizeof(" in lua_shared_dict zone \"\"") + shm_zone->shm.name.len; @@ -243,7 +243,7 @@ ngx_http_lua_shdict_lookup(ngx_shm_zone_t *shm_zone, ngx_uint_t hash, if (rc == 0) { ngx_queue_remove(&sd->queue); - ngx_queue_insert_head(&ctx->sh->queue, &sd->queue); + ngx_queue_insert_head(&ctx->sh->lru_queue, &sd->queue); *sdp = sd; @@ -299,11 +299,11 @@ ngx_http_lua_shdict_expire(ngx_http_lua_shdict_ctx_t *ctx, ngx_uint_t n) while (n < 3) { - if (ngx_queue_empty(&ctx->sh->queue)) { + if (ngx_queue_empty(&ctx->sh->lru_queue)) { return freed; } - q = ngx_queue_last(&ctx->sh->queue); + q = ngx_queue_last(&ctx->sh->lru_queue); sd = ngx_queue_data(q, ngx_http_lua_shdict_node_t, queue); @@ -692,8 +692,8 @@ ngx_http_lua_shdict_flush_all(lua_State *L) ngx_shmtx_lock(&ctx->shpool->mutex); - for (q = ngx_queue_head(&ctx->sh->queue); - q != ngx_queue_sentinel(&ctx->sh->queue); + for (q = ngx_queue_head(&ctx->sh->lru_queue); + q != ngx_queue_sentinel(&ctx->sh->lru_queue); q = ngx_queue_next(q)) { sd = ngx_queue_data(q, ngx_http_lua_shdict_node_t, queue); @@ -744,7 +744,7 @@ ngx_http_lua_shdict_flush_expired(lua_State *L) ngx_shmtx_lock(&ctx->shpool->mutex); - if (ngx_queue_empty(&ctx->sh->queue)) { + if (ngx_queue_empty(&ctx->sh->lru_queue)) { ngx_shmtx_unlock(&ctx->shpool->mutex); lua_pushnumber(L, 0); return 1; @@ -754,9 +754,9 @@ ngx_http_lua_shdict_flush_expired(lua_State *L) now = (uint64_t) tp->sec * 1000 + tp->msec; - q = ngx_queue_last(&ctx->sh->queue); + q = ngx_queue_last(&ctx->sh->lru_queue); - while (q != ngx_queue_sentinel(&ctx->sh->queue)) { + while (q != ngx_queue_sentinel(&ctx->sh->lru_queue)) { prev = ngx_queue_prev(q); sd = ngx_queue_data(q, ngx_http_lua_shdict_node_t, queue); @@ -840,7 +840,7 @@ ngx_http_lua_shdict_get_keys(lua_State *L) ngx_shmtx_lock(&ctx->shpool->mutex); - if (ngx_queue_empty(&ctx->sh->queue)) { + if (ngx_queue_empty(&ctx->sh->lru_queue)) { ngx_shmtx_unlock(&ctx->shpool->mutex); lua_createtable(L, 0, 0); return 1; @@ -852,9 +852,9 @@ ngx_http_lua_shdict_get_keys(lua_State *L) /* first run through: get total number of elements we need to allocate */ - q = ngx_queue_last(&ctx->sh->queue); + q = ngx_queue_last(&ctx->sh->lru_queue); - while (q != ngx_queue_sentinel(&ctx->sh->queue)) { + while (q != ngx_queue_sentinel(&ctx->sh->lru_queue)) { prev = ngx_queue_prev(q); sd = ngx_queue_data(q, ngx_http_lua_shdict_node_t, queue); @@ -874,9 +874,9 @@ ngx_http_lua_shdict_get_keys(lua_State *L) /* second run through: add keys to table */ total = 0; - q = ngx_queue_last(&ctx->sh->queue); + q = ngx_queue_last(&ctx->sh->lru_queue); - while (q != ngx_queue_sentinel(&ctx->sh->queue)) { + while (q != ngx_queue_sentinel(&ctx->sh->lru_queue)) { prev = ngx_queue_prev(q); sd = ngx_queue_data(q, ngx_http_lua_shdict_node_t, queue); @@ -1114,7 +1114,7 @@ ngx_http_lua_shdict_set_helper(lua_State *L, int flags) "size matched, reusing it"); ngx_queue_remove(&sd->queue); - ngx_queue_insert_head(&ctx->sh->queue, &sd->queue); + ngx_queue_insert_head(&ctx->sh->lru_queue, &sd->queue); sd->key_len = (u_short) key.len; @@ -1265,7 +1265,7 @@ ngx_http_lua_shdict_set_helper(lua_State *L, int flags) ngx_rbtree_insert(&ctx->sh->rbtree, node); - ngx_queue_insert_head(&ctx->sh->queue, &sd->queue); + ngx_queue_insert_head(&ctx->sh->lru_queue, &sd->queue); ngx_shmtx_unlock(&ctx->shpool->mutex); @@ -1376,7 +1376,7 @@ ngx_http_lua_shdict_incr(lua_State *L) "value size matched, reusing it"); ngx_queue_remove(&sd->queue); - ngx_queue_insert_head(&ctx->sh->queue, &sd->queue); + ngx_queue_insert_head(&ctx->sh->lru_queue, &sd->queue); dd("go to setvalue"); goto setvalue; @@ -1401,7 +1401,7 @@ ngx_http_lua_shdict_incr(lua_State *L) } ngx_queue_remove(&sd->queue); - ngx_queue_insert_head(&ctx->sh->queue, &sd->queue); + ngx_queue_insert_head(&ctx->sh->lru_queue, &sd->queue); dd("setting value type to %d", (int) sd->value_type); @@ -1484,7 +1484,7 @@ ngx_http_lua_shdict_incr(lua_State *L) ngx_rbtree_insert(&ctx->sh->rbtree, node); - ngx_queue_insert_head(&ctx->sh->queue, &sd->queue); + ngx_queue_insert_head(&ctx->sh->lru_queue, &sd->queue); setvalue: @@ -1758,7 +1758,7 @@ ngx_http_lua_shdict_push_helper(lua_State *L, int flags) ngx_queue_init(queue); ngx_queue_remove(&sd->queue); - ngx_queue_insert_head(&ctx->sh->queue, &sd->queue); + ngx_queue_insert_head(&ctx->sh->lru_queue, &sd->queue); dd("go to push_node"); goto push_node; @@ -1779,7 +1779,7 @@ ngx_http_lua_shdict_push_helper(lua_State *L, int flags) queue = ngx_http_lua_shdict_get_list_head(sd, key.len); ngx_queue_remove(&sd->queue); - ngx_queue_insert_head(&ctx->sh->queue, &sd->queue); + ngx_queue_insert_head(&ctx->sh->lru_queue, &sd->queue); dd("go to push_node"); goto push_node; @@ -1835,7 +1835,7 @@ ngx_http_lua_shdict_push_helper(lua_State *L, int flags) ngx_rbtree_insert(&ctx->sh->rbtree, node); - ngx_queue_insert_head(&ctx->sh->queue, &sd->queue); + ngx_queue_insert_head(&ctx->sh->lru_queue, &sd->queue); push_node: @@ -2083,7 +2083,7 @@ ngx_http_lua_shdict_pop_helper(lua_State *L, int flags) sd->value_len = sd->value_len - 1; ngx_queue_remove(&sd->queue); - ngx_queue_insert_head(&ctx->sh->queue, &sd->queue); + ngx_queue_insert_head(&ctx->sh->lru_queue, &sd->queue); } ngx_shmtx_unlock(&ctx->shpool->mutex); @@ -2164,7 +2164,7 @@ ngx_http_lua_shdict_llen(lua_State *L) } ngx_queue_remove(&sd->queue); - ngx_queue_insert_head(&ctx->sh->queue, &sd->queue); + ngx_queue_insert_head(&ctx->sh->lru_queue, &sd->queue); ngx_shmtx_unlock(&ctx->shpool->mutex); @@ -2255,7 +2255,7 @@ ngx_http_lua_shdict_set_expire(lua_State *L) } ngx_queue_remove(&sd->queue); - ngx_queue_insert_head(&ctx->sh->queue, &sd->queue); + ngx_queue_insert_head(&ctx->sh->lru_queue, &sd->queue); ngx_shmtx_unlock(&ctx->shpool->mutex); @@ -2434,7 +2434,7 @@ ngx_http_lua_ffi_shdict_store(ngx_shm_zone_t *zone, int op, u_char *key, "size matched, reusing it"); ngx_queue_remove(&sd->queue); - ngx_queue_insert_head(&ctx->sh->queue, &sd->queue); + ngx_queue_insert_head(&ctx->sh->lru_queue, &sd->queue); sd->key_len = (u_short) key_len; @@ -2572,7 +2572,7 @@ ngx_http_lua_ffi_shdict_store(ngx_shm_zone_t *zone, int op, u_char *key, ngx_memcpy(p, str_value_buf, str_value_len); ngx_rbtree_insert(&ctx->sh->rbtree, node); - ngx_queue_insert_head(&ctx->sh->queue, &sd->queue); + ngx_queue_insert_head(&ctx->sh->lru_queue, &sd->queue); ngx_shmtx_unlock(&ctx->shpool->mutex); return NGX_OK; @@ -2768,7 +2768,7 @@ ngx_http_lua_ffi_shdict_incr(ngx_shm_zone_t *zone, u_char *key, "value size matched, reusing it"); ngx_queue_remove(&sd->queue); - ngx_queue_insert_head(&ctx->sh->queue, &sd->queue); + ngx_queue_insert_head(&ctx->sh->lru_queue, &sd->queue); dd("go to setvalue"); goto setvalue; @@ -2791,7 +2791,7 @@ ngx_http_lua_ffi_shdict_incr(ngx_shm_zone_t *zone, u_char *key, } ngx_queue_remove(&sd->queue); - ngx_queue_insert_head(&ctx->sh->queue, &sd->queue); + ngx_queue_insert_head(&ctx->sh->lru_queue, &sd->queue); dd("setting value type to %d", (int) sd->value_type); @@ -2872,7 +2872,7 @@ ngx_http_lua_ffi_shdict_incr(ngx_shm_zone_t *zone, u_char *key, ngx_rbtree_insert(&ctx->sh->rbtree, node); - ngx_queue_insert_head(&ctx->sh->queue, &sd->queue); + ngx_queue_insert_head(&ctx->sh->lru_queue, &sd->queue); setvalue: @@ -2905,8 +2905,8 @@ ngx_http_lua_ffi_shdict_flush_all(ngx_shm_zone_t *zone) ngx_shmtx_lock(&ctx->shpool->mutex); - for (q = ngx_queue_head(&ctx->sh->queue); - q != ngx_queue_sentinel(&ctx->sh->queue); + for (q = ngx_queue_head(&ctx->sh->lru_queue); + q != ngx_queue_sentinel(&ctx->sh->lru_queue); q = ngx_queue_next(q)) { sd = ngx_queue_data(q, ngx_http_lua_shdict_node_t, queue); diff --git a/src/ngx_http_lua_shdict.h b/src/ngx_http_lua_shdict.h index 2d1178bffc..5c4ff36e6b 100644 --- a/src/ngx_http_lua_shdict.h +++ b/src/ngx_http_lua_shdict.h @@ -35,7 +35,7 @@ typedef struct { typedef struct { ngx_rbtree_t rbtree; ngx_rbtree_node_t sentinel; - ngx_queue_t queue; + ngx_queue_t lru_queue; } ngx_http_lua_shdict_shctx_t; From 0144d574abb7e6b1439c8e2ae2451ae7d5f7c369 Mon Sep 17 00:00:00 2001 From: doujiang24 Date: Tue, 2 Aug 2016 22:35:44 +0800 Subject: [PATCH 09/11] remove expire implement --- src/ngx_http_lua_shdict.c | 98 +-------------------------------------- t/062-count.t | 2 +- t/133-shdict-list.t | 23 +-------- 3 files changed, 4 insertions(+), 119 deletions(-) diff --git a/src/ngx_http_lua_shdict.c b/src/ngx_http_lua_shdict.c index b301fa30d0..19435331d8 100644 --- a/src/ngx_http_lua_shdict.c +++ b/src/ngx_http_lua_shdict.c @@ -41,7 +41,6 @@ static int ngx_http_lua_shdict_lpop(lua_State *L); static int ngx_http_lua_shdict_rpop(lua_State *L); static int ngx_http_lua_shdict_pop_helper(lua_State *L, int flags); static int ngx_http_lua_shdict_llen(lua_State *L); -static int ngx_http_lua_shdict_set_expire(lua_State *L); static ngx_inline ngx_shm_zone_t *ngx_http_lua_shdict_get_zone(lua_State *L, @@ -360,7 +359,7 @@ ngx_http_lua_inject_shdict_api(ngx_http_lua_main_conf_t *lmcf, lua_State *L) lua_createtable(L, 0, lmcf->shm_zones->nelts /* nrec */); /* ngx.shared */ - lua_createtable(L, 0 /* narr */, 19 /* nrec */); /* shared mt */ + lua_createtable(L, 0 /* narr */, 18 /* nrec */); /* shared mt */ lua_pushcfunction(L, ngx_http_lua_shdict_get); lua_setfield(L, -2, "get"); @@ -404,9 +403,6 @@ ngx_http_lua_inject_shdict_api(ngx_http_lua_main_conf_t *lmcf, lua_State *L) lua_pushcfunction(L, ngx_http_lua_shdict_llen); lua_setfield(L, -2, "llen"); - lua_pushcfunction(L, ngx_http_lua_shdict_set_expire); - lua_setfield(L, -2, "expire"); - lua_pushcfunction(L, ngx_http_lua_shdict_flush_all); lua_setfield(L, -2, "flush_all"); @@ -2179,98 +2175,6 @@ ngx_http_lua_shdict_llen(lua_State *L) } -static int -ngx_http_lua_shdict_set_expire(lua_State *L) -{ - int n; - ngx_str_t key; - uint32_t hash; - ngx_int_t rc; - ngx_http_lua_shdict_ctx_t *ctx; - ngx_http_lua_shdict_node_t *sd; - ngx_shm_zone_t *zone; - lua_Number exptime = 0; - ngx_time_t *tp; - - n = lua_gettop(L); - - if (n != 3) { - return luaL_error(L, "expecting 3 arguments, " - "but only seen %d", n); - } - - if (lua_type(L, 1) != LUA_TTABLE) { - return luaL_error(L, "bad \"zone\" argument"); - } - - zone = ngx_http_lua_shdict_get_zone(L, 1); - if (zone == NULL) { - return luaL_error(L, "bad \"zone\" argument"); - } - - ctx = zone->data; - - if (lua_isnil(L, 2)) { - lua_pushnil(L); - lua_pushliteral(L, "nil key"); - return 2; - } - - key.data = (u_char *) luaL_checklstring(L, 2, &key.len); - - if (key.len == 0) { - lua_pushnil(L); - lua_pushliteral(L, "empty key"); - return 2; - } - - if (key.len > 65535) { - lua_pushnil(L); - lua_pushliteral(L, "key too long"); - return 2; - } - - hash = ngx_crc32_short(key.data, key.len); - - exptime = luaL_checknumber(L, 3); - if (exptime < 0) { - exptime = 0; - } - - ngx_shmtx_lock(&ctx->shpool->mutex); - - rc = ngx_http_lua_shdict_lookup(zone, hash, key.data, key.len, &sd); - - dd("shdict lookup returned %d", (int) rc); - - if (rc == NGX_OK) { - - if (exptime > 0) { - tp = ngx_timeofday(); - sd->expires = (uint64_t) tp->sec * 1000 + tp->msec - + (uint64_t) (exptime * 1000); - - } else { - sd->expires = 0; - } - - ngx_queue_remove(&sd->queue); - ngx_queue_insert_head(&ctx->sh->lru_queue, &sd->queue); - - ngx_shmtx_unlock(&ctx->shpool->mutex); - - lua_pushboolean(L, 1); - return 1; - } - - ngx_shmtx_unlock(&ctx->shpool->mutex); - - lua_pushboolean(L, 0); - lua_pushliteral(L, "not found"); - return 2; -} - - ngx_shm_zone_t * ngx_http_lua_find_zone(u_char *name_data, size_t name_len) { diff --git a/t/062-count.t b/t/062-count.t index 1e36bd18e0..e16291b734 100644 --- a/t/062-count.t +++ b/t/062-count.t @@ -283,7 +283,7 @@ n = 4 --- request GET /test --- response_body -n = 19 +n = 18 --- no_error_log [error] diff --git a/t/133-shdict-list.t b/t/133-shdict-list.t index 24c4408d2d..ade228fed9 100644 --- a/t/133-shdict-list.t +++ b/t/133-shdict-list.t @@ -512,24 +512,18 @@ GET /test max = i break end - - local ok = dogs:expire(key, 0.01) - if not ok then - ngx.say("expire error") - end end local keys = dogs:get_keys(0) ngx.say("max-1 matched keys length: ", max-1 == #keys) - ngx.sleep(0.01) + dogs:flush_all() local keys = dogs:get_keys(0) ngx.say("keys all expired, left number: ", #keys) - -- some reused, some removed for i = 10000, 1, -1 do local key = string.format("%05d", i) @@ -650,19 +644,7 @@ two == number 2: true ngx.say("keys number: ", #keys) - for i = 1, #keys do - local ok, err = dogs:expire(keys[i], 0.01) - if not ok then - ngx.say("expire err: ", err) - end - end - - local ok, err = dogs:expire("not-exits", 1) - if not ok then - ngx.say("expire on not-exists, err: ", err) - end - - ngx.sleep(0.01) + dogs:flush_all() local keys = dogs:get_keys(0) @@ -673,7 +655,6 @@ two == number 2: true GET /test --- response_body keys number: 2 -expire on not-exists, err: not found keys number: 0 --- no_error_log [error] From 46a25b3255c347946202f90eaabf27d472e31cbe Mon Sep 17 00:00:00 2001 From: doujiang24 Date: Tue, 2 Aug 2016 22:47:46 +0800 Subject: [PATCH 10/11] remove expire doc --- README.markdown | 18 ------------------ doc/HttpLuaModule.wiki | 14 -------------- 2 files changed, 32 deletions(-) diff --git a/README.markdown b/README.markdown index 79d23f2d4e..491c79c082 100644 --- a/README.markdown +++ b/README.markdown @@ -3046,7 +3046,6 @@ Nginx API for Lua * [ngx.shared.DICT.lpop](#ngxshareddictlpop) * [ngx.shared.DICT.rpop](#ngxshareddictrpop) * [ngx.shared.DICT.llen](#ngxshareddictllen) -* [ngx.shared.DICT.expire](#ngxshareddictexpire) * [ngx.shared.DICT.flush_all](#ngxshareddictflush_all) * [ngx.shared.DICT.flush_expired](#ngxshareddictflush_expired) * [ngx.shared.DICT.get_keys](#ngxshareddictget_keys) @@ -6046,7 +6045,6 @@ The resulting object `dict` has the following methods: * [lpop](#ngxshareddictlpop) * [rpop](#ngxshareddictrpop) * [llen](#ngxshareddictllen) -* [expire](#ngxshareddictexpire) * [flush_all](#ngxshareddictflush_all) * [flush_expired](#ngxshareddictflush_expired) * [get_keys](#ngxshareddictget_keys) @@ -6390,22 +6388,6 @@ See also [ngx.shared.DICT](#ngxshareddict). [Back to TOC](#nginx-api-for-lua) -ngx.shared.DICT.expire ----------------------- -**syntax:** *ok, err = ngx.shared.DICT:expire(key, exptime)* - -**context:** *init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.** - -The `exptime` argument specifies a new expiration time (in seconds) for an existing key-value pair in the shm-based dictionary [ngx.shared.DICT](#ngxshareddict) specified by the `key` argument. The time resolution is 0.001 seconds. If the exptime takes the value 0, then the item will never expire. - -If key does not exist or has expired, it will return `false` and `"not found"`. Otherwise it will return `true`. - -This feature was first introduced in the `v0.*.*` release. - -See also [ngx.shared.DICT](#ngxshareddict). - -[Back to TOC](#nginx-api-for-lua) - ngx.shared.DICT.flush_all ------------------------- **syntax:** *ngx.shared.DICT:flush_all()* diff --git a/doc/HttpLuaModule.wiki b/doc/HttpLuaModule.wiki index 3d0c11da77..180d38fb4e 100644 --- a/doc/HttpLuaModule.wiki +++ b/doc/HttpLuaModule.wiki @@ -5058,7 +5058,6 @@ The resulting object dict has the following methods: * [[#ngx.shared.DICT.lpop|lpop]] * [[#ngx.shared.DICT.rpop|rpop]] * [[#ngx.shared.DICT.llen|llen]] -* [[#ngx.shared.DICT.expire|expire]] * [[#ngx.shared.DICT.flush_all|flush_all]] * [[#ngx.shared.DICT.flush_expired|flush_expired]] * [[#ngx.shared.DICT.get_keys|get_keys]] @@ -5352,19 +5351,6 @@ This feature was first introduced in the v0.*.* release. See also [[#ngx.shared.DICT|ngx.shared.DICT]]. -== ngx.shared.DICT.expire == -'''syntax:''' ''ok, err = ngx.shared.DICT:expire(key, exptime)'' - -'''context:''' ''init_by_lua*, set_by_lua*, rewrite_by_lua*, access_by_lua*, content_by_lua*, header_filter_by_lua*, body_filter_by_lua*, log_by_lua*, ngx.timer.*'' - -The exptime argument specifies a new expiration time (in seconds) for an existing key-value pair in the shm-based dictionary [[#ngx.shared.DICT|ngx.shared.DICT]] specified by the key argument. The time resolution is 0.001 seconds. If the exptime takes the value 0, then the item will never expire. - -If key does not exist or has expired, it will return false and "not found". Otherwise it will return true. - -This feature was first introduced in the v0.*.* release. - -See also [[#ngx.shared.DICT|ngx.shared.DICT]]. - == ngx.shared.DICT.flush_all == '''syntax:''' ''ngx.shared.DICT:flush_all()'' From f23de0bcac7d9229c312cd50b23294e7c7aa25aa Mon Sep 17 00:00:00 2001 From: doujiang24 Date: Wed, 3 Aug 2016 10:37:56 +0800 Subject: [PATCH 11/11] bugfix: maybe something wrong from rebase --- src/ngx_http_lua_shdict.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ngx_http_lua_shdict.c b/src/ngx_http_lua_shdict.c index 19435331d8..b987f7b8d0 100644 --- a/src/ngx_http_lua_shdict.c +++ b/src/ngx_http_lua_shdict.c @@ -2206,7 +2206,7 @@ ngx_http_lua_find_zone(u_char *name_data, size_t name_len) if (name->len == name_len && ngx_strncmp(name->data, name_data, name_len) == 0) { - return zone; + return &zone[i]; } }