Skip to content

v.2.6.0-2 release proposal #1041

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 29 commits into from
Apr 29, 2016
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
dfb1d93
Explicitly ask for the platform in the issue template
Apr 13, 2016
4e52981
Small Readme fixes
Apr 13, 2016
a11e0c5
Don't expose internal variables
Apr 13, 2016
228573b
Support __proto__ and similar as object attribute in hgetall
Apr 13, 2016
5e42302
Accept arbitrary arguments in the debug function
Apr 13, 2016
dfd493f
Update benchmark file
Apr 13, 2016
d2b8f2f
Add support for camelCase
Apr 13, 2016
8e24380
Add optional callback option to duplicate function
Apr 13, 2016
aff765a
Fix execution order
Apr 13, 2016
a9d565b
Fix auth regression
Apr 13, 2016
cd58e1f
Implement message_buffer and pmessage_buffer events
Apr 13, 2016
5d12659
Fix typos / comments
Apr 13, 2016
683815d
Refactor pipelining
Apr 13, 2016
0424cb0
Move pub sub command into individual commands and use call_on_write
Apr 13, 2016
3038c90
Make sure all individual handled command work in multi context the same
Apr 13, 2016
97ae788
Implement CLIENT REPLY ON|OFF|SKIP
Apr 13, 2016
a857829
Improve error handling
Apr 14, 2016
e58e310
Remove unnecessary unallocation. This is done by the queue itself
Apr 14, 2016
8308a3e
Update dependencies
Apr 14, 2016
5fac595
Fix async test executed sync
Apr 14, 2016
0dc45bd
Improve pub sub mode further
Apr 18, 2016
f500398
Run tests only with the js parser instead of hiredis and js parser fr…
Apr 18, 2016
625f14e
Fix address always set to 127.0.0.1:6379 in case the host/port is set…
Apr 19, 2016
f7c4d13
Remove jshint comments and update istanbul comments
Apr 21, 2016
eae1693
Add monitor transaction warning / error
Apr 21, 2016
ce1678c
Improve coverage; make tests ready for Redis 3.2
Apr 21, 2016
5368e74
Add changelog
Apr 22, 2016
bf39492
Use built-in error classes to make errors more specific
Apr 24, 2016
03f1a60
Improve error handling
Apr 24, 2016
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ to a small test case, but it's highly appreciated to have as much data as possib
Thank you!_

* **Version**: What node_redis and what redis version is the issue happening on?
* **Platform**: What platform / version? (For example Node.js 0.10 or Node.js 5.7.0)
* **Platform**: What platform / version? (For example Node.js 0.10 or Node.js 5.7.0 on Windows 7 / Ubuntu 15.10 / Azure)
* **Description**: Description of your issue, stack traces from errors and code that reproduces the issue

[gitter]: https://gitter.im/NodeRedis/node_redis?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge
92 changes: 68 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,7 @@ Install with:

npm install redis

## Usage

Simple example, included as `examples/simple.js`:
## Usage Example

```js
var redis = require("redis"),
Expand Down Expand Up @@ -51,6 +49,8 @@ This will display:
mjr:~/work/node_redis (master)$

Note that the API is entirely asynchronous. To get data back from the server, you'll need to use a callback.
From v.2.6 on the API supports camelCase and snack_case and all options / variables / events etc. can be used either way.
Copy link

@dirkbonhomme dirkbonhomme Apr 22, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps mention that it also supports UPPERCASE? ..but that camelCase is advised because of Nodejs conventions

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good hint. But all-caps is not supported for events, variables and options at the moment. Implementing this too would be easy, but I'm wondering if it might be worth thinking about removing support for it in v.3. It was implemented in the very first commit of node_redis but should it also be used that way? I'd disagree.
It is actually documented somewhere else but documenting it further might encourage people to use it, so removing it afterwards would be more work for users to switch.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the confusion, I totally agree that options, events etc etc shouldn't be passed as uppercase. I was only referring to the method names / redis commands. Probably should have read the whole readme before leaving this comment

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right now I'm actually thinking about removing the documentation about the UPPER_CASE commands support as this is similar to snack_case not typical in Node.js landscape.
In general conventions as I know them this is actually even more confusing as uppercase mostly stands for constants and this is clearly not the case here.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

It is recommended to use camelCase as this is the default for the Node.js landscape.

### Promises

Expand Down Expand Up @@ -83,7 +83,7 @@ return client.multi().get('foo').execAsync().then(function(res) {
Each Redis command is exposed as a function on the `client` object.
All functions take either an `args` Array plus optional `callback` Function or
a variable number of individual arguments followed by an optional callback.
Here are examples how to use the api:
Examples:

```js
client.hmset(["key", "test keys 1", "test val 1", "test keys 2", "test val 2"], function (err, res) {});
Expand Down Expand Up @@ -111,8 +111,6 @@ client.get("missingkey", function(err, reply) {

For a list of Redis commands, see [Redis Command Reference](http://redis.io/commands)

The commands can be specified in uppercase or lowercase for convenience. `client.get()` is the same as `client.GET()`.

Minimal parsing is done on the replies. Commands that return a integer return JavaScript Numbers, arrays return JavaScript Array. `HGETALL` returns an Object keyed by the hash keys. All strings will either be returned as string or as buffer depending on your setting.
Please be aware that sending null, undefined and Boolean values will result in the value coerced to a string!

Expand All @@ -139,6 +137,7 @@ are passed an object containing `delay` (in ms) and `attempt` (the attempt #) at
### "error"

`client` will emit `error` when encountering an error connecting to the Redis server or when any other in node_redis occurs.
If you use a command without callback and encounter a ReplyError it is going to be emitted to the error listener.

So please attach the error listener to node_redis.

Expand Down Expand Up @@ -182,7 +181,7 @@ __Tip:__ If the Redis server runs on the same machine as the client consider usi
| host | 127.0.0.1 | IP address of the Redis server |
| port | 6379 | Port of the Redis server |
| path | null | The UNIX socket string of the Redis server |
| url | null | The URL of the Redis server. Format: `[redis:]//[user][:password@][host][:port][/db-number][?db=db-number[&password=bar[&option=value]]]` (More info avaliable at [IANA](http://www.iana.org/assignments/uri-schemes/prov/redis)). |
| url | null | The URL of the Redis server. Format: `[redis:]//[[user][:password@]][host][:port][/db-number][?db=db-number[&password=bar[&option=value]]]` (More info avaliable at [IANA](http://www.iana.org/assignments/uri-schemes/prov/redis)). |
| parser | hiredis | If hiredis is not installed, automatic fallback to the built-in javascript parser |
| string_numbers | null | Set to `true`, `node_redis` will return Redis number values as Strings instead of javascript Numbers. Useful if you need to handle big numbers (above `Number.MAX_SAFE_INTEGER === 2^53`). Hiredis is incapable of this behavior, so setting this option to `true` will result in the built-in javascript parser being used no matter the value of the `parser` option. |
| return_buffers | false | If set to `true`, then all replies will be sent to callbacks as Buffers instead of Strings. |
Expand Down Expand Up @@ -295,6 +294,51 @@ client.get("foo_rand000000000000", function (err, reply) {

`client.end()` without the flush parameter set to true should NOT be used in production!

## Error handling (>= v.2.6)

All redis errors are returned as `ReplyError`.
All unresolved commands that get rejected due to what ever reason return a `AbortError`.
As subclass of the `AbortError` a `AggregateError` exists. This is emitted in case multiple unresolved commands without callback got rejected in debug_mode.
They are all aggregated and a single error is emitted in that case.

Example:
```js
var redis = require('./');
var assert = require('assert');
var client = redis.createClient();

client.on('error', function (err) {
assert(err instanceof Error);
assert(err instanceof redis.AbortError);
assert(err instanceof redis.AggregateError);
assert.strictEqual(err.errors.length, 2); // The set and get got aggregated in here
assert.strictEqual(err.code, 'NR_CLOSED');
});
client.set('foo', 123, 'bar', function (err, res) { // To many arguments
assert(err instanceof redis.ReplyError); // => true
assert.strictEqual(err.command, 'SET');
assert.deepStrictEqual(err.args, ['foo', 123, 'bar']);

redis.debug_mode = true;
client.set('foo', 'bar');
client.get('foo');
process.nextTick(function () {
client.end(true); // Force closing the connection while the command did not yet return
redis.debug_mode = false;
});
});

```

Every `ReplyError` contains the `command` name in all-caps and the arguments (`args`).

If node_redis emits a library error because of another error, the triggering error is added to the returned error as `origin` attribute.

___Error codes___

node_redis returns a `NR_CLOSED` error code if the clients connection dropped. If a command unresolved command got rejected a `UNERCTAIN_STATE` code is returned.
A `CONNECTION_BROKEN` error code is used in case node_redis gives up to reconnect.

## client.unref()

Call `unref()` on the underlying socket connection to the Redis server, allowing the program to exit once no more commands are pending.
Expand Down Expand Up @@ -363,7 +407,7 @@ client.HMSET(key1, "0123456789", "abcdefghij", "some manner of key", "a type of

## Publish / Subscribe

Here is a simple example of the API for publish / subscribe. This program opens two
Example of the publish / subscribe API. This program opens two
client connections, subscribes to a channel on one of them, and publishes to that
channel on the other:

Expand Down Expand Up @@ -412,6 +456,16 @@ Client will emit `pmessage` for every message received that matches an active su
Listeners are passed the original pattern used with `PSUBSCRIBE` as `pattern`, the sending channel
name as `channel`, and the message as `message`.

### "message_buffer" (channel, message)

This is the same as the `message` event with the exception, that it is always going to emit a buffer.
If you listen to the `message` event at the same time as the `message_buffer`, it is always going to emit a string.

### "pmessage_buffer" (pattern, channel, message)

This is the same as the `pmessage` event with the exception, that it is always going to emit a buffer.
If you listen to the `pmessage` event at the same time as the `pmessage_buffer`, it is always going to emit a string.

### "subscribe" (channel, count)

Client will emit `subscribe` in response to a `SUBSCRIBE` command. Listeners are passed the
Expand Down Expand Up @@ -529,7 +583,7 @@ Redis. The interface in `node_redis` is to return an individual `Batch` object b
The only difference between .batch and .multi is that no transaction is going to be used.
Be aware that the errors are - just like in multi statements - in the result. Otherwise both, errors and results could be returned at the same time.

If you fire many commands at once this is going to **boost the execution speed by up to 400%** [sic!] compared to fireing the same commands in a loop without waiting for the result! See the benchmarks for further comparison. Please remember that all commands are kept in memory until they are fired.
If you fire many commands at once this is going to boost the execution speed significantly compared to fireing the same commands in a loop without waiting for the result! See the benchmarks for further comparison. Please remember that all commands are kept in memory until they are fired.

## Monitor mode

Expand All @@ -539,7 +593,7 @@ across all client connections, including from other client libraries and other c
A `monitor` event is going to be emitted for every command fired from any client connected to the server including the monitoring client itself.
The callback for the `monitor` event takes a timestamp from the Redis server, an array of command arguments and the raw monitoring string.

Here is a simple example:
Example:

```js
var client = require("redis").createClient();
Expand Down Expand Up @@ -599,9 +653,10 @@ the second word as first parameter:
client.multi().script('load', 'return 1').exec(...);
client.multi([['script', 'load', 'return 1']]).exec(...);

## client.duplicate([options])
## client.duplicate([options][, callback])

Duplicate all current options and return a new redisClient instance. All options passed to the duplicate function are going to replace the original option.
If you pass a callback, duplicate is going to wait until the client is ready and returns it in the callback. If an error occurs in the meanwhile, that is going to return an error instead in the callback.

## client.send_command(command_name[, [args][, callback]])

Expand All @@ -615,27 +670,16 @@ All commands are sent as multi-bulk commands. `args` can either be an Array of a

Boolean tracking the state of the connection to the Redis server.

## client.command_queue.length
## client.command_queue_length

The number of commands that have been sent to the Redis server but not yet replied to. You can use this to
enforce some kind of maximum queue depth for commands while connected.

Don't mess with `client.command_queue` though unless you really know what you are doing.

## client.offline_queue.length
## client.offline_queue_length

The number of commands that have been queued up for a future connection. You can use this to enforce
some kind of maximum queue depth for pre-connection commands.

## client.retry_delay

Current delay in milliseconds before a connection retry will be attempted. This starts at `200`.

## client.retry_backoff

Multiplier for future retry timeouts. This should be larger than 1 to add more time between retries.
Defaults to 1.7. The default initial connection retry is 200, so the second retry will be 340, followed by 578, etc.

### Commands with Optional and Keyword arguments

This applies to anything that uses an optional `[WITHSCORES]` or `[LIMIT offset count]` in the [redis.io/commands](http://redis.io/commands) documentation.
Expand Down
42 changes: 22 additions & 20 deletions benchmarks/multi_bench.js
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ function returnArg (name, def) {
}
var num_clients = returnArg('clients', 1);
var run_time = returnArg('time', 2500); // ms
var pipeline = returnArg('pipeline', 1); // number of concurrent commands
var versions_logged = false;
var client_options = {
parser: returnArg('parser', 'hiredis'),
Expand All @@ -41,17 +42,18 @@ function lpad (input, len, chr) {

metrics.Histogram.prototype.print_line = function () {
var obj = this.printObj();
return lpad((obj.min / 1e6).toFixed(2), 6) + '/' + lpad((obj.max / 1e6).toFixed(2), 6) + '/' + lpad((obj.mean / 1e6).toFixed(2), 6);
return lpad((obj.mean / 1e6).toFixed(2), 6) + '/' + lpad((obj.max / 1e6).toFixed(2), 6);
};

function Test (args) {
this.args = args;
this.args.pipeline = +pipeline;
this.callback = null;
this.clients = [];
this.clients_ready = 0;
this.commands_sent = 0;
this.commands_completed = 0;
this.max_pipeline = this.args.pipeline || 50;
this.max_pipeline = +pipeline;
this.batch_pipeline = this.args.batch || 0;
this.client_options = args.client_options || {};
this.client_options.parser = client_options.parser;
Expand Down Expand Up @@ -206,7 +208,7 @@ Test.prototype.print_stats = function () {
var duration = Date.now() - this.test_start;
totalTime += duration;

console.log('min/max/avg: ' + this.command_latency.print_line() + ' ' + lpad(duration, 6) + 'ms total, ' +
console.log('avg/max: ' + this.command_latency.print_line() + lpad(duration, 5) + 'ms total, ' +
lpad(Math.round(this.commands_completed / (duration / 1000)), 7) + ' ops/sec');
};

Expand All @@ -217,55 +219,55 @@ large_buf = new Buffer(large_str);
very_large_str = (new Array((4 * 1024 * 1024) + 1).join('-'));
very_large_buf = new Buffer(very_large_str);

tests.push(new Test({descr: 'PING', command: 'ping', args: [], pipeline: 1}));
tests.push(new Test({descr: 'PING', command: 'ping', args: []}));
tests.push(new Test({descr: 'PING', command: 'ping', args: [], batch: 50}));

tests.push(new Test({descr: 'SET 4B str', command: 'set', args: ['foo_rand000000000000', small_str], pipeline: 1}));
tests.push(new Test({descr: 'SET 4B str', command: 'set', args: ['foo_rand000000000000', small_str]}));
tests.push(new Test({descr: 'SET 4B str', command: 'set', args: ['foo_rand000000000000', small_str], batch: 50}));

tests.push(new Test({descr: 'SET 4B buf', command: 'set', args: ['foo_rand000000000000', small_buf], pipeline: 1}));
tests.push(new Test({descr: 'SET 4B buf', command: 'set', args: ['foo_rand000000000000', small_buf]}));
tests.push(new Test({descr: 'SET 4B buf', command: 'set', args: ['foo_rand000000000000', small_buf], batch: 50}));

tests.push(new Test({descr: 'GET 4B str', command: 'get', args: ['foo_rand000000000000'], pipeline: 1}));
tests.push(new Test({descr: 'GET 4B str', command: 'get', args: ['foo_rand000000000000']}));
tests.push(new Test({descr: 'GET 4B str', command: 'get', args: ['foo_rand000000000000'], batch: 50}));

tests.push(new Test({descr: 'GET 4B buf', command: 'get', args: ['foo_rand000000000000'], pipeline: 1, client_opts: { return_buffers: true} }));
tests.push(new Test({descr: 'GET 4B buf', command: 'get', args: ['foo_rand000000000000'], client_opts: { return_buffers: true} }));
tests.push(new Test({descr: 'GET 4B buf', command: 'get', args: ['foo_rand000000000000'], batch: 50, client_opts: { return_buffers: true} }));

tests.push(new Test({descr: 'SET 4KiB str', command: 'set', args: ['foo_rand000000000001', large_str], pipeline: 1}));
tests.push(new Test({descr: 'SET 4KiB str', command: 'set', args: ['foo_rand000000000001', large_str]}));
tests.push(new Test({descr: 'SET 4KiB str', command: 'set', args: ['foo_rand000000000001', large_str], batch: 50}));

tests.push(new Test({descr: 'SET 4KiB buf', command: 'set', args: ['foo_rand000000000001', large_buf], pipeline: 1}));
tests.push(new Test({descr: 'SET 4KiB buf', command: 'set', args: ['foo_rand000000000001', large_buf]}));
tests.push(new Test({descr: 'SET 4KiB buf', command: 'set', args: ['foo_rand000000000001', large_buf], batch: 50}));

tests.push(new Test({descr: 'GET 4KiB str', command: 'get', args: ['foo_rand000000000001'], pipeline: 1}));
tests.push(new Test({descr: 'GET 4KiB str', command: 'get', args: ['foo_rand000000000001']}));
tests.push(new Test({descr: 'GET 4KiB str', command: 'get', args: ['foo_rand000000000001'], batch: 50}));

tests.push(new Test({descr: 'GET 4KiB buf', command: 'get', args: ['foo_rand000000000001'], pipeline: 1, client_opts: { return_buffers: true} }));
tests.push(new Test({descr: 'GET 4KiB buf', command: 'get', args: ['foo_rand000000000001'], client_opts: { return_buffers: true} }));
tests.push(new Test({descr: 'GET 4KiB buf', command: 'get', args: ['foo_rand000000000001'], batch: 50, client_opts: { return_buffers: true} }));

tests.push(new Test({descr: 'INCR', command: 'incr', args: ['counter_rand000000000000'], pipeline: 1}));
tests.push(new Test({descr: 'INCR', command: 'incr', args: ['counter_rand000000000000']}));
tests.push(new Test({descr: 'INCR', command: 'incr', args: ['counter_rand000000000000'], batch: 50}));

tests.push(new Test({descr: 'LPUSH', command: 'lpush', args: ['mylist', small_str], pipeline: 1}));
tests.push(new Test({descr: 'LPUSH', command: 'lpush', args: ['mylist', small_str]}));
tests.push(new Test({descr: 'LPUSH', command: 'lpush', args: ['mylist', small_str], batch: 50}));

tests.push(new Test({descr: 'LRANGE 10', command: 'lrange', args: ['mylist', '0', '9'], pipeline: 1}));
tests.push(new Test({descr: 'LRANGE 10', command: 'lrange', args: ['mylist', '0', '9']}));
tests.push(new Test({descr: 'LRANGE 10', command: 'lrange', args: ['mylist', '0', '9'], batch: 50}));

tests.push(new Test({descr: 'LRANGE 100', command: 'lrange', args: ['mylist', '0', '99'], pipeline: 1}));
tests.push(new Test({descr: 'LRANGE 100', command: 'lrange', args: ['mylist', '0', '99']}));
tests.push(new Test({descr: 'LRANGE 100', command: 'lrange', args: ['mylist', '0', '99'], batch: 50}));

tests.push(new Test({descr: 'SET 4MiB str', command: 'set', args: ['foo_rand000000000002', very_large_str], pipeline: 1}));
tests.push(new Test({descr: 'SET 4MiB str', command: 'set', args: ['foo_rand000000000002', very_large_str]}));
tests.push(new Test({descr: 'SET 4MiB str', command: 'set', args: ['foo_rand000000000002', very_large_str], batch: 20}));

tests.push(new Test({descr: 'SET 4MiB buf', command: 'set', args: ['foo_rand000000000002', very_large_buf], pipeline: 1}));
tests.push(new Test({descr: 'SET 4MiB buf', command: 'set', args: ['foo_rand000000000002', very_large_buf]}));
tests.push(new Test({descr: 'SET 4MiB buf', command: 'set', args: ['foo_rand000000000002', very_large_buf], batch: 20}));

tests.push(new Test({descr: 'GET 4MiB str', command: 'get', args: ['foo_rand000000000002'], pipeline: 1}));
tests.push(new Test({descr: 'GET 4MiB str', command: 'get', args: ['foo_rand000000000002']}));
tests.push(new Test({descr: 'GET 4MiB str', command: 'get', args: ['foo_rand000000000002'], batch: 20}));

tests.push(new Test({descr: 'GET 4MiB buf', command: 'get', args: ['foo_rand000000000002'], pipeline: 1, client_opts: { return_buffers: true} }));
tests.push(new Test({descr: 'GET 4MiB buf', command: 'get', args: ['foo_rand000000000002'], client_opts: { return_buffers: true} }));
tests.push(new Test({descr: 'GET 4MiB buf', command: 'get', args: ['foo_rand000000000002'], batch: 20, client_opts: { return_buffers: true} }));

function next () {
Expand Down
Loading