Node 11.9+ Exception on openssl digital envelope routines cipher parameter evp_lib.c

[MasterJames]

Version: 11.13 crashes but in 11.8 it works (fails/waiting since 11.9)
Platform: Linux 4.15.0-47-generic #50-Ubuntu SMP Wed Mar 13 10:44:52 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Subsystem: ../deps/openssl/openssl/crypto/evp/evp_lib.c:81

Error:
error:060CC07A:digital envelope routines:EVP_CIPHER_asn1_to_param:cipher parameter error:../deps/openssl/openssl/crypto/evp/evp_lib.c:81

It is a pretty cryptic error (ha ha no pun intended).
I guess it has to do with openssl encryption maybe there is something else I should update.
I pass some encrypted stuff over a socket to nodejs from Chrome and get this error message above, while in node 11.8 there is no problem.

This is via SSL only, and might be related to elliptic encryption.
Unrelated to node execution the keys are generated this way.
openssl ecparam -genkey -name secp384r1 -out server-key.pem

but it connects okay initially I guess it is more to do with sending over the socket.

(anonymous) | @ | VM81:2026
-- | -- | --
  | emit | @ | VM13 events.js:202
  | EventEmitter.emit | @ | VM146 domain.js:439
  | (anonymous) | @ | VM14 execution.js:128
  | TLSWRAP (async) |   |  
  | init | @ | VM34 inspector_async_hook.js:21
  | emitInitNative | @ | VM19 async_hooks.js:137
  | TLSSocket._wrapHandle | @ | VM126 _tls_wrap.js:427
  | TLSSocket | @ | VM126 _tls_wrap.js:327
  | tlsConnectionListener | @ | VM126 _tls_wrap.js:750
  | emit | @ | VM13 events.js:197
  | EventEmitter.emit | @ | VM146 domain.js:439
  | onconnection | @ | VM61 net.js:1501
  | TCPSERVERWRAP (async) |   |  
  | init | @ | VM34 inspector_async_hook.js:21
  | emitInitNative | @ | VM19 async_hooks.js:137
  | createServerHandle | @ | VM61 net.js:1180
  | setupListenHandle | @ | VM61 net.js:1236
  | listenInCluster | @ | VM61 net.js:1304
  | doListen | @ | VM61 net.js:1443
  | processTicksAndRejections | @ | VM30 next_tick.js:76
  | runNextTicks | @ | VM30 next_tick.js:51
  | Module.runMain | @ | VM46 loader.js:800
  | executeUserCode | @ | VM11 node.js:526
  | startMainThreadExecution | @ | VM11 node.js:439
  | TickObject (async) |   |  
  | init | @ | VM34 inspector_async_hook.js:21
  | emitInitNative | @ | VM19 async_hooks.js:137
  | emitInitScript | @ | VM19 async_hooks.js:336
  | TickObject | @ | VM30 next_tick.js:99
  | nextTick | @ | VM30 next_tick.js:130
  | lookup | @ | VM215 dns.js:132
  | lookupAndListen | @ | VM61 net.js:1438
  | Server.listen | @ | VM61 net.js:1388
  | listenUp | @ | VM81:2066
  | begin | @ | VM81:1993

[bnoordhuis]

When you say "crashes" you mean it throws an exception rather than abort the process outright?

Do you have an easy way to reproduce? What does the .opensslErrorStack property of the exception object contain?

[MasterJames]

Hmm? Line 170 in events.js at throw er has er in it is message stack and Symbol kEpandStackSymbol expands with:
name bound enhanceStackTrace

So nothing found like you asked for.

Going up the stack 'this' is a TLSSocket nothing called opensslErrorStack there either.
Maybe you could direct me further to what I'm looking for otherwise I guess that the answer is 'undefined'?

This is normal just a particular state?
authorizationError: "UNABLE_TO_GET_ISSUER_CERT"
encrypted is true

I am using the npm websocket library would be a step to reproduce.
https://www.npmjs.com/package/websocket
"websocket": "^1.0.28"
is
https://github.com/theturtle32/WebSocket-Node

[bnoordhuis]

How can I reproduce that 'cipher parameter error' exception that you mention?

New enough v11.x releases attach that .opensslErrorStack property, see here. The documentation in master is more comprehensive but that aside.

UNABLE_TO_GET_ISSUER_CERT means Node.js cannot verify the peer's TLS certificate because one or more certificates in its certificate chain are untrusted (e.g., unknown or self-signed CA.)

[sam-github]

attach that .opensslErrorStack property

To be exceedingly clear, they attach it to the Error object that is thrown (not the TLSSocket).

This would likely be easy to figure out if you posted a standalone reprodution. Without it, I'm not even clear from the bug report if this is a client or a server that is throwing. Are you sure this is v11.13.0? The first tls file:line ref in the report is to _tls_wrap.js:427: var handle;, which is not a call to EventEmitter.emit.

[MasterJames]

Okay thanks for the responses I will do my best to try and help get to the bottom of this.
Now I've switched to 11.14 and the problem still exists. Works in 11.8 fails in all versions after something changed there.
I have nodejs running in debug with the chrome inspector, and also the browser with it's dev tools open.

It fails first on the 'server' side as describe. Yes subsequently there's a problem on the browser side but it's a module load time type error as the server is frozen on the EventEmitter.prototype.emit = function emit(
as it's halted at the line 170 in events.js throw er; // unhandled 'error' event

I only mentioned the TLSSocket as I searched for opensslErrorStack in 'er' and it it is not there?

If I ask JSON.stringify(er) I get an empty object {}
type er in console it displays this
Error: 140559191078720:error:060CC07A:digital envelope routines:EVP_CIPHER_asn1_to_param:cipher parameter error:../deps/openssl/openssl/crypto/evp/evp_lib.c:81:

opening the err in the Scope inspector under Block is
message as shown
stack prepends "Error:" to the message
also err, inspect, stringifiedEr are listed but all undefined.

Console trace after error displayed as

(anonymous) | @ | VM83:2025
-- | -- | --
  | emit | @ | VM16 events.js:198
  | EventEmitter.emit | @ | VM148 domain.js:481
  | (anonymous) | @ | VM41 execution.js:123
  | TLSWRAP (async) |   |  
  | init | @ | VM46 inspector_async_hook.js:21
  | emitInitNative | @ | VM25 async_hooks.js:132
  | TLSSocket._wrapHandle | @ | VM128 _tls_wrap.js:447
  | TLSSocket | @ | VM128 _tls_wrap.js:349
  | tlsConnectionListener | @ | VM128 _tls_wrap.js:786
  | emit | @ | VM16 events.js:193
  | EventEmitter.emit | @ | VM148 domain.js:481
  | onconnection | @ | VM63 net.js:1504
  | TCPSERVERWRAP (async) |   |  
  | init | @ | VM46 inspector_async_hook.js:21
  | emitInitNative | @ | VM25 async_hooks.js:132
  | createServerHandle | @ | VM63 net.js:1183
  | setupListenHandle | @ | VM63 net.js:1239
  | listenInCluster | @ | VM63 net.js:1307
  | doListen | @ | VM63 net.js:1446
  | processTicksAndRejections | @ | VM26 task_queues.js:81
  | runNextTicks | @ | VM26 task_queues.js:56
  | Module.runMain | @ | VM49 loader.js:880
  | (anonymous) | @ | VM43 run_main_module.js:21
  | TickObject (async) |   |  
  | init | @ | VM46 inspector_async_hook.js:21
  | emitInitNative | @ | VM25 async_hooks.js:132
  | emitInitScript | @ | VM25 async_hooks.js:330
  | TickObject | @ | VM26 task_queues.js:104
  | nextTick | @ | VM26 task_queues.js:135
  | lookup | @ | VM218 dns.js:132
  | lookupAndListen | @ | VM63 net.js:1441
  | Server.listen | @ | VM63 net.js:1391
  | listenUp | @ | VM83:2065
  | begin | @ | VM83:1992
  | boot | @ | VM58 hotbawx.js:360
  | (anonymous) | @ | VM58 hotbawx.js:383
  | Module._compile | @ | VM49 loader.js:813
  | Module._extensions..js | @ | VM49 loader.js:827
  | Module.load | @ | VM49 loader.js:685
  | Module._load | @ | VM49 loader.js:620
  | Module.runMain | @ | VM49 loader.js:877
  | (anonymous) | @ | VM43 run_main_module.js:21

I think the message UNABLE_TO_GET_ISSUER_CERT I foolishly mentioned is just a distraction because I didn't attach a client key.
The Browser shows the Certificate Chain is Valid and it normally works that way, so that's what the Server defaults to maybe instead of a client CERT_SIGNATURE_FAILURE or what I'm not sure let's please forget about that for now. It's normal with 11.8 for me to see that at this sock.encrypted = true and sock.authorized = false in all version.

I would like to try distilling 2 years work in an example but it's server code I am unaware of any fiddle for that. If there is a way let me know but I really doubt I will be able to upload code that is reproducible.

Maybe I need a suggestion or an example of how to use sockets without this npm package 'websockets' as I guess your recent changes since 11.8 are no longer working with it I'm supposing.

Maybe this is a good question to ask?
Where is /deps/openssl/openssl/crypto/evp/evp_lib.c:81: and what changes occurred in 11.9?

If I look further into 'er' there is proto within proto but no opensslErrorStack to be found after extensive searching.
There is also Symbol kExpandStackSymbol but it is equally missing anything useful.

I can inspect that Symbol and it shows er["Symbol(kExpandStackSymbol)"] is a function but executing it fails and it's otherwise undefined so I guess it's crashed at that point.

As I move up the stack the next item is line 481 in domain.js
return Reflect.apply(eventEmit, this, args);
there 'this' is the TLSSocket

see next post ...

[MasterJames]

JSON.stringify(this) minus data buffers 'this' is the TLSSocket
{"_tlsOptions":{"secureContext":{"context":{},"alpnBuffer":{"type":"Buffer","data":[]}},"isServer":true,"server":{"_contexts":[],"requestCert":true,"rejectUnauthorized":false,"key":{"type":"Buffer","data":[]},"sessionIdContext":"1a3e89f76acfef2465334dbe8b8d7a27","_sharedCreds":{"context":{},"alpnBuffer":{"type":"Buffer","data":[]}},"_events":{},"_eventsCount":5,"_connections":7,"_handle":{"reading":false},"_usingWorkers":false,"_workers":[],"_unref":false,"allowHalfOpen":false,"pauseOnConnect":false,"httpAllowHalfOpen":false,"timeout":120000,"keepAliveTimeout":5000,"maxHeadersCount":null,"headersTimeout":40000,"_connectionKey":"6:2001:569:7561:3800:a00:28dd:fe2f:38ee:443"}},"_secureContext":{"context":{},"alpnBuffer":{"type":"Buffer","data":[8,104,116,116,112,47,49,46,49]}},"reading":true,"lastHandshakeTime":21029,"handshakes":0,"_consumed":true},"_requestCert":true,"_rejectUnauthorized":false,"timeout":0,"parser":null,"_paused":false,"_peername":{"address":"2001:569:7561:3900:202b:7822:164f:8482","family":"IPv6","port":50692},"aGnt":"Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.86 Safari/537.36","mobile":false,"resourceURL":{"protocol":null,"slashes":null,"auth":null,"host":null,"port":null,"hostname":null,"hash":null,"search":null,"query":{},"pathname":"/register","path":"/register","href":"/register"},"ip":"2001:569:7561:3900:202b:7822:164f:8482"}

Not that this would be much help I'm just showing you what information is available next in the stack trace.

from the doc you referenced https://nodejs.org/docs/v11.14.0/api/errors.html#errors_class_error

For crypto only, Error objects will include the OpenSSL error stack in a separate property called opensslErrorStack **if** it is available when the error is thrown.
I guess it is not available at this time for some reason.

[MasterJames]

Newly released Version 12 of nodejs also not working.

[sam-github]

Where is /deps/openssl/openssl/crypto/evp/evp_lib.c:81: and what changes occurred in 11.9

It is the node src, download and untar it, or look on github, the exact file path is in your question.

Between 11.8 and 11.9, OpenSSL was updated from 1.1.0 to 1.1.1a, you could look at the openssl changelog to see the differences, though I'd be surprised if it was illuminating.

You don't have a reproduction we can run, and I'm not really clear on your situation.

Server is node, using the websocket package.

Client is a browser? Using websockets from where?

And this happens during websocket connection? Or https? Or after connection, during data exchange?

Also, the error message suggests problems verifying a peer cert, so are you doing client auth, that is, the client has a cert?

I'd suggest trying to strip this down. I appreciate that your app is likely large, but building a tiny server that offers a websocket connection, and then seeing if it has the same problem when your browser client connects would be the first step in problem isolation. The second would be replacing the browser with a node client. Somewhere along there, you might get a working system, at which point you could ask yourself how its different from your large system where its not working.

Or, if your system doesn't work, you will end up with a repro we can run.

#27376 would probably help diagnose this, but it hasn't landed yet, much less gotten released.

[MasterJames]

Thanks for the input Sam.
The websocket is from npm by that name which is https://github.com/theturtle32/WebSocket-Node
I cross posted this issue there.
theturtle32/WebSocket-Node#352
The last release of that is from Sept 2018 v1.0.28 and no response to this issue.
I was thinking it might be wise to replace that part if node team is confident it's something else, and there's no response there.

My Ubuntu version shows it's openssl version is OpenSSL 1.1.0g 2 Nov 2017
I'll check the system for updates/upgrades... 32? Now there's not enough room to upgrade on 18.04
I will have to rebuild the entire server from scratch too, to Ubuntu latest 19.04.
This one will have OpenSSL 1.1.1b Feb 2019 I'll report back after some time for rebuilding and testing.

[sam-github]

My Ubuntu version shows it's openssl version is OpenSSL 1.1.0g 2 Nov 2017

This should not be relevant. node has its own builtin version of openssl, check that version with node -p process.versions. The result will show if you are using an ubuntu distro build of node that uses their shared openssl. If that is the case, then please reproduce the problem using node from nodejs.org downloads.