Keep Alive (NTLM)

[terrancesnyder]

I am trying to proxy content and the target server is running NTLM auth. The NTLM spec requires a certain handshake where the first 401 is sent, and the client responds.

http://www.innovation.ch/personal/ronald/ntlm.html

...this manifests itself in that the network connection 
must be kept alive during the second part of the 
handshake, i.e. between the receiving of the type-2 
message from the server (step 4) and the sending of the 
type-3 message (step 5). Each time the connection is closed 
this second part (steps 3 through 6) must be repeated over 
the new connection (i.e. it's not enough to just keep sending 
the last type-3 message). Also, once the connection is 
authenticated, the Authorization header need not be sent 
anymore while the connection stays open, no matter what 
resource is accessed.

For implementations wishing to work with M$'s software 
this means that they must make sure they use either HTTP/1.0 
keep-alive's or HTTP/1.1 persistent connections....

I've run wireshark and fiddler and I can see that the connection is NOT being re-used even though the req and res object include the required connection = 'keep-alive'.

I've attached screenshots of wireshark showing different ports being used during the multiple stages of NTLM authentication. Doing some research I got conflicting stories on if keep-alive actually does work, and if it does / does not work in node-http-proxy.

so far, running through squid-proxy, fiddler, and other proxies wireshark reports the correct re-using of the connection. The only one that does not show this is node-http-proxy (and it's also the only one that isn't able to auth the user correctly).

In addition it seems like the headers for www-authenticate are being mangled when there are multiple ones so I had to put in a patch... Without this Firefox, Chrome, IE were not prompting for NTLM authentication because there was only ONE www-authenticate header being returned rather than two separate ones.

    // WWW-Authenticate
    if (response.headers['www-authenticate']) {
      var l = [];
      for (var i=0; i<response.headers['www-authenticate'].split(',').length;i++) {
        l.push(response.headers['www-authenticate'].split(',')[i].trim());
      }
      response.headers['www-authenticate'] = l;
    }

Bad (through node-http-proxy)

As shown above you can see for each request we grab a new socket (which destroys NTLM auth and proves that it is not doing keep-alive).

Good (through fiddler, squid, etc)

The above capture shows that with a proxy running (fiddler, squid, etc) we get the re-use one would expect.

[porsager]

Is there any chance that node-http-proxy will support more than one request on a single connection in the future?
We would love to be customers at nodejitsu, but currently users of our client app that talks to our jitsu app can't get proper performance when making 10+ requests after oneanother. The handshake needed for https at each connection is adding up to the total request time.
Now that's the practical thing about it, the other thing is that on the nodejitsu frontpage it says If it works in localhost, it works for us!". That statement isn't quite true when persistent connections aren't allowed through node-http-proxy.

I don't know if it is wrong to bring up nodejitsu related stuff here, but after speaking with nuno on IRC, he told me to continue my questions here :)

Thanks.

[dscape]

Is your issue really with the keepalive or is it the https handshake performance?

I thought you were doing some advanced https1.1 stuff, but if its related to the handshake we actually would need to take different steps.

Anyway thanks for bumping up this, also do you remember the pull request that was related to this?

[porsager]

Our issue is with keepalive.

About handshake performance i don't think it is relavent since the problem is worst if latency is High which it often is on Mobile connections (and our api is used mainly by iphone users). Then improving the handshake wouldn't do anything about the trips back and forth.

It would be solved if note-http-proxy did persistent connections between proxy and client. I don't think there's an urgent need to implement it between server and proxy, since there's almost no latency and no handshake? Correct?

I'll see if I can find the pull request, when I'm back on my comp, but I think it was a smaller fix, and not an implementation for a solution.

[dscape]

Sounds awesome. Thanks for clarifying.

My point was that ssh performance in node is a far more understood topic. @indutny has a repo called tls-snappy that tackles it, but to the best of my knowledge nothing beats putting a tls terminal in front of node yet :(

I think you saw some discussion from @mranney and @mikeal on the topic on a request issue. Not entirely sure how voxer is currently dealing with this though.

[porsager]

Yeah, i've been reading through it, and found that Voxer made this module/agent that they're using:
https://github.com/ceejbot/keep-alive-agent

Unfortunately i don't have more time to look into this, cause we have to be ready for production within 2 weeks - so for now we will need to host our app by ourselves :(
That is - until (hopefully) nodejitsu get's support for it ;)

[GraemeF]

I'm seeing the same problem - without @terrancesnyder's patch (which I put in at https://github.com/nodejitsu/node-http-proxy/blob/master/lib/node-http-proxy/http-proxy.js#L262 ) there was no prompt for username and password, but I still see a new source port number for each part of the negotiation, which means NTLM doesn't work.

[GraemeF]

Another update after messing about some more yesterday - I substituted the default Node http agent with https://github.com/ceejbot/keep-alive-agent and that seemed to make the negotiation happen on the same connection, and it now gets as far as prompting for username and password (in Chrome). Unfortunately, it doesn't get any further than that because submitting the credentials results in another 401 and another prompt.

I don't know what other effect dropping keep-alive-agent in there might be having, either... 😕

[HelgeL]

@GraemeF, did you actually get any further with this? I'm trying to run NTLM authentication over an HTTPS-to-HTTP reverse proxy and see similar results. I have included the abovementioned patch but would probably need a hint on how to include the keep-alive-agent you mention. Are there any other choices to proxy NTLM authenticated calls? (have tried SQUID which didn't help, contrasting to what @terrancesnyder reports above) I just need to get one single call through that doesn't even have any content, it just authenticates a user and returns a token/cookie, and subsequent requests are normal http requests that use this token/cookie for authentication...

[GraemeF]

No I didn't, sorry.

[HelgeL]

Thanks anyway!

[kadishmal]

We're running node-http-proxy in front of our Web and API servers, and we critically need keep-alive. Any plan to support it in the near future? The old http-proxy@0.1.x used to support keep-alive. Since upgrade all connections with the client get instantly closed.

[ezaelbello]

Any updates on this? I've been testing node-http-proxy as a transparent proxy by simply running

var server = http.createServer(function(req, res) {
  console.log("proxy-ing " + req.url);
  proxy.web(req, res, { target: req.url });
});

But most of the content do not pass as connection is closed:

With proxy enabled:

With proxy disabled:

[mburbea]

Is there any sort of update as to how to fix or at least by pass this issue? I'm trying to proxy an ntlm authenticated server and running into the same issues.

[robertlevy]

+1