grpc.Dial does not return an error on bad certificates

[tamird]

When using mutual-TLS with mismatched certificates, calling grpc.Dial(..., grpc.WithBlock()) never returns. The client connection is happily banging its head against bad certificate errors and never reports an error to the caller.

Worse, it seems that WithBlock() is extremely unreliable. It blocks until the connection is connected, but rarely returns a non-nil error to the caller in the error case (connection refused is one case where it does the right thing).

[iamqizhao]

Connection error is treated as transient so that grpc internal keeps retrying until you set a timeout. I expect you can see something from log with mismatched certificates.

[iamqizhao]

s/until/unless

[dsymonds]

I could see the argument that cert errors are not transient, though. There's no point in retrying because it's expected to fail consistently.

[iamqizhao]

I can imagine that in some cases the name resolver or load balancer may direct you to another server which can accepts the cert. I am still debating with myself whether I should introduce some more tweaks into the connection errors.

[tamird]

There are some cases (like ours) where there is no load balancer in the
picture. There should be some way of treating this error as non
transient, even if it's not the default behaviour.

On Thu, Mar 31, 2016 at 10:03 PM, Qi Zhao notifications@github.com wrote:

I can imagine that in some cases the name resolver or load balancer may
direct you to another server which can accepts the cert. I am still
debating with myself whether I should introduce some more tweaks into the
connection errors.

—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
#622 (comment)

[iamqizhao]

Let me talk to our security guru to decide whether security handshake error should be treated as a fatal error first.

[iamqizhao]

After the discussion, we will treat this as a fatal error. A pull request will be out to address this soon.

[tamird]

@iamqizhao ping

[tamird]

@iamqizhao what's up with this?

[aaronlehmann]

This is a pain point for me as well. There doesn't seem to be a way to surface a TLS-related error to the calling code. It only goes to the log. This means an application where the user is not monitoring the logs directly can't signal the error condition.

[iamqizhao]

@menghanl is working on a PR to improve this. It should be out soon.

[stevvooe]

@iamqizhao Are there plans to handle this for use cases outside of WithBlock? Seems odd that we would just let an RPC method call hang forever. In general, it seems best to avoid WithBlock but there is no way to boil up errors on synchronous operations.