TLS/SSL test type(s) #3290
Description
This issue is for planning a new test type that exercises TLS (SSL), which was suggested as test type 10 on issue #133. I recommend reviewing the comment thread for #133 since it contains several prior thoughts concerning a TLS/SSL test type that remain applicable. Though I am not manually copying those comments over here, please feel free to reiterate points raised there.
Add any comments and questions that you believe can help us reach a consensus on reasonable requirements for this new test type.
Not all participants will agree that a TLS test is warranted in this project. Historically, we have not focused on the performance of web servers, intending our project to be about web application frameworks. However, our project is about measuring the high-water mark of performance using real-world web application technology stacks. And from that perspective, including a TLS test type is reasonable as long as there are some stacks for which TLS would be included "in-stack." And I know that is satisfied in the real-world because many stacks include nginx or another mainstream web server that has robust TLS capability.
Perhaps it is important to remember that, to whatever degree it is possible, we want the test implementations in this project to represent canonical usage of the platforms and frameworks we exercise. Many platforms and frameworks are of the opinion that TLS is an external concern to be handled by a front-end web server, load balancer, or other external system or service. That is perfectly fine and acceptable. Such platforms and frameworks may elect to bypass implementing this TLS test type. All test types in this project are optional, and omission should not be considered a failing or even evidence of lack of capability for a framework or platform. (Aside: We intend to eventually provide contributors a way to document the architectural approach that they have taken, explain the details of each test implementation they have provided, and in the case of omitting a test, explain why that test type is not applicable.)
I propose the following:
- This may be split into multiple test types since real-world TLS usage is applied to any and all request types. However, I am reluctant to duplicate many existing test types for a few reasons (e.g., additional test types increase the total runtime of the full suite). I tentatively propose TLS versions of two existing test types: Plaintext and Fortunes. Tentatively, TLS-Plaintext and TLS-Fortunes.
- If possible, I'd like to agree to a singular TLS cipher-suite configuration that is broadly supported and considered acceptable to a wide audience. If that is infeasible, I'd like for us to set some "minimum" acceptable level and illustrate the options that satisfy the minimum.
- If possible, I'd like to find and use a tool that validates TLS configuration that we can use in our suite's validation of implementations. Perhaps curl or similar can do this; I am not sure. But by way of comparison, I am imagining something that is approximately a command-line version of the SSL Labs tests.
Thoughts, questions, corrections, comments appreciated!