PathNotFound error when trying to validate request with query parameter

[stojan-jovic]

Found very strange bug, which is pretty critical for our work process, as it fully blocking endpoint validation (have no workaround for now).

I'm working with Falcon middleware at low level, but issue can be easy reproduced with even simpler example. I will provide most minimal working example that can be used for reproducing issue.

Falcon app (everything in single module, ugly, but only for testing purposes):

import logging
import json
import falcon
from wsgiref import simple_server

from openapi_core import create_spec
from openapi_spec_validator.schemas import read_yaml_file
from openapi_core.contrib.falcon.middlewares import FalconOpenAPIMiddleware

logging.basicConfig()
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)


class Resource(object):

    def on_get(self, req, resp, path_param=None):
        req_data = {}

        if path_param is not None:
            req_data['path'] = path_param

        query_param = req.get_param('validate')
        if query_param is not None:
            req_data['query'] = query_param

        resp.body = json.dumps(req_data)
        resp.status = falcon.HTTP_200


if __name__ == '__main__':
    spec_dict = read_yaml_file('swagger.yml')
    spec = create_spec(spec_dict)
    openapi_middleware = FalconOpenAPIMiddleware.from_spec(spec)

    # falcon_app = falcon.API()
    falcon_app = falcon.API(middleware=[openapi_middleware])

    test_resource = Resource()
    falcon_app.add_route('/v1/test', test_resource)
    falcon_app.add_route('/v1/test/{path_param}', test_resource)

    httpd = simple_server.make_server('127.0.0.1', 8000, falcon_app)
    logger.info('Now serving on port 8000')
    httpd.serve_forever()

Swagger spec file content (swagger.yml):

openapi: 3.0.0

info:
  version: 1.0.0
  title: Test PathNotFound
  description: Test PathNotFound
  contact:
    name: Test
    url: http://www.test.com
    email: [email protected]
  license:
    name: Test
    url: https://www.test.com/terms-of-use/

servers:
  - url: http://localhost:8000
  - url: /

tags:
  - name: Test
    description: Test

paths:
  /v1/test:
    get:
      tags:
        - Test
      summary: Test
      description: Test
      parameters:
        - name: validate
          in: query
          description: Test query parameter
          schema:
            type: string
      responses:
        200:
          description: Successful
        400:
          description: BadRequest error
        default:
          description: Unexpected error
  /v1/test/{path_param}:
    get:
      tags:
        - Test
      summary: Test
      description: Test
      parameters:
        - name: path_param
          in: path
          required: true
          description: Test path parameter
          schema:
            type: string
        - name: validate
          in: query
          description: Test query parameter
          schema:
            type: string
      responses:
        200:
          description: Successful
        400:
          description: BadRequest error
        default:
          description: Unexpected error

When I try to make simple GET request:

curl http://localhost:8000/v1/test?validate=query_param

Getting following error:

{
  "errors": [
    {
      "title": "Path not found for http://localhost:8000/v1/test?validate=query_param",
      "status": 404,
      "class": "<class 'openapi_core.templating.paths.exceptions.PathNotFound'>"
    }
  ]
}

Strange thing is that if I have some path parameter in request, query parameter being properly validated, for example GET request:

curl http://localhost:8000/v1/test/path_param?validate=query_param

Request passing and returning result:

{
  "path": "path_param",
  "query": "query_param"
}

If we send the same requests without query parameters, everything working fine:

curl http://localhost:8000/v1/test
curl http://localhost:8000/v1/test/path_param

Here there is stack trace for the same error (from my app, generated by my low-level middleware logic:):

Traceback (most recent call last):
  File "falcon_svc.py", line 395, in process_resource
    result.raise_for_errors()
  File "D:\Work\workspace2\venvs\.sim_venv\lib\site-packages\openapi_core\validation\datatypes.py", line 11, in raise_for_errors
    raise error
  File "D:\Work\workspace2\venvs\.sim_venv\lib\site-packages\openapi_core\validation\request\validators.py", line 29, in validate
    path, operation, _, path_result, _ = self._find_path(request)
  File "D:\Work\workspace2\venvs\.sim_venv\lib\site-packages\openapi_core\validation\validators.py", line 19, in _find_path
    return finder.find(request)
  File "D:\Work\workspace2\venvs\.sim_venv\lib\site-packages\openapi_core\templating\paths\finders.py", line 23, in find
    raise PathNotFound(request.full_url_pattern)
openapi_core.templating.paths.exceptions.PathNotFound: Path not found for http://localhost:8000/v1/test?validate=query_param

I did some very quick debugging and found pretty strange behavior of PathFinder's find method (where error is raised), for example, request that has path parameter and being properly validated, looks like follows when parsed:

TemplateResult(pattern='/v1/test/{path_param}', variables={'path_param': 'path_param?validate=query_param'})

Which does not looks to me as expected behavior and leads me to conclusion that it being validated with lucky circumstances.

My environment:

Package                  Version                      
------------------------ -----------------------------
falcon                   1.4.1
openapi-core             0.13.3

Not checked is this behavior the same with some other framework (Flask, Django), but I would say that this issue probably is not dependent on used framework. However, on the other side, it's strange that nobody found it till now?! Or I'm doing something wrong?

If there is no plans to debug this and provide fix, I would be happy to help with one PR with fix, if you can confirm that this is really bug and give me short directions where to look first!?

[stojan-jovic]

I did some deeper research and it seems that issue is with PathFinder class and the way how it working with request's full_url_pattern.
Specifically, when full_url_pattern contains query parameters and when it's passed to the parse.Parser's search method to match pattern, it behaves inconsistently:

• In the above case, when we do not have path parameters at all, it do not matching anything (i.e. returns None) and that leads to PathNotFound error.
• When we have path parameters in full_url_pattern, it will practically append last path parameter and everything behind it, which is again not proper behavior but it working (probably because it's always string, but just guessing).

However, solution is to use full_url_pattern without query parameters and everything working fine then. Not sure is this permanent solution, but when I tested it by adding following line:

full_url_pattern = full_url_pattern.split('?')[0]

at the beginning of PathFinder._get_paths_iter method, everything worked fine.

[dlarrick]

Please see #233 which probably fixes your issue.

[stojan-jovic]

Nice, thank you very much! This PR fixes all of my issues, even one that I have not reported (related to draining stream data from the original Falcon's request object). Really appreciate prompt feedback and quick fixes!

Btw, when new version will be released?

[dlarrick]

@stojan-jovic I am not a developer on this project, just hit the same issue in my own work and submitted a patch. I hope it will be considered for merge soon.

[stojan-jovic]

@dlarrick Oh, I see now. Thank you very much for PR! I hope that fixes will be approved and merged soon.
@p1c2u FYI.