Allow a CLI that does not handle Exceptions.

[Jacob-Stevens-Haas]

Is your feature request related to a problem? Please describe.
I would like to use my IDE's debugger to find out where SphinxErrors are coming from. In particular, I got the error toctree contains reference to nonexisting document when I didn't delete any documents from my repo. I'm not really familiar with PdB, so I tried putting an entry in my IDE's debugging config to run sphinx (This is VSCode, launch.json)

        {
            "name": "Documentation",
            "type": "python",
            "request": "launch",
            "module": "sphinx.cmd.build",
            "console": "integratedTerminal",
            "justMyCode": false,
            "args": [
                "-TEW",
                "-b",
                "html",
                "-d",
                " docs/_build/doctrees",
                "docs",
                "docs/_build/html"
            ]
        },

Unfortunately, all of the important work in the CLI gets passed through a try...except, and errors are sent to sphinx.cmd.build.handle_exception(), which just prints the error. So my debugger only catches the SystemExit exception. I can't get back into the call stack that generated the original exception and inspect variables.

Describe the solution you'd like
Move graceful exception handling to sphinx.__main__ so that people can debug sphinx.cmd.build with their IDE.

Describe alternatives you've considered

• This is also feels like slightly an issue with the -W flag, which doesn't actually raise an exception to the level of the python interpreter: it prints the exception, then sends sys.exit(2). That makes sense, and seems like the current behavior is necessary for --keep-going to be feasible. Perhaps sphinx.cmd.build.__main__() could handle the parsing, check for -W and --keep-going, and take over the call to handle_exception() from build_main(). That way, it could conditionally re-raise the exception.
• I can set my debugger to catch all exceptions, not just unhandled ones, but then I catch a variety of spurious exceptions.
• I can add a breakpoint in sphinx.cmd.build.handle_exceptions(), but I don't know if there's a way to pass the local exception variable to my IDE's debugger and view that stack?

[tk0miya]

Thank you for your proposal. I understand your concern and agree with you. +1 for improving Sphinx to debug it on IDE. But I don't still understand your proposal. Could you send a pull request, please? Then I'll review your proposal :-)

[AA-Turner]

I think this could be added as a --raise-raw-exceptions or etc flag -- there's a balance to play between the front end exposing Python implemention details and not, though.

A

[Jacob-Stevens-Haas]

Thank you for your proposal. I understand your concern and agree with you. +1 for improving Sphinx to debug it on IDE. But I don't still understand your proposal. Could you send a pull request, please? Then I'll review your proposal :-)

I'm new to the idea of event-driven programming, so my suggested design changes may not have been right. It'll take some time to get the hang of how sphinx works and submit a PR. In the interim, here's a simpler way of explaining the need: my debugger runs python -m sphinx -TE -b html . _build/html and stops/breakpoints upon unhandled exceptions. An extension generates the exception when a file isn't found. I'd like the debugger to be able to stop on that exception so I can inspect the variables that the extension sees, but it can't: sphinx handles that exception, prints it, and runs sys.exit(2). My debugger then stops on the SystemExit.

I think this could be added as a --raise-raw-exceptions or etc flag -- there's a balance to play between the front end exposing Python implemention details and not, though.

Good point about the balance. But an additional argument is additional complexity; if someone passes -W, I would assume they are prepared to deal with errors manually. Moreover, the interpreter already returns a nonzero exit code when it experiences an error:

~$ python -c 'print(foo)'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
NameError: name 'foo' is not defined
~$ echo $?
1

So I think the question is, do users of sphinx gain anything by having sphinx choose the exit code (2) rather than the default exit code for an error (1)?

[AA-Turner]

@Jacob-Stevens-Haas thank you for resurfacing this (in #13110).

We currently have --show-traceback. Perhaps we could move to --traceback=(show|raise) instead?

I wonder though if there's a better way of integrating with IDEs (such as PyCharm, VS Code, etc) that we should aim for first?

A

[Jacob-Stevens-Haas]

I see there's actually a new cli argument --exception-on-warning. This may have been what I was looking for, but I'm not sure yet. Basically, the ide debugger on 7.4.7 was sending me to a sys.exit call, rather than the point of the code that actually generated the error.