Formatter: Uses space indentation in docstrings when using `indent-style = "tab"`

[MichaReiser]

If I have indent-style set to "tab", when I run ruff format it is still converting indentation within docstring comments to be spaces, which then causes ruff check to fail on rule E101: Indentation contains mixed spaces and tabs

Before:

After:

Is there a format setting I can add to fix this? or is this a bug/intentional?

Originally posted by @1024Adam in #7310 (comment)

We can reproduce this behavior in our own tests:

ruff/crates/ruff_python_formatter/tests/snapshots/format@docstring.py.snap

Lines 538 to 544 in 3076d76

	def multiple_negatively_indented_docstring_lines():
	"""a
	b
	c
	d
	e
	"""

ruff/crates/ruff_python_formatter/tests/snapshots/format@docstring.py.snap

Lines 569 to 577 in 3076d76

	class IndentMeSome:
	def doc_string_without_linebreak_after_colon(self):
	"""This is somewhat strange
	a
	b
	We format this a is the docstring had started properly indented on the next
	line if the target indentation. This may we incorrect since source and target
	indentation can be incorrect, but this is also an edge case.
	"""

E101 has no special handling for docstring. It simply looks at every line and tests if the leading whitespace contains mixed spaces and tabs. Meaning, there can also be false positives if a multiline string contains leading whitespace that is a mix of spaces and tabs.

[MichaReiser]

@konstin you implemented the docstring formatting and have a lot of context on the Python interop. Could you take this on as a formatter goal sometime this month? The solution doesn't have to be to fix the docstring formatting, we could also warn about using E101 with the formatter.

I think what the formatter should support is that it preserves tabs when using indent-style=tab by using indent. It means it wouldn't fix mixed indentation in docstrings for you, but it, at least, doesn't result in mixed indentation.

The alternative is that we always use indent (and align if it isn't a multiple of tab-width). But I think there we have the issue that Python always assumes a tab-width of 8

[MichaReiser]

Depends on #8445

[Mutantpenguin]

Is there any way to circumvent this behaviour right now? Like telling the formatter to not format docstrings at all?

[konstin]

@Mutantpenguin What is the formatting that you want for docstring when using tab indentation? And if you use some documentation generator (like sphinx), do you know how they react to tab indent vs. space indent, and if indent size matters?

[Mutantpenguin]

We use handwritten docstrings so I can't say anything about sphinx and so on.

I want the same as MichaReiser:

• the formatter shouldn't use spaces when I specified tabs
• the comment shouldn't be indented more than before (as you can see in his screenshots)

E114 (which is in preview) should be taken into account too https://docs.astral.sh/ruff/rules/indentation-with-invalid-multiple-comment/

[Mutantpenguin]

I now have a nice testcase.

ruff.toml:

[format]
indent-style="tab"

This stays the same after formatting and doesn't get changed at all:

def test(arg1: str) -> None:
	"""
	Arguments:
	arg1: super duper arg with a tab in front
	"""

This stays the same after formatting and doesn't get changed at all too:

def test(arg1: str) -> None:
	"""
	Arguments:
	    arg1: super duper arg with a tab and 4 spaces in front
	"""

This

def test(arg1: str) -> None:
	"""
	Arguments:
		arg1: super duper arg with 2 tabs in front
	"""

gets butchered and formatted to this:

def test(arg1: str) -> None:
	"""
	Arguments:
	        arg1: super duper arg with 1 tab and 8 spaces in front
	"""

What the GitHub code formatting hides: we use a tab size of 4 in our editor so the last case looks expecially weird since the 8 spaces look like 2 indentation levels with tabs.

[MichaReiser]

@konstin are you still working on this? If not, feel free to un-assign.

[konstin]

No, i don't have a good solution for this. I'd probably go with replacing every 8 spaces in the beginning of a docstring with tabs, implementation-wise, even if whatever we do is bad because it we either mismatch with cpython (always unconditionally assumes tab width 8 and always converts to spaces) or we mismatch our configuration.

[BurntSushi]

Would it be possible to add a new config knob to the formatter that lets an end user specify their tabwidth? Then it seems like we might be able to get things right here.