Implement an indentation provider to support folding of docstrings, comments & multi line strings

[DonJayamanne]

We might want to consider disabling the folding provider in this release due to a number of issues in VS Code:

• [folding] Zombie folding regions after deletion vscode#50992
• Extension code folding provider always breaks default VS Code code folding behavior vscode#50991
• Empty list of FoldingRanges overrides VS Code folding providers vscode#50990

Came across this today and found it very annoying. Was unable to collapse a large python file, initially assumed it was an intermittent VSC issue, later looked into this and found that its worse than I suspected.

Basically us introducing a custom folding provider breaks the VSC default folding behavior we won't be able to collapse, any block of Python code

@brettcannon cc

[DonJayamanne]

Looks like we're getting some weird answers, I'd suggest we disable this new folding provider until things are sorted out in VSC (and I'm having a difficult time thus far).

[DonJayamanne]

Just realized that, the folding provider PR didn't have a news entry.

[DonJayamanne]

Based on feedback from VS Code team, we have to implement the indentation provider (i.e. we need to implement everything ourselves) microsoft/vscode#50991 (comment)

[beltekylevi]

Is there any update or alternative solution for this issue?

[brettcannon]

@beltekylevi no specific update. As for an alternative, you can check the Marketplace to see if someone has provided a separate extension for this.

[MicaelJarniac]

I've been looking for this as well, and until there's a better solution, we can use # region and # endregion around the docstring to explicitly inform VS Code that it should be a foldable region. From my testings, it worked pretty well, the docstring was still identified correctly, and all other foldable regions remained intact.

Another option is to use an extension such as https://github.com/zokugun/vscode-explicit-folding that allows us to manually define what should be used to delimit foldable regions, for example with a regex. A problem I've had with this one (although I haven't experimented much) was that it seems to completely replace the foldable regions, and not just add to them, so when I was using it, the only foldable regions I had were the ones between """s, that I manually defined. An example configuration for Python is provided on its readme.

Example with # region and # endregion

def greet(name: str) -> str:
    # region
    """Greets a person

    Parameters
    ----------
    name : str
        Name of the person to greet

    Returns
    -------
    str
        Greetings for said person
    """
    # endregion

    return f"Hello, {name}. Nice to meet you!"

[beltekylevi]

@MicaelJarniac Thank you, I didn't know about this region and endregion thing. It's perfect for now.

Unfortunately, I had the same issue with the vscode-explicit-folding extension.

[yuji96]

2021-03-28.22.18.43.mov

Here's how I solved it:

1. Copy the Python extension configuration files to own user directory.

source	/Applications/Visual Studio Code.app/Contents/Resources/app/extensions/python
target	~/.vscode/extensions/python

The following command is an example for macOS.

$ cp -r /Applications/Visual\ Studio\ Code.app/Contents/Resources/app/extensions/python ~/.vscode/extensions

2. Add a regex that matches docstrings to the markers in ~/.vscode/extensions/python/language-configuration.json.

{
  "folding": {
    "offSide": true,
    "markers": {
      "start": "^\\s*#\\s*region\\b|^\\ *\"{3}(?!.*\"{3}).+",
      "end": "^\\s*#\\s*endregion\\b|^\\ *\"{3}$"
    }
  }
}

This regex will match if we follow strict constraints such as OpenStack Style Guidelines - Docstrings.

# OK
"""[summary]
"""

# NG
"""
[summary]
"""

# NG
"""[summary]
"""  # comment

[jamilraichouni]

Nice one, thanks!

I have a little improvement to propose:

"start": "^\\s*#\\s*region\\b|^\\ *[r]?\"{3}(?!.*\"{3}).+"

That makes raw docstrings collapsable.

def func():
    r"""My docstring summary.

    Foo is located at ``C:\Windows\bar``

    """

[danieltuzes]

@yuji96 @jamilraichouni I copied the file to "C:\Users\username\.vscode\extensions\python\language-configuration.json" (the target folder python didn't exist, I created it) and modified the relevant lines in the new file to

    "folding": {
        "offSide": true,
        "markers": {
            "start": "^\\s*#\\s*region\\b|^\\ *\"{3}(?!.*\"{3}).+",
            "end": "^\\s*#\\s*endregion\\b|^\\ *\"{3}$"
        }
    },

but I still cannot fold the docstring. I am not sure if VS Code picks up the file. Is there something I missed? Is vscode-explicit-folding required to use this feature?

[jamilraichouni]

@danieltuzes in fact you end up with a directory named how ever you like it in the directory extensions.
In my case it looks like that:

So you see, that there are multiple files needed. The copy command from above
($ cp /Applications/Visual Studio Code.app/Contents/Resources/app/extensions/python ~/.vscode/extensions)
copies a directory.

You need to know that you must reload Visual Studio Code to see the hand made extension.
I named the extension by modifying the following in the file package.json:

"displayName": "Python docstring folding"

and I can see the extension:

[danieltuzes]

@jamilraichouni I indeed missed copying the whole folder, thank you for pointing it out. That's one real advantage of open source software and a strong community, thank you both, @yuji96 !

[sometowngeek]

I think #4345 was closed in error.

This issue is regarding collapsing/folding docstrings, comments, and multi-line strings, and #4345 is about being able to select the block. CTL+SHIFT+[-> or <-] doesn't work well with Python, and I think that's the thing #4345 is referring to.

I have the same issue as #4345.

Can we please consider re-opening that issue?

I'd appreciate it! Thank you!

[brettcannon]

@sometowngeek please open a new issue.

[jrom99]

I don't know if this is related to the issue at hand, but the folding of functions ending in comments isn't working.

def test_func():
    print("text")
    # hello world

folds as:

def test_func(): ...
    # hello world

System Information
Version: 1.63.2
Commit: 899d46d82c4c95423fb7e10e68eba52050e30ba3
Date: 2021-12-15T09:39:46.686Z
Electron: 13.5.2
Chromium: 91.0.4472.164
Node.js: 14.16.0
V8: 9.1.269.39-electron.0
OS: Linux x64 5.15.14-200.fc35.x86_64

Python: 3.10.1 (main, Jan 10 2022, 00:00:00) [GCC 11.2.1 20211203 (Red Hat 11.2.1-7)] on linux
Python Extension: v2021.12.1559732655
Pylance: v2022.1.1

[brettcannon]

@jrom99 this would get fixed by an indentation provider.

[LeonardoGentile]

Hello, 🖖
Is there any update on this?
Is it going to be implemented any time soon or should we rely on manual hacks as the ones proposed above?

[jamilraichouni]

I created a little extension to be able to fold multiline docstrings:

https://marketplace.visualstudio.com/items?itemName=jamilraichouni.python-docstring-folding

[jrom99]

Sorry for my lack of knowledge on this topic, but how was this handled before? I remember that single line comments at the end of functions used to fold correctly, given that the indentation was at least the same as the function's content. Is this due to microsoft/vscode#37682? And how does it keep up with multiple providers?

[brettcannon]

Our plan is to rely on microsoft/pylance-release#372 to fix this issue.

[ahallermed]

For someone, who also fell over this issue and wants to have a very nice customizable solution, I can highly recommend this extension: maptz.regionfolder (github).

It is additive to the VSCode folding and doesn't overwrite the settings as vscode-explicit-folding does as discussed above.

You can also define which kind of regex patterns should be folded at file opening, like I did below.

With the following section in the vscode settings.json:

    "maptz.regionfolder": {
        // Defaults per language can be found here: https://github.com/maptz/maptz.vscode.extensions.customfolding/blob/master/src/config/DefaultConfiguration.ts
        "[python]": {
            // regex cheat sheet: https://cheatography.com/davechild/cheat-sheets/regular-expressions/
            // ?! is a negative lookahead
            // works for: '#region'; '# region'; '# region Something else'; '""" + whitespace'; 'r""" + whitespace';
            // doesn't work for" '"""' without trailing whitespace, as start and end for regex would find the same token as start AND end
            // REGEX
            "foldStartRegex": "^[\\]s*#[\\s]*region[\\s]*(.*)[\\s]*$|^[\\ ]*[r]?\"{3}(?!.*\"{3}).+",
            "foldEndRegex": "^[\\s]*#[\\s]*endregion\\b|^[\\ ]*\"{3}$",
            "foldDefinitions": [
                // these imports get folded automatically after file is opened
                {
                    // region imports
                    "foldStartRegex": "^[\\]s*#[\\s]*region[\\s]*(imports)[\\s]*$",
                    "foldEndRegex": "^[\\s]*#[\\s]*endregion\\b",
                    "isFoldedByDefault": true
                },
                {
                    // region header
                    "foldStartRegex": "^[\\]s*#[\\s]*region[\\s]*(header)[\\s]*$",
                    "foldEndRegex": "^[\\s]*#[\\s]*endregion\\b",
                    "isFoldedByDefault": true
                }
            ],
            // INSERTION TAGS
            "foldStart": "# region [NAME]", //Text inserted at the start of the fold.  Use the `[NAME]` placeholder to indicate where the cursor should be placed after insertion
            "foldEnd": "# endregion", //Text inserted at the end of the fold
            "disableFolding": false
        }
    }

And customized keysettings with explanations:

    // folding regions in e.g. python code
    // -> fold regions -> which are collapsed by default or not, it doesn't matter
    {
        "key": "ctrl+r",
        "command": "regionfolder.collapseAllRegions",
        "when": "editorTextFocus"
    },
    // -> fold defaults -> which are defined as being folded on file opening.
    {
        "key": "ctrl+shift+/",
        "command": "regionfolder.collapseDefault",
        "when": "editorTextFocus"
    },
    {
        "key": "ctrl+shift+m ctrl+shift+r",
        "command": "-regionfolder.collapseDefault",
        "when": "editorTextFocus"
    },
    //*/
    // -> unfold all
    {
        "key": "ctrl+shift+r",
        "command": "editor.unfoldAll",
        "when": "editorTextFocus && foldingEnabled"
    },
    {
        "key": "ctrl+k ctrl+j",
        "command": "-editor.unfoldAll",
        "when": "editorTextFocus && foldingEnabled"
    },

And of course, thank you folks for discussing this issue and support with a solution/regex patterns.

[brettcannon]

More advanced folding is now being handled by the latest version of Pylance, so if there's any existing issues with it then please open an issue at https://github.com/microsoft/pylance-release .