You can input multiline text in `TextArea` when `multiline=False`

[eight04]

The app:

from prompt_toolkit import Application
from prompt_toolkit.buffer import Buffer
from prompt_toolkit.layout.containers import Window, HSplit
from prompt_toolkit.layout.controls import BufferControl
from prompt_toolkit.layout.layout import Layout

from prompt_toolkit.key_binding import KeyBindings
from prompt_toolkit.widgets import TextArea

kb = KeyBindings()

@kb.add('c-q')
def exit_(event):
  event.app.exit()

buffer1 = Buffer()  # Editable buffer.

root_container = HSplit([
  TextArea(
    prompt='Type something: ',
    multiline=False,
    wrap_lines=False,
    accept_handler=lambda buff: buffer1.insert_text(f"accept_handler called: {buff.text}\n"),
  ),
  Window(height=1, char='-'),
  Window(content=BufferControl(buffer=buffer1)),
])

layout = Layout(root_container)

app = Application(key_bindings=kb, layout=layout, full_screen=True)
app.run()

When multiline=False, pressing enter should trigger accept_handler.

However, when pasting multi-line text to the console, TextArea draws a new line then scroll down. In this case, accept_handler won't be triggered.

Another way to trigger this bug is to type very fast when the computer is slow. I guess it works similarly as copy-pasting.

Windows 10
prompt_toolkit 3.0.47
cmd.exe + Windows Terminal

[eight04]

It seems that the problem is: there is no "paste" event on Windows, and prompt-toolkit will try to guess a paste event when getting multiline text:

python-prompt-toolkit/src/prompt_toolkit/input/win32.py

Line 260 in 6695411

if self.recognize_paste and self._is_paste(all_keys):

However, since it is just a guess, it is not reliable. When the app is slow responding and the user presses enter multiple times, these chars will be treated as a paste event.

To avoid this behavior, add this line after creating the app:

app.input.console_input_reader.recognize_paste = False

1. I suggest documenting this behavior.
2. We should handle the paste event better. Since pasting multiline text into a single-line prompt doesn't really make sense, we can reject the paste event and re-feed those keys into input. On vt100, it should throw or replace \n with a space.

[Splines]

I think this bug is the reason for this IPython bug: ipython/ipython#13054. What we see on Linux and macOS:

In [2]: hello_world()

# Actually runs the hello world function, e.g. prints to the console

In [3]: 

What we see on Windows instead:

In [2]: hello_world()
    ..:

@eight04 Do you know any workaround that involves sending escape sequences to the terminal? Unfortunately, we cannot use your suggested workaround (app.input.console_input_reader.recognize_paste = False) since we write a VSCode extension that is just able to read and write to the in-built terminal that has a running IPython shell in it (InteractiveShellEmbed).

[Splines]

Since pasting multiline text into a single-line prompt doesn't really make sense

In the context of IPython it does make sense to send something like this to the terminal (programatically): hello_world()\n, which according to the mentioned logic of python-prompt-toolkit is already treated as multi-line-input, right? Otherwise, if we just send hello_world(), the cell won't execute, so the newline character is necessary here.

[eight04]

@eight04 Do you know any workaround that involves sending escape sequences to the terminal? Unfortunately, we cannot use your suggested workaround (app.input.console_input_reader.recognize_paste = False) since we write a VSCode extension that is just able to read and write to the in-built terminal that has a running IPython shell in it (InteractiveShellEmbed).

python-prompt-toolkit/src/prompt_toolkit/input/win32.py

Lines 260 to 281 in 6695411

	if self.recognize_paste and self._is_paste(all_keys):
	gen = iter(all_keys)
	k: KeyPress | None
	for k in gen:
	# Pasting: if the current key consists of text or \n, turn it
	# into a BracketedPaste.
	data = []
	while k and (
	not isinstance(k.key, Keys)
	or k.key in {Keys.ControlJ, Keys.ControlM}
	):
	data.append(k.data)
	try:
	k = next(gen)
	except StopIteration:
	k = None
	if data:
	yield KeyPress(Keys.BracketedPaste, "".join(data))
	if k is not None:
	yield k

I think you can try inserting a non-printable key at the start to break the loop e.g.

<C-L>hello_world()\n

However, there might be a chance that prompt-toolkit doesn't get the entire content at once and get

<C-L>hello
_world()\n

Then the second part will still be converted into a paste.

Also the additional control character may have side effect to the term.

[eight04]

No that might not work. It seems that prompt-toolkit will generate multiple pastes when getting non-printable chars in the middle?

[Splines]

I've written a monkey patch to set recognize_paste = False, see Manim-Notebook/manim-notebook#18 (comment). In CI/CD, it prints prompt_toolkit.input.win32 has been patched many times, so the patch itself seems to work, but we still experience the bug. I've now asked Windows users to try to set recognize_paste to False manually in the site_packages/prompt_toolkit/input/win32.py and await their response.

Update: Turns out it is probably this issue here, which is the reason for the IPython bug. It's just my monkey patch which doesn't work properly: Manim-Notebook/manim-notebook#18 (comment)

Update: a now working monkey patch can be found in this PR: Manim-Notebook/manim-notebook#117

[Splines]

Ok, so in case anybody looks for a patch to fix this problem: put the following at the top of your Python file (before you import anything else).

import sys


def monkey_patch_win32_recognize_paste():
    """
    Disables the "guessed bracketed mode" on Windows to avoid treating key
    strokes as paste events whenever the machine is slow to process input.

    Patch taken from:
    https://github.com/prompt-toolkit/python-prompt-toolkit/issues/1894#issuecomment-2601043935
    """
    if sys.platform != "win32":
        return

    try:
        print("Trying to patch prompt_toolkit.input.win32 ...")
        import prompt_toolkit.input.win32

        win32 = sys.modules["prompt_toolkit.input.win32"]
        original_class = win32.Win32Input
        original_init = original_class.__init__

        def patched_init(self, stdin=None, *args, **kwargs):
            print("Patched Win32Input __init__ called")
            original_init(self, stdin, *args, **kwargs)

            if hasattr(self, "console_input_reader"):
                print("Patching console_input_reader.recognize_paste to False")
                self.console_input_reader.recognize_paste = False

        original_class.__init__ = patched_init
        print("Win32Input.__init__ has been successfully patched.")
    except ImportError as exc:
        raise ImportError(
            "Module 'prompt_toolkit.input.win32' could not be imported."
        ) from exc


monkey_patch_win32_recognize_paste()

This will manually set the recognize_paste attribute of the console reader – which is instantiated here – to False.

It's a pity that this behavior is by default turned on (the docstring even says that this "bracketed paste guess" is not entirely correct), but at least this is a working solution for us now. @eight04 Thanks for raising this issue, otherwise I would never have thought of it.

---

Some tips for how to apply this patch

Determine the user site-packages folder via

python3 -c "import sysconfig; print(sysconfig.get_path('purelib'))"

In that folder, create two files:

# file: __windows_paste_patch.pth
import __windows_paste_patch

# file: __windows_paste_patch.py
# Here the patch from above

Don't use the sitecustomize.py file as that one is not supported in virtual environments. Instead, this approach with the .pth file works even in venv.

Here is a script that we use to perform this automatically:

Install patch script

import sysconfig
import os

PATCH_FILE_NAME = "__manim_notebook_windows_paste_patch"
PATCH_PTH = f"import {PATCH_FILE_NAME}"

PATCH = '''
import sys


def monkey_patch_win32_recognize_paste():
    """
    Disables the "guessed bracketed mode" on Windows to avoid treating key
    strokes as paste events whenever the machine is slow to process input.

    Patch provided in:
    https://github.com/prompt-toolkit/python-prompt-toolkit/issues/1894#issuecomment-2601043935

    If you want to disable this patch, simply remove this file and the
    corresponding __manim_notebook_windows_patch.pth file from your
    site-packages directory.
    """
    if sys.platform != "win32":
        return

    print("🐒 will try to patch...")

    try:
        import prompt_toolkit.input.win32

        win32 = sys.modules["prompt_toolkit.input.win32"]
        original_class = win32.Win32Input
        original_init = original_class.__init__

        def patched_init(self, stdin=None, *args, **kwargs):
            original_init(self, stdin, *args, **kwargs)
            if hasattr(self, "console_input_reader"):
                print("🐒 recognize_paste set to False")
                self.console_input_reader.recognize_paste = False

        original_class.__init__ = patched_init

    except ImportError as exc:
        raise ImportError(
            "Monkey-patching module 'prompt_toolkit.input.win32' failed as"
            + " we could not import it. Please report this issue to the"
            + " Manim Notebook developers on GitHub:"
            + " https://github.com/Manim-Notebook/manim-notebook"
        ) from exc

    except Exception as exc:
        raise Exception(
            "Monkey-patching module 'prompt_toolkit.input.win32' failed"
            + " due to an unexpected error. Please report this issue to the"
            + " Manim Notebook developers on GitHub:"
            + " https://github.com/Manim-Notebook/manim-notebook"
            + f" Original error: {exc}"
        ) from exc


monkey_patch_win32_recognize_paste()

'''


def install_patch():
    """
    Installs the Windows paste patch by copying the patch file to the
    site-packages directory and creating a .pth file that imports the patch.

    This patch will be applied whenever the Python interpreter is started.
    """
    # Find path to the site-packages directory
    # https://stackoverflow.com/a/52638888/
    site_packages_path = sysconfig.get_path("purelib")

    # Copy the patch file to the site-packages directory
    patch_file_path = os.path.join(site_packages_path, f"{PATCH_FILE_NAME}.py")
    with open(patch_file_path, "w", encoding="utf-8") as patch_file:
        patch_file.write(PATCH)

    # https://github.com/pypa/virtualenv/issues/1703#issuecomment-596618912
    # Create a .pth file in the site-packages directory that imports the patch
    # (necessary since sitecustomize.py does not work in virtual environments)
    pth_file_path = os.path.join(site_packages_path, f"{PATCH_FILE_NAME}.pth")
    with open(pth_file_path, "w", encoding="utf-8") as pth_file:
        pth_file.write(PATCH_PTH)


install_patch()
print("🐒 Manim Notebook: Windows paste patch applied")
print(
    "For more information, see https://github.com/Manim-Notebook/manim-notebook/wiki/%F0%9F%A4%A2-Troubleshooting#windows-paste-patch"
)