Feature Request: Alternate Interrupt method in instruct mode

[CRD716]

It would be nice to have a way to interrupt the model's output and inject your own words into the response, instead of starting a new instruction. This could be useful for writers, as well as generally just being cool.

[sw]

Maybe somewhat related to --author-mode proposed in #1040, at least in terms of intended user group.

[DannyDaemonic]

@CRD716 Are you talking about instruct mode here? If so, a clearer title might have been: "Feature Request: Alternate Interrupt method in instruct mode"

Assuming we're talking about instruct mode, I wonder if that feature is a good fit with --author-mode. It shouldn't be too hard to integrate that into it but it's a different codewise.

We'd have to store a state somewhere that said if instruction mode was interrupted by Ctrl+C so we'd know not to resume inference from wherever we were instead of starting a new "### Instruction:". Author mode would certainly help in resuming on the same line, but other than that, it's more about how the console interprets lines, not how it handles resets.

It could also be a different option that may be implied by --author-mode? Something like --instruct-interactive? I think the biggest downside to adding switches is that --help may get overwhelming. I do plan on making a pull request that separates the common options from the advanced options so this isn't as big of an issue.

Alternatively, as it currently is, when you Ctrl+C in instruct mode, perhaps people would just prefer if it always allowed them to interject on the answer?

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale.