[FEATURE]: Do you consider adding `plenary.job` like feature in a `nio` way?

[pysan3]

Ref: plenary.job.

As title. It'd be awesome if it could be implemented without using callbacks.

EDIT: New API

Ref: #7

 local first = nio.process.run({
   cmd = "printf", args = { "hello" }
 })

 local second = nio.process.run({
   cmd = "cat", stdin = first.stdout
 })

 local output = second.stdout.read()
 print(output)

OLD

-- Just my random thoughts.

nio.run(function()
  local job = nio.job.new("echo foo; echo bar; echo baz 1>&2")
  local iter = job:start()
  for line in iter do
    if line.type == "stdout" then
      on_stdout(line.content)
    elseif line.type == "stderr" then
      on_stderr(line.content)
    elseif line.type == "exit" then
      on_exit(line.exit_stat --[[@as integer]])
    end
  end
end)

[rcarriga]

Love the idea of having an abstraction over running subprocesses though would prefer a more flexible API.

I've got an initial implementation in #7, feel free to try it out (though it's still WIP)! The docs are quite sparse, I'd like to add more examples, but for now you can see the usage in the tests

[pysan3]

I'm not on my pc right now so I can't test it right away but I love the simplicity and also that I have the ability to pipe between multiple commands is absolutely awesome. Great job!!

One FR: would it be possible to loop through each line of stdout without waiting for the command to finish? For example the output from fd can be parsed one line at a time and processing one at a time.

[rcarriga]

Glad to hear you like it! I'll leave this change as the minimal interface for now. Reading lines (or until any separator) is relatively complex to do efficiently, and relatively trivial to do not efficiently. I'd rather get the interface "right" rather than half baked and have to deprecate things later. PRs welcome of course though

For reference, here's CPython's implementation of readuntil https://github.com/python/cpython/blob/7a470541e2bbc6f3e87a6d813e2ec42cf726de7a/Lib/asyncio/streams.py#L580 😅

[pysan3]

Thanks @rcarriga . I can officially recommend to neo-tree now :) nvim-neo-tree/neo-tree.nvim#1342

For reference, here's CPython's implementation of readuntil python/cpython@7a47054/Lib/asyncio/streams.py#L580 😅

I'm actually surprised that this mechanism was implemented in python and not directly modifying the stream inside internal C code.
Plenary's code looks really similar as well lol https://github.com/nvim-lua/plenary.nvim/blob/4f71c0c4a196ceb656c824a70792f3df3ce6bb6d/lua/plenary/job.lua#L291-L352

So, from what I understand, in the below code, lua will run until second.stdout without any block, and then blocked at .read()? And this code flow will be blocked until after cat prints a EOF to its stdout?

 local first = nio.process.run({
   cmd = "printf", args = { "hello" }
 })

 local second = nio.process.run({
   cmd = "cat", stdin = first.stdout
 })

 local output = second.stdout.read()
 print(output)

[rcarriga]

I can officially recommend to neo-tree now :) nvim-neo-tree/neo-tree.nvim#1342

Awesome, great to see it being used 😄

So, from what I understand, in the below code, lua will run until second.stdout without any block, and then blocked at .read()? And this code flow will be blocked until after cat prints a EOF to its stdout?

Yep that's correct 👌