Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(remote): add wait subcommands #17856

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

Conversation

groves
Copy link
Contributor

@groves groves commented Mar 25, 2022

This still needs more tests and handling of waiting for multiple edits. I was hoping to get some feedback on the approach in general as I work on those pieces.

@github-actions github-actions bot added lua stdlib remote remote UI, --remote commands, p2p / peer-to-peer labels Mar 25, 2022
vim.api.nvim_command(table.concat(command, ' '))
if flags['wait'] then
vim.api.nvim_create_augroup('_remote_' .. chan_id, {})
vim.api.nvim_create_autocmd('BufUnload', {callback=function()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll need to wait till all the open bufs are unloaded, not just the first one.

end
vim.fn.rpcrequest(rcid, 'nvim_command', table.concat(command, ' '))
local remote_chan_id = vim.fn.rpcrequest(rcid, 'nvim_get_api_info')[1]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a silly way to get the channel id for the remote side to callback with. I couldn't see a way to do that from the remote side though. Is there a way from inside an RPC to get the channel id for the other side of the connection? If not, would it make sense for me to add that?

src/nvim/main.c Outdated
if (wait) {
// If we're just waiting for the remote server to let us know editing is complete, flip flags
// to get us to listen for its response in as lightweight a way as possible
params.use_vimrc = "NONE";
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really I'd like to start the main loop enough to handle incoming RPC calls to say when I should exit. This works for that, but it's a little indirect. Is there a more direct way to kick off the main loop and do nothing else?

@Shougo
Copy link
Contributor

Shougo commented Mar 25, 2022

I have tested the feature as git commit editor with neovim terminal, but neovim is launched within neovim terminal.
I like neovim-remote behavior instead. It is possible?

@zeertzjq
Copy link
Member

Maybe this can be used:

nvim --server $NVIM_LISTEN_ADDRESS --remote-tab

@Shougo
Copy link
Contributor

Shougo commented Mar 26, 2022

I use it, but it launches within neovim terminal...

@groves
Copy link
Contributor Author

groves commented Mar 28, 2022

@Shougo Thanks for trying it out! Is the failure a regression with this PR, or is it failing with just the code on master? If it's failing on master, would you mind opening an issue, answering the below questions, and tagging me? I'd like to keep this PR focussed on changes in the PR.

In either case, a few more pieces of info would be helpful for me looking into this:

  1. how are you launching the Neovim server instance with the terminal in it?
  2. what command are you using as editor for git commit? How are you configuring git?
  3. what do you mean by it being "launched in the neovim terminal"? That it's a new instance of neovim started in the terminal, or that it's' a split in the terminal window?
  4. are there any error messages given? Is there anything in :messages in either the client or the server?

@Shougo
Copy link
Contributor

Shougo commented Mar 29, 2022

Thanks for trying it out! Is the failure a regression with this PR, or is it failing with just the code on master? If it's failing on master, would you mind opening an issue, answering the below questions, and tagging me? I'd like to keep this PR focussed on changes in the PR.

I have tested the PR. Because in master wait feature is not implemented yet.

I have tested it with neovim built in terminal.

I have set $EDITOR to nvim --server {servername} --remote-tab-wait-silent or nvr --remote-tab-wait-silent.
I have tested the editor by git commit command.

When nvr, the editor is neovim current tab(not terminal). It is my expected behavior.
When the PR, the editor is executed in terminal in vim. It is not expected result.

Note: I have tested it in Alacritty terminal. But I think it is same with neovim GUI client.

@Shougo
Copy link
Contributor

Shougo commented Mar 29, 2022

Note: I use nvim --listen ~/.cache/nvim/server.pipe from terminal.

@Shougo
Copy link
Contributor

Shougo commented Mar 29, 2022

what do you mean by it being "launched in the neovim terminal"? That it's a new instance of neovim started in the terminal, or that it's' a split in the terminal window?

It seems new instance in the terminal. But you can test it easily.

are there any error messages given? Is there anything in :messages in either the client or the server?

No error messages. I think it is not error.

@groves
Copy link
Contributor Author

groves commented Mar 29, 2022

@Shougo What are you setting {servername} to in your nvim --server {servername} --remote-tab-wait-silent example?

For it to connect to the pipe you're creating in the --listen command, it would need to be the full path i.e. ~/.cache/nvim/server.pipe. --remote doesn't support --servername yet, though I'd like to add that before this is released.

Also, using silent is going to suppress any warning about not being able to connect. Could you try with --remote-tab-wait and see if anything is printed to :messages?

@Shougo
Copy link
Contributor

Shougo commented Mar 30, 2022

What are you setting {servername} to in your nvim --server {servername} --remote-tab-wait-silent example?

I use nvim --listen ~/.cache/nvim/server.pipe from terminal.
I think the server feature works. But it is not I expected...

Also, using silent is going to suppress any warning about not being able to connect. Could you try with --remote-tab-wait and see if anything is printed to :messages?

I have tested it. No error messages.

@groves
Copy link
Contributor Author

groves commented Mar 30, 2022

I use nvim --listen ~/.cache/nvim/server.pipe from terminal.

I'm wondering about the other side of the connection i.e. where you specify --server not --listen. When you say you set $EDITOR to nvim --server {servername} --remote-tab-wait-silent, what value are you using for {servername}? $EDITOR should be nvim --server ~/.cache/nvim/server.pipe --remote-tab-wait-silent to connect to the Neovim listening on that pipe.

@Shougo
Copy link
Contributor

Shougo commented Mar 31, 2022

$EDITOR should be nvim --server ~/.cache/nvim/server.pipe --remote-tab-wait-silent to connect to the Neovim listening on that pipe.

Yes. I have set and it works. Because the editor is executed in the neovim terminal. Oh no.

I have tested it by deol.nvim.

https://github.com/Shougo/deol.nvim/blob/master/autoload/deol.vim#L238

@Shougo
Copy link
Contributor

Shougo commented Mar 31, 2022

Hm.... It seems the server feature does not work.

nvim --listen ~/.cache/nvim/server.pipe --clean

And

nvim --server ~/.cache/nvim/server.pipe --remote-tab-wait-silent ~/work/test2.ts --clean

It does not work. Because test2.ts is opened new neovim tab.
Please tell me the correct instruction.

@groves
Copy link
Contributor Author

groves commented Mar 31, 2022

To test this, your server starting command, nvim --listen ~/.cache/nvim/server.pipe --clean, looks perfect.

For the client, use this command:

nvim --clean --server ~/.cache/nvim/server.pipe --remote-tab-wait ~/work/test2.ts

Note that clean is moved before the remote command. Since remote treats the arguments after it like files, later arguments are ignored when sending files remote.

Most importantly, note that it's using --remote-tab-wait and not --remote-tab-wait-silent. Any of the silent options cause remote to edit locally without printing error messages if it can't connect.

After starting the server like that, please run :messages and report that output.

If that shows any errors, I'd also appreciate it if you could test on master and see if the same thing shows up. Since this branch is still under development, I may have broken something here.

@justinmk
Copy link
Member

justinmk commented Mar 31, 2022

I don't quite understand why we need all of the random variants of Vim's --remote-foo CLI args. We only really need one or two: --remote and maybe --remote-wait. Anything else should be easy to do via Lua or Vimscript. If it's not, then that should be fixed rather than having a bunch of clunky special-purpose CLI options.

@Shougo
Copy link
Contributor

Shougo commented Apr 1, 2022

@groves Hi. I have tested it.

E247: Failed to connect to '/home/shougo/.cache/nvim/server.pipe': connection refused. Editing locally

Why it is failed?

$ ~/src/neovim/build/bin/nvim --clean --listen ~/.cache/nvim/server.pipe
$ ~/src/neovim/build/bin/nvim --clean --server ~/.cache/nvim/server.pipe --remote-tab-wait ~/work/test2.ts
E247: Failed to connect to '/home/shougo/.cache/nvim/server.pipe': connection refused. Editing locally

Please see my command line.

@groves
Copy link
Contributor Author

groves commented Apr 1, 2022

I don't quite understand why we need all of the random variants of Vim's --remote-foo CLI args.

I'd also lose the tab and silent variants if I were designing this from scratch. I see keeping the variants as allowing ease of migration from Vim and to allow people to write scripts and instructions that work for both. This feels like part of the API that we keep in the same way that we keep a bunch of clunky bits of Vimscript and builtin functions.

@groves
Copy link
Contributor Author

groves commented Apr 1, 2022

@Shougo My guess would be that the server is failing to listen on that pipe. In the server instance you're running with ~/src/neovim/build/bin/nvim --clean --listen ~/.cache/nvim/server.pipe, run :edit $NVIM_LOG_FILE after it starts up. The server start commands should log a message to that file when they fail.

This does make me wonder if we should have a user visible message if we fail to start the server on an address given in --listen. Only logging if we fail to listen on the default address seems reasonable. If we fail to host like the user asked with --listen, they probably want to know about it. I'm happy to add that in a separate PR.

@Shougo
Copy link
Contributor

Shougo commented Apr 2, 2022

OK. I get it. It seems works.
But it has strange behavior.

If I open the file by ~/src/neovim/build/bin/nvim --clean --server ~/.cache/nvim/server.pipe --remote-tab-wait ~/work/test2.ts, the file is opened both remote neovim and local neovim. Why?

I think local neovim must wait until remote neovim file is closed like neovim-remote.

@Shougo
Copy link
Contributor

Shougo commented Apr 2, 2022

This does make me wonder if we should have a user visible message if we fail to start the server on an address given in --listen

I think --listen error message should be printed like --server error.

@groves groves force-pushed the remote_wait branch 2 times, most recently from 4eac586 to d4f507b Compare April 18, 2022 20:38
@groves
Copy link
Contributor Author

groves commented Apr 18, 2022

If I open the file by ~/src/neovim/build/bin/nvim --clean --server ~/.cache/nvim/server.pipe --remote-tab-wait ~/work/test2.ts, the file is opened both remote neovim and local neovim. Why?

This is a bug I introduced on this branch. Should be fixed now. Thanks for finding it!

Feel free to give that fix a try, but I should warn you that there are likely to be other issues. I still have a few things I need to address before bumping this PR from draft to ready.

@Shougo
Copy link
Contributor

Shougo commented Apr 19, 2022

Oh, thank you for the fix. I will try again later.

@Shougo
Copy link
Contributor

Shougo commented Apr 19, 2022

The feature is broken now...

$ ~/src/neovim/build/bin/nvim --clean --server ~/.cache/nvim/server.pipe --remote-tab-wait ~/work/test2.ts
Error executing lua: Vim:Error invoking 'nvim_exec_lua' on channel 3:
Error executing lua: [string "<nvim>"]:1: attempt to call field '_remote_server_edit' (a nil value)
stack traceback:
	[string "<nvim>"]:1: in main chunk
stack traceback:
	[C]: in function 'rpcrequest'
	vim/_editor.lua:742: in function <vim/_editor.lua:692>

@justinmk
Copy link
Member

justinmk commented Apr 23, 2022

I see keeping the variants as allowing ease of migration from Vim and to allow people to write scripts and instructions that work for both. This feels like part of the API that we keep in the same way that we keep a bunch of clunky bits of Vimscript and builtin functions.

Such a migration won't work except for trivial cases.

So I'm not seeing a strong reason to add anything more than --remote and --remote-wait. I'd even remove --remote-wait: make it so that --remote always waits (if someone wants "async" behavior that can be done with timer_start, vim.defer_fn, etc.)

Copy link
Member

@justinmk justinmk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before continuing with this, I'd like to see a table comparing the equivalent --remote foo command to the --remote-foo variant. For example:

Legacy (Vim) Minimal (Nvim, with always-blocking --remote)
--remote-send "foo" --remote "call nvim_input('foo')"
--remote-wait "foo" (blocking) --remote "foo"
--remote "foo" (nonblocking) --remote "call timer_start({-> foo})"
--remote-blah --remote "blah"

We should not add things just because Vim did it. There needs to be a good reason.

  • The Lua/Vimscript API has many more features today than it did when Vim added these things.
  • Providing examples of using Lua/Vimscript helps users understand and connect concepts.
    • Likewise, providing shallow "sugar" that does very little, actually harms users by increasing the burden of useless "knowledge" and decreasing the weight of composable concepts (the API).

@groves
Copy link
Contributor Author

groves commented Apr 26, 2022

I see keeping the variants as allowing ease of migration from Vim

Such a migration won't work except for trivial cases.
Providing examples of using Lua/Vimscript helps users understand and connect concepts. Likewise, providing shallow "sugar" that does very little, actually harms users by increasing the burden of useless "knowledge" and decreasing the weight of composable concepts (the API).

This makes a perfect sense to me if Neovim were a new editor. In that scenario, the best thing for users is to have a clear path from starting out to mastery of a simple, powerful set of customization tools.

In my understanding, Neovim's scenario has more nuance. At least as I've internalized Neovim's values, one of those is an easy transition for Vim users. That doesn't invalidate your point about helping users towards the best Neovim tools, but it adds a possibly conflicting desire to make existing Vim idioms and APIs work well.

Like with any big project, there are even more competing desires for Neovim's direction. To complete the picture as I understand it, this is what I roughly see as the Neovim desires in order of importance:

  • Ease of maintenance - does the addition make it easier to make future additions or at least not make future additions harder
  • Power and ease as an editor platform - does the addition allow users to do something new or do things with less fiddling
  • Drop-in Vim - does the addition let existing Vim users use Neovim without having to first learn Neovim before getting started

I'm not laying these out here to try and convince you that those are the right desires. I'm really trying to elicit what the desires are as envisaged by the Neovim maintainers. My having a better sense of the desires will help me make proposals that better track with those desires. If that's already on the web somewhere that I haven't seen, a link to that is plenty good.

What you're suggesting about comparing the Vim approach with a minimal Neovim approach makes sense to me as a way to decide if and how to proceed with this. I'd like to add in how well each of those items fit with the overall desires to make it a complete picture.

@clason
Copy link
Member

clason commented Apr 26, 2022

Not speaking for the rest of the team here, but I don't think

Drop-in Vim - does the addition let existing Vim users use Neovim without having to first learn Neovim before getting started

is really a strong project goal (anymore). What we do care about is backwards compatibility for the plugin ecosystem, so Neovim users can still use Vim plugins (even though they themselves are not expected to use legacy syntax, if better alternatives exist). So (only) if not supporting a particular Vim option would break a useful plugin, that would count as a valid argument in favor of including it. But that should be a concrete example.

@justinmk
Copy link
Member

justinmk commented Apr 26, 2022

At least as I've internalized Neovim's values, one of those is an easy transition for Vim users.

That doesn't apply to --remote, it's a very niche, one-off use-case. Typically used for setting up desktop shortcuts or small scripts. And those aren't going to be "drop in", no matter how much we aesthetically, superficially match the spelling of the old Vim --remote-x variants.

--remote should be restored inasmuch as it enables functionality that currently isn't possible out-of-the-box. I'm not seeing a strong use-case for one-to-one matching of Vim's --remote-x variants.

However, the table that I requested above, will be useful as:

  1. a reference for :help --remote
  2. it will drive thinking about ergonomic gaps. For example, if --remote is blocking, we might need Lua or Vimscript features that make it easy to "call without waiting". I've discussed previously something like a :async command, where :async foo executes :foo without waiting for the result.

@jcorbin

This comment was marked as off-topic.

@justinmk
Copy link
Member

justinmk commented Apr 27, 2022

There is a big use case for --remote-wait in my view:

Re-read my previous comments. I've no objection to --remote-wait functionality, in fact I am suggesting that --remote acts like --remote-wait. And the "non-blocking" nature of Vim's --remote can be achieved with things like timer_start() (we can make this more ergonomic by adding a command like :async or :job).

So to be clear, I'm suggesting that --remote is the only thing we need. We don't need the 17 other variants, they are shallow sugar.

@jcorbin
Copy link

jcorbin commented Apr 27, 2022

There is a big use case for --remote-wait in my view:

So to be clear, I'm suggesting that --remote is the only thing we need. We don't need the 17 other variants, they are shallow sugar.

Oh, I see! Thank you so much for clarifying, was confused after 0.7 shipped a non-blocking --remote, and hadn't quite read every comment fully, so I wasn't quite sure as an (albeit early adopter) user where this was all going, and wanted to provide some use case feedback :-)

@groves
Copy link
Contributor Author

groves commented Apr 28, 2022

At least as I've internalized Neovim's values, one of those is an easy transition for Vim users.

That doesn't apply to --remote, it's a very niche, one-off use-case. Typically used for setting up desktop shortcuts or small scripts. And those aren't going to be "drop in", no matter how much we aesthetically, superficially match the spelling of the old Vim --remote-x variants.

That was exactly my usage of --remote in Vim, I had a shell script that would let me edit a file in a running Vim from terminals I'd opened later that had nothing to do with that Vim. I bounced off of Neovim a couple years ago because there wasn't an obvious way to do that. Having --remote and something akin to --servername would have been a drop-in for my use case.

I don't understand the use of the tab and silent variants, and the send and expr variants are silly in the light of Neovim's API. I have no concern with dropping those, though as @jcorbin pointed out, the non wait versions of all of them shipped in 0.7. They add almost no code compared to the bulk of stuff needed for --remote-wait.

As it stands, there's a functional version of the wait variations on this branch now. I was planning on adding a couple more tests and finishing up the docs before marking it ready for review.

I'm happy to finish it up if someone wants to write up a design for where this should end up. I'm also perfectly happy if someone else wants to take it from here.

@Shougo
Copy link
Contributor

Shougo commented Apr 29, 2022

Before continuing with this, I'd like to see a table comparing the equivalent --remote foo command to the --remote-foo variant. For example:

If the same features are works, I don't need subcommands yeah.

But I want to use neovim remote feature as $EDITOR.

In $EDITOR, the feature must be {command} {filename}. Because {filename} is added by external command like git.

It works as expected in neovim-remote:
$EDITOR = 'nvr --remote-tab-wait-silent

It does not work in neovim remote features.

$EDITOR = 'nvim --clean --server ~/.cache/nvim/server.pipe --remote-send "tabnew | {filename}"'
filename cannot be added.

So I need the feature.

@justinmk
Copy link
Member

They add almost no code compared to the bulk of stuff needed for --remote-wait.

The main cost is the user interface (which includes documentation, and noise in --help).

Waiting to see the table requested in #17856 (review)

@Shougo
Copy link
Contributor

Shougo commented May 2, 2022

Waiting to see the table requested in #17856 (review)

Please see below table.

Legacy (Vim) Minimal (Nvim, with always-non-blocking --remote)
--remote "foo" --remote "foo"
--remote +cmd "foo" Not implemented yet
--remote-silent "foo" --remote-silent "foo"
--remote-silent +cmd "foo" Not implemented yet
--remote-wait "foo" (blocking) Not implemented yet
--remote-wait-silent "foo" (blocking) Not implemented yet
--remote-tab "foo" --remote-tab "foo"
--remote-tab-silent "foo" --remote-tab-silent "foo"
--remote-tab-wait (blocking) Not implemented yet
--remote-tab-wait-silent (blocking) Not implemented yet
--remote-send "foo" --remote-send "foo"

I think wait variants and +cmd support is only needed for neovim.
Other features are almost implemented.
So the PR is important.

@justinmk
Copy link
Member

justinmk commented May 4, 2022

@Shougo thanks. But that table doesn't give examples of what the interface would look like without the numerous --remote-x-y variants. I'm asking for this so that we have a concrete idea of why (or if) the variants are needed.

I've sketched out a proposed interface in #18414

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lua stdlib remote remote UI, --remote commands, p2p / peer-to-peer
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants