Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

move slider guis to meshcat #145

Closed
3 of 5 tasks
RussTedrake opened this issue Aug 11, 2021 · 2 comments
Closed
3 of 5 tasks

move slider guis to meshcat #145

RussTedrake opened this issue Aug 11, 2021 · 2 comments
Assignees

Comments

@RussTedrake
Copy link
Owner

RussTedrake commented Aug 11, 2021

Related to #144 . This is a possible work-around for lack of interaction in deepnote (and spotty support for ipywidgets more generally in the jupyter-like cloud notebook ecosystem).

  • Can use set_control, e.g. via this.set_control("test", this.mycallback, 50, 0, 100, 0.1); in the javascript to add a slider to the ui.
  • set_control messages are also parsed in the javascript, it's easy enough to pass them through the zmqserver.
  • Demonstrate that I can send something back to the zmqserver from the callback.
    Update: this works: this.set_control("test", '(value) => this.connection.send("slider=" + value)', 50, 0, 100, 0.1);
  • zmqserver should forward to other websocket clients that are not the sender (and store the values?). perhaps I need update_control for this? Or a variant of set_control that takes only the value.
  • Design protocol for sending UI updates from zmqserver => python. A few ideas?
    1) reuse the existing zmq connection. zmqserver already sends little ack messages back when it receives. I would have to be very careful to not interrupt those handshakes.
    2) open a second zmq connection, specifically for the ui callback traffic
    3) python could connect to zmqserver as a websocket, and listen only to the update_control messages.

Note that the python client receiving the ui updates need not be the original meschat.Visualizer() object. I'd be happy to have a separate meshcat.UISubscriber() type object instead. It might be cleaner.

@RussTedrake RussTedrake self-assigned this Aug 11, 2021
@RussTedrake
Copy link
Owner Author

RussTedrake commented Aug 11, 2021

Another alternative has always been to avoid the zmqserver. I have a working version of meshcat visualizer in python that relies only on ipython kernel comms as the transport. That works great on local machines, but not on colab / deepnote (both of which use a custom kernel implementation).

But perhaps I should consider a visualizer that hosts the websocket connection directly (as zmqserver does now). This would be instead of kernel comms, would have the advantage that it would work outside of jupyter, but would still cut out the zmq layer. This feels like perhaps the best solution. There are still many decisions to be made (do I support multiple clients? do I store the scene tree on the server and allow a new client to get the history, or act more like drake visualizer and require the viewer to connect before any important message is sent?).

Here's a burn-down list for this version

  • Spike test running zmqserver in the main jupyter notebook (not as subprocess). Verify I can still connect.
  • Spike test the WebSocketVisualizer case in python, taking the easy answer to all of the interesting questions (single client, no memory).
  • WebSocketVisualizer should get instantiated once (at the top of the notebook), and we pass a raw pointer or owned Pointer to the meshcat visualizer system. Perhaps it can even be a direct stand-in for the current visualizer. This is important so that users don't have to re-open their visualizer for each simulation.
  • Spike test a C++ version of the WebSocketVisualizer.

@RussTedrake
Copy link
Owner Author

I've decided to run with the Meshcat in C++ solution for this. Closing this since I've moved the discussion to RobotLocomotion/drake#13038

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant