FileUpload is not working with kernel gateway

[oztoprakmustafa]

Description

I am reporting an issue with the FileUpload function in ipywidgets when used in a notebook.When attempting to upload a file using the FileUpload widget in juypterlab, two main problems are observed:

1. The kernel connection is lost upon file upload, which prevents further cell executions.
2. The uploaded file is not recognized by the FileUpload widget (i.e., it returns an empty dict).

Reproduce

Expected behavior

I expected the FileUpload widget to successfully load the selected file and make it accessible for further processing, without affecting the kernel connection.

Context

• ipywidgets version : tested with both 7.0.6 and 8.0.4
• Operating System and version:
• Browser and version: Firefox
• python=3.8
• jupyterlab==3.6.1

Troubleshoot Output

Paste the output from running `jupyter troubleshoot` from the command line here.
You may want to sanitize the paths in the output.

Command Line Output

Paste the output from your command line running `jupyter lab` (or `jupyter notebook` if you use notebook) here, use `--debug` if possible.

Browser Output

Paste the output from your browser Javascript console here.

If using JupyterLab

• JupyterLab version:3.6.1

Installed Labextensions

Paste the output from your command line running `jupyter labextension list`.

[gerkensm]

Not sure if this is related, but I see similar behaviour when running the following minimal example in VSCode with ipywidgets version: 8.1.0:

from ipywidgets import widgets
uploader = widgets.FileUpload()
display(uploader)

After uploading any file, the server becomes unresponsive and I cannot execute any other cells. My only option is to restart the server.

[gerkensm]

Never mind, this might be due to other issues like microsoft/vscode-jupyter#8084 or microsoft/vscode-jupyter#8174

[edwardps]

When use the kernel gateway the upload widget cannot work properly.
I would like to add more details for reproducing this issue.

set up env

conda create --name myenv310 python=3.10
conda activate myenv310


pip3 install jupyterlab==3.6.1
pip3 install jupyter-kernel-gateway
pip3 install ipywidgets==8.1.1


# start kernel gateway server
jupyter kernelgateway --KernelGatewayApp.api=kernel_gateway.jupyter_websocket --JupyterWebsocketPersonality.list_kernels=True --port=10100

# start jupyter lab in the same conda env but a 2nd terminal 
jupyter lab --LabApp.token='' --gateway-url=http://127.0.0.1:10100

Issue

After the file is chosen, the uploader.value is still empty.

If simply remove "--gateway-url=http://127.0.0.1:10100" when start Jupyter to use local kernel, the upload widget works.
Seems this is caused by the remote kernel. See below screenshot.

[edwardps]

Thanks Piyush for helping move this forward and providing following suggestions.

I brought this up in the Jupyter server meeting today. Here is some more info to investigate this.
1. ipywidgets uses standard kernel communication channels (comms)
2. File upload widget sends binary messages from the frontend to the backend. Is the kernel gateway passing along binary messages?
3. Check if ImageWidget works, which sends binary messages from the kernel to the frontend. That at least checks that one way works. If ImageWidget works, check that binary kernel messages from the frontend to the backend work too.

I verified that it ImageWidget does not work with kernel gateway.
Following piece of code works for local mode but hangs for ever in remote kernel case.

import ipywidgets as widgets

file = open("test.png", "rb")
image = file.read()
widgets.Image(
    value=image,
    format='png',
    width=300,
    height=400,
)

Is the kernel gateway passing along binary messages?

According my troubleshooting, the event handling loop does not handle the binary message. It does call ipykernel event handling loop(process_one and dispatch_shell) for the binary message.

[ibdafna]

We discussed this issue in the 2023-10-10 ipywidgets dev meeting. I'll take a look to see where this could be failing.

[ibdafna]

I wanted to provide an update here as I looked into this issue today. I still don't have a clear picture, but I'm seeing a couple of issues here:

1. In the case of widgets.Image, the failure happens in this line due to what appears to be a corrupt message payload:

[E 2023-10-29 22:13:06.554 ServerApp] Exception in callback functools.partial(<bound method IOLoop._discard_future_result of <tornado.platform.asyncio.AsyncIOMainLoop object at 0x7f6044260b50>>, <Task finished name='Task-505' coro=<GatewayWebSocketConnection._read_messages() done, defined at /home/idafna/micromamba/envs/ipywidgets_support/lib/python3.10/site-packages/jupyter_server/gateway/connections.py:81> exception=UnicodeDecodeError('utf-8', b'\x00\x00\x00\x02\x00\x00\x00\x0c\x00\x00\x04`{"header": {"msg_id": "22935daa-b8f64abf74ba0dd21a134ff1_12996_69", "msg_type": "comm_open", "username": "username", "session": "22935daa-b8f64abf74ba0dd21a134ff1", "date": "2023-10-30T05:13:06.543415Z", "version": "5.3"}, "msg_id": "22935daa-b8f64abf74ba0dd21a134ff1_12996_69", "msg_type": "comm_open", "parent_header": {"date": "2023-10-30T05:13:06.533000Z", "msg_id": "e920d328-b000-4514-93c7-fc78d5394b9f", "msg_type": "execute_request", "session": "d2b77a37-29dc-4183-bca2-205713bfbe39", "username": "", "version": "5.2"}, "metadata": {"version": "2.1.0"}, "content": {"data": {"state": {"_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ImageModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ImageView", "format": "png", "height": "400", "layout": "IPY_MODEL_a9790d1b6f714581a7af04cbb9e47d63", "width": "300"}, "buffer_paths": [["value"]]}, "comm_id": "f52374b895c24751be0ef2c27e30a2de", "target_name": "jupyter.widget", "target_module": null}, "channel": "iopub"}\x89PNG\r\n\x1a\n\x00\x00\x00\rIHDR\x00\x00\x01\x7f\x00\x00\x00\xdb\x02\x03\x00\x00\x00\xf8\xd0y\xb8\x00\x00\x00\x0cPLTEDDD\xff\xff\xffDDD\xff\xdb\x00\xfb\xe6g\x93\x00\x00\x00\x02tRNS\x00\x00v\x93\xcd8\x00\x00\x00\x9aIDATx\xda\xed\xdc\xb1\r\x80 \x10@Q\n\x074F\xf7\xb1p\x1a\xa6p\x1e\x97\xc0J;(\x8c\x97\xa0y\xbf%\xe45GG.-Wc\x8ahH70\xe5\x886@_@c\x18\xf2Z?+{\xe3\x1e\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00>\x02<\xac\x1c\xc1\xdf\xe0\x00/\x02\xde\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xf4\x0bX\x98\xf7{ |qdL\xf3\t \xdc\x06\x9c%\xb1\xcb\xf7\x00\x00\x00\x00IEND\xaeB`\x82', 1120, 1121, 'invalid start byte')>)
    Traceback (most recent call last):
      File "/home/idafna/micromamba/envs/ipywidgets_support/lib/python3.10/site-packages/tornado/ioloop.py", line 738, in _run_callback
        ret = callback()
      File "/home/idafna/micromamba/envs/ipywidgets_support/lib/python3.10/site-packages/tornado/ioloop.py", line 762, in _discard_future_result
        future.result()
      File "/home/idafna/micromamba/envs/ipywidgets_support/lib/python3.10/site-packages/jupyter_server/gateway/connections.py", line 103, in _read_messages
        message = message.decode("utf8")
    UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 1120: invalid start byte

2. In the case of widget.FileUpload, the message is None.

I'll update here as I finish this investigation tomorrow.

[edwardps]

Hi @ibdafna,

Happy new year. I am trying to circle back and see if there is any progress. Thanks

[werpachowski-msft]

Not working for me either with ipywidgets 8.0.6.