wit-component: IndexMap: key not found

[dragunOnCode]

Hi, everyone! I'm using wasi-sdk-21 and wasm-tools 1.0.55 to build my C project which calls Preview 2 wasi-sockets function. I write this code:

#include <stdio.h>
#include <stdbool.h>
#include <wasi/preview2.h>

int main() {
    tcp_create_socket_own_tcp_socket_t sock;
    tcp_create_socket_error_code_t create_err;
    bool ret = tcp_create_socket_create_tcp_socket(NETWORK_IP_ADDRESS_FAMILY_IPV4, &sock, &create_err);

    if (ret) {
        printf("created socket\n");
    }
    return 0;
}

Then I try to use the clang from wasi-sdk and wasm-tools to create wasm module. But I get the following error from wasm-tools component new:

[root@localhost multifunc]# $CC --target=wasm32-wasi-preview2 cclient.c -o cclient.wasm
[root@localhost multifunc]# wasm-tools component new --adapt ~/.cargo/bin/wasi_snapshot_preview1.command.wasm cclient.wasm -o cclient.wasm 
thread 'main' panicked at /root/.cargo/registry/src/gitpro.ttaallkk.top-1ecc6299db9ec823/wit-component-0.19.1/src/encoding.rs:1815:50:
IndexMap: key not found
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

I enable the debugging log, which shows the following information:

......
[2024-01-17T11:57:11Z DEBUG] moving interface Some("network")
[2024-01-17T11:57:11Z DEBUG] moving interface Some("instance-network")
[2024-01-17T11:57:11Z DEBUG] moving interface Some("udp")
[2024-01-17T11:57:11Z DEBUG] moving interface Some("udp-create-socket")
[2024-01-17T11:57:11Z DEBUG] moving interface Some("tcp")
[2024-01-17T11:57:11Z DEBUG] moving interface Some("tcp-create-socket")
[2024-01-17T11:57:11Z DEBUG] moving interface Some("ip-name-lookup")
[2024-01-17T11:57:11Z DEBUG] moving interface Some("random")
[2024-01-17T11:57:11Z DEBUG] moving interface Some("insecure")
[2024-01-17T11:57:11Z DEBUG] moving interface Some("insecure-seed")
[2024-01-17T11:57:11Z DEBUG] moving world root
[2024-01-17T11:57:11Z DEBUG] moving world imports
[2024-01-17T11:57:11Z DEBUG] moving world command
[2024-01-17T11:57:11Z DEBUG] shim 0 is import `wasi:random/random@0.2.0-rc-2023-11-10` lowering 0 `get-random-bytes`
thread 'main' panicked at /root/.cargo/registry/src/gitpro.ttaallkk.top-1ecc6299db9ec823/wit-component-0.19.1/src/encoding.rs:1815:50:
IndexMap: key not found
stack backtrace:
   0: rust_begin_unwind
             at /rustc/5ac4c8a63ee305742071ac6dd11817f7c24adce2/library/std/src/panicking.rs:645:5
   1: core::panicking::panic_fmt
             at /rustc/5ac4c8a63ee305742071ac6dd11817f7c24adce2/library/core/src/panicking.rs:72:14
   2: core::panicking::panic_display
             at /rustc/5ac4c8a63ee305742071ac6dd11817f7c24adce2/library/core/src/panicking.rs:196:5
   3: core::panicking::panic_str
             at /rustc/5ac4c8a63ee305742071ac6dd11817f7c24adce2/library/core/src/panicking.rs:171:5
   4: core::option::expect_failed
             at /rustc/5ac4c8a63ee305742071ac6dd11817f7c24adce2/library/core/src/option.rs:1980:5
   5: wit_component::encoding::Shims::append_indirect
   6: wit_component::encoding::EncodingState::encode_core_instantiation
   7: wit_component::encoding::ComponentEncoder::encode
   8: wasm_tools::component::Opts::run
   9: wasm_tools::main

I'm not sure if I'm using wasm-tools in the right way. How can I create a wasm from C calling the Preview 2 function?

[pchickey]

Thanks for the bug report. Can you please attach your cclient.wasm, as well as your copy of wasi_snapshot_preview1.command.wasm, so we can reproduce and debug?

[pchickey]

cc @dicej

[dicej]

@dragunOnCode Preview 2 support has only been added to wasi-sdk after the v21 release, so I assume you're using the main branch of wasi-sdk. Anyway, I just reproduced this using the main branch, plus wasm-tools 1.0.55 and https://github.com/bytecodealliance/wasmtime/releases/download/v16.0.0/wasi_snapshot_preview1.command.wasm . I'll investigate.

[dicej]

Okay, I think the issue here is that wasi-sdk main doesn't fully support Preview 2 yet -- it's a work-in-progress. The maintainers of wasi-sdk have asked that I break WebAssembly/wasi-libc#449 up into a series of small PRs, which is what I've been doing. Only a handful of those PRs have landed so far.

In this case, the problem is that the preview2_component_type.o is not included in libc.a, and thus the final binary does not include a component type custom section for wit-component to use. Arguably, wit-component should do a better job of error reporting here, so it's a bug in that sense, but the underlying issue is that Preview 2 support in wasi-sdk is not yet complete.

In the meantime, feel free to experiment with my sockets branch, which is "complete" in that it can handle various real-world tests. Note that it does not (yet) expose preview2.h as a public header, though, so your code won't work as-is -- you'd need to use the POSIX-level APIs or else use wit-bindgen c to generate the header file yourself.

Sorry about the incomplete state of things right now. Getting stuff upstreamed into wasi-sdk is a slow process, unfortunately.

[alexcrichton]

Thanks for the report! I agree with @dicej's conclusion and I've opened #1378 to fix the panic here and return a proper error instead.