input.harbor TLS not exiting after first connection

[metaswirl]

Description

I can connect to input.harbor via TLS, when I disconnect the source, nothing shows in the log, and when I try and re-connect this appears:

2024/09/20 18:15:49 [harbor:4] Harbor.Make(T).Mount_taken

I have tried various clients including liquidsoap output.icecast tls, mixxx and butt, all have the same issues.

Steps to reproduce

transport = http.transport.tls(
certificate = "/opt/liquidsoap/cert/fullchain.pem",
key = "/opt/liquidsoap/cert/privkey.pem"
)

stream_live_tls = input.harbor("live-tls", port=9016, auth=auth, transport=transport, id="live-tls",
icy=false, icy_metadata_charset="UTF-8", metadata_charset="UTF-8", replay_metadata=true,
on_connect=live_connect_tls, on_disconnect=live_disconnect_tls, buffer=5.0, max=10.0)

Expected behavior

input.harbor TLS does not recognize when source has disconnected making it impossible to re-connect.

Liquidsoap version

Liquidsoap 2.2.5+dev
Copyright (c) 2003-2024 Savonet team
Liquidsoap is open-source software, released under GNU General Public License.
See <http://liquidsoap.info> for more information.

Liquidsoap build config

* Liquidsoap version  : 2.2.5+dev

 * Compilation options
   - Release build       : false
   - Git SHA             : (none)
   - OCaml version       : 5.2.0
   - OS type             : Unix
   - Libs versions       : angstrom=0.16.0 asn1-combinators=0.2.6 astring=0.8.5 base64=3.5.1 bigarray-compat=1.1.0 bigstringaf=0.10.0 bos=0.2.1 bytes=[distributed with OCaml 4.02 or above] ca-certs=v0.2.3 camlp-streams camomile.lib=2.0 cry=1.0.3 cstruct=6.2.0 ctypes=0.23.0 ctypes-foreign=0.23.0 ctypes.stubs=0.23.0 curl=0.9.2 domain-name=0.4.0 dtools=0.4.5 dune-build-info=3.16.0 dune-private-libs.dune-section=3.16.0 dune-site=3.16.0 dune-site.private=3.16.0 duppy=0.9.4 eqaf=0.9 eqaf.bigstring=0.9 eqaf.cstruct=0.9 ffmpeg-av=1.1.11 ffmpeg-avcodec=1.1.11 ffmpeg-avdevice=1.1.11 ffmpeg-avfilter=1.1.11 ffmpeg-avutil=1.1.11 ffmpeg-swresample=1.1.11 ffmpeg-swscale=1.1.11 fileutils=0.6.4 fmt=0.9.0 fpath=0.7.3 gen=1.1 gmap=0.3.0 hkdf=1.0.4 integers ipaddr=5.6.0 ladspa=0.2.2 liquidsoap-lang=2.2.5 liquidsoap-lang.console=2.2.5 liquidsoap_builtins liquidsoap_core liquidsoap_ffmpeg liquidsoap_ladspa liquidsoap_optionals liquidsoap_oss liquidsoap_runtime liquidsoap_srt liquidsoap_stereotool liquidsoap_taglib liquidsoap_tls logs=0.7.0 macaddr=5.6.0 magic-mime=1.3.1 menhirLib=20240715 metadata=0.3.0 mirage-crypto=0.11.3 mirage-crypto-ec=0.11.3 mirage-crypto-pk=0.11.3 mirage-crypto-rng=0.11.3 mirage-crypto-rng.unix=0.11.3 mm=0.8.5 mm.audio=0.8.5 mm.base=0.8.5 mm.image=0.8.5 mm.midi=0.8.5 mm.video=0.8.5 pbkdf pcre=7.5.0 posix-base=2.0.2 posix-socket=2.0.2 posix-socket.constants=2.0.2 posix-socket.stubs=2.0.2 posix-socket.types=2.0.2 ppx_string.runtime=v0.17.0 ptime=1.1.0 ptime.clock.os=1.1.0 re=1.12.0 rresult=0.7.0 sedlex=3.2 seq=[distributed with OCaml 4.07 or above] sexplib0=v0.17.0 srt=0.3.0 srt.constants=0.3.0 srt.stubs=0.3.0 srt.stubs.locked=0.3.0 srt.types=0.3.0 stdlib-shims=0.3.0 stereotool str=5.2.0 stringext=1.6.0 taglib=0.3.10 threads=5.2.0 tls=0.17.3 unix=5.2.0 uri=4.4.0 x509=0.16.5 zarith=1.14
   - architecture        : amd64
   - host                : x86_64-pc-linux-gnu
   - target              : x86_64-pc-linux-gnu
   - system              : linux
   - ocamlopt_cflags     : -O2 -fno-strict-aliasing -fwrapv -pthread -fPIC -pthread
   - native_c_compiler   : gcc -O2 -fno-strict-aliasing -fwrapv -pthread -fPIC -pthread -D_FILE_OFFSET_BITS=64
   - native_c_libraries  : -lm -ldl -lpthread

 * Configured paths
   - mode              : default
   - standard library  : (set by dune-site)
   - scripted binaries : (set by dune-site)
   - rundir            : (set by dune-site)
   - logdir            : (set by dune-site)
   - camomile files    : (set by dune-site)

 * Supported input formats
   - MP3               : no (requires mad)
   - AAC               : no (requires faad)
   - Ffmpeg            : yes
   - Flac (native)     : no (requires flac)
   - Flac (ogg)        : no (requires ogg)
   - Opus              : no (requires opus)
   - Speex             : no (requires speex)
   - Theora            : no (requires theora)
   - Vorbis            : no (requires vorbis)

 * Supported output formats
   - FDK-AAC           : no (requires fdkaac)
   - Ffmpeg            : yes
   - MP3               : no (requires lame)
   - MP3 (fixed-point) : no (requires shine)
   - Flac (native)     : no (requires flac)
   - Flac (ogg)        : no (requires ogg)
   - Opus              : no (requires opus)
   - Speex             : no (requires speex)
   - Theora            : no (requires theora)
   - Vorbis            : no (requires vorbis)

 * Tags
   - Taglib (ID3 tags) : yes
   - Vorbis            : no (requires vorbis)

 * Input / output
   - ALSA              : no (requires alsa)
   - AO                : no (requires ao)
   - FFmpeg            : yes
   - GStreamer         : no (requires gstreamer)
   - JACK              : no (requires bjack)
   - OSS               : yes
   - Portaudio         : no (requires portaudio)
   - Pulseaudio        : no (requires pulseaudio)
   - SRT               : yes

 * Audio manipulation
   - FFmpeg            : yes
   - LADSPA            : yes
   - Lilv              : no (requires lilv)
   - Samplerate        : no (requires samplerate)
   - SoundTouch        : no (requires soundtouch)
   - StereoTool        : yes

 * Video manipulation
   - camlimages        : no (requires camlimages)
   - FFmpeg            : yes
   - frei0r            : no (requires frei0r)
   - ImageLib          : no (requires imagelib)
   - SDL               : no (requires tsdl-image & tsdl-ttf)

 * MIDI manipulation
   - DSSI              : no (requires dssi)

 * Visualization
   - GD                : no (requires gd)
   - Graphics          : no (requires graphics)
   - SDL               : no (requires tsdl-image & tsdl-ttf)

 * Additional libraries
   - FFmpeg filters    : yes
   - FFmpeg devices    : yes
   - inotify           : no (requires inotify)
   - irc               : no (requires irc-client-unix)
   - jemalloc          : no (requires jemalloc)
   - lastfm            : no (requires lastfm)
   - lo                : no (requires lo)
   - memtrace          : no (requires memtrace)
   - mem_usage         : no (requires mem_usage)
   - osc               : no (requires osc-unix)
   - ssl               : no (requires ssl)
   - tls               : yes
   - posix-time2       : no (requires posix)
   - windows service   : no (requires winsvc)
   - YAML support      : no (requires yaml)
   - XML playlists     : no (requires xmlplaylist)

 * Monitoring
   - Prometheus        : no (requires prometheus)

Installation method

From OPAM

Additional Info

No response

[vitoyucepi]

Hi @metaswirl,

I'm able to reproduce this problem. Also, I've found another problem when trying to connect to it.

So here's my reproduction script

1. compose.yaml

   services:
     ls22:
       image: savonet/liquidsoap:v2.2.5
       command: /tmp/test/main.liq
       volumes:
         - ./:/tmp/test/
       ports:
         - 8000:8000

2. main.liq

   settings.log.level := 5
   
   transport = http.transport.tls(
     certificate = "/tmp/test/cert.pem",
     key = "/tmp/test/key.pem",
   )
   
   live = input.harbor(
     "live",
     port=8000,
     buffer=3.,
     transport=transport,
   )
   
   output.dummy(live, fallible=true)

3. openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -sha256 -days 365 -subj '/CN=localhost' -nodes;
   chmod 0644 key.pem;
   docker compose up;

4. ffmpeg -re -f lavfi -i "sine=frequency=200:duration=2000" -b:a 320k -c:a mp3 -f mp3 -tls 1 icecast://source:hackme@localhost:8000/live

But that's not working properly, the stream doesn't even start. So I've configured reverse proxy and got the error.

Log

2024/09/24 01:45:53 [input.harbor:5] Not ready: need more buffering (0/132300).
2024/09/24 01:46:13 [harbor:4] New client on port 8000: 172.18.0.1
2024/09/24 01:46:13 [harbor:4] Method: PUT, uri: /live, protocol: HTTP/1.1
2024/09/24 01:46:13 [harbor:4] Header: User-Agent, value: Lavf/61.1.100.
2024/09/24 01:46:13 [harbor:4] Header: Accept, value: */*.
2024/09/24 01:46:13 [harbor:4] Header: Expect, value: 100-continue.
2024/09/24 01:46:13 [harbor:4] Header: Connection, value: close.
2024/09/24 01:46:13 [harbor:4] Header: Host, value: 192.168.0.1:8080.
2024/09/24 01:46:13 [harbor:4] Header: Content-Type, value: audio/mpeg.
2024/09/24 01:46:13 [harbor:4] Header: Icy-MetaData, value: 1.
2024/09/24 01:46:13 [harbor:4] Header: Ice-Public, value: 0.
2024/09/24 01:46:13 [harbor:4] Client logged in.
2024/09/24 01:46:13 [harbor:4] PUT (source) request on /live.
2024/09/24 01:46:13 [harbor:4] Adding source on mountpoint "/live" with type "audio/mpeg".
2024/09/24 01:46:13 [harbor:5] Relaying HTTP/1.1.
2024/09/24 01:46:13 [decoder:4] Available decoders:
2024/09/24 01:46:13 [decoder:4] ffmpeg (priority: 10)
2024/09/24 01:46:13 [decoder:4] mad (priority: 1)
2024/09/24 01:46:13 [decoder:4] Selected decoder ffmpeg for mime-type audio/mpeg with expected content {audio=pcm(stereo)}
2024/09/24 01:46:13 [threads:4] Created thread "harbor source feeding" (2 total).
2024/09/24 01:46:13 [input.harbor:3] Decoding...
2024/09/24 01:46:14 [input.harbor:5] Not ready: need more buffering (57600/132300).
...
2024/09/24 01:46:15 [input.harbor:5] Not ready: need more buffering (123264/132300).
2024/09/24 01:46:15 [input.harbor:5] Not ready: need more buffering (125568/132300).
2024/09/24 01:46:15 [input.harbor:5] Not ready: need more buffering (127872/132300).
2024/09/24 01:46:15 [input.harbor:5] Not ready: need more buffering (130176/132300).
2024/09/24 01:46:15 [input.harbor:5] Not ready: need more buffering (131328/132300).
2024/09/24 01:46:15 [input.harbor:5] Got metadata at position 0: calling handlers...
2024/09/24 01:46:27 [input.harbor:4] End of track.
2024/09/24 01:46:27 [input.harbor:4] Buffer emptied, buffering needed.
2024/09/24 01:46:27 [input.harbor:5] Not ready: need more buffering (0/132300).
2024/09/24 01:46:27 [dummy:3] Source failed (no more tracks) stopping output...
2024/09/24 01:46:42 [harbor:4] New client on port 8000: 172.18.0.1
2024/09/24 01:46:42 [harbor:4] Method: PUT, uri: /live, protocol: HTTP/1.1
2024/09/24 01:46:42 [harbor:4] Header: User-Agent, value: Lavf/61.1.100.
2024/09/24 01:46:42 [harbor:4] Header: Accept, value: */*.
2024/09/24 01:46:42 [harbor:4] Header: Expect, value: 100-continue.
2024/09/24 01:46:42 [harbor:4] Header: Connection, value: close.
2024/09/24 01:46:42 [harbor:4] Header: Host, value: 192.168.0.1:8080.
2024/09/24 01:46:42 [harbor:4] Header: Content-Type, value: audio/mpeg.
2024/09/24 01:46:42 [harbor:4] Header: Icy-MetaData, value: 1.
2024/09/24 01:46:42 [harbor:4] Header: Ice-Public, value: 0.
2024/09/24 01:46:42 [harbor:4] Client logged in.
2024/09/24 01:46:42 [harbor:4] PUT (source) request on /live.
2024/09/24 01:46:42 [harbor:4] Adding source on mountpoint "/live" with type "audio/mpeg".
2024/09/24 01:46:42 [harbor:5] Relaying HTTP/1.1.
2024/09/24 01:46:42 [harbor:4] Harbor.Make(T).Mount_taken

If you need a quick fix, then set up a reverse proxy in front of liquidsoap and configure it to strip encryption in TCP mode. #3659 (comment)

[toots]

Thanks for the report, I will have a pass at it.

Is it also happening in 2.3.0-rc1 or main?

Also worth noting we do have SSL as well. A reverse proxy is never a bad idea if that suits your infrastructure.

[metaswirl]

Thank you Toots!

http.transport.ssl works without any issue! http.transport.tls is buggy.

[vitoyucepi]

Both http.transport.ssl and http.transport.tls have their own quirks.
Ref: #3574