CI: Coverage flake //test/integration:multiplexed_integration_test

[asraa]

Title: One line description
Coverage build flake with test failure, see
https://dev.azure.com/cncf/envoy/_build/results?buildId=93566&view=logs&jobId=bbe4b42d-86e6-5e9c-8a0b-fea01d818a24&j=bbe4b42d-86e6-5e9c-8a0b-fea01d818a24&t=e00c5a13-c6dc-5e9a-6104-69976170e881

Description:

Describe the desired behavior, what scenario it enables and how it
would be used.

[optional Relevant Links:]

Any extra documentation required to understand the issue.

Failure in 2021-11-03T04:21:13.0695512Z [ RUN ] IpVersions/Http2IntegrationTest.InvalidTrailers/IPv4_Http3Downstream_HttpUpstream
Trace logs show:

2021-11-03T04:21:13.1903428Z [2021-11-03 04:18:25.589][30956][trace][main] [source/common/event/dispatcher_impl.cc:112] clearing deferred deletion list (size=1)
2021-11-03T04:21:13.1906593Z [2021-11-03 04:18:25.590][30956][trace][misc] [source/common/network/utility.cc:561] sendmsg bytes 59
2021-11-03T04:21:13.1909521Z [2021-11-03 04:18:25.590][30956][debug][client] [source/common/http/codec_client.cc:106] [C13534158527669248406] disconnect. resetting 0 pending requests
2021-11-03T04:21:13.1912673Z [2021-11-03 04:18:25.594][11222][trace][udp] [source/common/network/udp_listener_impl.cc:54] Listener at 127.0.0.1:43831 :socket event: 3
2021-11-03T04:21:13.1914914Z [2021-11-03 04:18:25.594][11222][trace][udp] [source/common/network/udp_listener_impl.cc:66] Listener at 127.0.0.1:43831 :handleReadCallback
2021-11-03T04:21:13.1917556Z [2021-11-03 04:18:25.594][11222][trace][misc] [source/common/network/utility.cc:661] starting recvmmsg with packets=16 max=1500
2021-11-03T04:21:13.1920110Z [2021-11-03 04:18:25.595][11222][trace][misc] [source/common/network/utility.cc:668] recvmmsg read 2 packets
2021-11-03T04:21:13.1922429Z [2021-11-03 04:18:25.595][11222][debug][misc] [source/common/network/utility.cc:678] Receive a packet with 35 bytes from 127.0.0.1:53931
2021-11-03T04:21:13.1924807Z [2021-11-03 04:18:25.596][11222][debug][misc] [source/common/network/utility.cc:678] Receive a packet with 59 bytes from 127.0.0.1:53931
2021-11-03T04:21:13.1927079Z [2021-11-03 04:18:25.600][11222][trace][misc] [source/common/network/utility.cc:661] starting recvmmsg with packets=16 max=1500
2021-11-03T04:21:13.1929388Z [2021-11-03 04:18:25.601][11222][trace][udp] [source/common/network/udp_listener_impl.cc:98] Listener at 127.0.0.1:43831 :handleWriteCallback
2021-11-03T04:21:13.1931903Z [2021-11-03 04:18:25.611][30956][critical][assert] [test/integration/autonomous_upstream.cc:35] assert failure: complete(). Details: Found that end_stream is not true
2021-11-03T04:21:13.1934302Z [2021-11-03 04:18:25.611][30956][critical][backtrace] [./source/server/backtrace.h:104] Caught Aborted, suspect faulting address 0x3e9000078ec
2021-11-03T04:21:13.1936214Z #0 Envoy::SignalAction::sigHandler() [0x7f5568949400]
2021-11-03T04:21:13.1937409Z #1 __restore_rt [0x7f555622d980]
2021-11-03T04:21:13.1938625Z #2 Envoy::AutonomousStream::~AutonomousStream() [0x7f55ff0ad2e0]
2021-11-03T04:21:13.1939917Z #3 Envoy::AutonomousStream::~AutonomousStream() [0x7f55ff0ad349]
2021-11-03T04:21:13.1941025Z #4 [0xcdf75e]
2021-11-03T04:21:13.1941933Z #5 [0xcdf6b0]
2021-11-03T04:21:13.1942796Z #6 [0xbdcb9b]
2021-11-03T04:21:13.1943959Z #7 [0xce80bb]
2021-11-03T04:21:13.1945019Z #8 std::__1::allocator_traits<>::destroy<>() [0xce807f]
2021-11-03T04:21:13.1946056Z #9 [0xce802c]
2021-11-03T04:21:13.1946923Z #10 [0xce7f2a]
2021-11-03T04:21:13.1948002Z #11 std::__1::__vector_base<>::~__vector_base() [0xce7d0b]
2021-11-03T04:21:13.1949355Z #12 [0x7f55ff1150d4]
2021-11-03T04:21:13.1950610Z #13 Envoy::AutonomousHttpConnection::~AutonomousHttpConnection() [0x7f55ff115060]
2021-11-03T04:21:13.1951982Z #14 Envoy::AutonomousHttpConnection::~AutonomousHttpConnection() [0x7f55ff0bc060]
2021-11-03T04:21:13.1953505Z #15 Envoy::AutonomousHttpConnection::~AutonomousHttpConnection() [0x7f55ff0bc089]
2021-11-03T04:21:13.1954750Z #16 [0x7f55ff15ff9e]
2021-11-03T04:21:13.1955704Z #17 [0x7f55ff15fef0]
2021-11-03T04:21:13.1956619Z #18 [0x7f55ff0b995b]
2021-11-03T04:21:13.1957604Z #19 [0x7f55ff1597db]
2021-11-03T04:21:13.1958731Z #20 std::__1::allocator_traits<>::destroy<>() [0x7f55ff15979f]
2021-11-03T04:21:13.1959894Z #21 [0x7f55ff15974c]
2021-11-03T04:21:13.1960854Z #22 [0x7f55ff15964a]
2021-11-03T04:21:13.1961826Z #23 [0x7f55ff0b944b]
2021-11-03T04:21:13.1963081Z #24 Envoy::AutonomousUpstream::~AutonomousUpstream() [0x7f55ff0aeb09]
2021-11-03T04:21:13.1964491Z #25 Envoy::AutonomousUpstream::~AutonomousUpstream() [0x7f55ff0aec59]
2021-11-03T04:21:13.1965687Z #26 [0xc3a20e]
2021-11-03T04:21:13.1966653Z #27 [0xc3a1c0]
2021-11-03T04:21:13.1967617Z #28 [0xc3876b]
2021-11-03T04:21:13.1968382Z #29 [0xc39efb]
2021-11-03T04:21:13.1969497Z #30 std::__1::allocator_traits<>::destroy<>() [0xc39ebf]
2021-11-03T04:21:13.1970644Z #31 [0x7f56002f355c]
2021-11-03T04:21:13.1971753Z #32 [0x7f56002f345a]
2021-11-03T04:21:13.1972899Z #33 std::__1::__vector_base<>::~__vector_base() [0x7f56002f323b]
2021-11-03T04:21:13.1974069Z #34 [0x7f56002f2e34]
2021-11-03T04:21:13.1975267Z #35 Envoy::BaseIntegrationTest::~BaseIntegrationTest() [0x7f56002be300]
2021-11-03T04:21:13.1977246Z #36 Envoy::HttpIntegrationTest::~HttpIntegrationTest() [0x7f5600277fd7]
2021-11-03T04:21:13.1978397Z #37 [0xc3827d]
2021-11-03T04:21:13.1979264Z #38 [0xc38045]
2021-11-03T04:21:13.1980179Z #39 [0xbe60f5]
2021-11-03T04:21:13.1981076Z #40 [0xbe6119]
2021-11-03T04:21:13.1982005Z #41 testing::Test::DeleteSelf_() [0x7f55572a27aa]
2021-11-03T04:21:13.1983700Z #42 testing::internal::HandleSehExceptionsInMethodIfSupported<>() [0x7f55572bb3bd]
2021-11-03T04:21:13.1985627Z #43 testing::internal::HandleExceptionsInMethodIfSupported<>() [0x7f55572a1d9e]
2021-11-03T04:21:13.1987036Z #44 testing::TestInfo::Run() [0x7f5557289e3b]
2021-11-03T04:21:13.1988271Z #45 testing::TestSuite::Run() [0x7f555728a621]
2021-11-03T04:21:13.1989589Z #46 testing::internal::UnitTestImpl::RunAllTests() [0x7f555729888e]
2021-11-03T04:21:13.1991076Z #47 testing::internal::HandleSehExceptionsInMethodIfSupported<>() [0x7f55572c1fed]
2021-11-03T04:21:13.1992629Z #48 testing::internal::HandleExceptionsInMethodIfSupported<>() [0x7f55572a4dae]
2021-11-03T04:21:13.1994068Z #49 testing::UnitTest::Run() [0x7f55572982d2]
2021-11-03T04:21:13.1995268Z #50 RUN_ALL_TESTS() [0x7f557b93c543]
2021-11-03T04:21:13.1996495Z #51 Envoy::TestRunner::RunTests() [0x7f557b93b830]
2021-11-03T04:21:13.1997759Z #52 main [0x7f557b8d5a70]
2021-11-03T04:21:13.1998708Z #53 __libc_start_main [0x7f5555e4bbf7]
2021-11-03T04:21:13.2000242Z external/bazel_tools/tools/test/collect_coverage.sh: line 170: 30956 Aborted                 (core dumped) "$@"

[asraa]

Some updates:

I can repro with docker, but not running bazel coverage with the flags used in test/run_envoy_bazel_coverage.sh

[rojkov]

Still //test/integration:multiplexed_integration_test is flaking in CI for coverage and for compile_time_options builds (1 and 2)

[rojkov]

The actual error seems to be different this time. Opened a new issue instead: #19040.