TextIO.inputAll segfaults when reading 980M file

[serpent7776]

Version

110.99.4 (Latest)

Operating System

• Any
• Linux
• macOS
• Windows
• Other Unix

OS Version

Linux arc 5.10.194-1-MANJARO #1 SMP PREEMPT Sat Sep 2 20:55:57 UTC 2023 x86_64 GNU/Linux

Processor

• Any
• Arm (using Rosetta)
• PowerPC
• Sparc
• x86 (32-bit)
• x86-64 (64-bit)
• Other

System Component

Basis Library

Severity

Major

Description

inputAll segfaults when reading 980M file with TextIo.inputAll.

Transcript

$ truncate -s 980M input
$ smlnj
fgrep: warning: fgrep is obsolescent; using grep -F
Standard ML of New Jersey (64-bit) v110.99.4 [built: Tue Aug 08 11:25:21 2023]
- val f = TextIO.openIn "input";
[autoloading]
[library $SMLNJ-BASIS/basis.cm is stable]
[library $SMLNJ-BASIS/(basis.cm):basis-common.cm is stable]
[autoloading done]
val f = - : TextIO.instream
- val s = TextIO.inputAll f;
/usr/lib/smlnj/bin/sml: Fatal error -- bogus fault not in ML: pc = 0x55723ef19685, sig = 11

Expected Behavior

file to be read correctly, no segfault.

Steps to Reproduce

$ truncate -s 980M input
$ sml
- val f = TextIO.openIn "input";
- val s = TextIO.inputAll f;
/usr/lib/smlnj/bin/sml: Fatal error -- bogus fault not in ML: pc = 0x55723ef19685, sig = 11

Additional Information

Can reproduce this on FreeBSD DaemONX 12.4-RELEASE-p3 FreeBSD 12.4-RELEASE-p3 GENERIC amd64 with version Standard ML of New Jersey (64-bit) v110.98 [built: Sun Aug 21 21:39:28 2022]

Email address

serpent7776 \at gmail \dot com

[JohnReppy]

I think that the bug is in runtime/gc/flip.c; it appears that the requested size of the new string arena is too small.

[JohnReppy]

The actual threshold of failure is actually slightly below 384M = (0x18000000). If we try 402653176 (i.e., 8 bytes smaller), then we get a different error:

/usr/local/smlnj/bin/sml: Fatal error -- bad string tag 1, obj = 0x121240000, desc = 0x86

A simpler test that shows the bug is to use Unsafe.CharVector.create to create a large string object. For example:

Standard ML of New Jersey (64-bit) v110.99.4 [built: Tue Aug 01 16:07:38 2023]
- Unsafe.CharVector.create 402653176;
[autoloading]
[library $SMLNJ-BASIS/basis.cm is stable]
[library $SMLNJ-BASIS/(basis.cm):basis-common.cm is stable]
[autoloading done]
val it =
  "\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@\^@#"
Memory fault

[JohnReppy]

The problem actually turned out to be that we were still using 32-bit ints to represent some size information. Switching to using the Word_t type for array/vector lengths in the runtime system appears to have fixed the problem. The fix will be included in 110.99.5.