Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OutOfMemoryErrors when reading files #6988

Closed
robertpanzer opened this issue Jan 8, 2022 · 1 comment
Closed

OutOfMemoryErrors when reading files #6988

robertpanzer opened this issue Jan 8, 2022 · 1 comment
Milestone

Comments

@robertpanzer
Copy link

robertpanzer commented Jan 8, 2022

Environment Information

Provide at least:

  • JRuby version 9.3.2.0
  • Operating system and platform (e.g. uname -a)
Darwin MBP-von-Robert 21.2.0 Darwin Kernel Version 21.2.0: Sun Nov 28 20:28:41 PST 2021; root:xnu-8019.61.5~1/RELEASE_ARM64_T6000 arm64

Other relevant info you may wish to add:

  • Installed or activated gems: Running embedded in a Java process
  • Application/framework version (e.g. Rails, Sinatra): None
  • Environment variables: Nothing additional

I realized when trying to upgrade JRuby for AsciidoctorJ to the latest version 9.3.2.0 that the tests started failing with OutOfMemoryErrors. It most often failed when Asciidoctor tries to load a file with IO.read.
I could reproduce this to a smaller project without Asciidoctor at https://github.com/robertpanzer/asciidoctorj-jruby-test-posix-reproducer.

It looks like since JRuby 9.3.2.0 PosixShim.size does not return the correct file size anymore, but much higher values instead.
JRuby then allocates memory for the entire file, and that can fail.

The following screenshot shows that:
Bildschirmfoto 2022-01-08 um 15 36 40

The file documentwithtoc.adoc has a size of 156 bytes. But PosixShim.size() returns 249568530 instead in this run. And a next run might for example return 654663914.

Expected Behavior

JRuby should only allocate the required memory when loading a small file.
This is also what I observe on JRuby 9.2.20.1

Actual Behavior

JRuby 9.3.2.0 allocates several 100MB even for loading a file with just 156 bytes.

@robertpanzer
Copy link
Author

I tested with jnr-posix 3.1.15 and that seems to fix the problem.
Thank you!

@enebo enebo added this to the JRuby 9.3.3.0 milestone Mar 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants