Zlib: update external library, introduces 'Deflate'

[sciecode]

This PR is intended to update the current Zlib.js external library used, with the purpose of including the "Deflate" option.

The reason for this is to prepare for an incoming EXR Exporter, which makes use of zlib to compress data internally.

The way I see it, we have two options on how to approach this, that's why I have this PR set-up as a draft.

• Either we go with this PRs approach of simply updating the zlib.js build to one that includes both inflate and deflate.
• Or we use distinct zlib.js builds that only export one function per module.

The downside of the second approach is that it may cause a namespace conflict either for the module ( Zlib ) or global variable when using the non-modular version. Given the minimal difference in file-size overall, I felt like it was better to go with the former.

The initial draft of EXRExporter is ready, I'm just waiting for this to get resolved. Would be nice if we could settle on this ASAP, so that we can get some proper testing on EXRExporter before the next release.

[looeee]

Have you seen Pako? It's supposed to be faster than zlib and creates identical output, at the expense of a few extra kb of JS. It's what JSZip uses internally.

There's also Uzip by @photopea which is supposed to have an 0-50% 20-50% faster inflate method and similar deflate. That's only 15kb minified and could possibly replace all of inflate/deflate/JSZip. It's less well tested though.

[sciecode]

Oh, didn't know about these, looks promising.

I will run some tests on these libs and make sure they are actually compatible with the internal format expected by the loaders, and most importantly the exporter, because we need to make sure the compressed data can be correctly inflated by the OpenEXR implementation.

[looeee]

I've tested out UZIP against JSZip on my own project. It seems to work flawlessly and it's a lot faster. There's a couple of bytes difference at equivalent compression levels but both zips seem to be fine.

The main difference is that with UZIP I have to convert files to Uint8Array manually.

Results for a folder containing 12 text files and two PNGS.

No compression:

UZIP:
Time: 312ms
Size: 1_408_363bytes

JSZip:
Time: 5112ms
Size: 1_411_202bytes

Compression Level 6:

UZIP:
Time: 403ms
Size: 438_818bytes

JSZip:
Time: 5456ms
Size: 432_121bytes

Difference in library size:

UZIP.min.js: 15kb
JSZip.min.js: 93.6kb

[Mugen87]

The downside of the second approach is that it may cause a namespace conflict either for the module ( Zlib ) or global variable when using the non-modular version.

This disadvantage can be neglected since examples/js is already deprecated and will be removed in a few months anyway. Since the project is heading for a module-only codebase, I think I favor option two.

[sciecode]

Since the project is heading for a module-only codebase, I think I favor option two.

In that case, should we rename the exported module to the name of function instead? So that we don't have two different Zlib modules being exported by two different files?

So instead of using this:

import { Zlib } from "../libs/inflate.module.min.js";
//...
new Zlib.Inflate();

We use this:

import { Inflate } from "../libs/inflate.module.min.js";
//...
new Inflate(); // maybe zInflate / zDeflate, to make it obvious it's a zlib implementation

This way, if we ever need both Deflate and Inflate modules in the same file, we don't run into problems like the following

import { Zlib } from "../libs/inflate.module.min.js";
import { Zlib } from "../libs/deflate.module.min.js"; 
// although we can always import one of them with
// import * as Zlib2 from "../libs/deflate.module.min.js"; 

[Mugen87]

Sounds good.

[sciecode]

Just pushed an update with the discussed changes, preliminary tests on EXR, FBX and VTK loaders shows everything is in order.
We should be good to go.

@looeee I haven't discarded using other libs, just for the sake of getting the EXR Exporter PR out there asap, I'll go with this approach for now and we can continue testing zlib.js alternatives.

[photopea]

Hi guys, I am really glad that you use UZIP.js . I created it because I was not happy with the speed and size of pako.js .

To have extremely fast decompression (inflate), give it a target Uint8Array, into which the data should be decompressed (otherwise, the target array must be re-allocated several times during the decompression, as the final size of data is not known).

Also, UZIP.js can very easily parse and create .ZIP files (unlike pako.js).

[sciecode]

To have extremely fast decompression (inflate), give it a target Uint8Array, into which the data should be decompressed (otherwise, the target array must be re-allocated several times during the decompression, as the final size of data is not known).

When inflating EXR pixel data, we have the privilege of knowing exactly the target Uint8Array raw size. So that sounds really good for me, I'll definitely run some tests to check it out.

[mrdoob]

Thanks!