Improve streaming API

CHANGELOG.md

@@ -1,3 +1,12 @@
+## 0.8.2
+- Fixed broken UMD build
+- Fixed edge-case causing skipped data during streaming compression
+- Fixed bug in GZIP streaming on member boundary
+- Improved streaming performance on inconsistent chunk sizes
+- Improved `unzip` performance on undercompressed archives
+- Added flushing support into streaming API
+- Added backpressure support into async streaming API
+  - Use new `ondrain` handler and `queuedSize`
 ## 0.8.1
 - Fixed reallocating on pre-supplied buffer in `inflateSync` and `unzlibSync`
 - Minor documentation fixes

README.md

@@ -536,9 +536,9 @@ Before you decide that `fflate` is the end-all compression library, you should n
 ## What about `CompressionStream`?
 Like `fflate`, the [Compression Streams API](https://developer.mozilla.org/en-US/docs/Web/API/Compression_Streams_API) provides DEFLATE, GZIP, and Zlib compression and decompression support. It's a good option if you'd like to compress or decompress data without installing any third-party libraries, and it wraps native Zlib bindings to achieve better performance than what most JavaScript programs can achieve.
 
-However, browsers do not offer any native non-streaming compression API, and `CompressionStream` has surprisingly poor performance on data already loaded into memory; `fflate` tends to be faster even for files that are dozens of megabytes large. Similarly, `fflate` is much faster for files under a megabyte because it avoids marshalling overheads. Even when streaming hundreds of megabytes of data, the native API usually only performs between 5% slower and 10% faster than `fflate`. And Compression Streams have many other disadvantages - no ability to control compression level, poor support for older browsers, no ZIP support, etc.
+However, browsers do not offer any native non-streaming compression API, and `CompressionStream` has surprisingly poor performance on data already loaded into memory; `fflate` tends to be faster even for files that are dozens of megabytes large. Similarly, `fflate` is much faster for files under a megabyte because it avoids marshalling overheads. Even when streaming hundreds of megabytes of data, the native API usually performs between 30% faster and 10% slower than `fflate`. And Compression Streams have many other disadvantages - no ability to control compression level, poor support for older browsers, no ZIP support, etc.
 
-If you'd still prefer to depend upon a native browser API, you can use an `fflate`-based [Compression Streams ponyfill](https://github.com/101arrowz/compression-streams-polyfill) for a seamless transition to `CompressionStream` and `DecompressionStream` if ever they become substantially faster than `fflate`.
+If you'd still prefer to depend upon a native browser API but want to support older browsers, you can use an `fflate`-based [Compression Streams ponyfill](https://github.com/101arrowz/compression-streams-polyfill).
 
 ## Browser support
 `fflate` makes heavy use of typed arrays (`Uint8Array`, `Uint16Array`, etc.). Typed arrays can be polyfilled at the cost of performance, but the most recent browser that doesn't support them [is from 2011](https://caniuse.com/typedarrays), so I wouldn't bother.

docs/README.md

@@ -71,6 +71,7 @@
 
 ### Type Aliases
 
+- [AsyncFlateDrainHandler](README.md#asyncflatedrainhandler)
 - [AsyncFlateStreamHandler](README.md#asyncflatestreamhandler)
 - [AsyncZippableFile](README.md#asynczippablefile)
 - [FlateCallback](README.md#flatecallback)
@@ -135,6 +136,28 @@ Renames and re-exports [gzipSync](README.md#gzipsync)
 
 ## Type Aliases
 
+### AsyncFlateDrainHandler
+
+Ƭ **AsyncFlateDrainHandler**: (`size`: `number`) => `void`
+
+#### Type declaration
+
+▸ (`size`): `void`
+
+Handler for the asynchronous completion of (de)compression for a data chunk
+
+##### Parameters
+
+| Name | Type | Description |
+| :------ | :------ | :------ |
+| `size` | `number` | The number of bytes that were processed. This is measured in terms of the input (i.e. compressed bytes for decompression, uncompressed bytes for compression.) |
+
+##### Returns
+
+`void`
+
+___
+
 ### AsyncFlateStreamHandler
 
 Ƭ **AsyncFlateStreamHandler**: (`err`: [`FlateError`](interfaces/FlateError.md) \| ``null``, `data`: `Uint8Array`, `final`: `boolean`) => `void`

docs/classes/AsyncDecompress.md

@@ -11,6 +11,8 @@ Asynchronous streaming GZIP, Zlib, or raw DEFLATE decompression
 ### Properties
 
 - [ondata](AsyncDecompress.md#ondata)
+- [ondrain](AsyncDecompress.md#ondrain)
+- [queuedSize](AsyncDecompress.md#queuedsize)
 
 ### Methods
 
@@ -49,6 +51,22 @@ Creates an asynchronous decompression stream
 
 The handler to call whenever data is available
 
+___
+
+### ondrain
+
+• `Optional` **ondrain**: [`AsyncFlateDrainHandler`](../README.md#asyncflatedrainhandler)
+
+The handler to call whenever buffered source data is processed (i.e. `queuedSize` updates)
+
+___
+
+### queuedSize
+
+• **queuedSize**: `number`
+
+The number of compressed bytes buffered in the stream
+
 ## Methods
 
 ### push

docs/classes/AsyncDeflate.md

@@ -11,10 +11,13 @@ Asynchronous streaming DEFLATE compression
 ### Properties
 
 - [ondata](AsyncDeflate.md#ondata)
+- [ondrain](AsyncDeflate.md#ondrain)
+- [queuedSize](AsyncDeflate.md#queuedsize)
 - [terminate](AsyncDeflate.md#terminate)
 
 ### Methods
 
+- [flush](AsyncDeflate.md#flush)
 - [push](AsyncDeflate.md#push)
 
 ## Constructors
@@ -52,6 +55,22 @@ The handler to call whenever data is available
 
 ___
 
+### ondrain
+
+• `Optional` **ondrain**: [`AsyncFlateDrainHandler`](../README.md#asyncflatedrainhandler)
+
+The handler to call whenever buffered source data is processed (i.e. `queuedSize` updates)
+
+___
+
+### queuedSize
+
+• **queuedSize**: `number`
+
+The number of uncompressed bytes buffered in the stream
+
+___
+
 ### terminate
 
 • **terminate**: [`AsyncTerminable`](../interfaces/AsyncTerminable.md)
@@ -61,6 +80,19 @@ push() will silently fail.
 
 ## Methods
 
+### flush
+
+▸ **flush**(): `void`
+
+Flushes buffered uncompressed data. Useful to immediately retrieve the
+deflated output for small inputs.
+
+#### Returns
+
+`void`
+
+___
+
 ### push
 
 ▸ **push**(`chunk`, `final?`): `void`

docs/classes/AsyncGunzip.md

@@ -11,7 +11,9 @@ Asynchronous streaming single or multi-member GZIP decompression
 ### Properties
 
 - [ondata](AsyncGunzip.md#ondata)
+- [ondrain](AsyncGunzip.md#ondrain)
 - [onmember](AsyncGunzip.md#onmember)
+- [queuedSize](AsyncGunzip.md#queuedsize)
 - [terminate](AsyncGunzip.md#terminate)
 
 ### Methods
@@ -53,6 +55,14 @@ The handler to call whenever data is available
 
 ___
 
+### ondrain
+
+• `Optional` **ondrain**: [`AsyncFlateDrainHandler`](../README.md#asyncflatedrainhandler)
+
+The handler to call whenever buffered source data is processed (i.e. `queuedSize` updates)
+
+___
+
 ### onmember
 
 • `Optional` **onmember**: [`GunzipMemberHandler`](../README.md#gunzipmemberhandler)
@@ -61,6 +71,14 @@ The handler to call whenever a new GZIP member is found
 
 ___
 
+### queuedSize
+
+• **queuedSize**: `number`
+
+The number of compressed bytes buffered in the stream
+
+___
+
 ### terminate
 
 • **terminate**: [`AsyncTerminable`](../interfaces/AsyncTerminable.md)

docs/classes/AsyncGzip.md

@@ -11,10 +11,13 @@ Asynchronous streaming GZIP compression
 ### Properties
 
 - [ondata](AsyncGzip.md#ondata)
+- [ondrain](AsyncGzip.md#ondrain)
+- [queuedSize](AsyncGzip.md#queuedsize)
 - [terminate](AsyncGzip.md#terminate)
 
 ### Methods
 
+- [flush](AsyncGzip.md#flush)
 - [push](AsyncGzip.md#push)
 
 ## Constructors
@@ -52,6 +55,22 @@ The handler to call whenever data is available
 
 ___
 
+### ondrain
+
+• `Optional` **ondrain**: [`AsyncFlateDrainHandler`](../README.md#asyncflatedrainhandler)
+
+The handler to call whenever buffered source data is processed (i.e. `queuedSize` updates)
+
+___
+
+### queuedSize
+
+• **queuedSize**: `number`
+
+The number of uncompressed bytes buffered in the stream
+
+___
+
 ### terminate
 
 • **terminate**: [`AsyncTerminable`](../interfaces/AsyncTerminable.md)
@@ -61,6 +80,19 @@ push() will silently fail.
 
 ## Methods
 
+### flush
+
+▸ **flush**(): `void`
+
+Flushes buffered uncompressed data. Useful to immediately retrieve the
+GZIPped output for small inputs.
+
+#### Returns
+
+`void`
+
+___
+
 ### push
 
 ▸ **push**(`chunk`, `final?`): `void`

docs/classes/AsyncInflate.md

@@ -11,6 +11,8 @@ Asynchronous streaming DEFLATE decompression
 ### Properties
 
 - [ondata](AsyncInflate.md#ondata)
+- [ondrain](AsyncInflate.md#ondrain)
+- [queuedSize](AsyncInflate.md#queuedsize)
 - [terminate](AsyncInflate.md#terminate)
 
 ### Methods
@@ -52,6 +54,22 @@ The handler to call whenever data is available
 
 ___
 
+### ondrain
+
+• `Optional` **ondrain**: [`AsyncFlateDrainHandler`](../README.md#asyncflatedrainhandler)
+
+The handler to call whenever buffered source data is processed (i.e. `queuedSize` updates)
+
+___
+
+### queuedSize
+
+• **queuedSize**: `number`
+
+The number of compressed bytes buffered in the stream
+
+___
+
 ### terminate
 
 • **terminate**: [`AsyncTerminable`](../interfaces/AsyncTerminable.md)

docs/classes/AsyncUnzlib.md

@@ -11,6 +11,8 @@ Asynchronous streaming Zlib decompression
 ### Properties
 
 - [ondata](AsyncUnzlib.md#ondata)
+- [ondrain](AsyncUnzlib.md#ondrain)
+- [queuedSize](AsyncUnzlib.md#queuedsize)
 - [terminate](AsyncUnzlib.md#terminate)
 
 ### Methods
@@ -52,6 +54,22 @@ The handler to call whenever data is available
 
 ___
 
+### ondrain
+
+• `Optional` **ondrain**: [`AsyncFlateDrainHandler`](../README.md#asyncflatedrainhandler)
+
+The handler to call whenever buffered source data is processed (i.e. `queuedSize` updates)
+
+___
+
+### queuedSize
+
+• **queuedSize**: `number`
+
+The number of compressed bytes buffered in the stream
+
+___
+
 ### terminate
 
 • **terminate**: [`AsyncTerminable`](../interfaces/AsyncTerminable.md)

docs/classes/AsyncZlib.md

@@ -11,10 +11,13 @@ Asynchronous streaming Zlib compression
 ### Properties
 
 - [ondata](AsyncZlib.md#ondata)
+- [ondrain](AsyncZlib.md#ondrain)
+- [queuedSize](AsyncZlib.md#queuedsize)
 - [terminate](AsyncZlib.md#terminate)
 
 ### Methods
 
+- [flush](AsyncZlib.md#flush)
 - [push](AsyncZlib.md#push)
 
 ## Constructors
@@ -52,6 +55,22 @@ The handler to call whenever data is available
 
 ___
 
+### ondrain
+
+• `Optional` **ondrain**: [`AsyncFlateDrainHandler`](../README.md#asyncflatedrainhandler)
+
+The handler to call whenever buffered source data is processed (i.e. `queuedSize` updates)
+
+___
+
+### queuedSize
+
+• **queuedSize**: `number`
+
+The number of uncompressed bytes buffered in the stream
+
+___
+
 ### terminate
 
 • **terminate**: [`AsyncTerminable`](../interfaces/AsyncTerminable.md)
@@ -61,6 +80,19 @@ push() will silently fail.
 
 ## Methods
 
+### flush
+
+▸ **flush**(): `void`
+
+Flushes buffered uncompressed data. Useful to immediately retrieve the
+zlibbed output for small inputs.
+
+#### Returns
+
+`void`
+
+___
+
 ### push
 
 ▸ **push**(`chunk`, `final?`): `void`

docs/classes/Deflate.md

@@ -14,6 +14,7 @@ Streaming DEFLATE compression
 
 ### Methods
 
+- [flush](Deflate.md#flush)
 - [push](Deflate.md#push)
 
 ## Constructors
@@ -51,6 +52,19 @@ The handler to call whenever data is available
 
 ## Methods
 
+### flush
+
+▸ **flush**(): `void`
+
+Flushes buffered uncompressed data. Useful to immediately retrieve the
+deflated output for small inputs.
+
+#### Returns
+
+`void`
+
+___
+
 ### push
 
 ▸ **push**(`chunk`, `final?`): `void`