Skip to content

Commit

Permalink
fix: flakey trim-cache test (#53)
Browse files Browse the repository at this point in the history
It appears the trim-cache test using tablet compression occasionally
fails in CI/CD. I suspect this is due to the `save` function returning
before the GZip stream has bene closed, resulting in the subsequent read
attempting to consume an incomplete GZip object.

This adds the necessary provisions to ensure the GZip stream is closed
(or failed) before `save` returns.

Forward-ported from aws/jsii#4043

---

By submitting this pull request, I confirm that my contribution is made
under the terms of the [Apache 2.0 license].

[Apache 2.0 license]: https://www.apache.org/licenses/LICENSE-2.0
  • Loading branch information
RomainMuller authored Apr 5, 2023
1 parent feb117f commit 190656a
Show file tree
Hide file tree
Showing 2 changed files with 9 additions and 7 deletions.
11 changes: 7 additions & 4 deletions src/json.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { Readable, Writable, pipeline } from 'node:stream';
import { Readable, pipeline } from 'node:stream';
import { promisify } from 'node:util';
import { parser } from 'stream-json';
import * as Assembler from 'stream-json/Assembler';
Expand Down Expand Up @@ -36,12 +36,15 @@ export function parse(reader: Readable): Promise<any> {
* better performance.
*
* @param value the value to be serialized.
* @param writer the write in which to write the JSON text.
* @param writers the sequence of write streams to use to output the JSON text.
*/
export function stringify(value: any, writer: Writable): Promise<void> {
export async function stringify(
value: any,
...writers: Array<NodeJS.ReadWriteStream | NodeJS.WritableStream>
): Promise<void> {
const reader = new Readable({ objectMode: true });
reader.push(value);
reader.push(null);

return asyncPipeline(reader, disassembler(), stringer(), writer);
return asyncPipeline(reader, disassembler(), stringer(), ...writers);
}
5 changes: 2 additions & 3 deletions src/tablets/tablets.ts
Original file line number Diff line number Diff line change
Expand Up @@ -182,14 +182,13 @@ export class LanguageTablet {
* Saves the tablet schema to a file. If the compress option is passed, then
* the schema will be gzipped before writing to the file.
*/
public async save(filename: string, compress = false) {
public async save(filename: string, compress = false): Promise<void> {
await fs.mkdir(path.dirname(filename), { recursive: true });

const writeStream: Writable = createWriteStream(filename, { flags: 'w' });
const gzip = compress ? zlib.createGzip() : undefined;
gzip?.pipe(writeStream, { end: true });

return stringify(this.toSchema(), gzip ?? writeStream);
return stringify(this.toSchema(), ...(gzip ? [gzip] : []), writeStream);
}

private toSchema(): TabletSchema {
Expand Down

0 comments on commit 190656a

Please sign in to comment.