Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore: release 2.0.0 #405

Merged
merged 2 commits into from
Sep 23, 2021
Merged

chore: release 2.0.0 #405

merged 2 commits into from
Sep 23, 2021

Conversation

bee-worker
Copy link
Collaborator

@bee-worker bee-worker commented Aug 30, 2021 •
edited by AuHau
Loading

🤖 I have created a release *beep* *boop*

2.0.0 (2021-09-20)

This is our first major version bump as we did a big revamp of bee-js internals and fixed a few things and shortcomings that required breaking changes.

🤖 HTTP client swap (timeout and retries support)

In the JS browser ecosystem, there are two main HTTP clients: old XMLHttpRequest (XHR) API and new modern fetch API.

We originally used axios library that employs the XHR client, but XHR is old and will not get any new features as it is superseded with fetch API that is actively developed by the WHATWG group, and hence it has its limitations. Many limitations can be overcome using polyfills etc. but a hard stop is networking that only browsers decide what to allow (usually based on the specification). In the case of XHR the limitation is streaming support.

We have therefore decided to use fetch based library ky that supports streaming downloads and hopefully in close future will support also streaming uploads (see whatwg/fetch#966, there is also already functional experiment that enables this in Chrome). fetch is also more future-proof.

This change, unfortunately, does not come without a cost and that is support for tracking upload progress, that fetch still does not have (if interested please comment on the relevant whatwg/fetch#607 issue to raise importance).
If this feature is crucial for you, we have devised a workaround thanks to @mattiaz9, which is demonstrated in our example upload-progress.

This change unfortunately is breaking as we originally exposed AxiosOptions on our API. We have refactored this into more generic HTTP options that should be more future-proof. Thanks to ky we also now have support for retries of failed requests (only for non-POST requests, defaults are seen here) and timeouts. Both are possible to set generally for the Bee instance and/or override it for each method call.

🎏 Streaming revamp

As part of the HTTP client revamp, we had a deeper look at how we handle streams. In the JS land, there are two main types of streams the NodeJS Readable and the browser WHATWG ReadableStream. As our design mindset is browser-first and polyfill the rest in NodeJs, we have unified all returned streams into the WHATWG ReadableStream no matter what platform you are on. Most probably you will want to use NodeJs Readable on NodeJs platform, so we have included utility function readableWebToNode that converts WHATWG stream into NodeJs. There are also more stream-related utility functions that you can check out.

For stream inputs, we accept both types of streams and convert them internally.

⏎ Upload results refactor

One of our short-coming was dropping the returned object from upload methods in favor of the simple string Reference. Later on, we discovered that there is actually a need to return more information from upload operations because Bee automatically creates a Tag for each upload that we could return. Hence we have introduced back UploadResult interface that all the upload methods will now return.

🫓 Utility namespace flatting and filtering

We have merged all the Utils.* namespaces directly into Utils and we have filtered out the functions only to those that make sense to expose in order to minimize the public API and possible future breaking changes.

🗾 uploadCollection() method

We have introduced new uploadCollection method that is more flexible in uploading collection if you do not want to use our convenience methods like uploadFilesFromDirectory() or uploadFiles. This new method accepts the Collection<Uint8Array | Readable> interface.

⚠ BREAKING CHANGES

Other changes:

  • All upload methods that used to accept NodeJS's Readable now accept both NodeJS and WHATWG Readable(Stream).
  • Usage of ReadableStreams in a browser is now possible. Be aware that real support for streaming browsers has only for download, but not upload. When Readable is passed to upload methods it is first fully buffered before making the request. With this be aware of uploading enormous files as they have to be first loaded into memory, which might degrade the user's browser performance! We are in search of optimization for this use case.

Features

  • HTTP request options is possible to override per method call (#411) (9eac5cd)
  • return UploadResult for upload methods (#408) (e58b8e8)

Bug Fixes

  • pss target length verification (#384) (fd032a8)
  • remove check for pinned content in reupload (#412) (6032a22)

Code Refactoring

This PR was generated with Release Please. See documentation.

@bee-runner
Copy link

bee-runner bot commented Aug 30, 2021 •
edited by AuHau
Loading

🚀 Release checklist

Before release

  • Ensure the release works with the intended bee version
  • Verify all the example code works with the release code
  • Verify all user documentation is up to date
  • Have two approval for this PR
  • Write small summary for the release and verify changelog
  • Verify that changes to release PR comment is synchronized to CHANGELOG with manual commit

After release

  • Update API docs
  • Update example's bee-js version

@AuHau
Copy link
Contributor

AuHau commented Aug 30, 2021 •
edited
Loading

Release notes:

2.0.0 (2021-09-20)

This is our first major version bump as we did a big revamp of bee-js internals and fixed a few things and shortcomings that required breaking changes.

🤖 HTTP client swap (timeout and retries support)

In the JS browser ecosystem, there are two main HTTP clients: old XMLHttpRequest (XHR) API and new modern fetch API.

We originally used axios library that employs the XHR client, but XHR is old and will not get any new features as it is superseded with fetch API that is actively developed by the WHATWG group, and hence it has its limitations. Many limitations can be overcome using polyfills etc. but a hard stop is networking that only browsers decide what to allow (usually based on the specification). In the case of XHR the limitation is streaming support.

We have therefore decided to use fetch based library ky that supports streaming downloads and hopefully in close future will support also streaming uploads (see whatwg/fetch#966, there is also already functional experiment that enables this in Chrome). fetch is also more future-proof.

This change, unfortunately, does not come without a cost and that is support for tracking upload progress, that fetch still does not have (if interested please comment on the relevant whatwg/fetch#607 issue to raise importance).
If this feature is crucial for you, we have devised a workaround thanks to @mattiaz9, which is demonstrated in our example upload-progress.

This change unfortunately is breaking as we originally exposed AxiosOptions on our API. We have refactored this into more generic HTTP options that should be more future-proof. Thanks to ky we also now have support for retries of failed requests (only for non-POST requests, defaults are seen here) and timeouts. Both are possible to set generally for the Bee instance and/or override it for each method call.

🎏 Streaming revamp

As part of the HTTP client revamp, we had a deeper look at how we handle streams. In the JS land, there are two main types of streams the NodeJS Readable and the browser WHATWG ReadableStream. As our design mindset is browser-first and polyfill the rest in NodeJs, we have unified all returned streams into the WHATWG ReadableStream no matter what platform you are on. Most probably you will want to use NodeJs Readable on NodeJs platform, so we have included utility function readableWebToNode that converts WHATWG stream into NodeJs. There are also more stream-related utility functions that you can check out.

For stream inputs, we accept both types of streams and convert them internally.

⏎ Upload results refactor

One of our short-coming was dropping the returned object from upload methods in favor of the simple string Reference. Later on, we discovered that there is actually a need to return more information from upload operations because Bee automatically creates a Tag for each upload that we could return. Hence we have introduced back UploadResult interface that all the upload methods will now return.

🫓 Utility namespace flatting and filtering

We have merged all the Utils.* namespaces directly into Utils and we have filtered out the functions only to those that make sense to expose in order to minimize the public API and possible future breaking changes.

🗾 uploadCollection() method

We have introduced new uploadCollection method that is more flexible in uploading collection if you do not want to use our convenience methods like uploadFilesFromDirectory() or uploadFiles. This new method accepts the Collection<Uint8Array | Readable> interface.

⚠ BREAKING CHANGES

Other changes:

  • All upload methods that used to accept NodeJS's Readable now accept both NodeJS and WHATWG Readable(Stream).
  • Usage of ReadableStreams in a browser is now possible. Be aware that real support for streaming browsers have only for download, but not upload. When Readable is passed to upload methods it is first fully buffered before making the request. With this be aware of uploading enormous files as they have to be first loaded into memory, which might degrade the user's browser performance! We are in search of optimization for this use case.

Features

  • HTTP request options is possible to override per method call (#411) (9eac5cd)
  • return UploadResult for upload methods (#408) (e58b8e8)

Bug Fixes

  • pss target length verification (#384) (fd032a8)
  • remove check for pinned content in reupload (#412) (6032a22)

Code Refactoring

@bee-worker bee-worker force-pushed the release-v2.0.0 branch 3 times, most recently from 14ba5b7 to c222acf Compare September 3, 2021 06:02
@bee-worker bee-worker force-pushed the release-v2.0.0 branch 2 times, most recently from ee5df4a to 3765004 Compare September 20, 2021 08:22
@Cafe137
Copy link
Collaborator

Cafe137 commented Sep 23, 2021

The https://github.com/ethersphere/examples-js/tree/master/upload-progress link is a 404 for me

@Cafe137
Copy link
Collaborator

Cafe137 commented Sep 23, 2021

I welcome the possibility to add retries to failed requests, but I am wondering if this can cause double spends, when a costly request succeeds partially but returns non 2xx HTTP status code. Not sure if there should be an action about this, and even a warning at most.

@Cafe137
Copy link
Collaborator

Cafe137 commented Sep 23, 2021

BTW I think it is extremely well-written. On my first read I was not sure if we should link other open issues in release notes, but it is really thorough this way, so I am fine with it.

agazso
agazso approved these changes Sep 23, 2021
View reviewed changes
Copy link
Member

@agazso agazso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice description of the changes

@AuHau
Copy link
Contributor

AuHau commented Sep 23, 2021

Yeah the links are not working, because they are yet to be merged (your case) or some of the API Docs are still not released.

I welcome the possibility to add retries to failed requests, but I am wondering if this can cause double spends, when a costly request succeeds partially but returns non 2xx HTTP status code. Not sure if there should be an action about this, and even a warning at most.

Damn, I forgot to write about that. Thanks! The thing is that the retries work only for some methods and does not work for POST so it should be safe ;-)
See: https://github.com/sindresorhus/ky#retry

nugaon
nugaon approved these changes Sep 23, 2021
View reviewed changes
Copy link
Member

@nugaon nugaon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great description of the release, thanks for your work @AuHau !

The browser-upload with axios-fetch by the examples-js upload example crashed my computer at trying to upload with 1GB data. I think either here or in the examples we should put a notice about do not upload big files from browser - because as you wrote it does not support streaming at uploads.

@AuHau AuHau merged commit 5b112bf into master Sep 23, 2021
@AuHau AuHau deleted the release-v2.0.0 branch September 23, 2021 16:27
@bee-worker
Copy link
Collaborator Author

🤖 Release is at https://github.com/ethersphere/bee-js/releases/tag/v2.0.0 🌻

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@agazso agazso agazso approved these changes

@AuHau AuHau AuHau approved these changes

@nugaon nugaon nugaon approved these changes

Assignees
No one assigned
Labels
autorelease: tagged type:pull-request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@bee-worker @AuHau @Cafe137 @agazso @nugaon