Replace axios with fetch based alternative

[AuHau]

Currently, we are using axios library to make HTTP requests. It is based on an XHR client in a browser that is old and obsolete and is not further developed. fetch is the current standard for making modern requests. It for example supports streaming responses from server, which XHR does not support. And in future it will most probably support even streaming requests. We should therefore switch to either native fetch or some fetch's based library.

I propose to use ky (or its isomorphic version ky-universal) that shares a lot of the same API with fetch but adds some small but important behaviors and functionality like:

• Simpler API
• Method shortcuts (ky.post())
• Treats non-2xx status codes as errors (after redirects)
• Retries failed requests
• JSON option
• Timeout support
• URL prefix option
• Instances with custom defaults
• Hooks

[mattiaz9]

No upload progress is a big deal breaker for us, we'll have to drop this dependency now.
Why not giving the option to provide a custom fetch implementation? similar to what other packages do, like ccxt.

[AuHau]

@mattiaz9 sorry to hear that, but I am happy you raised your issue!

Well, we could do that, but I am not sure if it would solve anything. Are you developing on browser-side or NodeJS?

To my knowledge, it is not possible to provide nor mock the upload tracking in browsers. If you know about a way please let me know. Well, the only thing I can think of is an XHR wrapper that has a fetch-like API 😅 That sounds like a scary movie thought 🙈

It is possible to do this in NodeJS and it is something we plan to do as per this issue: #397

[mattiaz9]

@AuHau I'm developing on browser not Node.
I'm not sure how it could be done, or how easy It'd be, since, if I'm not mistaken, you used a custom fetch implementation.

I found this: https://github.com/lifeomic/axios-fetch which should build a fetch like function using axios. So I believe it could be possible, but you'll need to provide an option to override the default implementation.

[AuHau]

Well, I have not exactly used custom fetch per se, but rather a utility that extends bit its functionality (ky). That said I think it allows to use of a custom fetch implementation on its own. See it for example here: https://github.com/sindresorhus/ky-universal/blob/main/index.js#L8

But even if ky would use this custom axios-fetch implementation you would still not get any information about the upload progress right? We would have to expose that somehow...

[AuHau]

Or I guess you could use its "Transforming requests" functionality to catch the request and append an upload-progress callback to it and then track the progress? 🤔

That said, this workaround would for example break the download streaming which was one of the reason for adopting fetch.

[AuHau]

Btw. please also comment on this issue to push the WHATWG to implement proper upload progress tracking for fetch.

[mattiaz9]

Or I guess you could use its "Transforming requests" functionality to catch the request and append an upload-progress callback to it and then track the progress? 🤔

That said, this workaround would for example break the download streaming which was one of the reason for adopting fetch.

I'm not familiar with the download streaming. That being said the option to override fetch could be implemented in only the api that allows that. Regarding the upload progress, that would be managed by the axios instance so in theory you wouldn't need to expose anything.

Example:

const axiosInstance = axios.create({
  onUploadProgress: () => {...}
})
const fetch = buildAxiosFetch(axiosInstance, ...)
const fileHash = await bee.uploadData(batchId, "Bee is awesome!",  { fetch })

[AuHau]

I see, that is interesting. We will discuss this. Thanks for the input!

[AuHau]

@mattiaz9 we discussed this and we are on-board for this solution. We will validate that the solution you propose indeed will yield the wanted result and if so, then we will implement it! 👍

[mattiaz9]

@AuHau sounds great, thanks!

[AuHau]

@mattiaz9 just to update. I was able to create a prototype example that made working this approach although the axios-fetch needed bit hacking, which is now put into PR. Feel free to endorse it. So will move further with implementing support for it in bee-js.