Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP]Fix calling Buffer.toString with (offset, length, encoding) #3467

Merged
merged 6 commits into from
Jul 2, 2023

Conversation

Hanaasagi
Copy link
Collaborator

Close: #3085

  • Fix calling Buffer.toString with (offset, length, encoding)
  • Fix utf8Slice, latin1Slice, ...

export function utf8Slice(this: BufferExt, offset, length) {
return this.toString(offset, length, "utf8");
export function utf8Slice(this: BufferExt, start, end) {
return this.toString("utf8", start, end);
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

start and end are used instead of offset and length to maintain compatibility with nodejs.

expect(buf.ucs2Slice(2, 6)).toStrictEqual("いう");
});

it("Buffer.base64Slice()", () => {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get a segmentfault in the base64-related tests on my local machine. I don't know what happened here.

Copy link
Collaborator Author

@Hanaasagi Hanaasagi Jul 1, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Attempting to debug a coredump file using gdb, the arguments passed to WTF__toBase64URLStringValue looks like strange.

2023-07-01_15-28

Updated:

I build the latest main branch, also find this problem. toString('base64) will segment fault. But if i use the canary release bun, not find this problem.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I vaguely remember @cirospaciari running into strange issues with .toBase64URL as well... I think we should switch to using the SIMD base64 library that Node.js also uses. Our current implementation uses a mix of WebKit's internal atob function and Zig's standard library. I would guess the cause of this is the Zig standard library usage, but it's hard to say without digging deeper

offset = static_cast<uint32_t>(istart);
length = (length > offset) ? (length - offset) : 0;
} else {
int32_t ioffset = arg1.toInt32(lexicalGlobalObject);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

toInt32 can throw, we should technically check for exceptions here (we did not before)


expect(buf.asciiSlice()).toStrictEqual("0123456789");
expect(buf.asciiSlice(3)).toStrictEqual("3456789");
expect(buf.asciiSlice(3, 4)).toStrictEqual("3");
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These tests are failing

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh its failing because we need to commit regenerated builtins

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you. I had mistakenly thought that the generated code was not required for commit.

@Jarred-Sumner Jarred-Sumner merged commit 4720fa1 into oven-sh:main Jul 2, 2023
14 of 19 checks passed
@Jarred-Sumner
Copy link
Collaborator

Manually tested, looks good.

Thank you for this!

@Hanaasagi Hanaasagi deleted the fix-toString branch July 2, 2023 12:14
@Jarred-Sumner
Copy link
Collaborator

Reading through node's implementation, do they use this interface? I don't see it

image

@Hanaasagi
Copy link
Collaborator Author

@Jarred-Sumner
As I mentioned in issue #3085, Node.js does not support the offset and length parameters. If you are certain that Bun does not rely on them internally, it would be better to remove them(personal opinion).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fileupload error using Express/Multer (busboy)
2 participants