Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Getting unexpected eof after upgrade to 0.11.8 #1396

Closed
mathstuf opened this issue Dec 8, 2017 · 9 comments
Closed

Getting unexpected eof after upgrade to 0.11.8 #1396

mathstuf opened this issue Dec 8, 2017 · 9 comments
Labels
A-client Area: client. C-bug Category: bug. Something is wrong. This is bad!

Comments

@mathstuf
Copy link
Contributor

mathstuf commented Dec 8, 2017

After upgrading our application (which only uses hyper directly for the header! macro in a dependent library and reqwest otherwise) to 0.11.8, we've started getting this error showing up:

pipeline error: unexpected end of file

which comes from tokio-proto. We've narrowed it down to hyper changing in commit 95e0164. Is there some way that we're using reqwest improperly that would cause this?

All of our calls using reqwest are contained in this file and this is a long-lived structure, though we're seeing errors right after startup. Does reqwest need changes related to its connection pool?

Cc: @bradking

@seanmonstar
Copy link
Member

Does the connection send bytes it shouldn't? Such as, is there a body to a head request, or some extra bytes after the end of a response body?

@mathstuf
Copy link
Contributor Author

mathstuf commented Dec 8, 2017

I doubt it; we're talking to Gitlab which is using Grape for its REST API and I'd expect it to be well-behaved in that respect. I'll check the logs though to see if that's happening.

@mathstuf
Copy link
Contributor Author

mathstuf commented Dec 8, 2017

I'm only seeing 200 OK responses with full JSON data objects.

@seanmonstar
Copy link
Member

Would it be possible to turn on logs from hyper and (after sanitizing) paste the ones that appear around the error log? I'm setting up a super simple program to just hit some public API end points on Gitlab and not seeing this error log, so I could use a little more on how to investigate this.

@mathstuf
Copy link
Contributor Author

mathstuf commented Dec 8, 2017

We see it when running the queries through Rayon, so it could be something to do with saturating reqwest's connection pool. Currently dumping logs out (there's lots of activity and it's taking a while to get a week's worth of logs).

@mathstuf
Copy link
Contributor Author

mathstuf commented Dec 8, 2017

Is there an email address that would be good to send it to? I anonymized it by removing all of the received data bits (they now just say Object() or Array()), but it is 220K which I'd rather not upload directly to github.

@seanmonstar
Copy link
Member

You could send it to sean@seanmonstar.com.

@seanmonstar
Copy link
Member

The most likely cause for this was that the connection was being closed immediately after the end of a response body was read. The internal implementation would not have adjusted its state that an EOF was OK at that point (only doing so in flush), and the tokio-proto dispatcher was calling poll again (since the previous one succeeded).

Merged to master is a fix to update its state that the message has completed and so an EOF is not unexpected. Would you be able to test using hyper master, to confirm?

@seanmonstar seanmonstar added A-client Area: client. C-bug Category: bug. Something is wrong. This is bad! labels Dec 9, 2017
@bradking
Copy link

Would you be able to test using hyper master, to confirm?

Yes, it works as of v0.11.9, thanks!

seeekr pushed a commit to productioneer/rust-gitlab that referenced this issue Apr 15, 2019
This release fixes an issue that occurred frequently with Gitlab
communication.

See hyperium/hyper#1396 for details.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-client Area: client. C-bug Category: bug. Something is wrong. This is bad!
Projects
None yet
Development

No branches or pull requests

3 participants