Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

IllegalStateException: cannot send more responses than requests #6775

Closed
devoncrouse opened this issue Apr 4, 2016 · 3 comments
Closed

IllegalStateException: cannot send more responses than requests #6775

devoncrouse opened this issue Apr 4, 2016 · 3 comments

Comments

@devoncrouse
Copy link

We have some existing infrastructure automation for Elasticsearch (that we like), and Kibana seems to break upon upgrading to ES 2.3/Kibana 4.5:

  • Cluster status is green (checked via curl from Kibana host, using same address provided to -e arg)
  • Head plugin loads and also shows green, with .kibana index created
  • Kibana loads into Oops! page with fatal error: Courier Fetch Error: unhandled courier request error: socket hang up

Kibana log output on each request of /:

{"type":"response","@timestamp":"2016-04-04T23:08:59+00:00","tags":[],"pid":5,"method":"get","statusCode":304,"req":{"url":"/bundles/node_modules/font-awesome/fonts/fontawesome-webfont.woff2","method":"get","headers":{"host":"10.100.1.213:22281","connection":"keep-alive","cache-control":"max-age=0","origin":"http://10.100.1.213:22281","if-modified-since":"Mon, 28 Mar 2016 23:29:56 GMT","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.110 Safari/537.36","accept":"*/*","referer":"http://10.100.1.213:22281/app/kibana","accept-encoding":"gzip, deflate, sdch","accept-language":"en-US,en;q=0.8"},"remoteAddress":"192.168.57.51","userAgent":"192.168.57.51","referer":"http://10.100.1.213:22281/app/kibana"},"res":{"statusCode":304,"responseTime":2,"contentLength":9},"message":"GET /bundles/node_modules/font-awesome/fonts/fontawesome-webfont.woff2 304 2ms - 9.0B"}
{"type":"response","@timestamp":"2016-04-04T23:09:01+00:00","tags":[],"pid":5,"method":"get","statusCode":200,"req":{"url":"/","method":"get","headers":{"host":"10.100.1.213:22281","connection":"keep-alive","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8","upgrade-insecure-requests":"1","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.110 Safari/537.36","accept-encoding":"gzip, deflate, sdch","accept-language":"en-US,en;q=0.8"},"remoteAddress":"192.168.57.51","userAgent":"192.168.57.51"},"res":{"statusCode":200,"responseTime":2,"contentLength":9},"message":"GET / 200 2ms - 9.0B"}
{"type":"response","@timestamp":"2016-04-04T23:09:01+00:00","tags":[],"pid":5,"method":"get","statusCode":200,"req":{"url":"/app/kibana","method":"get","headers":{"host":"10.100.1.213:22281","connection":"keep-alive","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8","upgrade-insecure-requests":"1","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.110 Safari/537.36","referer":"http://10.100.1.213:22281/","accept-encoding":"gzip, deflate, sdch","accept-language":"en-US,en;q=0.8"},"remoteAddress":"192.168.57.51","userAgent":"192.168.57.51","referer":"http://10.100.1.213:22281/"},"res":{"statusCode":200,"responseTime":4,"contentLength":9},"message":"GET /app/kibana 200 4ms - 9.0B"}
{"type":"response","@timestamp":"2016-04-04T23:09:02+00:00","tags":[],"pid":5,"method":"get","statusCode":304,"req":{"url":"/bundles/commons.style.css?v=9889","method":"get","headers":{"host":"10.100.1.213:22281","connection":"keep-alive","accept":"text/css,*/*;q=0.1","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.110 Safari/537.36","referer":"http://10.100.1.213:22281/app/kibana","accept-encoding":"gzip, deflate, sdch","accept-language":"en-US,en;q=0.8","if-none-match":"\"def64d76817ecadf60b9fd69e48a547847fb9d5e-gzip\"","if-modified-since":"Mon, 28 Mar 2016 23:29:56 GMT"},"remoteAddress":"192.168.57.51","userAgent":"192.168.57.51","referer":"http://10.100.1.213:22281/app/kibana"},"res":{"statusCode":304,"responseTime":4,"contentLength":9},"message":"GET /bundles/commons.style.css?v=9889 304 4ms - 9.0B"}
{"type":"response","@timestamp":"2016-04-04T23:09:02+00:00","tags":[],"pid":5,"method":"get","statusCode":304,"req":{"url":"/bundles/kibana.style.css?v=9889","method":"get","headers":{"host":"10.100.1.213:22281","connection":"keep-alive","accept":"text/css,*/*;q=0.1","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.110 Safari/537.36","referer":"http://10.100.1.213:22281/app/kibana","accept-encoding":"gzip, deflate, sdch","accept-language":"en-US,en;q=0.8","if-none-match":"\"94def40ee55a84529a383c1277923ea08a31e75c-gzip\"","if-modified-since":"Mon, 28 Mar 2016 23:29:56 GMT"},"remoteAddress":"192.168.57.51","userAgent":"192.168.57.51","referer":"http://10.100.1.213:22281/app/kibana"},"res":{"statusCode":304,"responseTime":5,"contentLength":9},"message":"GET /bundles/kibana.style.css?v=9889 304 5ms - 9.0B"}
{"type":"response","@timestamp":"2016-04-04T23:09:02+00:00","tags":[],"pid":5,"method":"get","statusCode":304,"req":{"url":"/bundles/commons.bundle.js?v=9889","method":"get","headers":{"host":"10.100.1.213:22281","connection":"keep-alive","accept":"*/*","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.110 Safari/537.36","referer":"http://10.100.1.213:22281/app/kibana","accept-encoding":"gzip, deflate, sdch","accept-language":"en-US,en;q=0.8","if-none-match":"\"e2a5dea356ee1a329b1a9a57379d5559b20f493e-gzip\"","if-modified-since":"Mon, 28 Mar 2016 23:29:56 GMT"},"remoteAddress":"192.168.57.51","userAgent":"192.168.57.51","referer":"http://10.100.1.213:22281/app/kibana"},"res":{"statusCode":304,"responseTime":7,"contentLength":9},"message":"GET /bundles/commons.bundle.js?v=9889 304 7ms - 9.0B"}
{"type":"response","@timestamp":"2016-04-04T23:09:02+00:00","tags":[],"pid":5,"method":"get","statusCode":304,"req":{"url":"/bundles/kibana.bundle.js?v=9889","method":"get","headers":{"host":"10.100.1.213:22281","connection":"keep-alive","accept":"*/*","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.110 Safari/537.36","referer":"http://10.100.1.213:22281/app/kibana","accept-encoding":"gzip, deflate, sdch","accept-language":"en-US,en;q=0.8","if-modified-since":"Mon, 28 Mar 2016 23:29:56 GMT"},"remoteAddress":"192.168.57.51","userAgent":"192.168.57.51","referer":"http://10.100.1.213:22281/app/kibana"},"res":{"statusCode":304,"responseTime":1,"contentLength":9},"message":"GET /bundles/kibana.bundle.js?v=9889 304 1ms - 9.0B"}
{"type":"response","@timestamp":"2016-04-04T23:09:03+00:00","tags":[],"pid":5,"method":"get","statusCode":304,"req":{"url":"/bundles/src/ui/public/images/kibana.svg","method":"get","headers":{"host":"10.100.1.213:22281","connection":"keep-alive","accept":"image/webp,image/*,*/*;q=0.8","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.110 Safari/537.36","referer":"http://10.100.1.213:22281/app/kibana","accept-encoding":"gzip, deflate, sdch","accept-language":"en-US,en;q=0.8","if-modified-since":"Mon, 28 Mar 2016 23:29:56 GMT"},"remoteAddress":"192.168.57.51","userAgent":"192.168.57.51","referer":"http://10.100.1.213:22281/app/kibana"},"res":{"statusCode":304,"responseTime":2,"contentLength":9},"message":"GET /bundles/src/ui/public/images/kibana.svg 304 2ms - 9.0B"}
{"type":"response","@timestamp":"2016-04-04T23:09:04+00:00","tags":[],"pid":5,"method":"post","statusCode":502,"req":{"url":"/elasticsearch/_mget?timeout=0&ignore_unavailable=true&preference=1459811342955","method":"post","headers":{"host":"10.100.1.213:22281","connection":"keep-alive","content-length":"62","accept":"application/json, text/plain, */*","origin":"http://10.100.1.213:22281","kbn-version":"4.5.0","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.110 Safari/537.36","content-type":"application/json;charset=UTF-8","referer":"http://10.100.1.213:22281/app/kibana","accept-encoding":"gzip, deflate","accept-language":"en-US,en;q=0.8"},"remoteAddress":"192.168.57.51","userAgent":"192.168.57.51","referer":"http://10.100.1.213:22281/app/kibana"},"res":{"statusCode":502,"responseTime":11,"contentLength":9},"message":"POST /elasticsearch/_mget?timeout=0&ignore_unavailable=true&preference=1459811342955 502 11ms - 9.0B"}

This automation has been pretty stable throughout the 2.x upgrades, so I thought I'd raise this in case something bigger happened here. All nodes are Dockerized using bridged networking and cloud-aws EC2 discovery.

Also, I spun up the official kibana:4.5 Docker image with "cmd": "kibana -e http://my-es:9200" with the same result, despite the green status from my-es:9200...

Does anything above stand out, by chance? Thanks in advance!
@devoncrouse
Copy link
Author

Remaining output on Kibana error page:

Version: 4.5.0
Build: 9889

Error: unhandled courier request error: socket hang up
    at handleError (http://10.100.3.89:5601/bundles/kibana.bundle.js?v=9889:89249:23)
    at DocRequest.AbstractReqProvider.AbstractReq.handleFailure (http://10.100.3.89:5601/bundles/kibana.bundle.js?v=9889:89169:15)
    at http://10.100.3.89:5601/bundles/kibana.bundle.js?v=9889:89063:18
    at Array.forEach (native)
    at http://10.100.3.89:5601/bundles/kibana.bundle.js?v=9889:89061:19
    at processQueue (http://10.100.3.89:5601/bundles/commons.bundle.js?v=9889:41836:29)
    at http://10.100.3.89:5601/bundles/commons.bundle.js?v=9889:41852:28
    at Scope.$eval (http://10.100.3.89:5601/bundles/commons.bundle.js?v=9889:43080:29)
    at Scope.$digest (http://10.100.3.89:5601/bundles/commons.bundle.js?v=9889:42891:32)
    at Scope.$apply (http://10.100.3.89:5601/bundles/commons.bundle.js?v=9889:43188:25)

@spalger
Copy link
Contributor

spalger commented Apr 5, 2016

@devoncrouse the "socket hang up" error is definitely coming from a misconfigured component outside of Kibana. This type of support request is better served at the kibana user forum where you should be able to find help debugging this. I would start by tracking down the actual http response coming out of the Kibana server, it may have additional information to help debug the issue.

@spalger spalger closed this as completed Apr 5, 2016
@jpgil
Copy link

jpgil commented May 7, 2016

This problem as been appearing in many forums, it should be nice if a more useful link is posted here. I also upgrade from ES 2.2 to 2.3, Kibana 4.3 to 4.5, and got the same issue.

@jpgil jpgil mentioned this issue May 8, 2016
Closed
cee-chen added a commit that referenced this issue May 31, 2023
@1Copenut @stratoula
@kibanamachine @cee-chen
Upgrade EUI to v81.0.0 (#158330)
aa1d266 
## Summary

`@elastic/eui@80.0.0` ⏩ `@elastic/eui@81.0.0`

---

## [`81.0.0`](https://github.com/elastic/eui/tree/v81.0.0)

- Added ability to set `options.checked` to "mixed" in `EuiSelectable`
([#6774](elastic/eui#6774))

**Bug fixes**

- Portalled components (e.g. `EuiPopover`, `EuiModal`, `EuiFlyout`) will
correctly inherit text color from its nearest `EuiThemeProvider` parent.
`<EuiText color="default">` is no longer needed.
([#6775](elastic/eui#6775))

**Breaking changes**

- `EuiSelectable` no longer renders a `data-test-selected` attribute on
its list items. Use the `aria-checked` property instead
([#6774](elastic/eui#6774))
- Nested `EuiThemeProvider`s now render a wrapping `<span>` element in
order to correctly set the inherited text `color` of all descendants.
`<EuiText color="default">` is no longer needed.
([#6775](elastic/eui#6775))

---------

Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Constance Chen <constance.chen@elastic.co>
sloanelybutsurely pushed a commit to sloanelybutsurely/kibana that referenced this issue Jun 6, 2023
@1Copenut @stratoula
@kibanamachine @cee-chen
Upgrade EUI to v81.0.0 (elastic#158330)
36edac5 
## Summary

`@elastic/eui@80.0.0` ⏩ `@elastic/eui@81.0.0`

---

## [`81.0.0`](https://github.com/elastic/eui/tree/v81.0.0)

- Added ability to set `options.checked` to "mixed" in `EuiSelectable`
([elastic#6774](elastic/eui#6774))

**Bug fixes**

- Portalled components (e.g. `EuiPopover`, `EuiModal`, `EuiFlyout`) will
correctly inherit text color from its nearest `EuiThemeProvider` parent.
`<EuiText color="default">` is no longer needed.
([elastic#6775](elastic/eui#6775))

**Breaking changes**

- `EuiSelectable` no longer renders a `data-test-selected` attribute on
its list items. Use the `aria-checked` property instead
([elastic#6774](elastic/eui#6774))
- Nested `EuiThemeProvider`s now render a wrapping `<span>` element in
order to correctly set the inherited text `color` of all descendants.
`<EuiText color="default">` is no longer needed.
([elastic#6775](elastic/eui#6775))

---------

Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Constance Chen <constance.chen@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jpgil @devoncrouse @spalger