Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug: failed to get large body #114

Closed
zdzh opened this issue Nov 14, 2022 · 6 comments
Closed

bug: failed to get large body #114

zdzh opened this issue Nov 14, 2022 · 6 comments
Assignees
Labels
bug Something isn't working

Comments

@zdzh
Copy link

zdzh commented Nov 14, 2022

Issue description

我用request.Body来获取请求的body,但是当请求请求体过大时,该函数会提示获取失败,数据被截断。

Environment

  • APISIX Go Plugin Runner's version:
    v0.3.0
  • APISIX version:
    2.15.0
  • Go version:
    go version go1.15.1 linux/amd64
  • OS (cmd: uname -a):
    Linux fefcfe0549f3 4.18.0 #1 SMP Tue Sep 27 10:33:52 CST 2022 x86_64 x86_64 x86_64 GNU/Linux

Minimal test code / Steps to reproduce the issue

构造一个足够大的body,然后在go-runner中通过req.Body获取body

What's the actual result? (including assertion message & call stack if applicable)

传入的body首先提示存入一个临时文件,然后在调用req.Body时获取body失败,提示被截断。
觉得是存入临时文件的数据没有正确传递给go-runner,但是在ext-plugin/init.lua查看body的长度好像没问题

2022/11/14 17:15:43 [warn] 85510#85510: *444 a client request body is buffered to a temporary file /usr/local/apisix/client_body_temp/0000000001, client: 10.131.115.28, server: _, request: "POST ******** HTTP/1.1", host: "127.0.0.1:9080"
2022/11/14 17:15:43 [warn] 85510#85510: *444 [lua] init.lua:319: handle_extra_info(): request body len 900938, client: 10.131.115.28, server: _, request: "POST ****** HTTP/1.1", host: "127.0.0.1:9080"
2022/11/14 17:15:43 [warn] 85514#85514: *49 [lua] init.lua:915: 2022-11-14T17:15:43.465+0800    INFO    http/request.go:390     receive rpc type: 3 data length: 900964
, context: ngx.timer
2022/11/14 17:15:43 [warn] 85514#85514: *49 [lua] init.lua:915: 2022-11-14T17:15:43.466+0800    ERROR   util/msg.go:55  read: truncated, only get the first 219260 bytes
github.com/apache/apisix-go-plugin-runner/internal/util.ReadErr
        /home/go/pkg/mod/github.com/apache/apisix-go-plugin-runner@v0.3.0/internal/util/msg.go:55
github.com/apache/apisix-go-plugin-runner/internal/http.(*Request).askExtraInfo
        /home/go/pkg/mod/github.com/apache/apisix-go-plugin-runner@v0.3.0/internal/http/request.go:394
github.com/apache/apisix-go-plugin-runner/internal/http.(*Request).Body
        /home/go/pkg/mod/github.com/apache/apisix-go-plugin-runner@v0.3.0/internal/http/request.go:183

What's the expected result?

获取body成功

@soulbird
Copy link
Contributor

I'll check it out later

@zdzh
Copy link
Author

zdzh commented Nov 17, 2022

func (r *Request) askExtraInfo(builder *flatbuffers.Builder,
	infoType ei.Info, info flatbuffers.UOffsetT) ([]byte, error) { 
       ......
	ty := header[0]
	header[0] = 0
	length := binary.BigEndian.Uint32(header)

	log.Infof("receive rpc type: %d data length: %d", ty, length)

	buf := make([]byte, length)
	n, err = c.Read(buf)
	if util.ReadErr(n, err, int(length)) {
		return nil, common.ErrConnClosed
	}
        ....
}

这里从conn读取数据的时候有可能没有读取完全,需要循环读取

	var tmp int
	for n < length {
		tmp, err = c.Read(buf[n:])
		if err != nil {
			break
		}
		n += tmp
	}

@soulbird
Copy link
Contributor

func (r *Request) askExtraInfo(builder *flatbuffers.Builder,
	infoType ei.Info, info flatbuffers.UOffsetT) ([]byte, error) { 
       ......
	ty := header[0]
	header[0] = 0
	length := binary.BigEndian.Uint32(header)

	log.Infof("receive rpc type: %d data length: %d", ty, length)

	buf := make([]byte, length)
	n, err = c.Read(buf)
	if util.ReadErr(n, err, int(length)) {
		return nil, common.ErrConnClosed
	}
        ....
}

这里从conn读取数据的时候有可能没有读取完全,需要循环读取

	var tmp int
	for n < length {
		tmp, err = c.Read(buf[n:])
		if err != nil {
			break
		}
		n += tmp
	}

This is read in a loop.

for n < length

@soulbird
Copy link
Contributor

soulbird commented Dec 9, 2022

@zdzh Looks like it's buggy, would you be interested in submiting a PR to fix it?

@An-DJ
Copy link
Contributor

An-DJ commented Dec 20, 2022

@spacewander Please assign to me.

An-DJ added a commit to An-DJ/apisix-go-plugin-runner that referenced this issue Dec 26, 2022
soulbird pushed a commit that referenced this issue Dec 27, 2022
* fix(#114): fix transfering large body failed
@An-DJ
Copy link
Contributor

An-DJ commented Dec 28, 2022

@zdzh This issue has been solved by #124 .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants