allow proxy-rewrite plugin to rewrite x-forwarded-port request header

[eastearth]

Issue description

If the upstream service through the APISIX proxy returns 302 redirect, the client redirected the port to the 9080 port of apisix,

For example, client access http://a.com:80/，

Normally, it would be redirected to the http://a.com:80/login

But it’s actually redirected to the http://a.com:9080/login

Environment

• apisix version: 2.7.0(helm chart installed,and chart version is apisix-0.3,6)

• apisixroute:

apiVersion: apisix.apache.org/v2beta1
kind: ApisixRoute
metadata:
  name: jenkins
  namespace: jenkins
spec:
  http:
  - name: jenkins
    match:
      hosts:
      - eks-jenkins-test.invik.com
      paths:
      - /*
    backend:
      serviceName: jenkins2
      servicePort: 8080
    plugins:
    - name: ip-restriction
      enable: true
      config:
        whitelist:
        - 39.106.1.281/32
        - 192.168.0.0/16

Service:

apiVersion: v1
kind: Service
metadata:
  name: jenkins2
  namespace: jenkins
spec:
  ports:
  - name: web
    port: 8080
    protocol: TCP
    targetPort: web
  - name: agent
    port: 50000
    protocol: TCP
    targetPort: agent
  selector:
    app: jenkins2
  sessionAffinity: None
  type: ClusterIP

Steps to reproduce

1.client access http://a.com:80/，

2.Normally, upstream response 302 to the http://a.com:80/login

3.But it’s actually redirected to the http://a.com:9080/login

Actual result

3.But it’s actually redirected to the http://a.com:9080/login
and the browser is waitting

Error log

the browser return:
无法访问此网站eks-jenkins-test.invik.com 的响应时间过长。

Expected result

No response

[tokers]

What's the result of requesting to the backend directly?

[eastearth]

What's the result of requesting to the backend directly?

it is normal，can redirect to the login api and logged in.

[chloe88]

the response is not processed by the gateway (apisix).

See the nginx conf

the following x-forwarded-* headers is to send to upstream server

        set $var_x_forwarded_for        $remote_addr;
        set $var_x_forwarded_proto      $scheme;
        set $var_x_forwarded_host       $host;
        set $var_x_forwarded_port       $server_port;

        if ($http_x_forwarded_for != "") {
            set $var_x_forwarded_for "${http_x_forwarded_for}, ${realip_remote_addr}";
        }
        if ($http_x_forwarded_proto != "") {
            set $var_x_forwarded_proto $http_x_forwarded_proto;
        }
        if ($http_x_forwarded_host != "") {
            set $var_x_forwarded_host $http_x_forwarded_host;
        }
        if ($http_x_forwarded_port != "") {
            set $var_x_forwarded_port $http_x_forwarded_port;
        }

        proxy_set_header   X-Forwarded-For      $var_x_forwarded_for;
        proxy_set_header   X-Forwarded-Proto    $var_x_forwarded_proto;
        proxy_set_header   X-Forwarded-Host     $var_x_forwarded_host;
        proxy_set_header   X-Forwarded-Port     $var_x_forwarded_port;

so you can add some header to the previous proxy. set the X-Forwarded-Port to 80 to resolve it

[tokers]

the response is not processed by the gateway (apisix).

See the nginx conf

the following x-forwarded-* headers is to send to upstream server

        set $var_x_forwarded_for        $remote_addr;
        set $var_x_forwarded_proto      $scheme;
        set $var_x_forwarded_host       $host;
        set $var_x_forwarded_port       $server_port;

        if ($http_x_forwarded_for != "") {
            set $var_x_forwarded_for "${http_x_forwarded_for}, ${realip_remote_addr}";
        }
        if ($http_x_forwarded_proto != "") {
            set $var_x_forwarded_proto $http_x_forwarded_proto;
        }
        if ($http_x_forwarded_host != "") {
            set $var_x_forwarded_host $http_x_forwarded_host;
        }
        if ($http_x_forwarded_port != "") {
            set $var_x_forwarded_port $http_x_forwarded_port;
        }

        proxy_set_header   X-Forwarded-For      $var_x_forwarded_for;
        proxy_set_header   X-Forwarded-Proto    $var_x_forwarded_proto;
        proxy_set_header   X-Forwarded-Host     $var_x_forwarded_host;
        proxy_set_header   X-Forwarded-Port     $var_x_forwarded_port;

so you can add some header to the previous proxy. set the X-Forwarded-Port to 80 to resolve it

Good job, let see whether it can help @eastearth to solve his problems. If so, I think we need to add a FAQ entry to record this.

[eastearth]

the response is not processed by the gateway (apisix).

See the nginx conf

the following x-forwarded-* headers is to send to upstream server

        set $var_x_forwarded_for        $remote_addr;
        set $var_x_forwarded_proto      $scheme;
        set $var_x_forwarded_host       $host;
        set $var_x_forwarded_port       $server_port;

        if ($http_x_forwarded_for != "") {
            set $var_x_forwarded_for "${http_x_forwarded_for}, ${realip_remote_addr}";
        }
        if ($http_x_forwarded_proto != "") {
            set $var_x_forwarded_proto $http_x_forwarded_proto;
        }
        if ($http_x_forwarded_host != "") {
            set $var_x_forwarded_host $http_x_forwarded_host;
        }
        if ($http_x_forwarded_port != "") {
            set $var_x_forwarded_port $http_x_forwarded_port;
        }

        proxy_set_header   X-Forwarded-For      $var_x_forwarded_for;
        proxy_set_header   X-Forwarded-Proto    $var_x_forwarded_proto;
        proxy_set_header   X-Forwarded-Host     $var_x_forwarded_host;
        proxy_set_header   X-Forwarded-Port     $var_x_forwarded_port;

so you can add some header to the previous proxy. set the X-Forwarded-Port to 80 to resolve it

my previous proxy is Layer 4 proxy,can not add headers,and I think this is a bug,cant fix it?

[eastearth]

Is it possible to specify X-Forwarded-Port by configuring configmap? The corresponding configuration is not found. Where should I configure it? For example, which parameter should be set for this configuration?

nginx_config:                     # config for render the template to genarate nginx.conf
  error_log: "/dev/stderr"
  error_log_level: "warn"         # warn,error
  worker_rlimit_nofile: 20480     # the number of files a worker process can open, should be larger than worker_connections
  event:
    worker_connections: 10620
  http:
    access_log: "/dev/stdout"
    keepalive_timeout: 60s         # timeout during which a keep-alive client connection will stay open on the server side.
    client_header_timeout: 60s     # timeout for reading client request header, then 408 (Request Time-out) error is returned to the client
    client_body_timeout: 60s       # timeout for reading client request body, then 408 (Request Time-out) error is returned to the client
    send_timeout: 10s              # timeout for transmitting a response to the client.then the connection is closed
    underscores_in_headers: "on"   # default enables the use of underscores in client request header fields
    real_ip_header: "X-Forwarded-For"    # http://nginx.org/en/docs/http/ngx_http_realip_module.html#real_ip_header
    X-Forwarded-Port
    real_ip_from:                  # http://nginx.org/en/docs/http/ngx_http_realip_module.html#set_real_ip_from
      - 127.0.0.1
      - 192.168.0.0/16
      - 10.0.0.0/8
      - 172.16.0.0/12
      - 'unix:'

[nic-chen]

@eastearth

You could take a test for it.

Does your APISIX use port 9080? What is the relationship between port 80 and 9080?

[chloe88]

@eastearth Can your layer 4 proxy be configured in a transparent manner?

[chloe88]

@eastearth tell me wechat

[eastearth]

@eastearth tell me wechat

jxd1075943283

[tokers]

是否能通过配置configmap来指定X-Forwarded-Port呢，没有找到对应的配置，该在哪里配置呢，例如这个配置，应该设置哪一个参数呢？

nginx_config:                     # config for render the template to genarate nginx.conf
  error_log: "/dev/stderr"
  error_log_level: "warn"         # warn,error
  worker_rlimit_nofile: 20480     # the number of files a worker process can open, should be larger than worker_connections
  event:
    worker_connections: 10620
  http:
    access_log: "/dev/stdout"
    keepalive_timeout: 60s         # timeout during which a keep-alive client connection will stay open on the server side.
    client_header_timeout: 60s     # timeout for reading client request header, then 408 (Request Time-out) error is returned to the client
    client_body_timeout: 60s       # timeout for reading client request body, then 408 (Request Time-out) error is returned to the client
    send_timeout: 10s              # timeout for transmitting a response to the client.then the connection is closed
    underscores_in_headers: "on"   # default enables the use of underscores in client request header fields
    real_ip_header: "X-Forwarded-For"    # http://nginx.org/en/docs/http/ngx_http_realip_module.html#real_ip_header
    X-Forwarded-Port
    real_ip_from:                  # http://nginx.org/en/docs/http/ngx_http_realip_module.html#set_real_ip_from
      - 127.0.0.1
      - 192.168.0.0/16
      - 10.0.0.0/8
      - 172.16.0.0/12
      - 'unix:'

How did you install APISIX?

See https://github.com/apache/apisix/blob/master/apisix/cli/ngx_tpl.lua, currently, the related items are not configurable.

[eastearth]

是否能通过配置configmap来指定X-Forwarded-Port呢，没有找到对应的配置，该在哪里配置呢，例如这个配置，应该设置哪一个参数呢？

nginx_config:                     # config for render the template to genarate nginx.conf
  error_log: "/dev/stderr"
  error_log_level: "warn"         # warn,error
  worker_rlimit_nofile: 20480     # the number of files a worker process can open, should be larger than worker_connections
  event:
    worker_connections: 10620
  http:
    access_log: "/dev/stdout"
    keepalive_timeout: 60s         # timeout during which a keep-alive client connection will stay open on the server side.
    client_header_timeout: 60s     # timeout for reading client request header, then 408 (Request Time-out) error is returned to the client
    client_body_timeout: 60s       # timeout for reading client request body, then 408 (Request Time-out) error is returned to the client
    send_timeout: 10s              # timeout for transmitting a response to the client.then the connection is closed
    underscores_in_headers: "on"   # default enables the use of underscores in client request header fields
    real_ip_header: "X-Forwarded-For"    # http://nginx.org/en/docs/http/ngx_http_realip_module.html#real_ip_header
    X-Forwarded-Port
    real_ip_from:                  # http://nginx.org/en/docs/http/ngx_http_realip_module.html#set_real_ip_from
      - 127.0.0.1
      - 192.168.0.0/16
      - 10.0.0.0/8
      - 172.16.0.0/12
      - 'unix:'

How did you install APISIX?

See https://github.com/apache/apisix/blob/master/apisix/cli/ngx_tpl.lua, currently, the related items are not configurable.

actually,I install apisix by helm in k8s,
now i configure apisix deployment like this,it is working now,
" lifecycle:
postStart:
exec:
command:
- /bin/sh
- -c
- sed -i 's/proxy_set_header X-Forwarded-Port $var_x_forwarded_port/#proxy_set_header
X-Forwarded-Port $var_x_forwarded_port/g'
/usr/local/apisix/conf/nginx.conf
"

but i think this still need a configurable item,
because according to the introduction of k8s spec.containers.lifecycle.poststart,

The operation performed by poststart does not necessarily complete before the main program entry enterpoint starts

this is k8s official document introduction:
"Kubernetes sends the postStart event immediately after the Container is created. There is no guarantee, however, that the postStart handler is called before the Container's entrypoint is called. The postStart handler runs asynchronously relative to the Container's code, but Kubernetes' management of the container blocks until the postStart handler completes. The Container's status is not set to RUNNING until the postStart handler completes.

"

[tokers]

是否能通过配置configmap来指定X-Forwarded-Port呢，没有找到对应的配置，该在哪里配置呢，例如这个配置，应该设置哪一个参数呢？

nginx_config:                     # config for render the template to genarate nginx.conf
  error_log: "/dev/stderr"
  error_log_level: "warn"         # warn,error
  worker_rlimit_nofile: 20480     # the number of files a worker process can open, should be larger than worker_connections
  event:
    worker_connections: 10620
  http:
    access_log: "/dev/stdout"
    keepalive_timeout: 60s         # timeout during which a keep-alive client connection will stay open on the server side.
    client_header_timeout: 60s     # timeout for reading client request header, then 408 (Request Time-out) error is returned to the client
    client_body_timeout: 60s       # timeout for reading client request body, then 408 (Request Time-out) error is returned to the client
    send_timeout: 10s              # timeout for transmitting a response to the client.then the connection is closed
    underscores_in_headers: "on"   # default enables the use of underscores in client request header fields
    real_ip_header: "X-Forwarded-For"    # http://nginx.org/en/docs/http/ngx_http_realip_module.html#real_ip_header
    X-Forwarded-Port
    real_ip_from:                  # http://nginx.org/en/docs/http/ngx_http_realip_module.html#set_real_ip_from
      - 127.0.0.1
      - 192.168.0.0/16
      - 10.0.0.0/8
      - 172.16.0.0/12
      - 'unix:'

How did you install APISIX?
See https://github.com/apache/apisix/blob/master/apisix/cli/ngx_tpl.lua, currently, the related items are not configurable.

actually,I install apisix by helm in k8s,
now i configure apisix deployment like this,it is working now,
" lifecycle: postStart: exec: command: - /bin/sh - -c - sed -i 's/proxy_set_header X-Forwarded-Port $var_x_forwarded_port/#proxy_set_header X-Forwarded-Port $var_x_forwarded_port/g' /usr/local/apisix/conf/nginx.conf "

but i think this still need a configurable item,
because according to the introduction of k8s spec.containers.lifecycle.poststart,

The operation performed by poststart does not necessarily complete before the main program entry enterpoint starts

this is k8s official document introduction:
"Kubernetes sends the postStart event immediately after the Container is created. There is no guarantee, however, that the postStart handler is called before the Container's entrypoint is called. The postStart handler runs asynchronously relative to the Container's code, but Kubernetes' management of the container blocks until the postStart handler completes. The Container's status is not set to RUNNING until the postStart handler completes.

"

That's right, the postStart hook is not a reliable way to do this. Would you like to submit a PR to let these X-Forwared-* items be configurable? Thanks!

[kika-jiaxudong]

是否能通过配置configmap来指定X-Forwarded-Port呢，没有找到对应的配置，该在哪里配置呢，例如这个配置，应该设置哪一个参数呢？

nginx_config:                     # config for render the template to genarate nginx.conf
  error_log: "/dev/stderr"
  error_log_level: "warn"         # warn,error
  worker_rlimit_nofile: 20480     # the number of files a worker process can open, should be larger than worker_connections
  event:
    worker_connections: 10620
  http:
    access_log: "/dev/stdout"
    keepalive_timeout: 60s         # timeout during which a keep-alive client connection will stay open on the server side.
    client_header_timeout: 60s     # timeout for reading client request header, then 408 (Request Time-out) error is returned to the client
    client_body_timeout: 60s       # timeout for reading client request body, then 408 (Request Time-out) error is returned to the client
    send_timeout: 10s              # timeout for transmitting a response to the client.then the connection is closed
    underscores_in_headers: "on"   # default enables the use of underscores in client request header fields
    real_ip_header: "X-Forwarded-For"    # http://nginx.org/en/docs/http/ngx_http_realip_module.html#real_ip_header
    X-Forwarded-Port
    real_ip_from:                  # http://nginx.org/en/docs/http/ngx_http_realip_module.html#set_real_ip_from
      - 127.0.0.1
      - 192.168.0.0/16
      - 10.0.0.0/8
      - 172.16.0.0/12
      - 'unix:'

How did you install APISIX?
See https://github.com/apache/apisix/blob/master/apisix/cli/ngx_tpl.lua, currently, the related items are not configurable.

actually,I install apisix by helm in k8s,
now i configure apisix deployment like this,it is working now,
" lifecycle: postStart: exec: command: - /bin/sh - -c - sed -i 's/proxy_set_header X-Forwarded-Port $var_x_forwarded_port/#proxy_set_header X-Forwarded-Port $var_x_forwarded_port/g' /usr/local/apisix/conf/nginx.conf "
but i think this still need a configurable item,
because according to the introduction of k8s spec.containers.lifecycle.poststart,
The operation performed by poststart does not necessarily complete before the main program entry enterpoint starts
this is k8s official document introduction:
"Kubernetes sends the postStart event immediately after the Container is created. There is no guarantee, however, that the postStart handler is called before the Container's entrypoint is called. The postStart handler runs asynchronously relative to the Container's code, but Kubernetes' management of the container blocks until the postStart handler completes. The Container's status is not set to RUNNING until the postStart handler completes.
"

That's right, the postStart hook is not a reliable way to do this. Would you like to submit a PR to let these X-Forwared-* items be configurable? Thanks!

Yes,I would very much want to let these items to be configurable,

What should I do?

[tokers]

是否能通过配置configmap来指定X-Forwarded-Port呢，没有找到对应的配置，该在哪里配置呢，例如这个配置，应该设置哪一个参数呢？

nginx_config:                     # config for render the template to genarate nginx.conf
  error_log: "/dev/stderr"
  error_log_level: "warn"         # warn,error
  worker_rlimit_nofile: 20480     # the number of files a worker process can open, should be larger than worker_connections
  event:
    worker_connections: 10620
  http:
    access_log: "/dev/stdout"
    keepalive_timeout: 60s         # timeout during which a keep-alive client connection will stay open on the server side.
    client_header_timeout: 60s     # timeout for reading client request header, then 408 (Request Time-out) error is returned to the client
    client_body_timeout: 60s       # timeout for reading client request body, then 408 (Request Time-out) error is returned to the client
    send_timeout: 10s              # timeout for transmitting a response to the client.then the connection is closed
    underscores_in_headers: "on"   # default enables the use of underscores in client request header fields
    real_ip_header: "X-Forwarded-For"    # http://nginx.org/en/docs/http/ngx_http_realip_module.html#real_ip_header
    X-Forwarded-Port
    real_ip_from:                  # http://nginx.org/en/docs/http/ngx_http_realip_module.html#set_real_ip_from
      - 127.0.0.1
      - 192.168.0.0/16
      - 10.0.0.0/8
      - 172.16.0.0/12
      - 'unix:'

How did you install APISIX?
See https://github.com/apache/apisix/blob/master/apisix/cli/ngx_tpl.lua, currently, the related items are not configurable.

actually,I install apisix by helm in k8s,
now i configure apisix deployment like this,it is working now,
" lifecycle: postStart: exec: command: - /bin/sh - -c - sed -i 's/proxy_set_header X-Forwarded-Port $var_x_forwarded_port/#proxy_set_header X-Forwarded-Port $var_x_forwarded_port/g' /usr/local/apisix/conf/nginx.conf "
but i think this still need a configurable item,
because according to the introduction of k8s spec.containers.lifecycle.poststart,
The operation performed by poststart does not necessarily complete before the main program entry enterpoint starts
this is k8s official document introduction:
"Kubernetes sends the postStart event immediately after the Container is created. There is no guarantee, however, that the postStart handler is called before the Container's entrypoint is called. The postStart handler runs asynchronously relative to the Container's code, but Kubernetes' management of the container blocks until the postStart handler completes. The Container's status is not set to RUNNING until the postStart handler completes.
"

That's right, the postStart hook is not a reliable way to do this. Would you like to submit a PR to let these X-Forwared-* items be configurable? Thanks!

Yes,I would very much want to let these items to be configurable,

What should I do?

The ngx_tpl.lua is a template to customize the nginx.conf file, so just let these items to be templated. You can see more in apisix/cli/ directory.

[kika-jiaxudong]

是否能通过配置configmap来指定X-Forwarded-Port呢，没有找到对应的配置，该在哪里配置呢，例如这个配置，应该设置哪一个参数呢？

nginx_config:                     # config for render the template to genarate nginx.conf
  error_log: "/dev/stderr"
  error_log_level: "warn"         # warn,error
  worker_rlimit_nofile: 20480     # the number of files a worker process can open, should be larger than worker_connections
  event:
    worker_connections: 10620
  http:
    access_log: "/dev/stdout"
    keepalive_timeout: 60s         # timeout during which a keep-alive client connection will stay open on the server side.
    client_header_timeout: 60s     # timeout for reading client request header, then 408 (Request Time-out) error is returned to the client
    client_body_timeout: 60s       # timeout for reading client request body, then 408 (Request Time-out) error is returned to the client
    send_timeout: 10s              # timeout for transmitting a response to the client.then the connection is closed
    underscores_in_headers: "on"   # default enables the use of underscores in client request header fields
    real_ip_header: "X-Forwarded-For"    # http://nginx.org/en/docs/http/ngx_http_realip_module.html#real_ip_header
    X-Forwarded-Port
    real_ip_from:                  # http://nginx.org/en/docs/http/ngx_http_realip_module.html#set_real_ip_from
      - 127.0.0.1
      - 192.168.0.0/16
      - 10.0.0.0/8
      - 172.16.0.0/12
      - 'unix:'

How did you install APISIX?
See https://github.com/apache/apisix/blob/master/apisix/cli/ngx_tpl.lua, currently, the related items are not configurable.

actually,I install apisix by helm in k8s,
now i configure apisix deployment like this,it is working now,
" lifecycle: postStart: exec: command: - /bin/sh - -c - sed -i 's/proxy_set_header X-Forwarded-Port $var_x_forwarded_port/#proxy_set_header X-Forwarded-Port $var_x_forwarded_port/g' /usr/local/apisix/conf/nginx.conf "
but i think this still need a configurable item,
because according to the introduction of k8s spec.containers.lifecycle.poststart,
The operation performed by poststart does not necessarily complete before the main program entry enterpoint starts
this is k8s official document introduction:
"Kubernetes sends the postStart event immediately after the Container is created. There is no guarantee, however, that the postStart handler is called before the Container's entrypoint is called. The postStart handler runs asynchronously relative to the Container's code, but Kubernetes' management of the container blocks until the postStart handler completes. The Container's status is not set to RUNNING until the postStart handler completes.
"

That's right, the postStart hook is not a reliable way to do this. Would you like to submit a PR to let these X-Forwared-* items be configurable? Thanks!

Yes,I would very much want to let these items to be configurable,
What should I do?

The ngx_tpl.lua is a template to customize the nginx.conf file, so just let these items to be templated. You can see more in apisix/cli/ directory.

Yes,I see,you mean that I modify ngx_tpl.lua to be configurable myself and change the X-Forwarded-Port in k8s configmap?

Actually, my lua is not good,I can't do it myself.

Can you help?

[kika-jiaxudong]

I have a request, can someone help me to change the x-forward-* in the https://github.com/apache/apisix/blob/master/apisix/cli/ngx_tpl.lua to configurable?

[kika-jiaxudong]

Especially the X-Forwarded-Port

[kika-jiaxudong]

@eastearth

You could take a test for it.

Does your APISIX use port 9080? What is the relationship between port 80 and 9080?

eg. I request this url http://eks-jenkins-test.invik.com:80

when jenkins response 302 to browser,

and the browser request http://eks-jenkins-test.invik.com:9080/login

The port of apisix is 9080,

and 80 is the port of aws LB layer 4

[tokers]

I have a request, can someone help me to change the x-forward-* in the https://github.com/apache/apisix/blob/master/apisix/cli/ngx_tpl.lua to configurable?

As an example, you can see how the keepalive_* was set to be configurable, see https://github.com/apache/apisix/blob/master/apisix/cli/ngx_tpl.lua#L328-L330 for more details.

[nic-chen]

@eastearth

you can use proxy-rewrite plugin to set the header. see:

https://github.com/apache/apisix/blob/master/docs/en/latest/plugins/proxy-rewrite.md

[z-yuxie]

@eastearth @nic-chen @tokers
Hello everyone, has this issue been resolved? I encountered the same problem with

I checked the instructions about reverse proxy in jenkins’ own documentation.
↓Screenshot of the description of this content in the above document

According to the description, Jenkins will redirect itself through the 302 status code in some steps, and return the address to be redirected in the Location of the http response header. However, currently, the port of the Localtion returned by Jenkins every time is apisix. There are two ways to handle the port, one of which requires us to set the following parameters in the request header:
X-Forwarded-Host: jenkins.xxx.xxx (the domain name of its own jenkins)
X-Forwarded-Port: 80
X-Forwarded-Proto: https

Regarding the way to set the request header, this is the method I have tried, but it has no effect. Jenkins still returns the redirection address with port 9080:

1. Set the request header through the "Request Header Rewrite" function in the "Routes" of the interface
   
   ↓This is how the routing configuration formed in this way looks like in the "Data Editor"
   
2. Set the request header through the "proxy-rewrite" plug-in in the "service" of the interface
   
   ↓This is how the service configuration formed in this way looks like in the "Data Editor"
   

Then because I am not sure whether this is a problem with Jenkins itself or a problem with the apisix plug-in, I now want to try another method of modifying the Location in the response header. Is there any way that I can modify the Location in the response header? Do you perform regular matching and modify it? The desired effect is roughly this:
response header:
Location: https://jenkins.xxx.xxx:9080/*
↓
response header:
Location: https://jenkins.xxx.xxx/*

I know that there is a "response-rewrite" plug-in that can modify the Location mentioned above, but because the plug-in provides The instructions of How should I configure it? Thank you.

In addition, one method that I think is probably effective is to open port 9080, and then reconfigure a route to identify requests from port 9080, and redirect them to port 80, but To be honest, I don't really want to open a new port unless I have to, and this way of handling it will add one more http request, which doesn't always feel good.

[nic-chen]

hi @z-yuxie
Are there other routes configured in your APISIX? Need to confirm whether the requests hit other routes.

[z-yuxie]

@nic-chen 您好，我有配置别的路由，但是jenkins能命中的路由仅有这一个，因为我的每个路由都是针对独立的域名配置的

My routing strategy is based on the domain name. Although there are multiple routes configured, Jenkins will only hit its own route and not the others.

[tokers]

@z-yuxie Try to use the serverless-functions plugins to rewrite headers in the header_filter phase, you can inject your own logics there.

[nic-chen]

@z-yuxie
After testing it, I found that X-Forwarded-Port cannot be rewritten directly.
It is recommended that jenkins and APISIX use the same port number, and there is no such problem.

Or use a temporary solution, comment out these lines https://github.com/apache/apisix/blob/master/apisix/cli/ngx_tpl.lua#L612-L613, and restart APISIX.

[haozileung]

I have encountered exactly the same problem here, using k8s + apisix ingress + jenkins.
Changing the port is not a good approach and modified the source code could work but annoying.

[zsjinwei]

@z-yuxie Try to use the serverless-functions plugins to rewrite headers in the header_filter phase, you can inject your own logics there.

I have this issue too, could you show me how to rewrite using serverless-functions?

[return200]

@eastearth @nic-chen @tokers 您们好，请问这个问题有被解决么？我遇到了与之相同的问题

我去查看了jenkins自己的文档中关于反向代理的说明， ↓上述文档中关于此内容描述的截图

根据其中的描述，是jenkins在某些步骤会自己通过302状态码进行重定向，并在http响应头的Location返回要重定向到的地址，但是目前jenkins方面每次返回的Localtion的端口都是apisix的端口，有两种处理方式，其中一种是需要我们在请求头中设置以下参数： X-Forwarded-Host: jenkins.xxx.xxx（自身jenkins的域名） X-Forwarded-Port: 80 X-Forwarded-Proto: https

关于设置请求头的这种方式，这是我尝试过的方法，但都没有效果，jenkins依然返回了带有9080端口的重定向地址： 1、通过界面的“路由”(routes)中的“请求头重写”功能设置请求头 ↓这是通过该方式形成的路由配置在“数据编辑器”中展示的样子 2、通过界面的“服务”(service)中的“proxy-rewrite”插件设置请求头 ↓这是通过该方式形成的服务配置在“数据编辑器”中展示的样子

然后因为我不太确定这是jenkins自身的问题还是apisix插件的问题，于是我现在想要试试另一种修改响应头中的Location的方法，请问有什么方式能让我对响应头中的Location进行正则匹配，并进行修改的么？想要的效果大概是这样的： response header: Location：https://jenkins.xxx.xxx:9080/* ↓ response header: Location：https://jenkins.xxx.xxx/*

我知道有个“response-rewrite”插件可以对上面所说的Location进行修改，但因为该插件提供的说明中并没有对其进行较多的描述，也没有提供类似的改写范例，我并不确定他是否能做到我上面说的这种基于正则的改写，如果可以的话，能否给我说下我应该怎么去配置，谢谢。

除此之外，我目前觉得大概能有效的一种方法是，我将9080端口也开放出去，然后再重新配置一个路由去识别来自9080端口的请求，并将其重新重定向到80端口，但老实说不到万不得已，我并不是太想新开放一个端口，而且这样的处理方式会让多增加一次http的请求，总感觉不太美好。

你字多，就回复你吧。你域名解析到哪里了，是 nginx 还是 slb？如果是 Nginx 加一个 proxy_set_header X-Forwarded-Port 443; slb 应该有单独的选项可以设置。

[return200]

Issue description

If the upstream service through the APISIX proxy returns 302 redirect, the client redirected the port to the 9080 port of apisix,

For example, client access http://a.com:80/，

Normally, it would be redirected to the http://a.com:80/login

But it’s actually redirected to the http://a.com:9080/login

Environment

• apisix version: 2.7.0(helm chart installed,and chart version is apisix-0.3,6)
• apisixroute:

apiVersion: apisix.apache.org/v2beta1 kind: ApisixRoute metadata: name: jenkins namespace: jenkins spec: http:

• name: jenkins
  match:
  hosts:

  • eks-jenkins-test.invik.com
    paths:

  • /*
    backend:
    serviceName: jenkins2
    servicePort: 8080
    plugins:

  • name: ip-restriction
    enable: true
    config:
    whitelist:

    • 39.106.1.281/32
    • 192.168.0.0/16

Service:

apiVersion: v1 kind: Service metadata: name: jenkins2 namespace: jenkins spec: ports:

• name: web
  port: 8080
  protocol: TCP
  targetPort: web
• name: agent
  port: 50000
  protocol: TCP
  targetPort: agent
  selector:
  app: jenkins2
  sessionAffinity: None
  type: ClusterIP

Steps to reproduce

1.client access http://a.com:80/，

2.Normally, upstream response 302 to the http://a.com:80/login

3.But it’s actually redirected to the http://a.com:9080/login

Actual result

3.But it’s actually redirected to the http://a.com:9080/login and the browser is waitting

Error log

the browser return: 无法访问此网站eks-jenkins-test.invik.com 的响应时间过长。

Expected result

No response

where does your domain a.com resolve to? Add proxy_set_header X-Forwarded-Port $server_port; if nginx; if it's a slb/alb, there should have a configuration to add the header X-Forwarded-Port.

[zhoutaoo]

I have the same problem, using k8s + apisix ingress + jenkins.

[zhoutaoo]

I have the same problem, using k8s + apisix ingress + jenkins.

{
  "uris": [
    "/",
    "/*"
  ],
  "name": "ingress_devops-jenkins-base.xxxxx.com_/",
  "desc": "Created by apisix-ingress-controller, DO NOT modify it manually",
  "host": "devops-jenkins-base.xxxxx.com",
  "plugins": {
    "response-rewrite": {
      "headers": {
        "Location": "//devops-jenkins-base.xxxxx.com"
      },
      "vars": [
        [
          "status",
          "==",
          302
        ]
      ]
    }
  },
  "upstream_id": "767e1afc",
  "labels": {
    "managed-by": "apisix-ingress-controller"
  },
  "status": 1
}

[haozileung]

For anyone needs workaround:
Apply plugin serverless-pre-function to the route config
Sample code:

  "plugins": {
        "serverless-pre-function": {
            "phase": "rewrite",
            "functions": [
                "return function(conf, ctx) ngx.var.var_x_forwarded_port = 443 end"
            ]
        }
    },

[spacewander]

Maybe we can allow proxy-rewrite plugin to rewrite x_forwarded_port. What about everyone's opinion?

[soulbird]

Maybe we can allow proxy-rewrite plugin to rewrite x_forwarded_port. What about everyone's opinion?

LGTM

[tzssangglass]

Maybe we can allow proxy-rewrite plugin to rewrite x_forwarded_port. What about everyone's opinion?

agree +1

[kwanhur]

Maybe we can allow proxy-rewrite plugin to rewrite x_forwarded_port. What about everyone's opinion?

What's the action rewrite meaning? Like add update delete ? It seems proxy-rewrite plugin can do it now, see the doc.

[spacewander]

Currently this header is generated via var_x_forwarded_port:

apisix/apisix/cli/ngx_tpl.lua

Line 683 in d7e49c9

proxy_set_header X-Forwarded-Port $var_x_forwarded_port;

[spacewander]

We can apply the same handling like:

apisix/apisix/init.lua

Line 228 in d7e49c9

local hdr = core.request.header(api_ctx, "X-Forwarded-Proto")

[kwanhur]

Maybe I still don't get the accurate meanings, I guess that take plugin proxy-rewrite to rewrite attributes x_forwarded_for x_forwarded_proto x_forwarded_host x_forwarded_port like the host scheme attributes.

If these extensible headers supported one by one, it seems not a good idea. Why not reuse the existed attribute headers?

apisix/docs/en/latest/plugins/proxy-rewrite.md

Line 43 in 2800a06

| headers | object | False | | | New Upstream headers. Headers are overwritten if they are already present otherwise, they are added to the present headers. To remove a header, set the header value to an empty string. The values in the header can contain Nginx variables like `$remote_addr` and `$client_addr`. |

[tzssangglass]

If these extensible headers supported one by one, it seems not a good idea. Why not reuse the existed attribute headers?

We don't need to extend the schema with new attributes
just that proxy-rewrite currently can't rewrite the x_forwarded_port in request headers and we need to support this feature.

[tokers]

Once we let proxy-rewrite plugin support modifying these headers, also take care the precedence between it and proxy_set_header.

[tzssangglass]

If proxy-rewrite can modify x-forward-port, then the following scenario occurs (to be confirmed)

1. client access http://a.com:80/

2. upstream response 302 to the http://a.com:80/login

3. it redirected to the http://a.com:80/login

My questions:

1. What port is 80? The port exposed by APISIX in the container, like -p 80:9080?
2. Does the 80 port have anything to do with Jenkins?

[hotbaby]

If Jenkins is deployed in kubernetes, you can add another Nginx container to the pod, forward the traffic to Jenkins through Nginx, and set X-Forwarded-Port to 80. The test can avoid this problem.

server {
    listen       80;
    listen  [::]:80;
    server_name  localhost;

    #access_log  /var/log/nginx/host.access.log  main;

    location / {
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Forwarded-Host $host;
        proxy_set_header X-Forwarded-Port 80;
        proxy_pass http://localhost:8080;
    }
}

[niuguy]

For anyone needs workaround: Apply plugin serverless-pre-function to the route config Sample code:

  "plugins": {
        "serverless-pre-function": {
            "phase": "rewrite",
            "functions": [
                "return function(conf, ctx) ngx.var.var_x_forwarded_port = 443 end"
            ]
        }
    },

Thank you so much bro.
btw for guys who use ingress controller

    plugins:
      - name: serverless-pre-function
        enable: true
        config:
          phase: "rewrite"
          functions: ["return function(conf, ctx) ngx.var.var_x_forwarded_port = 443 end"]

[brentmjohnson]

Another alternative is to modify the listen ports for apisix to the following:

apisix:    # universal configurations
  node_listen: 80    # APISIX listening port
  ssl:
    enable: true
      listen:
        - port: 443

Since these are privileged ports, in kubernetes if you don't want to run as root, this will require you to use the following securityContext on the pod (not container):

securityContext:
    sysctls:
      - name: net.ipv4.ip_unprivileged_port_start
        value: "1"

[acuteaura]

for anyone from the future looking to have flattened X-Forwarded-For headers with a list of explicit good IPs, this is what we ended up doing:

apiVersion: apisix.apache.org/v2
kind: ApisixGlobalRule
metadata:
  name: real-ip
spec:
  plugins:
    - name: real-ip
      enable: true
      config:
        recursive: true
        source: http_x_forwarded_for
        trusted_addresses:
          - 173.245.48.0/20
          - 103.21.244.0/22
          - 103.22.200.0/22
          - 103.31.4.0/22
          - 141.101.64.0/18
          - 108.162.192.0/18
          - 190.93.240.0/20
          - 188.114.96.0/20
          - 197.234.240.0/22
          - 198.41.128.0/17
          - 162.158.0.0/15
          - 104.16.0.0/13
          - 104.24.0.0/14
          - 172.64.0.0/13
          - 131.0.72.0/22
    - name: proxy-rewrite
      enable: true
      config:
        headers:
          remove:
            - X-Forwarded-For
          set:
            X-Forwarded-Port: 443

This parses XFF and sets $remote_addr to the first untrusted IP, then unsets XFF so it's not appended to in the nginx snippet and X-Forwarded-Port just seems to be trusted blindly.