[bug] COmpressHandler is broken

[max202021]

Everything was working till yesterday. it starts failing today. I am using the google cloud build , so it takes the latest mux from repo. Local mux which is old is working fine.

I am using the compress handler like this

http.ListenAndServe(":8080", handlers.CompressHandler(r))

and serving static files.

but all the browser are not able to read the content. I tested with Mozzila , chrome and Microsoft EDGE.

Following error is shown

When I use without compress , things works but now my server slow without compression

http.ListenAndServe(":8080",r)

[elithrar]

Can you please share the request & response headers for the failed connection?

The relevant part of your application code will help too.

We merged #187 last night but unless your site is doing something with the Upgrade header, there should have been no change.

[max202021]

MuxCompression.zip

This zip has sammple web app , golang code with gorilla mux and dockerfile.

I am changing the X-Frame-Options header, but this is required to avoid the clickjacking. Code is available in zip

No header information is shown when this error occurs. please find screenshot below.

[elithrar]

Can you please provide your (minimal) code in code blocks here? I would prefer not to download a zip file.

e.g. like this. 

[max202021]

This is the server code

package main

import (
	"fmt"
	"net/http"
	"os"
	"path/filepath"

	"github.com/gorilla/handlers"
)

func main() {
	var files []string

	root := "/opt"
	err := filepath.Walk(root, func(path string, info os.FileInfo, err error) error {
		files = append(files, path)
		return nil
	})
	if err != nil {
		panic(err)
	}
	for _, file := range files {
		fmt.Println(file)
	}

	r := http.NewServeMux()
	changeHeaderThenServe := func(h http.Handler) http.HandlerFunc {
		return func(w http.ResponseWriter, r *http.Request) {
			// Set some header.
			w.Header().Add("X-Frame-Options", "DENY")
			// Serve with the actual handler.
			h.ServeHTTP(w, r)
		}
	}

	r.Handle("/", changeHeaderThenServe(http.FileServer(http.Dir("/opt/web"))))

	http.ListenAndServe(":9000", handlers.CompressHandler(r))

}

[max202021]

It is serving one index.html and minified javascript which is of size 38k lines , so cant copy it here
you can create a flutter project and then build using flutter build web --release to get that index and javascript file

https://flutter.dev/docs/get-started/web

[max202021]

@elithrar Are you able to replicate the issue? Is there any workaround?

[andig]

Same problem here. After upgrading handlers 1.4.2 to 1.5.0 my application is broken. Local requests are working fine (no gzip), all gzip requests are broken (appear red in Safari). In that sense, 1.5.0 is broken!

This is the Safari Request:

GET /js/app.js HTTP/1.1
Cookie: grafana_session=1ea9a20f4916d51b24b3db58bb558c52; smid=PKGUyGljsDnmBbv_lasldo0bkP1spWfn3CWT3ZFb_3OGgydosWfpIiNr13bhxT_f6BFeyH0UlLz8UhQuQ4rd9A; stay_login=1
Accept: */*
Accept-Encoding: gzip, deflate
Host: nas.fritz.box:7070
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.1.2 Safari/605.1.15
Accept-Language: en-us
Referer: http://nas.fritz.box:7070/
Connection: keep-alive
Proxy-Connection: keep-alive

Response headers look ok, but response body apparently isn't. I would suspect either #187 or #193.

Update: I'm not able to force the problem on localhost, not sure why:

curl -H "Accept-Encoding: gzip, deflate" -H "Connection: keep-alive" -H "Proxy-Connection: keep-alive" -H "Accept: */*" -H "Host: localhost" -v --compressed http://localhost:7070

...is always working fine.

[max202021]

Agree with andig, if the easy solution is not in sight then alteast handler code rollback shold happen. It takes a lot of time for user to understand what is wrong with the application. Nobody thinks that there can be anything wrong with golang or http provider. We start checking current code, old code, current deployment , old deployment , environment configuration and so on. It is very time consuming.

[andig]

@max202021 can you repro this when running on localhost? If yes we could git revert the two prs selectively and see which one is the culprit. You can always rollback yourself using 1.4.2 for time being.

[max202021]

I am able to reproduce this locally. When "github.com/felixge/httpsnoop" added to compressor.go it starts failing. Please refer screenshot below.

for now using localcopy. But not able to use the cloud build since it always refer to the latest gorilla handler code.

[andig]

How do you execute your test to repro?

[andig]

/cc @muirdm @elithrar

[max202021]

@andig , Remove the old gorilla Handler from local gopath . ( Take Backup ) I have attached a zip file which has dummy website and server.go with Dockerfile which uses gorilla mux. when we run that it will take the latest gorilla mux and then you can replicate the issue

MuxCompression.zip

[max202021]

@andig use firefox browser . it will show content encoding error.

[zeisss]

We are running the same problem in our setup where we run the testapp behind nginx. The endpoint that broke is also the net/http file server. Reproducable locally by just using go run . and with curl.

• curl --compress <url> fails (no content)
• curl <url> which works

[elithrar]

This should be addressed in #197 thanks to @muirdm.

Please pull from head until I can cut a fix later this week.

[andig]

@elithrar it would be good to release this asap. In my case I had dependabot running and upgraded when the pr was opened (stupid me) bit I think it would be good to release before more users might get bitten.

[yangjuncode]

same problem and several hour lost, please release 1.5.1

[max202021]

@yangjuncode How did you find that it is issue with handler and not anything else?

[yangjuncode]

@max202021

1. tester report failed, we don't know why since we have changed lots of our code.
2. our http server serve http and https, https run fine,only http can't open in browser, we run wget with http url and the file seems fine.
3. we check network, reboot browser,reboot server, downgrade go1.15 to 1.14,1.13, nothing help.
4. we rollback version until a go mod update of handlers from 1.4.2 to 1.5.0 thats several days ago

[andig]

I can only emphasize the need for a release. Its really urgent as it breaks production systems (and has just done again) ...

[elithrar]

Please pin to the latest commit hash. I do not have time to make a release this week. e.g. go get github.com/gorilla/handlers@abc123

[andig]

I've put a warning on r/golang to help spread the word.

@elithrar if I can help with releasing (looks like tagging + cutting a github release) please let me know if I can support. I've also setup a couple of projects with Travis CI and golang releaser and could provide the steps here (requires org access though).

[elithrar]

v1.5.1 is a tagged release that includes the fixes in d453eff.