Redactors in v0.71.1 consume too much memory

[banjoh]

Bug Description

In certain scenarios, running redactor when collector a support bundle can consume a lot of memory leading to OOM signals killing pods.

Here is are memory profiles of this happening in KOTS .

With v0.71.1

With v0.70.3

NOTE: This has not been reproduced when using the CLI. From comments below

• Checkout v0.71.1 of troubleshoot
• Build binaries make build
• Collect and redact support bundle - ./bin/support-bundle ./support-bundle.yaml --memprofile=mem.conf --redactors ./redactor.yaml with support-bundle.yaml and redactor.yaml
• Analyse memory allocations - go tool pprof -http :8888 -alloc_space mem.prof will start an http server on port 8888

Expected Behavior

Redaction code should not consume excess memory.

Steps To Reproduce

Include the commands to reproduce the issue including any output. Any information that will help us to understand the problem is useful. Feel free to paste long output into a Github gist and include the link here.

To be fleshed

Additional Context

Include the following information.

• Troubleshoot version. If you built from source, note that including the version of Go you used to build with.
• Operating system
• Operating system version
• Other details that might be helpful in diagnosing the problem

[arcolife]

For context, as first seen in testsuite while bumping troubleshoot to v0.71.1 in kots replicatedhq/kots#4031

And PR relevant to investigation #1291

Potentially helpful tooling #1301

Discussion (internal - all info is summarized above) https://replicated.slack.com/archives/C016DU6CXNE/p1694185342225949

[chris-sanders]

@banjoh did you get the images backwards?

[cbodonnell]

The higher memory consumption can also be reproduced from the CLI:

1. Modify the cmd/troubleshoot/main.go function so that it generates a memory profile (might be a better way to do this without rebuild the binary, but this is how I got it working):

func main() {
	cli.InitAndExecute()

	memProfileFile, err := os.Create("mem.prof")
	if err != nil {
		panic(err)
	}
	defer memProfileFile.Close()

	if err := pprof.WriteHeapProfile(memProfileFile); err != nil {
		panic(err)
	}
}

2. make support-bundle
3. ./bin/support-bundle ./support-bundle.yaml --redactors ./redactor.yaml with support-bundle.yaml and redactor.yaml
4. go tool pprof -http :8888 -alloc_space mem.prof will start an http server on port 8888 where you can view the details