‘no such file or directory’ running coverage

[plastikfan]

A common problem I come across when I run go tool cover, are coverage failures due to coverage location not being present. I have found that to resolve this issue, I need to create coverage directories first.

I am using Taskfile to run my tasks so here is an example coverage task:

  cover:
    cmds:
      - mkdir -p ./coverage
      - mkdir -p ./collections/coverage
      - mkdir -p ./core/coverage/
      - mkdir -p ./cycle/coverage
      - mkdir -p ./enums/coverage
      - mkdir -p ./internal/feat/filter/coverage
      - mkdir -p ./internal/feat/hiber/coverage
      - mkdir -p ./internal/feat/resume/coverage
      - mkdir -p ./internal/feat/sampling/coverage
      - mkdir -p ./internal/filtering/coverage
      - mkdir -p ./internal/kernel/coverage
      - mkdir -p ./internal/level/coverage
      - mkdir -p ./internal/measure/coverage
      - mkdir -p ./internal/override/coverage
      - mkdir -p ./internal/services/coverage
      - mkdir -p ./internal/level/coverage
      - mkdir -p ./internal/third/bus/coverage
      - mkdir -p ./internal/third/lo/coverage
      - mkdir -p ./internal/types/coverage
      - mkdir -p ./locale/coverage
      - mkdir -p ./nfs/coverage
      - mkdir -p ./pref/coverage
      - mkdir -p ./tapable/coverage
      - ginkgo --json-report
        ./ginkgo.report
        -coverpkg=./...
        -coverprofile=./coverage/coverage.out -r
      - go tool cover -html=./coverage/coverage.out -o ./coverage/coverage.html
      - open ./coverage/coverage.html

This does fix my issue, but it is 'unnecessarily' verbose. Is there a way to configure coverage to create the coverage directory automatically if it doesn't exist, so that I can remove all the 'mkdir -p' that are currently required in my setup, to avoid 'no such file or directory' errors.

For larger projects, with many directories, I baulk at the prospect of having litter my task file with these mkdir commands, so I hope that I am indeed missing something that I have so far not found a solution for.

PS: an example output showing the error is:

 λ  task cover
[1725513806] Traverse Suite - 20/20 specs •••••••••••••••••••• SUCCESS! 772.136431ms PASS
error generating coverage report: internal error: opening coverage data output file "/Users/plastikfan/dev/github/snivilised/traverse/coverage/coverage.out": open /Users/plastikfan/dev/github/snivilised/traverse/coverage/coverage.out: no such file or directory
ginkgo run failed

I have also tried adding:

    --output-dir=./coverage/

but this made no difference. What am I missing?

[onsi]

heyo @plastikfan - the docs for cover profiles are here

Ginkgo won’t make directories for you. “Standard” usage is to just generate the coverage files in the package directory itself. When you are running multiple suites Ginkgo will, by default, generate a single coverage file and put it at the location from which the specs were run and then delete the individual cover profiles (I believe…) if you wan to keep them separate you can use `—keep-separate-cover profiles.

So, you should be able to run:

ginkgo —json-report=ginkgo.report -coverpkg=./… -coverprofile=coverage.out -r

if you don’t want the coverprofile in the root directory then you can

midir -p ./reports
ginkgo —json-report=ginkgo.report -coverpkg=./… -coverprofile=coverage.out -output-dir=./reports -r

this will put both ginkgo.report and a single coverage.out in ./reports

[plastikfan]

Hi @onsi, I am still having major problems with this (particularly in ci) and I am wondering if there is actually an issue here that needs adressing.

Compare these 2 approaches, the first uses the standard go test command to generate coverage, and the second uses ginkgo (github actions):

  - name: Run tests and generate coverage profile with Go test
    run: |
      go test ./... -coverprofile=${{ github.workspace }}/coverage/coverage.out

  - name: Run tests and generate coverage profile with Ginkgo
    run: ginkgo ./... --json-report ./ginkgo.report \
      -coverpkg=$(go list github.com/snivilised/traverse/...) \
      -coverprofile=${{ github.workspace }}/coverage/coverage.out -r

The first one works (ish, explained below), but the ginkgo version does not; the coverage file can't be found:

/home/runner/work/traverse/traverse/coverage/coverage.out

As you can see, both use the same path for the coverage.out file, but for some reason, ginkgo is not creating this file.

Now the issue I have with the first version using the go test command, is that it is not producing very credible results, there are a lot of files being reported as having 0% coverage, which I know is incorrect. When I run ginkgo locally, I am getting better results. So I am trying to re-create what I have got running locally to run the same in ci with ginkgo, but ginkgo is not producing the .out file so that it can be published; the strange thing is that ginkgo appears to behave differently in ci than to a local run.

Run ginkgo ./... --json-report ./ginkgo.report \ -coverpkg=$(go list github.com/snivilised/traverse/...) \ -coverprofile=/home/runner/work/traverse/traverse/coverage/coverage.out -r
  ginkgo ./... --json-report ./ginkgo.report \ -coverpkg=$(go list github.com/snivilised/traverse/...) \ -coverprofile=/home/runner/work/traverse/traverse/coverage/coverage.out -r
  shell: /usr/bin/bash -e {0}
go: downloading github.com/pkg/errors v0.9.1
go: downloading golang.org/x/exp v0.0.0-20240808152545-0cdaa3abc0fa
go: downloading github.com/snivilised/li18ngo v0.1.4
go: downloading github.com/snivilised/pants v0.1.2
go: downloading github.com/onsi/gomega v1.34.1
go: downloading github.com/nicksnyder/go-i18n/v2 v2.4.0
go: downloading github.com/go-logr/logr v1.4.2
go: downloading github.com/google/go-cmp v0.6.0
go: downloading golang.org/x/text v0.17.0
go: downloading golang.org/x/net v0.28.0
go: downloading golang.org/x/sys v0.24.0
go: downloading gopkg.in/yaml.v3 v3.0.1
go: downloading github.com/fortytw2/leaktest v1.3.0

[1725914735] Traverse Suite - 20/20 specs •••••••••••••••••••• SUCCESS! 953.564595ms PASS
[1725914735] Collections Suite - 69/69 specs ••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••• SUCCESS! 18.294528ms PASS
[1725914735] Core Suite - 14/14 specs •••••••••••••• SUCCESS! 876.838µs PASS
[1725914735] Cycle Suite - 31/31 specs ••••••••••••••••••••••••••••••• SUCCESS! 2.054738ms PASS
[1725914735] Filter Suite - 0/0 specs  SUCCESS! 104.936µs PASS
[1725914735] Hibernate Suite - 9/9 specs ••••••••• SUCCESS! 106.182896ms PASS
[1725914735] Sampling Suite - 29/29 specs ••••••••••••••••••••••••••••• SUCCESS! 69.725032ms PASS
[1725914735] Filtering Suite - 49/49 specs ••••••••••••••••••••••••••••••••••••••••••••••••• SUCCESS! 191.310909ms PASS
[1725914735] Kernel Suite - 13/13 specs ••••••••••••• SUCCESS! 36.518666ms PASS
[1725914735] Broker Suite - 22/22 specs •••••••••••••••••••••• SUCCESS! 2.310857ms PASS
[1725914735] Locale Suite - 6/6 specs •••••• SUCCESS! 7.097119ms PASS
[1725914735] Nfs Suite - 11/11 specs ••••••••••• SUCCESS! 2.306619ms PASS
[1725914735] Pref Suite - 0/0 specs  SUCCESS! 51.967µs PASS
[1725914735] Tapable Suite - 15/15 specs ••••••••••••••• SUCCESS! 6.682485ms PASS

Ginkgo ran 14 suites in 11.660211078s
Test Suite Passed
0s
Run if [ -f "/home/runner/work/traverse/traverse/coverage/coverage.out" ]; then
Coverage file not found
Error: Process completed with exit code 1.

[onsi]

heyo,

what happens if you do:

 ginkgo ./... --json-report ./ginkgo.report \
      -coverpkg=$(go list github.com/snivilised/traverse/...) \
      -coverprofile=coverage.out -r

can you try that?

[plastikfan]

heyo,

what happens if you do:

 ginkgo ./... --json-report ./ginkgo.report \
      -coverpkg=$(go list github.com/snivilised/traverse/...) \
      -coverprofile=coverage.out -r

can you try that?

sorry, that results in the exact same error!

[onsi]

is this open source? i can look at the code + action?

this should generate a single coverage.out in the same location as the json report file

[onsi]

so... /home/runner/work/traverse/traverse/coverage.out i think?

[plastikfan]

yeah sure, my repo is at: traverse and you'll see from the actions tab, the history of failed runs of me trying to fix the issue.

[onsi]

thanks for the link. this:

https://github.com/snivilised/traverse/actions/runs/10781150231/job/29898448042#step:10:1

is still looking for the coverage file under traverse/coverage/coverage.out instead of traverse/coverage.out

[plastikfan]

hey @onsi, thanks for trying to help me out on this. I dont think this is a ginkgo issue. Doing this type of thing in gh-actions is a real pain in the arse. I've tried to simplify this by defining a variable and only using the variable, but even getting this working is pain because there are so many restrictions on how to use a variable and different syntaxes depending on the context.

If you have any workflows already defined that use ginkgo from a workflow, I'd be interested to see it. Thanks again for your help.

The only other thing I would request is for there to be a fix for ginkgo to create the required intermediate directories instead of failing. That way, we wouldn't have to create them manually (eg mkdir -p ./internal/filtering/coverage as seen in my original post) and have some pain taken away.

[plastikfan]

ok, I think there is in fact an issue with ginkgo afterall. I'm going close this issue and raise another as I have managed to reproduce the issue found in ci, locally, so it has nothing to do with ci.