Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

go mod graph list dependencies that aren't expected and contradicts go mod why #28171

Closed
didrocks opened this issue Oct 12, 2018 · 5 comments
Closed

go mod graph list dependencies that aren't expected and contradicts go mod why #28171

didrocks opened this issue Oct 12, 2018 · 5 comments
Labels
FrozenDueToAge modules

Comments

@didrocks
Copy link

What version of Go are you using (go version)?

go version go1.11.1 linux/amd64

Does this issue reproduce with the latest release?

Yes (1.11.1)

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/home/didrocks/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/didrocks/.go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build108021283=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Clone https://github.com/ubuntu/ubuntu-report (outside of GOPATH)

# outside of GOPATH
$ git clone https://github.com/ubuntu/ubuntu-report 
$ cd ubuntu-report
$ $ go mod graph
github.com/ubuntu/ubuntu-report github.com/cpuguy83/go-md2man@v1.0.8
github.com/ubuntu/ubuntu-report github.com/davecgh/go-spew@v1.1.1
github.com/ubuntu/ubuntu-report github.com/inconshreveable/mousetrap@v1.0.0
github.com/ubuntu/ubuntu-report github.com/onsi/gomega@v1.4.2
github.com/ubuntu/ubuntu-report github.com/pkg/errors@v0.8.0
github.com/ubuntu/ubuntu-report github.com/pmezard/go-difflib@v1.0.0
github.com/ubuntu/ubuntu-report github.com/russross/blackfriday@v1.5.1
github.com/ubuntu/ubuntu-report github.com/sirupsen/logrus@v1.0.4
github.com/ubuntu/ubuntu-report github.com/spf13/cobra@v0.0.1
github.com/ubuntu/ubuntu-report github.com/spf13/pflag@v1.0.0
github.com/ubuntu/ubuntu-report github.com/stretchr/objx@v0.1.1
github.com/ubuntu/ubuntu-report github.com/stretchr/testify@v1.2.2
github.com/ubuntu/ubuntu-report golang.org/x/crypto@v0.0.0-20180308185624-c7dcf104e3a7
github.com/ubuntu/ubuntu-report gopkg.in/airbrake/gobrake.v2@v2.0.9
github.com/ubuntu/ubuntu-report gopkg.in/gemnasium/logrus-airbrake-hook.v2@v2.1.2
github.com/onsi/gomega@v1.4.2 github.com/fsnotify/fsnotify@v1.4.7
github.com/onsi/gomega@v1.4.2 github.com/golang/protobuf@v1.2.0
github.com/onsi/gomega@v1.4.2 github.com/hpcloud/tail@v1.0.0
github.com/onsi/gomega@v1.4.2 github.com/onsi/ginkgo@v1.6.0
github.com/onsi/gomega@v1.4.2 golang.org/x/net@v0.0.0-20180906233101-161cd47e91fd
github.com/onsi/gomega@v1.4.2 golang.org/x/sync@v0.0.0-20180314180146-1d60e4601c6f
github.com/onsi/gomega@v1.4.2 golang.org/x/sys@v0.0.0-20180909124046-d0be0721c37e
github.com/onsi/gomega@v1.4.2 golang.org/x/text@v0.3.0
github.com/onsi/gomega@v1.4.2 gopkg.in/fsnotify.v1@v1.4.7
github.com/onsi/gomega@v1.4.2 gopkg.in/tomb.v1@v1.0.0-20141024135613-dd632973f1e7
github.com/onsi/gomega@v1.4.2 gopkg.in/yaml.v2@v2.2.1
gopkg.in/yaml.v2@v2.2.1 gopkg.in/check.v1@v0.0.0-20161208181325-20d25e280405

# I wasn't expecting gomega to show up (or having all indirect dependencies to show up)
$ grep -r gomega *
go.mod:	github.com/onsi/gomega v1.4.2 // indirect
go.sum:github.com/onsi/gomega v1.4.2 h1:3mYCb7aPxS/RU7TI1y4rkEn1oKmPRjNJLNEXgw7MH2I=
go.sum:github.com/onsi/gomega v1.4.2/go.mod h1:ex+gbHU/CVuBBDIJjb2X0qEXbFg53c61hWP/1CpauHY=
$ go mod why github.com/onsi/gomega
# github.com/onsi/gomega
github.com/ubuntu/ubuntu-report/cmd/ubuntu-report
github.com/sirupsen/logrus
github.com/sirupsen/logrus.test
gopkg.in/gemnasium/logrus-airbrake-hook.v2
gopkg.in/airbrake/gobrake.v2
gopkg.in/airbrake/gobrake.v2.test
github.com/onsi/gomega

# coming thus from the .test package from logrus (indirect dep, as a stated on go.mod)

# This is inconsistent with testify, which doesn't show up in go mod graph, and requires -m in go mod why:
$ grep -r testify *
go.mod:	github.com/stretchr/testify v1.2.2 // indirect
go.sum:github.com/stretchr/testify v1.2.2 h1:bSDNvY7ZPG5RlJ8otE/7V6gMiyenm9RtJ7IUVIAoJ1w=
go.sum:github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
$ go mod why github.com/stretchr/testify
# github.com/stretchr/testify
(main module does not need package github.com/stretchr/testify)
$ go mod why -m github.com/stretchr/testify
# github.com/stretchr/testify
github.com/ubuntu/ubuntu-report/cmd/ubuntu-report
github.com/sirupsen/logrus
github.com/sirupsen/logrus.test
github.com/stretchr/testify/assert

What did you expect to see?

I was expecting to see similar behavior for testify and gomega:

  • Both shown in go mod graph output or none (I was expecting "none" as those are indirect dependencies)
  • go mod why to require -m for both, as both are needed directly or indirectly by logrus.test.

What did you see instead?

I can't spot why there is a difference in those outputs for similar kind of indirect imports to me.
@mvdan mvdan added the modules label Oct 12, 2018
@myitcv
Copy link
Member

myitcv commented Nov 5, 2018

go mod graph simply shows dependencies between all modules, or put in terms of the documentation:

Graph prints the module requirement graph (with replacements applied)
in text form.

Hence go mod graph with a main module of github.com/ubuntu/ubuntu-report/cmd/ubuntu-report shows all of the requirements for modules (transitively) required by github.com/ubuntu/ubuntu-report/cmd/ubuntu-report. Some of the "module" dependencies aren't yet modules, hence some dependencies are visible as // indirect as you note.

Refreshing on the documentation for go mod why:

Why shows a shortest path in the import graph from the main module to
each of the listed packages. If the -m flag is given, why treats the
arguments as a list of modules and finds a path to any package in each
of the modules.

So in summary:

  • go mod graph simply reflects the module requirement graph in text form, nothing else
  • go mod why -m github.com/stretchr/testify tries to find a package path from the main module to the github.com/stretchr/testify module:
$ go mod why -m github.com/stretchr/testify
# github.com/stretchr/testify
github.com/ubuntu/ubuntu-report/cmd/ubuntu-report
github.com/sirupsen/logrus
github.com/sirupsen/logrus.test
github.com/stretchr/testify/assert
  • github.com/stretchr/testify/assert is one such package in the github.com/stretchr/testify module.. Hence go mod why github.com/stretchr/testify will show us the same path:
$ go mod why github.com/stretchr/testify/assert
# github.com/stretchr/testify/assert
github.com/ubuntu/ubuntu-report/cmd/ubuntu-report
github.com/sirupsen/logrus
github.com/sirupsen/logrus.test
github.com/stretchr/testify/assert
  • Same story for github.com/onsi/gomega:
$ go mod why -m github.com/onsi/gomega
# github.com/onsi/gomega
github.com/ubuntu/ubuntu-report/cmd/ubuntu-report
github.com/sirupsen/logrus
github.com/sirupsen/logrus.test
gopkg.in/gemnasium/logrus-airbrake-hook.v2
gopkg.in/airbrake/gobrake.v2
gopkg.in/airbrake/gobrake.v2.test
github.com/onsi/gomega
$ go mod why github.com/onsi/gomega
# github.com/onsi/gomega
github.com/ubuntu/ubuntu-report/cmd/ubuntu-report
github.com/sirupsen/logrus
github.com/sirupsen/logrus.test
gopkg.in/gemnasium/logrus-airbrake-hook.v2
gopkg.in/airbrake/gobrake.v2
gopkg.in/airbrake/gobrake.v2.test
github.com/onsi/gomega

Everything appears to be working as expected here so I'm going to speculatively close this issue, but please shout if I've missed anything.

@myitcv myitcv closed this as completed Nov 5, 2018
@didrocks
Copy link
Author

didrocks commented Nov 5, 2018

Thanks for the answer @myitcv. That makes a lot more sense.

So, go mod why can't be used to assert that a module is included in a package binary from what I get.
If I take back your example of testify:

$ go mod why -m github.com/stretchr/testify/assert
# github.com/stretchr/testify/assert
github.com/ubuntu/ubuntu-report/cmd/ubuntu-report
github.com/sirupsen/logrus
github.com/sirupsen/logrus.test
github.com/stretchr/testify/assert

As part of the path, there is logrus.test and I can assume logrus doesn't import logrus.test, so I can only infer that testify/assert isn't imported.

Same for gopkg.in/check.v1:

go mod why -m gopkg.in/check.v1
# gopkg.in/check.v1
github.com/ubuntu/ubuntu-report/cmd/ubuntu-report
github.com/ubuntu/ubuntu-report/cmd/ubuntu-report.test
github.com/spf13/cobra/doc
gopkg.in/yaml.v2
gopkg.in/yaml.v2.test
gopkg.in/check.v1

So, I guess there is no flag to get the list of imported packages, according to build flags (no test, excuded files) apart from building the binary itself and opening the package manifest inside the binary introduced by Go 1.11?

@myitcv
Copy link
Member

myitcv commented Nov 5, 2018

So, I guess there is no flag to get the list of imported packages, according to build flags (no test, excuded files) apart from building the binary itself and opening the package manifest inside the binary introduced by Go 1.11?

I think you're after go list -json, specifically the .Deps field:

$ go list -json github.com/ubuntu/ubuntu-report/cmd/ubuntu-report
{
        "Dir": "/tmp/ubuntu-report/cmd/ubuntu-report",
        "ImportPath": "github.com/ubuntu/ubuntu-report/cmd/ubuntu-report",
        "Name": "main",
...

To get a list of non-standard library dependencies you need to get a bit more creative:

$ go list -deps -f "{{if not .Standard}}{{.ImportPath}}{{end}}" github.com/ubuntu/ubuntu-report/cmd/ubuntu-report
golang.org/x/sys/unix
golang.org/x/crypto/ssh/terminal
github.com/sirupsen/logrus
github.com/spf13/pflag
github.com/spf13/cobra
github.com/pkg/errors
github.com/ubuntu/ubuntu-report/internal/sender
github.com/ubuntu/ubuntu-report/internal/utils
github.com/ubuntu/ubuntu-report/internal/metrics
github.com/ubuntu/ubuntu-report/pkg/sysmetrics
github.com/ubuntu/ubuntu-report/cmd/ubuntu-report

I've been planning to do a "Go Modules By Example" on the use of go mod why, go mod why -m, go mod graph and go list for a while... Will try and do that today.

@didrocks
Copy link
Author

didrocks commented Nov 5, 2018

Makes a lot of sense now! Yeah, a Go Modules By Example would be awesome! 👍 Thanks again for your answers.

@bcmills
Copy link
Contributor

bcmills commented Nov 5, 2018

See also #27900.

@golang golang locked and limited conversation to collaborators Nov 5, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge modules
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@didrocks @myitcv @mvdan @bcmills @gopherbot