19

I've been going though a set of projects, dealing with updating dependencies and there is one thing I don't have a clear answer to and that is why the generated sum file lists so many older versions of each dependency.

in our project we had some vulnerabilities introduced though an older version of golang.org/x/crypto that we resolved though a replace directive to a package release with security fixes but this doesn't feel very correct and could lock us into an insecure version of a package.

now I have gone though and updated the package which depended on an older version of golang.org/x/crypto and looped back to the package with the the replace directive and attempted an update but I still see the older packages listed.

I'm wondering what this means for our project and how I can find why these are included in the first place?

running a simple go mod why -m golang.org/x/crypto reveals that the only project dependent on golang.org/x/crypto was the one that I had updated.

Adam M.
  • 1,023
  • 1
  • 10
  • 21
  • 3
    [Why does 'go.sum' include information for module versions I am no longer using?](https://github.com/golang/go/wiki/Modules#is-gosum-a-lock-file-why-does-gosum-include-information-for-module-versions-i-am-no-longer-using) and [go.sum files](https://golang.org/ref/mod#go-sum-files): "The go command may need to load go.mod files from multiple versions of a dependency in order to perform minimal version selection." – JimB Nov 03 '21 at 13:21
  • try to run `go mod tidy` command https://golang.org/ref/mod#go-mod-tidy – Matteo Nov 03 '21 at 13:32
  • 1
    @JimB thanks for the link, I missed that in my search. So in regards to the vulnerability scan which indicated that version X.Y.Z of `crypto` had vulns, would it mean that the scan may be indicating a vuln incorrectly as long as my projects include a later version with the patch to the vuln? – Adam M. Nov 03 '21 at 13:45
  • 3
    A vulnerability scanner should be checking `go.mod` for the exact versions being used. The `go.sum` is only a record of package versions, not what is being compiled. It seems like you should be able to update the crypto package without resorting to `replace`, but it should make no difference to a scanner that understands go dependencies. – JimB Nov 03 '21 at 13:47
  • So perhaps this can be solved with `go get golang.org/x/crypto@latest` (latest has the fix lets say) and the indirect inclusion in the mod file may supersede the inclusion in the sum file from an indirect dependency. In the end that would just be to make the scanner (and project lead) happy I believe. Again, thank you for the insight! – Adam M. Nov 03 '21 at 13:54
  • 2
    It depends on the scanner, but may help if you update your `go.mod` to `go1.17`, which will then include all indirect dependencies for static verification. – JimB Nov 03 '21 at 14:25

1 Answers1

23

@JimB offered some documentation on go sum with the following statement

The go.sum file may contain hashes for multiple versions of a module. The go command may need to load go.mod files from multiple versions of a dependency in order to perform minimal version selection. go.sum may also contain hashes for module versions that aren’t needed anymore (for example, after an upgrade). go mod tidy will add missing hashes and will remove unnecessary hashes from go.sum.

And the resulting set of packages defined in a go sum comes from the minimal version selection process which seems to be a deep topic.

An example would be importing "google.golang.org/grpc/metadata" into a module, running go mod tidy in the module and a small portion of a resulting sum file would be the following:

github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.3/go.mod h1:vzj43D7+SQXF/4pzW/hwtAqwc6iTitCiVSaWz5lYuqw=
github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8=
github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA=
github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs=
github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w=
github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=
github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8=
github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=

each reference to a version indictates a node in the minimal version selection algorithm graph

adding the following to the mod file

replace github.com/golang/protobuf => github.com/golang/protobuf v1.4.3

and running a go mod tidy, the resulting sum entry for protobuf changes to:

github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=

because the replace directive indicates a replacement of all versions so all nodes in the dependency graph are replaced with v1.4.3 which just simplifies to the inclusion of the single version of the dependency v1.4.3

As for the issue I had with the vulnerability scanner, it seems the author of it was unaware of of how dependencies should be checked for Golang and upgrading the module to go 1.17 where indirect dependencies are listed in the mod file didn't stop the sum entries from flagging the project for vulnerabilities.

Dharman
  • 30,962
  • 25
  • 85
  • 135
Adam M.
  • 1,023
  • 1
  • 10
  • 21