2

I have made a fork of a repository (specifically, github.com/ethereum/go-ethereum) and am trying to make sure that I can build the main application (cmd/geth) before developing on the fork. However, due to the upstream source importing internal packages, the build does not work in my fork.

I have tried this both using GOPATH and Go modules. In each case, when I try to run go install github.com/<me>/go-ethereum/cmd/geth) I get the following error:

cmd/geth/main.go:40:2: use of internal package github.com/ethereum/go-ethereum/internal/debug not allowed

In the Go module approach, I have tried adding a replace directive to use my fork instead:

replace github.com/ethereum/go-ethereum => github.com/<me>/go-ethereum master

But this leads to the same error.

I am happy to provide more information (e.g., build output) if that would be helpful, but it seems like there is something straightforward that I am missing, rather than simply replacing all instances of ethereum/go-ethereum/internal with <me>/go-ethereum/internal.

Stipe Matic
  • 324
  • 1
  • 2
  • 5
  • Clone your fork to $GOPATH/src/github.com/ethereum/go-ethereum. – Charlie Tumahai Sep 06 '19 at 22:17
  • Possible duplicate of [Using forked package import in Go](https://stackoverflow.com/questions/14323872/using-forked-package-import-in-go). – Charlie Tumahai Sep 06 '19 at 23:24
  • This doesn't really solve the problem, though. That would also mean that anyone who clones my fork would *also* have to clone it to the ethereum directory, and it's hard to believe that there isn't a more canonical and portable way. – Stipe Matic Sep 07 '19 at 04:51
  • If you are creating a new package that you expect others to import, then you need to replace the old import paths (`github.com/ethereum/go-ethereum...`) with the import path of your new package (`github.com/yourUserId/go-ethereum...`). The internal import paths will work after this change. My previous comments assumed that you are fixing bugs or doing some other work where there is no intent to create a new package. – Charlie Tumahai Sep 07 '19 at 05:03
  • It's true that replacing these would work, but this would then create hurdles when merging changes to the upstream repository. I'm fine with doing this, but it seems like there should be a cleaner solution in Go. Is that really not the case? – Stipe Matic Sep 09 '19 at 17:38
  • Follow https://stackoverflow.com/questions/14323872/using-forked-package-import-in-go if your goal is to merge changes back to the upstream. – Charlie Tumahai Sep 09 '19 at 18:16
  • Sorry, should have been clearer. The idea is to merge changes from the upstream repository to my fork. It's unlikely that changes in my fork will be merged back into the upstream repo. – Stipe Matic Sep 09 '19 at 18:47

4 Answers4

0

If you are forking you should change the import paths as well.

nrxr
  • 59
  • 4
0

By default, GOPATH is set to $HOME/gopath, and your repository source code is placed in $GOPATH/github.com/user/repo (or $GOPATH/bitbucket.org/user/repo).

When working on a forked repo, in order to import internal packages, the path of the original repo should be added to the go import path.

As an example, on Travis-CI, my forked repo failed on the same problem (use of internal package not allowed). So I just set go_import_path in travis.yml - and it passed.

Noam Manos
  • 15,216
  • 3
  • 86
  • 85
0

Root of the Issue

It turns out this issue has nothing to do with Go modules. It is raised because importing from internal folders is disallowed in Go if you aren't in the same tree.

How to Solve It

The only way to fix this issue is to not use an import statement to the source's internal folder, but instead replace it with an import statement to the fork's.

If you've already used a replace directive the way OP has, you do not need to change any other import statement - just the one that's giving you trouble.

Walkthrough of the Solution

Suppose you have source github.com/source/s and you have made a fork github.com/fork/f, and you've cloned the fork locally before making changes.

The go.mod for the fork will look like

module github.com/source/s

go 1.11

require (
...
)

and assume the file that's throwing the internal error will look like

package main

import (
   "github.com/source/s/something"
   "github.com/source/s/internal/somethingelse"
)
...

Add a replace directive and change the module directive to mirror your fork:

module github.com/fork/f

go 1.11

replace github.com/source/s => github.com/fork/f

require (
...
)

Now run go clean -modcache (to clear any older modules already imported by this module) and run go mod tidy. This gets everything installed cleanly, which removes any other sources of possible issues.

Go to the folder containing the problematic file, and run go install (as the OP did). You will see an error:

main.go: use of internal package source/s/internal/somethingelse not allowed

Just replace the problematic file's contents with:

package main

import (
   "github.com/source/s/something"
   "github.com/fork/f/internal/somethingelse"
)
...

and rerun go clean -modcache and go mod tidy.

Now you should be able to run go install with no issues, and have replace work the way you want.

(Personally, this was a very frustrating issue for me to solve! I was working on modifying a fork of a core package in the Go ecosystem, and the sheer number of dependencies that would break if you tried changing import paths naively made me tear my hair out. I am now able to sleep peacefully at night knowing the answer to this question.)

Akshat Mahajan
  • 9,543
  • 4
  • 35
  • 44
0

You are doing that correctly already, but need to run go mod tidy after adding the replace.

For anyone trying to do that in 2022 (or later). Tested with go 1.17.

replace github.com/ethereum/go-ethereum => github.com/<me>/go-ethereum master

Then run, go mod tidy, and it will automatically replace the master to a version+hash, like: v1.0.0-0.20220709042708-bfee27c57c7c. Check this example:

go.mod

module mymodule.com/module_name

go 1.17

require (
    gorm.io/gorm v1.23.8
)

replace (
    gorm.io/driver/postgres v1.3.8 => github.com/rwrz/postgres v1.3.9-0.20220709042708-bfee27c57c7c
)

On this example above, I'm replacing an indirect dependency of gorm module with my own fork. This number v1.3.9-0.20220709042708-bfee27c57c7c was replaced by the master that I was using.

Rodrigo Boratto
  • 1,053
  • 13
  • 13