145

Suppose you have a repository at github.com/someone/repo and you fork it to github.com/you/repo. You want to use your fork instead of the main repo, so you do a

go get github.com/you/repo

Now all the import paths in this repo will be "broken", meaning, if there are multiple packages in the repository that reference each other via absolute URLs, they will reference the source, not the fork.

Is there a better way as cloning it manually into the right path?

git clone git@github.com:you/repo.git $GOPATH/src/github.com/someone/repo
  • 1
    _No_ import path in the new fork will be broken which were not broken already before the forking. – zzzz Jan 14 '13 at 19:23
  • 15
    Sorry to disappoint you, but that's not true. If a sub-package is referenced in the imports via it's absolute url, this import will be broken in the fork (or at least reference the wrong package). –  Jan 14 '13 at 20:49
  • 3
    E.g. [goamz](https://code.launchpad.net/goamz). It has internal references all over the place. –  Jan 14 '13 at 21:00
  • What is an "internal reference"? A package cannot import itself. If it imports any other package than the import in the fork still imports that package. I don't think that "broken import" means what you think it means. – zzzz Jan 14 '13 at 21:13
  • 2
    Look at the `ec2` package - it has an `launchpad.net/goamz/aws` import. Both, the `aws` and the `ec2` packages reside in the SAME repository, so when forked, will not reference the correct package (the one in the fork). –  Jan 14 '13 at 21:33
  • 1
    The fork will reference the same package as the fork's source. What's incorrect in that? The fork will compile, it will build, it will do the same thing as before. What's the definition of 'incorrect package' then? Note that the Go language, so as its build system, has _no_ awareness of repositories, only packages. – zzzz Jan 14 '13 at 21:39
  • @zzz, I think subpackages/subfolders are used to organize code, and make internal packages... Referencing an internal package from another repository is definitely wrong. I note that "net/http/internal" is an example of an what I presume is an internal package. – jonasfj May 17 '15 at 06:39
  • 2
    Go dependency management is straight-up BS. Use go modules then some dependency becomes incompatible, don't use go modules you're forced to work in $GOPATH/src/github.com/blah/blahblah instead of where you want the project to be. – Phani Rithvij May 08 '20 at 08:04
  • You might consider changing the accepted answer to [this new one](https://stackoverflow.com/a/56792766/119527) which is correct in the new go module world. – Jonathon Reinhart Mar 15 '21 at 00:41

12 Answers12

171

If you are using go modules. You could use replace directive

The replace directive allows you to supply another import path that might be another module located in VCS (GitHub or elsewhere), or on your local filesystem with a relative or absolute file path. The new import path from the replace directive is used without needing to update the import paths in the actual source code.

So you could do below in your go.mod file

module some-project

go 1.12

require (
    github.com/someone/repo v1.20.0
)

replace github.com/someone/repo => github.com/you/repo v3.2.1

where v3.2.1 is tag on your repo. Also can be done through CLI

go mod edit -replace="github.com/someone/repo@v0.0.0=github.com/you/repo@v1.1.1"
Yogesh
  • 4,546
  • 2
  • 32
  • 41
  • 10
    worked great. i think the only reason this doesn't have more upvotes is because folks are not using go modules yet. I also used this trick to point to a file location to another directory on my workstation where i had local edits i was working on. I would just remove my "replace" line once i push my local edits in github. – lazieburd Aug 30 '19 at 13:45
  • ^ 100% agree. Vote people. – Andrew Arrow Oct 27 '19 at 06:19
  • 4
    oh, but "master" did not work for me. I had to write v0.0.1 or some specific version there. – Andrew Arrow Oct 27 '19 at 06:31
  • 2
    You can also [`go mod edit -replace`](https://golang.org/cmd/go/#hdr-Edit_go_mod_from_tools_or_scripts) directly on the command line: `go mod edit -replace="github.com/someone/repo@v0.0.0=github.com/you/repo@v1.1.1"`. Both `@v...` are optional. – Joel Purra Nov 23 '19 at 12:17
  • 2
    Wouldn't it be cool to have a `go.mod.local` or `go.mod.dev` whose role is to actually replace the import path for local development? I mean, you would never forget to remove the ugly "replace" because you wouldn't have to. – Manuel Jul 08 '20 at 17:59
  • For testing, I recomment you to remove `github.com/someone/repo` and `github.com/you/repo` under `$GOPATH/pkg/mod/`. If `go build` downloading `github.com/you/repo`, means working. – coanor Dec 14 '20 at 05:50
  • @Yogesh I used this method but now I'm getting the error `go: github.com/you/repo@v1.0.0 used for two different module paths (github.com/someone/repo and github.com/you/repo)` any idea how to resolve this? – Jeet Nov 16 '22 at 06:04
  • 1
    There is open issue on golang since 2018 https://github.com/golang/go/issues/26904 problem seems to be from transitive dependencies – Yogesh Nov 16 '22 at 09:17
85

To handle pull requests

  • fork a repository github.com/someone/repo to github.com/you/repo
  • download original code: go get github.com/someone/repo
  • be there: cd "$(go env GOPATH)/src"/github.com/someone/repo
  • enable uploading to your fork: git remote add myfork https://github.com/you/repo.git
  • upload your changes to your repo: git push myfork

http://blog.campoy.cat/2014/03/github-and-go-forking-pull-requests-and.html

To use a package in your project

https://github.com/golang/go/wiki/PackageManagementTools

kubanczyk
  • 5,184
  • 1
  • 41
  • 52
Ivan Rave
  • 1,759
  • 18
  • 15
21

One way to solve it is that suggested by Ivan Rave and http://blog.campoy.cat/2014/03/github-and-go-forking-pull-requests-and.html -- the way of forking.

Another one is to workaround the golang behavior. When you go get, golang lays out your directories under same name as in the repository URI, and this is where the trouble begins.

If, instead, you issue your own git clone, you can clone your repository onto your filesystem on a path named after the original repository.

Assuming original repository is in github.com/awsome-org/tool and you fork it onto github.com/awesome-you/tool, you can:

cd $GOPATH
mkdir -p {src,bin,pkg}
mkdir -p src/github.com/awesome-org/
cd src/github.com/awesome-org/
git clone git@github.com:awesome-you/tool.git # OR: git clone https://github.com/awesome-you/tool.git
cd tool/
go get ./...

golang is perfectly happy to continue with this repository and doesn't actually care some upper directory has the name awesome-org while the git remote is awesome-you. All import for awesome-org are resovled via the directory you have just created, which is your local working set.

In more length, please see my blog post: Forking Golang repositories on GitHub and managing the import path

edit: fixed directory path

Shlomi Noach
  • 9,073
  • 1
  • 23
  • 20
  • 4
    I agree this is the "best" solution for this. But it would be really nice to see how people manage this workflow when running the Go app in a Docker container. I am learning golang and wanted to add a tiny feature to a library I am using when I ran into this headache with testing it before creating a Pull Request. – Joakim Jul 18 '16 at 16:20
  • 7 years later, still no good solution for Docker container that I can find online. – Zhivko Draganov Aug 04 '23 at 10:40
4

If your fork is only temporary (ie you intend that it be merged) then just do your development in situ, eg in $GOPATH/src/launchpad.net/goamz.

You then use the features of the version control system (eg git remote) to make the upstream repository your repository rather than the original one.

It makes it harder for other people to use your repository with go get but much easier for it to be integrated upstream.

In fact I have a repository for goamz at lp:~nick-craig-wood/goamz/goamz which I develop for in exactly that way. Maybe the author will merge it one day!

Nick Craig-Wood
  • 52,955
  • 12
  • 126
  • 132
  • 1
    Just so I understand the implications of doing this, if I went this route, when someone does a `go get` from my repo, all of my import statements and such will still reflect `github.com/original_author` and thus be broken... correct? – parker.sikand Oct 04 '14 at 20:30
  • @parker.sikand yes that is correct. This technique is best for stuff you intend to get merged upstream, not for go get use. If you intend to fork the package permanently then use the other answer's technique. – Nick Craig-Wood Oct 05 '14 at 14:30
3

Here's a way to that works for everyone:

Use github to fork to "my/repo" (just an example):

go get github.com/my/repo
cd ~/go/src/github.com/my/repo
git branch enhancement
rm -rf .
go get github.com/golang/tools/cmd/gomvpkg/…
gomvpkg <<oldrepo>> ~/go/src/github.com/my/repo
git commit

Repeat each time when you make the code better:

git commit
git checkout enhancement
git cherry-pick <<commit_id>>
git checkout master

Why? This lets you have your repo that any go get works with. It also lets you maintain & enhance a branch that's good for a pull request. It doesn't bloat git with "vendor", it preserves history, and build tools can make sense of it.

kubanczyk
  • 5,184
  • 1
  • 41
  • 52
user1212212
  • 1,311
  • 10
  • 6
  • Slight correction: go run github.com/golang/tools/cmd/gomvpkg/main.go and this command moves .git, so save that elsewhere & restore it afterward. – user1212212 Aug 24 '17 at 15:29
  • also it is possible just use mvn-golang plugin which makes some automation in processing of dependencies like in the example https://github.com/raydac/mvn-golang/tree/master/mvn-golang-examples/test-git-cvs – Igor Maznitsa Jan 01 '18 at 18:21
2

Instead of cloning to a specific location, you can clone wherever you want. Then, you can run a command like this, to have Go refer to the local version:

go mod edit -replace github.com/owner/repo=../repo

https://golang.org/cmd/go#hdr-Module_maintenance

Zombo
  • 1
  • 62
  • 391
  • 407
1

The answer to this is that if you fork a repo with multiple packages you will need to rename all the relevant import paths. This is largely a good thing since you've forked all of those packages and the import paths should reflect this.

Jeremy Wall
  • 23,907
  • 5
  • 55
  • 73
  • 3
    I burned more time than I care to admit diagnosing this in my first contribution to a Go project. "All tests pass, including the ones I wrote to exhaustively test new functionality. What's wrong?!" Are you aware of any available tooling to ease this stumbling point for beginners? – Sage Mitchell May 30 '14 at 16:37
  • 3
    Once I figured it out it was easy to solve using `find`, `xargs`, and `sed`, but it would help to have a pain-free workflow that consistently works for everyone. – Sage Mitchell May 30 '14 at 16:39
  • @JakeMitchell [`gomvpkg`](https://godoc.org/golang.org/x/tools/cmd/gomvpkg) can do the renames easier/better. `go get golang.org/x/tools/cmd/gomvpkg` then `gomvpkg -help`. – Dave C Mar 20 '15 at 17:06
  • 3
    This answer strikes me as complety impractical. Sed-ing project files out of a forked project, that's insane? What do you do when you create a pull request? The answer by Ivan Rave looks like a much better solution to me. – Ivan P Mar 27 '15 at 05:58
  • So common git flow seems to be `fork`, `hack`, `push to fork` and `file a pr`, if one has to change import paths how one file a PR? And if one doesn't change import paths how can one modify/test the code and it running in CI (travis, etc)? – jonasfj May 17 '15 at 06:36
  • 8
    Is this still how Go-lang is working? This is just so insane, that it is not funny... Either be upstream-friendly, or downstream-friendly, but not both. It is a huge design flaw in my not so humble opinion, probably done by people who doesn't collaborate too much cross-projects. #FAIL #GOLANG – Niclas Hedhman Jul 15 '16 at 12:41
  • How can this be a good thing, unless you are working in a vacuum? – talonx Aug 04 '17 at 05:20
  • Good answer for this important scenario: `fork`, then `hack`, then... sit back and let people `go get` and run your awesome code. No PR, just a permanent fork. – kubanczyk May 10 '19 at 10:58
0

Use vendoring and submodules together

  1. Fork the lib on github (go-mssqldb in this case)
  2. Add a submodule which clones your fork into your vendor folder but has the path of the upstream repo
  3. Update your import statements in your source code to point to the vendor folder, (not including the vendor/ prefix). E.g. vendor/bob/lib => import "bob/lib"

E.g.

cd ~/go/src/github.com/myproj

mygithubuser=timabell
upstreamgithubuser=denisenkom
librepo=go-mssqldb

git submodule add "git@github.com:$mygithubuser/$librepo" "vendor/$upstreamgithubuser/$librepo"

Why

This solves all the problems I've heard about and come across while trying to figure this out myself.

  • Internal package refs in the lib now work because the path is unchanged from upstream
  • A fresh checkout of your project works because the submodule system gets it from your fork at the right commit but in the upstream folder path
  • You don't have to know to manually hack the paths or mess with the go tooling.

More info

Tim Abell
  • 11,186
  • 8
  • 79
  • 110
0

The modern answer (go 1.15 and higher, at least).

go mod init github.com/theirs/repo

Make an explicit init arg that is the ORIGINAL package names. If you don't include the repo name, it will assume the one in gopath. But when you use go modules, they no longer care where they are on disk, or where git actually pulls dependencies from.

Rob
  • 1,387
  • 1
  • 13
  • 18
-1

To automate this process, I wrote a small script. You can find more details on my blog to add a command like "gofork" to your bash.

function gofork() {
  if [ $# -ne 2 ] || [ -z "$1" ] || [ -z "$2" ]; then
    echo 'Usage: gofork yourFork originalModule'
    echo 'Example: gofork github.com/YourName/go-contrib github.com/heirko/go-contrib'
    return
  fi
   echo "Go get fork $1 and replace $2 in GOPATH: $GOPATH"
   go get $1
   go get $2
   currentDir=$PWD
   cd $GOPATH/src/$1
   remote1=$(git config --get remote.origin.url)
   cd $GOPATH/src/$2
   remote2=$(git config --get remote.origin.url)
   cd $currentDir
   rm -rf $GOPATH/src/$2
   mv $GOPATH/src/$1 $GOPATH/src/$2
   cd $GOPATH/src/$2
   git remote add their $remote2
   echo Now in $GOPATH/src/$2 origin remote is $remote1
   echo And in $GOPATH/src/$2 their remote is $remote2
   cd $currentDir
}

export -f gofork
heralight
  • 920
  • 1
  • 7
  • 15
-2

You can use command go get -f to get you a forked repo

Brian Tompsett - 汤莱恩
  • 5,753
  • 72
  • 57
  • 129
kevin
  • 1
  • 1
-3

in your Gopkg.toml file add these block below

[[constraint]]
  name = "github.com/globalsign/mgo"
  branch = "master"
  source = "github.com/myfork/project2"

So it will use the forked project2 in place of github.com/globalsign/mgo

msonowal
  • 1,553
  • 3
  • 17
  • 36
  • 1
    The `Gopkg.toml` file is only used by `dep` which this question doesn't mention at all. New Go projects should be using Go modules instead (and IMO existing dep based projects should migrate as well). – Dave C Aug 21 '19 at 16:54
  • 1
    I didn't know about this dep feature, and your answer surely helped me out :) – Veger Oct 25 '19 at 12:06