1

In the company I work at, we have several backend systems written in Go and some of the code is shared between them. The backend systems need to be deployed separately, potentially on different machines. All of these projects are still under active development and change quite often.

We are trying to come up with a good way of managing our git repositories and the dependencies between them.

At the moment we have one repository for the shared code, lets call it backend-shared. Then we have a separate repository for each backend system, lets call them backend1 and backend2. In turn each backend has a Godep dependency on backend-shared.

As far as I understand, the preferred method of dependency management in Golang is via vendoring, according to which all dependencies are copied into the /vendor directory, and should be checked into the version control. This way, all dependencies are locked to specific versions.

This works fine for our external dependencies, however for the internal dependency on backend-shared, it becomes quite cumbersome since it isn't unusual for a developer to have to simultaneously make changes to both a specific backend system as well as backend-shared.

Right now, if a change is made to the backend-shared that resides in the developer's GOPATH, that change wouldn't be visible inside the backend1 (also in GOPATH) because backend1 will first look at the stale copy of backend-shared inside its /vendor directory.

Thus, we either have to re-vendor backend1 in order to copy the new version of backend-shared, or we would have to temporarily remove backend-shared from the /vendor directory in order for the imports to point to the version inside the GOPATH. Both of these options feel potentially dirty and I am not sure if they are the way Go is meant to be used.

My question is if there is a better way to keep our current repositories and simplify the development of multiple projects at once?

Or should we combine all repositories into a single one, given that right now their development life-cycles are rather intertwined even if dependency wise backend1 and backend2 are separate?

The main reason we didn't begin with a single repository containing backend1, backend2 and backend-shared is that backend1 and backend2 must be deployed separately, so we wanted to have their code also physically separated.

e.nikolov
  • 127
  • 1
  • 11
  • 1
    It's probably unnecessary to vendor internal dependencies; vendoring is most commonly for third-party dependencies (those from an outside vendor, hence the name) where you have no control over changes made to the library. – Adrian Apr 28 '17 at 18:51
  • @Adrian Doesn't godep automatically vendor all of the dependencies, including the internal ones? I didn't find a way to tell godep to ignore some of them. Or do I manually remove them from the /vendor directory? If say I wanted to deploy backend1 to some machine, I still need some way of getting backend-shared to be put there as well. If I don't use godep for that, wouldn't I have to manually git clone each internal dependency? – e.nikolov Apr 28 '17 at 18:58
  • @Adrian also godep seems to have a problem with cloning private repositories, which vendoring seems to help with. – e.nikolov Apr 28 '17 at 19:02
  • I'm not closely familiar enough with godep to say; go get works with private repositories, so sounds like there's a defect specific to godep. As far as deployment, why are you deploying any of the code? One of the big joys of Go is that it produces statically-linked native binaries, so wherever you deploy to doesn't need to know anything about your dependencies. – Adrian Apr 28 '17 at 19:14
  • @Adrian We are using bitbucket's Pipelines, which automatically builds the code and runs the tests on it. Also we are trying to set up the pipelines to automatically build an executable and deploy it on our server if the tests pass. – e.nikolov Apr 28 '17 at 19:24
  • OK, I can get that as a build issue - easily solved in something like Jenkins but I don't know about BB pipelines. I was thrown off by "to deploy backend1 to some machine, I still need some way of getting backend-shared to be put there as well" – Adrian Apr 28 '17 at 19:26

3 Answers3

0

Maybe govendor can meet your needs. It can add one dependency package instead of all dependency package by following command:

cd your_project_path
govendor init                         // run only first time
govendor add path_to_external_package // like github.com/astaxie/beego
CNT
  • 1
0

I think that Glide would work better for you, since you can safely ignore vendoring your internal shared package and use the version that lives in the GOPATH (this will work if your backend1 and backend2 must compile with the latest version of backend, if it's not the case you maybe should prefer vendoring your package and using SemVer tags to remark the breaking changes made in backend, then lock to the versions that work for each project).

The glide.yaml file names the packages you're vendoring, there you can add the packages you want to ignore

glide.yaml docs

Juan Carlos Garcia
  • 541
  • 2
  • 4
0

Not wanting to indulge to much in favourite dependency tools, I will however mention Glock (https://github.com/robfig/glock) which manages dependencies without 'vendoring' per se. I use it on my own projects where I control the upstream repos too. It might be a good fit for your company for the same reason.

I also have a totally different suggestion. Use a single repo for all your projects, with a sub directory for each one. Although this is best when the overall codebase remains small, it does allow co-development organisation much better than having many repos.

For example, any developer can branch (containing everything of course) and work on inter-related components, possibly involving breaking changes. When it's finished and stable, it can be merged back all in one go. There are never any 'race conditions' with this method because the commits are atomic.

Rick-777
  • 9,714
  • 5
  • 34
  • 50