I'll go ahead and answer my own question since I figured it out:
cd to gogs directory
nano gogs.go
Look at the first line for me it was
// +build go1.5
This means you will at least need go 1.5 to build. Now look at your go version:
go --version
If this number is smaller, this is the reason your build won't even start.
In my case ubuntu was using an older version located at
whereis go
I deleted the old binary folder (probably something you shouldn't do) and reinstalled go from scratch, which gave me the corret version and the build worked fine.