You have the wrong arguments order, what you want is git branch --set-upstream mainline origin/mainline
.
see:
-t, --track
When creating a new branch, set up configuration to mark the
start-point branch as "upstream" from the new branch. This
configuration will tell git to show the relationship between the two
branches in git status and git branch -v. Furthermore, it directs git
pull without arguments to pull from the upstream when the new branch
is checked out.
This behavior is the default when the start point is a remote-tracking
branch. Set the branch.autosetupmerge configuration variable to false
if you want git checkout and git branch to always behave as if
--no-track were given. Set it to always if you want this behavior when the start-point is either a local or remote-tracking branch.
--set-upstream
If specified branch does not exist yet or if --force has been given,
acts exactly like --track. Otherwise sets up configuration like
--track would when creating the branch, except that where branch points to is not changed.
So what your had did is creating a local branch named origin/mainline
which tracks mainline
branch.