5

I have asked the same question on github but no response. So, I think here may be a better place to ask this kind of coding questions because this is the problem when I tried to do some coding work.

I would like to set up airflow with docker on MacBook Pro with macOS 10.14.x.

I have got zpencerq/docker-airflow:1.10.2 and followed the instructions at https://docs.docker.com/compose/install/#install-compose

Also, I have set up the ssh agent forward with the instructions at https://github.com/uber-common/docker-ssh-agent-forward

When I run:

 docker-compose up -d

I got:

Collecting my-data-store from git+ssh://git@github.com/my-data-store.git@v.xx.xx.x#egg=my-data-store

Cloning ssh://git@github.com/my-data-store.git to /my/local/path

Permission denied (publickey).

fatal: Could not read from remote repository.

I have created ssh public key and added the ssh public key in my git profile. I have also added my private key.

I have tried some answers for this kind of question, but none of them work.

If I run the following command in Mac terminal, it works well.

git clone -q ssh://git@github.com/my_path/my-data-store.git /tmp/my_folder

Why docker-compose cannot do the same thing to access github ?

Do I miss something ?

Alexis Wilke
  • 19,179
  • 10
  • 84
  • 156
user3448011
  • 1,469
  • 1
  • 17
  • 39

3 Answers3

7

Sounds like you haven't added your keys. Try this from inside your OSX terminal...

ssh-add

Then try again.

allencoded
  • 7,015
  • 17
  • 72
  • 126
1

Why docker-compose cannot do the same thing to access github ?

Because it (docker) will look for your keys in $HOME/.ssh/id_rsa(.pub).

When you execute git clone -q ssh://git@github.com/... from your Mac, $HOME is your regular home directory.

But from a container, it is /root by default, or the one of the user.

You should mount those keys in your docker-compose.yml (or use secrets).

VonC
  • 1,262,500
  • 529
  • 4,410
  • 5,250
  • The location of the key on the virtual machine (created by virtualbox) is at /Users/my_id/.ssh. After adding it in the docker-compose.yml file, I run "docker-compose up -d" and got "WARNING: Connection pool is full, discarding connection:". I have run "docker network prune" but same error. – user3448011 Aug 18 '19 at 01:39
  • @user3448011 I am not talking about any SSH key created by VirtualBox. When you say "I have created ssh public key and added the ssh public key in my git profile", you should mean that you have created dedicated SSH keys. – VonC Aug 18 '19 at 05:48
  • The SSH key is not created by VirtualBox. It is created by the host (my local machine) by following https://help.github.com/en/articles/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent. – user3448011 Aug 18 '19 at 20:38
  • The SSH key has been forwarded to the docker-machine (created by a virtualbox). I just added the path of the key in the docker-machine into the docker-compose.yml by following your link https://stackoverflow.com/a/34933181/6309. – user3448011 Aug 18 '19 at 20:40
0

You can add to a container environment 

-e GIT_SSH_COMMAND="ssh -o StrictHostKeyChecking=no -o User=<github user> -i <private key>"

When is full key text making by ssh-keygen

ozlevka
  • 1,988
  • 16
  • 28