SSH not working using Gitea on docker with traefik

Question

I have installed gitea on docker (docker-compose) with traefik (v2.3) as reverse proxy. I'm trying to set up ssh but it's failing, both to SSH into and (mainly) to perform git clone and push.

I get

<user>@<domain>: Permission denied (publickey).

I have tried specifying ports in the docker-compose file for the traefik container

ports:
  - "22:22"

and in the docker-compose for gitea I have the following labels:

  # SSH
  - "traefik.http.routers.gitea_ssh.rule=HOST(`gitea.localhost`)"
  - "traefik.http.routers.gitea_ssh.entrypoints=ssh"
  - "traefik.http.routers.gitea_ssh.service=gitea_ssh"
  # Services
  - "traefik.http.services.gitea_ssh.loadbalancer.server.port=22"

I have similar setup for http and https, http is redirected to https.

Entrypoints is defined in traefik.yml as ":22"

This however doesn't work. I figured signal flow would go like this:

ssh request -> server port 1234 -> docker port 22 -> traefik redirects -> gitea container port 22 

I have uploaded public key to

The response I get with this setup for ssh connection request is:

<login on computer>:/ <user>$ ssh -v <address to gitea>
OpenSSH_8.1p1, LibreSSL 2.7.3
debug1: Reading configuration data /<Path to config>/config
debug1: /<Path to config>/config line 12: Applying options for <address to gitea>
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 47: Applying options for *
debug1: Connecting to <address to gitea> port 1234.
debug1: Connection established.
debug1: identity file /<Path to private key>/private-key type 0
debug1: identity file /<Path to private key>/private-key-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.1
debug1: kex_exchange_identification: banner line 0: HTTP/1.1 400 Bad Request
debug1: kex_exchange_identification: banner line 1: Content-Type: text/plain; charset=utf-8
debug1: kex_exchange_identification: banner line 2: Connection: close
debug1: kex_exchange_identification: banner line 3: 
kex_exchange_identification: Connection closed by remote host

And when I try to access git clone:

<login on computer>:/ <user>$ git clone git@<address to gitea>:<path to repo>.git
Cloning into 'some-repo'...
kex_exchange_identification: Connection closed by remote host
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Answer 1

Example for gitea ssh available on port 222 because port 22 is used for host ssh.

1. Create a entrypoint for traefik called ssh for port 222
2. Add port config to traefik container 222:222/tcp (udp, sctp not needed)
3. for gitea you dont need to expose ports
4. configure gitea, add environment variables:

# details: https://docs.gitea.io/en-us/config-cheat-sheet/#admin-admin
# start gitea only ssh server, default use system which didn't work for me in any way
GITEA__server__START_SSH_SERVER: "true"
# this port is used in git clone, if not 22 this will add ssh:// to clone url which... i needed to remove everytime
GITEA__server__SSH_PORT: 22
# gitea ssh listen port
GITEA__server__SSH_LISTEN_PORT: 222

4. configure traefik for gitea with labels:

# host resolving doesn't work for ssh, so you can only use "*"
- "traefik.tcp.routers.gitea-ssh.rule=HostSNI(`*`)"
- "traefik.tcp.routers.gitea-ssh.entrypoints=ssh"
- "traefik.tcp.routers.gitea-ssh.service=gitea-ssh-svc"
- "traefik.tcp.services.gitea-ssh-svc.loadbalancer.server.port=222"

5. add your ssh key to your account, check if its a valid type, minimum bit aso. https://docs.gitea.io/en-us/config-cheat-sheet/#ssh-minimum-key-sizes-sshminimum_key_sizes
6. create a simple repository named Example
7. update on client side ~/.ssh/config:

Host MyGiteaInstance
  HostName git.example.de
  IdentityFile ~/.ssh/gitea
  User username
  Port 222

8. clone git@git.example.de:user/Example.git, for me it failed if there is port information in the url or if there is the protocol prefix like ssh://

Debugging tipps:

1. check traefik logs if there is an error like entrypoint not found
2. do a clone with verbose ssh: GIT_SSH_COMMAND="ssh -v" git clone git@git.example.de:user/Example.git

• if the host is unreachable -> wrong ssh port on your side or entrypoint not set in traefik
• host authenticity request = traefik is fine and client port config is maybe fine (multiple ssh server)
• permission denied -> ssh server in gitea started, correct port set, check gitea logs or ssh key not added to user

Complete not minimal example, if I missed something above:

networks:
  proxy:
    external: false
    internal: true
  web:
    external: false
  gitea:
    external: false
    internal: true

volumes:
  gitea:
  gitea_db:

services:
  traefik:
    image: traefik:v2.5
    command: --api.insecure=true --providers.docker
    ports:
      # entrypoint http
      - "80:80"
      # entrypoint https
      - "443:443"
      # entrypoint ssh
      - "222:222/tcp"
      # todo maybe behind vpn?
      # - "8080:8080"
    # todo create docker.sock proxy
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./traefik:/etc/traefik:ro
      - ./acme.json:/certs/acme.json
      - /etc/timezone:/etc/timezone:ro
      - /etc/localtime:/etc/localtime:ro
    networks: 
      - proxy
      - web
  
  watchtower:
    image: containrrr/watchtower:latest
    environment:
      WATCHTOWER_CLEANUP: "true"
      WATCHTOWER_REVIVE_STOPPED: "true"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /etc/timezone:/etc/timezone:ro
      - /etc/localtime:/etc/localtime:ro
    # disable for container?
    # LABEL com.centurylinklabs.watchtower.enable="false"
  

  gitea_db:
    image: postgres:14
    restart: unless-stopped
    environment:
      POSTGRES_USER: gitea
      POSTGRES_PASSWORD: ${GITEA_DB_PASSWD}
      POSTGRES_DB: gitea
    networks:
      - gitea
    volumes:
      - gitea_db:/var/lib/postgresql/data
      - /etc/timezone:/etc/timezone:ro
      - /etc/localtime:/etc/localtime:ro

  gitea:
    image: gitea/gitea:1.15
    depends_on:
      - gitea_db
    # https://docs.gitea.io/en-us/config-cheat-sheet/
    environment:
      USER_UID: 1000
      USER_GID: 1000
      # configuration:
      # some values are created from the documentation but untested, therefore uncommented
      # https://docs.gitea.io/en-us/config-cheat-sheet/#repository---local-repositorylocal
      ## default
      GITEA__default__RUN_MODE: prod
      GITEA__default__APP_NAME: "Gitea: Git with a cup of tea"
      ## database
      GITEA__database__DB_TYPE: postgres
      GITEA__database__HOST: gitea_db:5432
      GITEA__database__NAME: gitea
      GITEA__database__USER: gitea
      GITEA__database__PASSWD: ${GITEA_DB_PASSWD}
      ## server
      GITEA__server__DOMAIN: ${GITEA_DOMAIN}
      # ensuring unneccessary port isnt added
      GITEA__server__ROOT_URL: "https://%(DOMAIN)s"
      GITEA__server__HTTP_PORT: 80
      # if true SSH_LISTEN_PORT needs to be != 22
      GITEA__server__START_SSH_SERVER: "true"
      # SSH port in clone URL (needs to be 22 to remove ssh:// from clone url)
      GITEA__server__SSH_PORT: 22
      # disable forced ssh:// prefix of clone url
      GITEA__repository__USE_COMPAT_SSH_URI: "false"
      # SSH port for built-in SSH server (e.g. docker run ... -p SSH_PORT:LISTEN_PORT)
      GITEA__server__SSH_LISTEN_PORT: 222
      
      # mailer
      GITEA__mailer__ENABLED: "true"
      GITEA__mailer__HOST: "${GITEA_MAIL_HOST}"
      GITEA__mailer__FROM: "${GITEA_MAIL}"
      GITEA__mailer__USER: "${GITEA_MAIL}"
      # require email confirmation to register, enable email notifications
      # no effect: GITEA__server__REGISTER_EMAIL_CONFIRM: "true"
      # allows mails as notifications on updates
      # no effect: GITEA__server__ENABLE_NOTIFY_MAIL: "true"
      #
      # open-id sign
      GITEA__openid__ENABLE_OPENID_SIGNIN: "true"
      # disable self-registration
      GITEA__service__DISABLE_REGISTRATION: "true"
      # require sign in to view pages
      GITEA__service__REQUIRE_SIGNIN_VIEW: "true"
      # password hash argon2
      GITEA__security__PASSWORD_HASH_ALGO: "argon2"
      #
      # change default branch from master to x
      GITEA__repository__DEFAULT_BRANCH: "main"
      #
      # set manually during install, keys don't work for unknown reason:
      # Email Settings:
      # SMTP Password:
      # GITEA__mailer__PASSWD: "${GITEA_MAIL_PASSWORD}"
      # [x] Require Email Confirmation to Register
      # [x] Enable Email Notifications (to watch repos / issues aso.)
      #
      # set manually, no config option found:
      # Server and Third-Party Service Settings:
      # [x] Enable Local Mode (disable all third party content)
      #
      # Admin Account:
      # admin: ${GITEA_ADMIN_NAME}
      # pw: "${GITEA_MAIL_PASSWORD}"
      # mail: ${GITEA_MAIL}
      #
      # check configuration here:
      # https://${GITEA_DOMAIN}/admin/config
      # 
      # remember correct port isn't part of ssh clone url, but works with ~/.ssh/config

    restart: unless-stopped
    networks:
      - gitea
      - proxy
    volumes:
      - gitea:/data
      - /etc/timezone:/etc/timezone:ro
      - /etc/localtime:/etc/localtime:ro
    labels:
      - "traefik.enable=true"
      - "traefik.http.services.gitea.loadbalancer.server.port=80"
      - "traefik.http.routers.gitea_insecure.rule=Host(`${GITEA_DOMAIN}`)"
      - "traefik.http.routers.gitea_insecure.entrypoints=http"
      - "traefik.http.routers.gitea_insecure.middlewares=https-redirect@file"
      - "traefik.http.routers.gitea.rule=Host(`${GITEA_DOMAIN}`)"
      - "traefik.http.routers.gitea.entrypoints=https"
      - "traefik.http.routers.gitea.tls.certresolver=tlsChallenge"
# https://community.traefik.io/t/routing-ssh-traffic-with-traefik-v2/717 
      # ssh cant be set on specific domain
      - "traefik.tcp.routers.gitea-ssh.rule=HostSNI(`*`)"
      - "traefik.tcp.routers.gitea-ssh.entrypoints=ssh"
      - "traefik.tcp.routers.gitea-ssh.service=gitea-ssh-svc"
      - "traefik.tcp.services.gitea-ssh-svc.loadbalancer.server.port=222"

.env file:

GITEA_DOMAIN=git.example.de
GITEA_DB_PASSWD=securePassword
GITEA_ADMIN_NAME=admin name
GITEA_MAIL_PASSWORD=securePassword
GITEA_MAIL_HOST=smtp.mymail.de:465
GITEA_MAIL=yourEmail

./traefik/dynamic.yml

http:
  middlewares:
    https-redirect:
      redirectScheme:
        scheme: https

./traefik/traefik.yml

global:
  sendAnonymousUsage: false

log:
  level: INFO

entryPoints:
  http:
    address: :80
  https:
    address: :443
  ssh:
    address: :222

defaultEntryPoints:
  - https

api:
  insecure: true
  dashboard: true

providers:
  docker:
    endpoint: unix:///var/run/docker.sock
    watch: true
    exposedByDefault: false
    network: shared_proxy
  file:
    filename: /etc/traefik/dynamic.yml
    watch: true

certificatesResolvers:
  tlsChallenge:
    acme:
      email: MyEmailAdress
      storage: /certs/acme.json
      tlsChallenge: {}
      # https://doc.traefik.io/traefik/https/acme/#caserver
      # caServer: https://acme-staging-v02.api.letsencrypt.org/directory # For test certificates

Answer 2

I don't have first hand experience with proxying ssh via traefik but i've looked at this article before:

https://www.georglutz.de/blog/2020/06/20/homeassistant-with-traefik-and-ssh/

and makes sense to recommend it, since it's the same setup.

and i would point out that you're wrongly using the http router, you need the tcp one.

I use Gitea and I would advise just exposing SSH, on a different port; Traefik will only give you overhead.