Testcontainers keycloak error when run in gitlab pipeline

Question

I am using Testcontainers to run some tests against keycloak. This works fine on my development machine but fails in gitlab pipeline.

The error I get is: Error connecting to http://docker:32769/realms/clienttest/.well-known/openid-configuration. HTTPS required. This error comes from the GetDiscoveryDocumentAsync extension method from the IdentityModel library which I use to get the bearer token for accessing the keycloak api. However HTTPS should not be required as the container is started with start-dev.

Creation of container

            new ContainerBuilder()
            .WithName("Keycloak-Client-Test-" + Guid.NewGuid().ToString("D"))
            .WithImage("quay.io/keycloak/keycloak:21.0.1")
            .WithPortBinding(privateKeycloakPort, true)
            .WithResourceMapping(new FileInfo("ContainerData/clienttest-realm.json"), new FileInfo("/opt/keycloak/data/import/realm.json"))
            .WithCommand("start-dev", "--import-realm")
            .WithWaitStrategy(Wait.ForUnixContainer().UntilHttpRequestIsSucceeded(r => r.ForPort(privateKeycloakPort)))
            .Build();

Test stage of pipeline

test:
  image: mcr.microsoft.com/dotnet/sdk:6.0
  stage: test
  services: 
    - name: docker:dind
      command: ["--tls=false"]
  variables:
    DOCKER_HOST: "tcp://docker:2375"
    DOCKER_TLS_CERTDIR: ""
    DOCKER_DRIVER: overlay2
  script:
    - dotnet test --logger:"junit;LogFilePath=UnitTestResult.xml"
  artifacts:
    paths:
      - "*/UnitTestResult.xml"
    reports:
      junit: "*/UnitTestResult.xml"

Does perhaps your imported realm require ssl? This is a property to be set on realm-level. — dasniko, Jul 04 '23 at 11:07
BTW: Do you know https://github.com/dasniko/testcontainers-keycloak (disclaimer: I'm the maintainer of this library) — dasniko, Jul 04 '23 at 11:08
@dasniko Require SSL for the realm was indeed set to external, changed to none but still the same error in the pipeline. — Eelke, Jul 04 '23 at 12:29
Maybe the port mapping is not sufficient. If you access the server with HTTP through the HTTPS port, it won't work. Have you mapped both ports? I only see one port binding in your code... — dasniko, Jul 04 '23 at 13:42

Eelke · Accepted Answer · 2023-07-07T05:31:40.873

Turns out the error was given by IdentityModel because its default policy only allows http on loopback which worked for my development machine but not on a pipeline with docker in docker.

I changed the GetDiscoveryDocumentAsync call from

    await loginClient.GetDiscoveryDocumentAsync(config.TokenIssuer);

to

    await loginClient.GetDiscoveryDocumentAsync(
        new DiscoveryDocumentRequest
        {
            Address = config.TokenIssuer,
            Policy = new()
            {
                RequireHttps = config.RequireHttps,
                AllowHttpOnLoopback = true,
            }
        });

Ofcourse the desired security level I pass in through the config so in production it stays secure and only in the tests is lowered.

Note per dasniko's comment I changed the realm Require SSL setting to none before I found out the source of the problem was in IdentityModel. I strongly suspect this is also needed but have not verified it.

Happy you found the root cause! Most likely it is not required to set `requireSSL` to `none`, should also work with `external`, as you are accessing the container through a private ip address. I just wanted to make sure that it is not set to `all`. — dasniko, Jul 07 '23 at 06:12

Testcontainers keycloak error when run in gitlab pipeline

1 Answers1