0

I'm trying to setup cloud native buildpacks (paketo builder) behind a corporate proxy, and I have "x509: certificate signed by unknown authority". This is because the proxy rewrites certificates.

I tried follwing https://paketo.io/docs/howto/configuration/#ca-certificates (create a "binding" folder in the project with the .pem file of the root CA of my proxy and a "type" file containing "ca-certificate"), but this had no effect.

  • Hints?
  • Is it possible to disable this check (workaroud for testing)

Logs

pack build test --builder paketobuildpacks/builder:base -e BP_JVM_VERSION=17
base: Pulling from paketobuildpacks/builder
Digest: sha256:855aaa00a7eadca9c6cdba72550889f7c618c2bf1489c29fb9e7e01588665db7
Status: Image is up to date for paketobuildpacks/builder:base
base-cnb: Pulling from paketobuildpacks/run
Digest: sha256:1e2a282085f1b7a0fa1d96bd8509378270ab752c6d8042472a95df2bf5e1d6ea
Status: Image is up to date for paketobuildpacks/run:base-cnb
Previous image with name "test" not found
===> DETECTING
9 of 24 buildpacks participating
paketo-buildpacks/ca-certificates   3.2.4
paketo-buildpacks/bellsoft-liberica 9.4.0
paketo-buildpacks/syft              1.13.0
paketo-buildpacks/maven             6.6.0
paketo-buildpacks/executable-jar    6.2.4
paketo-buildpacks/apache-tomcat     7.3.7
paketo-buildpacks/liberty           1.1.2
paketo-buildpacks/dist-zip          5.2.4
paketo-buildpacks/spring-boot       5.13.0
===> RESTORING
===> BUILDING

Paketo CA Certificates Buildpack 3.2.4
  https://github.com/paketo-buildpacks/ca-certificates
  Launch Helper: Contributing to layer
    Creating /layers/paketo-buildpacks_ca-certificates/helper/exec.d/ca-certificates-helper

Paketo BellSoft Liberica Buildpack 9.4.0
  https://github.com/paketo-buildpacks/bellsoft-liberica
  Build Configuration:
    $BP_JVM_JLINK_ARGS           --no-man-pages --no-header-files --strip-debug --compress=1  configure custom link arguments (--output must be omitted)
    $BP_JVM_JLINK_ENABLED        false                                                        enables running jlink tool to generate custom JRE
    $BP_JVM_TYPE                 JRE                                                          the JVM type - JDK or JRE
    $BP_JVM_VERSION              17                                                           the Java version
  Launch Configuration:
    $BPL_DEBUG_ENABLED           false                                                        enables Java remote debugging support
    $BPL_DEBUG_PORT              8000                                                         configure the remote debugging port
    $BPL_DEBUG_SUSPEND           false                                                        configure whether to suspend execution until a debugger has attached
    $BPL_HEAP_DUMP_PATH                                                                       write heap dumps on error to this path
    $BPL_JAVA_NMT_ENABLED        true                                                         enables Java Native Memory Tracking (NMT)
    $BPL_JAVA_NMT_LEVEL          summary                                                      configure level of NMT, summary or detail
    $BPL_JFR_ARGS                                                                             configure custom Java Flight Recording (JFR) arguments
    $BPL_JFR_ENABLED             false                                                        enables Java Flight Recording (JFR)
    $BPL_JMX_ENABLED             false                                                        enables Java Management Extensions (JMX)
    $BPL_JMX_PORT                5000                                                         configure the JMX port
    $BPL_JVM_HEAD_ROOM           0                                                            the headroom in memory calculation
    $BPL_JVM_LOADED_CLASS_COUNT  35% of classes                                               the number of loaded classes in memory calculation
    $BPL_JVM_THREAD_COUNT        250                                                          the number of threads in memory calculation
    $JAVA_TOOL_OPTIONS                                                                        the JVM launch flags
    Using Java version 17 from BP_JVM_VERSION
  BellSoft Liberica JDK 17.0.3: Contributing to layer
    Downloading from https://github.com/bell-sw/Liberica/releases/download/17.0.3.1+2/bellsoft-jdk17.0.3.1+2-linux-amd64.tar.gz
unable to invoke layer creator
unable to get dependency jdk
unable to download https://github.com/bell-sw/Liberica/releases/download/17.0.3.1+2/bellsoft-jdk17.0.3.1+2-linux-amd64.tar.gz
unable to request https://github.com/bell-sw/Liberica/releases/download/17.0.3.1+2/bellsoft-jdk17.0.3.1+2-linux-amd64.tar.gz
Get "https://objects.githubusercontent.com/github-production-release-asset-2e65be/115621629/40b89c3a-208c-434b-8604-3dd53f5dfde1?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20220713%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220713T111101Z&X-Amz-Expires=300&X-Amz-Signature=24802ad67a837b74ba5d682a5891905dd219f38f524af3d3b19b714800b40243&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=115621629&response-content-disposition=attachment%3B%20filename%3Dbellsoft-jdk17.0.3.1%2B2-linux-amd64.tar.gz&response-content-type=application%2Foctet-stream": x509: certificate signed by unknown authority
ERROR: failed to build: exit status 1
ERROR: failed to build: executing lifecycle: failed with status code: 51
Remy
  • 502
  • 6
  • 19

1 Answers1

1

You need to pass the binding to the pack build command, so it's present at build time.

Your command is pack build test --builder paketobuildpacks/builder:base -e BP_JVM_VERSION=17, you need to include the --volume mapping.

pack build test --builder paketobuildpacks/builder:base -e BP_JVM_VERSION=17 --volume $PWD/bindings:/platform/bindings

Where $PWD/bindings references the bindings directory you created.

$ tree bindings/
bindings/
├── ca-certificates
│   ├── support-labs-ca.crt
│   └── type

When your build runs, you'll know it's working because the CA Certificates buildpack will look like this:


Paketo CA Certificates Buildpack 3.2.4
  https://github.com/paketo-buildpacks/ca-certificates
  Launch Helper: Contributing to layer
    Creating /layers/paketo-buildpacks_ca-certificates/helper/exec.d/ca-certificates-helper
  CA Certificates: Contributing to layer
    Added 1 additional CA certificate(s) to system truststore
    Writing env.build/SSL_CERT_DIR.append
    Writing env.build/SSL_CERT_DIR.delim
    Writing env.build/SSL_CERT_FILE.default

You can see it's Added 1 additional CA certificate(s) to system truststore. That's your CA cert.

You can also confirm by looking at the JVM buildpack, when it installs the JDK tools, it will add your certificate:

  BellSoft Liberica JDK 11.0.15: Contributing to layer
    Downloading from https://github.com/bell-sw/Liberica/releases/download/11.0.15.1+2/bellsoft-jdk11.0.15.1+2-linux-amd64.tar.gz
    Verifying checksum
    Expanding to /layers/paketo-buildpacks_bellsoft-liberica/jdk
    Adding 128 container CA certificates to JVM truststore
    Writing env.build/JAVA_HOME.override
    Writing env.build/JDK_HOME.override

You can see it's Adding 128 container CA certificates to JVM truststore (by default there are 127 certificates, so it's default + 1).

You might also want to take a look at binding-tool, which is a small CLI you can install that will help you manage bindings, including CA Certs.

  1. It can be used to easily create the binding folder structure. Just give it your certificate file & it'll create the structure.

  2. You can also integrate it with your shell and it'll automatically augment your pack build command to add the --volume argument so you don't need to think about that.

It's not a requirement to use it, but thought I'd point it out as it can make working with pack and bindings easier.

Daniel Mikusa
  • 13,716
  • 1
  • 22
  • 28
  • Great answer, thank you! But I don't got the line "Added 1 additional CA certificate(s)...". pack build test --builder paketobuildpacks/builder:base -e BP_JVM_VERSION=17 --volume bindings:/platform/bindings [...] ===> DETECTING 9 of 24 buildpacks participating paketo-buildpacks/ca-certificates 3.2.4 [...] Paketo CA Certificates Buildpack 3.2.4 https://github.com/paketo-buildpacks/ca-certificates Launch Helper: Contributing to layer Creating /layers/paketo-buildpacks_ca-certificates/helper/exec.d/ca-certificates-helper Paketo BellSoft Liberica Buildpack 9.4.0 [...] – Remy Jul 15 '22 at 08:01
  • 1
    If you don't see that then something is either a.) wrong with your binding (type isn't right or the folder structure is wrong) causing the buildpack to not see the binding or b.) the binding is not being mounted into the container. My suspicion is b.) since I don't see a `--volume` flag in your `pack build` command. You can turn on debug logging for the buildpack and it's possible you might get some more clues. You'd need to add `-v -e BP_DEBUG=true` to your `pack build` command. Attach that if you're still stuck. – Daniel Mikusa Jul 16 '22 at 19:57
  • You're right, the volume command was ignored. Actually, it didn't find the bindings directory with a relative path. With an absolute path, it works: pack build test --builder paketobuildpacks/builder:base -e BP_JVM_VERSION=17 --volume %CD%\bindings:/platform/bindings – Remy Jul 19 '22 at 10:40
  • @DanielMikusa after enabling BP_DEBUG=true, I am getting debug logs. but not sure if its adding the certificate and binding in system keystore: logs: Buildpack contents: [...] Buildpack: {API:0.7 Info:{ID:paketo-buildpacks/ca-certificates Name:Paketo Buildpack for CA Certificates Version:3.5.1 Platform contents: [. bindings bindings/ca-certificates env env/BP_DEBUG env/BP_EMBED_CERTS env/BP_JVM_VERSION env/SSL_CERT_DIR] Platform Bindings: [{Name: ca-certificates Path: /platform/bindings/ca-certificates Type: Provider: Secret: []}] – Sanket wani Feb 24 '23 at 17:45
  • `Platform contents: [..]` shows the files in `/platform`, so you should see your binding there if it's in the right place and mounted correctly. In this case, it just shows `bindings bindings/ca-certificates` which means you have two directories and no files. I have seen this in the past and it's usually a problem with the binding, permissions or the Docker server. The *Docker Server* needs access to the files you want to mount. Most of the time Docker Client/Server are effectively the same, like with Docker Desktop but if you're in CI that's not always true. – Daniel Mikusa Feb 24 '23 at 19:39
  • Thanks for explanation Daniel, I have added correct folder contents now and binding is reflecting at buildtime like this:Platform Bindings: [{Name: ca-certificates Path: /platform/bindings/ca-certificates Type: ca-certificates Provider: Secret: [AAACertificateServices.pem]}] but I don't see certificate getting added in jvm truststore. Thanks in advance – Sanket wani Feb 28 '23 at 09:06
  • Does the ca-certs buildpack pick it up? That's the first step. Do you see `Added X additional CA certificate(s) to system truststore`, where `X` is the number of ca certs you're adding? – Daniel Mikusa Feb 28 '23 at 19:13
  • Hi Daniel, Yes its picking up the certificates... I added 2 bindings and I can see line : Added 2 additional CA certificate(s) to system truststore, but total number is not increasing in JVM buildpack(124 default ca-certificates) but I expect 126 in jvm buildpack. – Sanket wani Mar 01 '23 at 10:54
  • I've not seen that. If the CA certs are loaded as system certs, then the JVM buildpack should find them. Sounds like a different issue. I would suggest you start a new question and provide the full output of your build log, as well as the structure of your bindings. – Daniel Mikusa Mar 01 '23 at 13:33