gitlab mulitple caches - global cache is not considered

Question

I want to use multiple caches in my .gitlab.yml file

1. global cache for maven dependencies, gradle wrapper, etc.
2. local cache for each build (share gradle build output)

It seems like the global cache is not considered when executing the build/test stage as gradle always loads the wrapper. How do i enable the global cache on the build stage ?

image:
  name: gradle:jdk11

before_script:
  - export GRADLE_USER_HOME=`pwd`/.gradle_home

cache: &global_cache
  key: gradle_global_cache
  #    files:
  #      - build.gradle
  paths:
    - $GRADLE_USER_HOME/caches
    - $GRADLE_USER_HOME/wrapper

build:
  stage: build
  script: ./gradlew assemble

  cache:
    key: "$CI_COMMIT_REF_NAME"
    policy: pull-push
    paths:
      - build
      - .gradle

test:
  stage: test
  script: ./gradlew check
  cache:
    key: "$CI_COMMIT_REF_NAME"
    policy: pull
    paths:
      - build
      - .gradle