Using Amazon ECR as a Docker build cache in Jenkins pipelines

Backend
1

I’m trying to set up a Jenkins pipeline that builds Docker images using Amazon ECR as a remote cache. My agents are temporary, so I want to boost cache hits. Here’s what I’ve got so far:

myBuilder = 'ecr-cache-builder'

pipeline {
  stages {
    stage('Build') {
      steps {
        script {
          // Set up a custom builder
          sh """
            docker buildx inspect ${myBuilder} || \
              docker buildx create \
                --driver=docker-container \
                --name=${myBuilder}
          """

          // Check that the builder exists
          sh 'docker buildx ls'

          // Build with the ECR registry
          docker.withRegistry("https://${ECR_URL}", "ecr:${REGION}:${ECR_CREDS}") {
            docker.build("${IMAGE_NAME}_${BRANCH}_${BUILD_NUM}",
             "--builder ${myBuilder} \
              --cache-to mode=max,image-manifest=true,oci-mediatypes=true,type=registry,ref=${ECR_URL}/${IMAGE_NAME}:cache \
              --cache-from type=registry,ref=${ECR_URL}/${IMAGE_NAME}:cache \
              ."
            )
          }
        }
      }
    }
  }
}

I set up a Docker Builder with a docker-container driver to support the use of ECR as a cache. The default docker driver doesn’t offer this functionality. Although the builder appears when I run docker buildx ls, the docker.build() step fails with a message saying the builder wasn’t found. Any thoughts on what might be going wrong?

2

Having encountered similar issues, I’d suggest a few troubleshooting steps. First, ensure your Jenkins agents have the latest Docker version supporting buildx. Sometimes, older versions cause compatibility problems.

Secondly, try explicitly setting the builder context before the build step:

sh "docker buildx use ${myBuilder}"

This ensures the correct builder is active.

Also, consider using the buildx command directly instead of docker.build():

sh "docker buildx build --builder ${myBuilder} [other options] ."

This approach gives you more control and verbose output for debugging.

Lastly, check if your Jenkins environment preserves the Docker context between steps. If not, you might need to recreate the builder in each stage or use a persistent volume for the Docker context.

3

hey there! have u tried using the buildx command directly instead of docker.build()? like this:

sh “docker buildx build --builder ${myBuilder} [your options] .”

it might give u more control and better error messages. also, make sure ur jenkins agents have the latest docker version that supports buildx properly. older versions can be a pain sometimes!

what other things have u tried so far? curious to hear more about ur setup!

4

hey, have u tried running buildx directly? try: sh “docker buildx build --builder ${myBuilder} [options] .”. also, check if jenkins agents run latest docker version supporting buildx; older versions cause issues.