Using ECR for Docker build cache: Encountering manifest issues

Backend
1

I’m switching my Docker build cache from S3 to ECR to reduce costs. My images are stored in ECR and deployed to ECS. The first build runs without issues, but the second one fails with a 400 Bad Request error when it attempts to write the manifest blob.

Here’s a revised version of my build command:

docker buildx build --push \
    --cache-to type=registry,ref=myecr.amazonaws.com/cache:service-env,mode=max \
    --cache-from type=registry,ref=myecr.amazonaws.com/cache:service-env \
    --tag "myimage:latest" .

I have enabled both oci-mediatypes and image-manifest as recommended by the documentation, yet the ECR repository only displays images with their digest and tag, and no manifests. Why does this issue occur with ECR rather than with S3, and what might be missing in my configuration?

2

hmm, interesting issue! have u tried using a different tag for the cache repo? maybe ECR is getting confused with the ‘service-env’ tag. also, whats ur ECR repository policy look like? could there be permissions issues? just brainstorming here - curious to hear if u’ve explored these angles?

3

yo, had similar probs. try tweakin ur ECR lifecycle policy. sometimes it messes w/ cache layers. also, double-check ur IAM perms for ECR - might need ‘ecr:BatchCheckLayerAvailability’ n ‘ecr:PutImage’. if that dont work, maybe switch to a multi-stage build? good luck man

4

I encountered a similar issue when transitioning to ECR for cache storage. One solution that worked for me was implementing a separate repository specifically for caching. This approach helps avoid conflicts with your main image repository.

Additionally, ensure your Docker version is up-to-date, as older versions may have compatibility issues with ECR’s manifest handling. If the problem persists, consider adjusting the ‘mode’ parameter in your cache-to flag. Using ‘mode=max,oci-mediatypes=true’ explicitly might resolve the manifest writing issue.

Lastly, verify that your ECR repository doesn’t have any size limitations that could be preventing the cache layers from being stored properly. These steps should help address the manifest issues you’re experiencing.