9

Every build artifact is 39MB, and every time I submit a build, it adds another 39MB artifact to the codepipeline S3 bucket. Is there a way to automatically delete old artifacts?

Improve this question

2 Answers 2

Reset to default
11

You can use S3 lifecycle policies to automatically expire old objects: http://docs.aws.amazon.com/AmazonS3/latest/dev/object-lifecycle-mgmt.html

Two things to keep in mind when picking an expiration time:

  • Deployment providers like CodeDeploy might need access to old versions to scale or roll back
  • Your pipeline may take many days or weeks to run depending on how it's configured

So long as you deploy regularly then it should be fine to pick an expiration in the order of a month or so.

If your pipeline rarely runs then it might be harder to expire objects based on age because you might expire a version that's still current.

Improve this answer
3
  • We have a similar issue, where we build infrequently enough that lifecycle policies could potentially remove recent builds. We are considering creating a Lambda function that will effectively delete all but the N most recently modified objects in the artifact prefix, and set it to run nightly using CloudWatch Events. It's a relatively hands-off solution that should also be in the Lambda free tier for most people. However, it's not bulletproof -- if too many builds fail, the failed artifacts could push "good" builds out of the N-most-recent list and get purged.
    – cdhowie
    Nov 1, 2017 at 19:15
  • I suspect there's a CodePipeline API that could be used to determine which artifacts are from good vs. bad runs, so the logic could instead be to delete all but the N most recent good and N most recent bad builds.
    – cdhowie
    Nov 1, 2017 at 19:19
  • It seems to dangerous to rely on s3 lifecycle rules for the reason stated by cdhowie, I would think the best way would be to trawl for artifacts that are unreferenced - e.g. for lambda, presumably there's an API you can use to retrieve the artifact corresponding to a specific lambda function so any artifact not referenced by any lambda is safe to delete. However, I'm guessing that CodeBuild/CodePipeline may generate other artifacts you're not aware of, so a bit of trial and error might be necessary
    – Andy
    Sep 10, 2022 at 12:53
0

Do you really need to keep old build artifacts at all?

If you're building lambdas, once the function has been created, lambda keeps its own copy of the deployment package so you can delete it from the artifact bucket entirely (and you can even retrieve a copy from lambda using the 'export function' menu)

Caveat: this is based on my own experience using nodejs. I can't find anything in the documentation that guarantees this but the fact that lambda allows you to download the package, and doesn't store a reference to the original build artifact would seem to back this up.

Improve this answer
2
  • 1
    I think you might run into problems if you are using something like CloudFormation. Even if it were true that lambda permanently caches the image (which I doubt anyway), if you delete the zip source code artifact and some future deployment of lambda fails, CFn will try to rollback to the previous source code, but that is now no longer present in the artifacts path in the CFn template. So the update-rollback will fail too and that's a nasty state to be in.
    – ustulation
    Nov 1, 2022 at 19:33
  • good point - I'd agree that in the event of a CloudFormation rolback you would need the previous artifacts. Having said that I still believe that lamba keeps its own copy which is more than just a cache - as far as I can see, lambda doesn't even remember the original artifact location.
    – Andy
    Nov 9, 2022 at 10:06

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .