Automating AWS S3 Bucket Cleanup with CloudFormation Lambda Hooks

As organizations increasingly adopt Infrastructure as Code (IaC) practices, managing the lifecycle of cloud resources becomes both more automated and more complex. One common challenge is handling S3 buckets when tearing down CloudFormation stacks. In this post, I’ll share a solution that leverages AWS CloudFormation Lambda Hooks to automatically empty S3 buckets before deletion.

GitHub Repo: https://github.com/jftuga/cloudformation-s3-auto-empty-hook

The S3 Bucket Deletion Challenge

Anyone who’s worked with AWS CloudFormation knows the frustration: you try to delete a stack, but it fails because an S3 bucket isn’t empty. The standard workflow becomes:

Manually empty the bucket, including any versioned objects
Retry the stack deletion
Hope no one uploaded new files in the meantime

This manual step breaks automation and can cause delays in CI/CD pipelines or development workflows.

CloudFormation Stack Architecture: Separation of Concerns

The solution I’m proposing uses a “separation of concerns” approach with two distinct CloudFormation stacks:

Lambda Hook Infrastructure Stack: A long-lived, foundational stack that creates the Lambda function and IAM roles needed for the hook mechanism.
S3 Bucket Resources Stack: A more ephemeral stack containing the actual S3 bucket and the hook configuration that references the Lambda.

This separation provides several benefits:

The hook infrastructure can be deployed once and reused across multiple bucket stacks
Different teams can deploy bucket stacks without needing permissions to create Lambda functions or IAM roles
Updating the hook logic affects all buckets without needing to update each bucket stack

How the Solution Works

When you initiate a deletion of a CloudFormation stack that contains an S3 bucket:

CloudFormation triggers the Lambda hook before it attempts to delete the bucket
The Lambda function empties the bucket by deleting all objects and versions
Once empty, CloudFormation proceeds with deleting the bucket itself

This makes stack deletion a single-step, fully automated process. No more manual intervention!

The Lambda Hook Infrastructure

The first stack (lambda-hook-infrastructure.yaml) defines:

# Key components of the hook infrastructure
CfnHookEmptyS3BucketLambda:
  Type: AWS::Serverless::Function
  Properties:
    Description: CFN hook to empty S3 buckets
    CodeUri: src/lambda-hooks/empty-s3-bucket
    FunctionName: !Ref CfnHookLambdaName
    # IAM policies to allow emptying buckets

CfnHookEmptyS3BucketInvokerRole:
  Type: AWS::IAM::Role
  Properties:
    # Role that allows CloudFormation to invoke the Lambda

Full implementation: lambda-hook-infrastructure.yaml

A note on permissions: The Lambda function is granted permissions to operate on all S3 buckets within the same AWS account (using the aws:ResourceAccount condition). While this permission scope is broader than the principle of least privilege might suggest, it’s appropriate for this use case since:

The hook may need to empty different buckets across various stacks
The bucket names might be dynamically generated
The function’s only capability is to empty buckets, not to access or modify their contents for other purposes
The Lambda includes validation checks to ensure it can only be invoked by CloudFormation through the defined hook mechanism and specifically for DELETE operations

These validation checks provide an additional layer of security by confirming that the Lambda function is only being called in the expected context, even if someone were to attempt to invoke it directly. You can further restrict these permissions if your organization’s security posture requires it.

The S3 Bucket Resources

The second stack (s3-bucket-resources.yaml) defines:

S3Bucket:
  Type: AWS::S3::Bucket
  DependsOn: LambdaEmptyS3BucketHook
  Properties:
    # Bucket configuration with security best practices

LambdaEmptyS3BucketHook:
  Type: AWS::CloudFormation::LambdaHook
  Properties:
    # Hook configuration that points to the Lambda function
    # Configured to run before DELETE operations

Full implementation: s3-bucket-resources.yaml

The Lambda Function

The core of this solution is the Lambda function that empties the S3 bucket. Here’s an abridged version of Python code that powers it:

import boto3

s3_client = boto3.client("s3")
s3 = boto3.resource('s3')

def empty_bucket(bucket_name: str) -> dict:
    bucket = s3.Bucket(bucket_name)
    delete_response = bucket.objects.delete()
    bucket.object_versions.delete()
    return delete_response

def lambda_handler(event, context):
    # Validate that this is being called by CloudFormation Hooks
    if not event.get("hookTypeName") or not event.get("actionInvocationPoint") or not event.get("clientRequestToken"):
        error_msg = "This Lambda function can only be invoked by CloudFormation Hooks"
        return {"hookStatus": "FAILURE", "errorCode": "Unauthorized", "message": error_msg}

    # Validate the action is specifically a DELETE pre-provision operation
    if event.get("actionInvocationPoint") != "DELETE_PRE_PROVISION":
        error_msg = f"This Lambda function is only authorized for DELETE_PRE_PROVISION operations, received: {event.get('actionInvocationPoint')}"
        return {"hookStatus": "FAILURE", "errorCode": "Unauthorized", "message": error_msg, "clientRequestToken": event["clientRequestToken"]}

    try:
        bucket_name = event["requestData"]["targetModel"]["resourceProperties"]["BucketName"]
    except KeyError as err:
        return {"hookStatus": "FAILURE", "errorCode": "NonCompliant", "message": f"{err}", "clientRequestToken": event["clientRequestToken"]}

    empty_bucket(bucket_name)
    return {"hookStatus": "SUCCESS", "message": "compliant", "clientRequestToken": event["clientRequestToken"]}

Full implementation: empty-s3-bucket/index.py

The Lambda function:

Validates that it’s being called by CloudFormation Hooks
Ensures it’s only used for DELETE operations
Extracts the bucket name from the CloudFormation event
Deletes all objects in the bucket, including versioned objects using the AWS SDK
Returns success to CloudFormation to continue with bucket deletion

Lambda Hooks vs. DeletionPolicy

AWS CloudFormation offers a built-in DeletionPolicy attribute for resources, which has two primary options relevant to S3 buckets:

DeletionPolicy: Retain

Setting DeletionPolicy: Retain tells CloudFormation to leave the bucket intact when the stack is deleted.

Pros:

Simple to implement
No additional code needed
Protects against accidental data loss

Cons:

Leaves “orphaned” resources after stack deletion
Requires manual cleanup later
Can cause naming conflicts when redeploying stacks

DeletionPolicy: Delete (Default)

The default behavior attempts to delete the bucket.

Pros:

Clean removal of all resources
No orphaned resources to manage

Cons:

Fails if the bucket contains objects
Requires manual emptying before deletion

Lambda Hooks Approach

Pros:

Fully automated deletion process
No manual intervention required
Clean separation of infrastructure from application resources
Reusable across multiple stacks

Cons:

More complex to set up initially
Requires managing additional resources (Lambda, IAM roles)
Potential Lambda execution costs (minimal in most cases)

When to Use Each Approach

Use DeletionPolicy: Retain when:

You need to preserve data regardless of stack lifecycle
The bucket contains critical data that should never be automatically deleted
Regulatory requirements mandate data preservation
You’re doing a temporary stack update and want to preserve the data

Use DeletionPolicy: Delete with Lambda Hooks when:

You want fully automated stack cleanup, including in CI/CD pipelines
Development environments need frequent recreation
You want to enforce the “infrastructure as code” principle completely
Test data in buckets is disposable when the stack is removed

Use Both Together when:

You want protection against accidental deletions but also want automation
Different buckets in the same stack have different data sensitivity
You’re transitioning from a manual process to full automation

Implementation

The implementation consists of three files:

lambda-hook-infrastructure.yaml: Defines the Lambda function and IAM role
s3-bucket-resources.yaml: Defines the S3 bucket and hook configuration
deploy-stacks.sh: Deployment script with command-line options

Since there is a wide variety of CI systems ranging from Jenkins to GitHub Actions, for the simplicity of this blog post, I am using a shell script because it is the lowest common denominator.

The deployment script is designed to be flexible, allowing you to deploy just the hook infrastructure, just the bucket resources, or both, with different AWS profiles and regions.

# Example deployments
./deploy-stacks.sh                      # Deploy both stacks
./deploy-stacks.sh --hook-only          # Deploy only hook infrastructure
./deploy-stacks.sh --bucket-only        # Deploy only bucket resources
./deploy-stacks.sh --profile prod       # Use a specific AWS profile

Conclusion

Automating S3 bucket emptying with CloudFormation Lambda Hooks provides a robust solution that maintains the integrity of your Infrastructure as Code approach. By separating the hook infrastructure from the bucket resources, you create a reusable component that can be leveraged across multiple projects and teams.

This pattern exemplifies an important principle in cloud architecture: building automated solutions that remove manual steps and human error from your deployment processes. While it requires a bit more upfront investment than simply setting a DeletionPolicy, the long-term benefits in developer productivity and system reliability are substantial.

For organizations heavily invested in AWS and CloudFormation, implementing patterns like this one helps standardize resource lifecycle management and ensures consistent behavior across environments. Whether you’re managing dev, test, or production resources, having predictable, automated cleanup processes is essential for maintaining a well-governed cloud environment.

The complete code for this solution is available in the GitHub repository linked below. Give it a try in your own environment, and feel free to adapt it to your specific requirements!

https://github.com/jftuga/cloudformation-s3-auto-empty-hook

The S3 Bucket Deletion Challenge#

CloudFormation Stack Architecture: Separation of Concerns#

How the Solution Works#

The Lambda Hook Infrastructure#

The S3 Bucket Resources#

The Lambda Function#

Lambda Hooks vs. DeletionPolicy#

DeletionPolicy: Retain#

DeletionPolicy: Delete (Default)#

Lambda Hooks Approach#

When to Use Each Approach#

Use DeletionPolicy: Retain when:#

Use DeletionPolicy: Delete with Lambda Hooks when:#

Use Both Together when:#

Implementation#

Conclusion#

The S3 Bucket Deletion Challenge

CloudFormation Stack Architecture: Separation of Concerns

How the Solution Works

The Lambda Hook Infrastructure

The S3 Bucket Resources

The Lambda Function

Lambda Hooks vs. DeletionPolicy

DeletionPolicy: Retain

DeletionPolicy: Delete (Default)

Lambda Hooks Approach

When to Use Each Approach

Use DeletionPolicy: Retain when:

Use DeletionPolicy: Delete with Lambda Hooks when:

Use Both Together when:

Implementation

Conclusion