Delete all files from aws S3 bucket

Amazon Web Services (AWS) Simple Storage Service (S3) is a Storage-as-a-Service provided by Amazon.  It’s a general purpose object store, the objects are grouped under a name space called “buckets.” The buckets are unique across the entire AWS S3.

Boto library is the official Python SDK for software development and it provides APIs to work with AWS services like Dynamodb, S3, and others.

In this article, I will focus on how to delete all files in Amazon S3 bucket using Python Boto3 library.
First let’s use profile name as credential to access AWS S3

s3 = boto3.session.Session(profile_name='dev').resource('s3', region_name='us-east-1')
my_bucket = s3.Bucket(AWS_BUCKET_NAME)
for my_bucket_object in my_bucket.objects.all():
    if not my_bucket_object.key.find(AWS_S3_FILE_KEY):
        KEY = my_bucket_object.key
        BUCKET_NAME = my_bucket_object.bucket_name
        obj = s3.Object(BUCKET_NAME, KEY)
        obj.delete()

Second you can also use access and secret key as credential to access AWS S3

AWS_ACCESS_KEY         = "YOUR ACCESS KEY"
AWS_ACCESS_SECRET_KEY  = "YOU SECRET KEY"
    s3 = boto3.resource('s3', aws_access_key_id=AWS_ACCESS_KEY,aws_secret_access_key=AWS_ACCESS_SECRET_KEY)
for my_bucket_object in my_bucket.objects.all():
    if not my_bucket_object.key.find(AWS_S3_FILE_KEY):
        KEY = my_bucket_object.key
        BUCKET_NAME = my_bucket_object.bucket_name
        obj = s3.Object(BUCKET_NAME, KEY)
        obj.delete()

let’s wrap up everything into a function.

AWS_ACCESS_KEY         = "YOUR ACCESS KEY"
AWS_ACCESS_SECRET_KEY  = "YOU SECRET KEY"
AWS_BUCKET_NAME        = "YOUR BUCKET NAME"
def delete_from_aws_s3_bucket(AWS_BUCKET_NAME,AWS_ACCESS_KEY,AWS_ACCESS_SECRET_KEY):
    s3 = boto3.resource('s3', aws_access_key_id=AWS_ACCESS_KEY,aws_secret_access_key=AWS_ACCESS_SECRET_KEY)
    my_bucket = s3.Bucket(AWS_BUCKET_NAME)
    for my_bucket_object in my_bucket.objects.all():
        if not my_bucket_object.key.find(AWS_S3_FILE_KEY):
            KEY = my_bucket_object.key
            BUCKET_NAME = my_bucket_object.bucket_name
            obj = s3.Object(BUCKET_NAME, KEY)
            obj.delete()
delete_from_aws_s3_bucket(AWS_BUCKET_NAME,AWS_ACCESS_KEY,AWS_ACCESS_SECRET_KEY)

Conclusion:
Python Boto3 library makes it very easy for us to delete a single or all file(s) in amazon s3 bucket. Click here if you want to know how to upload files to aws s3 bucket.

Published by Jean Joseph

Jean Joseph is a Database, Big Data, Data Warehouse Platform, Data Pipeline, Database Architecture Solutions Provider as well as a Data Engineer enthusiast among other disciplines.

One thought on “Delete all files from aws S3 bucket

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: