Amazon Web Services (AWS) Simple Storage Service (S3) is a Storage-as-a-Service provided by Amazon. It’s a general purpose object store, the objects are grouped under a name space called “buckets.” The buckets are unique across the entire AWS S3.
Boto library is the official Python SDK for software development and it provides APIs to work with AWS services like Dynamodb, S3, and others.
In this article, I will focus on how to delete all files in Amazon S3 bucket using Python Boto3 library.
First let’s use profile name as credential to access AWS S3
s3 = boto3.session.Session(profile_name='dev').resource('s3', region_name='us-east-1')
my_bucket = s3.Bucket(AWS_BUCKET_NAME)
for my_bucket_object in my_bucket.objects.all():
if not my_bucket_object.key.find(AWS_S3_FILE_KEY):
KEY = my_bucket_object.key
BUCKET_NAME = my_bucket_object.bucket_name
obj = s3.Object(BUCKET_NAME, KEY)
obj.delete()
Second you can also use access and secret key as credential to access AWS S3
AWS_ACCESS_KEY = "YOUR ACCESS KEY"
AWS_ACCESS_SECRET_KEY = "YOU SECRET KEY"
s3 = boto3.resource('s3', aws_access_key_id=AWS_ACCESS_KEY,aws_secret_access_key=AWS_ACCESS_SECRET_KEY)
for my_bucket_object in my_bucket.objects.all():
if not my_bucket_object.key.find(AWS_S3_FILE_KEY):
KEY = my_bucket_object.key
BUCKET_NAME = my_bucket_object.bucket_name
obj = s3.Object(BUCKET_NAME, KEY)
obj.delete()
let’s wrap up everything into a function.
AWS_ACCESS_KEY = "YOUR ACCESS KEY"
AWS_ACCESS_SECRET_KEY = "YOU SECRET KEY"
AWS_BUCKET_NAME = "YOUR BUCKET NAME"
def delete_from_aws_s3_bucket(AWS_BUCKET_NAME,AWS_ACCESS_KEY,AWS_ACCESS_SECRET_KEY):
s3 = boto3.resource('s3', aws_access_key_id=AWS_ACCESS_KEY,aws_secret_access_key=AWS_ACCESS_SECRET_KEY)
my_bucket = s3.Bucket(AWS_BUCKET_NAME)
for my_bucket_object in my_bucket.objects.all():
if not my_bucket_object.key.find(AWS_S3_FILE_KEY):
KEY = my_bucket_object.key
BUCKET_NAME = my_bucket_object.bucket_name
obj = s3.Object(BUCKET_NAME, KEY)
obj.delete()
delete_from_aws_s3_bucket(AWS_BUCKET_NAME,AWS_ACCESS_KEY,AWS_ACCESS_SECRET_KEY)
Conclusion:
Python Boto3 library makes it very easy for us to delete a single or all file(s) in amazon s3 bucket. Click here if you want to know how to upload files to aws s3 bucket.
bigdatadriven.org 
One thought on “Delete all files from aws S3 bucket”