In this article, I will focus on how python Boto3 library makes it easy for us to upload a single or all file(s) in amazon s3 bucket. Amazon Web Services (AWS) Simple Storage Service (S3) is a Storage-as-a-Service provided by Amazon. It’s a general purpose object store, the objects are grouped under a name space called “buckets.” The buckets are unique across the entire AWS S3.
Copy and past below scripts to create the function.
def upload_local_file_to_aws_s3 (FILE_NAME_DIR, AWS_BUCKET_NAME, AWS_ACCESS_KEY, AWS_ACCESS_SECRET_KEY):
from boto3.s3.transfer import S3Transfer
#have all the variables populated which are required below
client = boto3.client('s3', aws_access_key_id=AWS_ACCESS_KEY,aws_secret_access_key=AWS_ACCESS_SECRET_KEY)
transfer = S3Transfer(client)
Key = AWS_S3_FILE_KEY +'/' + os.path.basename(FILE_NAME_DIR)
transfer.upload_file(FILE_NAME_DIR, AWS_BUCKET_NAME, Key)
Don’t forget to change your bucket and directory name, access and secret key before execute the function.
AWS_ACCESS_KEY = "YOUR ACCESS KEY"
AWS_ACCESS_SECRET_KEY = "YOU SECRET KEY"
AWS_BUCKET_NAME = "YOUR BUCKET NAME"
FILE_NAME_DIR = "YOU LOCAL DIRECTORY"
def upload_local_file_to_aws_s3 (FILE_NAME_DIR, AWS_BUCKET_NAME, AWS_ACCESS_KEY, AWS_ACCESS_SECRET_KEY)
Conclusion:
Python Boto3 library makes it bery easy for us to upload a single or all file(s) from our local directory to amazon s3 bucket. Click here if you want to know how to delete all files in aws s3 bucket.
bigdatadriven.org 