download file from web and unzip it

In this tutorial, you will learn how to download files from the web using wget Python modules and unzip it.

Run pip install wget if you have not yet install wget on your system. Below function will create a new local directory if not exit, download the web file then unzip it.
import wget
import os
def download_web_file_locally (URL, LOCAL_DIR_FILE, WORKING_DIR):
    # CREATE LOCALL DIRECTORY IF NOT EXISTS
    if not os.path.exists(WORKING_DIR):
        os.makedirs(WORKING_DIR)
    # START DOWNLOADING THE FILE FROM THE WEB
    DOWNLOAD_FILE = WORKING_DIR + LOCAL_DIR_FILE
    wget.download(URL, DOWNLOAD_FILE)
    # UNZIP THE ZILE TO A DIFFERENT DIRECTORY NAME
    print('unzipping gtfs file')
    with zipfile.ZipFile(DOWNLOAD_FILE,'r') as zip_ref:
        zip_ref.extractall(WORKING_DIR)

Now, let’s execute the function to download and unzip the file.

URL             = "YOU WEB URL"
LOCAL_DIR_FILE  = "YOUR LOCAL DIRECTORY NAME"
WORKING_DIR     = "YOUR WORKING DIRECTORY NAME" # WHERE TO UNZIP THE FILE
def download_web_file_locally (URL, LOCAL_DIR_FILE, WORKING_DIR)

Published by Jean Joseph

Jean Joseph is a Database, Big Data, Data Warehouse Platform, Data Pipeline, Database Architecture Solutions Provider as well as a Data Engineer enthusiast among other disciplines.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: