Scripting: Work Smarter, Not Harder

Have you ever been given a task that seemed so repetitive or manual that you just knew there must be a better way to accomplish the task? If your answer is yes, then I have been in your shoes before. Anytime I am given a project that involves manually searching through files, converting data from one format to another, or pulling reports from data that isn’t setup for reporting, I ask myself some questions such as “how could we accomplish this quicker?” or “could this task be scripted?”

Python has been my language of choice for writing utility scripts like the ones I am referring to above. Python has a clean and concise syntax, an incredible amount of packages you can add, and a large community that supports it. In this post, I will show two different scripts I have used recently to automate some incredibly mundane tasks.

#1: Find staging domains in AWS Route53

Several months ago, we were doing some infrastructure updates to websites we host. As part of that process, we created staging domains for most of the sites. We needed to do some cleanup by removing the old staging domains (i.e. staging.example.com). With numerous domain names to check, this could be quite a chore if I was to check each domain name in the console. Below is the Python script I used to automate this task.

import boto3


def main():
    client = boto3.client('route53')

    # Get list of zones
    zones = client.list_hosted_zones()['HostedZones']

    for zone in zones:
        # Get the record sets for the current zone
        record_sets = client.list_resource_record_sets(HostedZoneId=zone['Id'])['ResourceRecordSets']

        # Search for the string "staging" in each record set
        for record_set in record_sets:
            if 'staging' in record_set['Name']:
                print('Found: ' + record_set['Name'])


if __name__ == '__main__':
    main()

#2: Mass migration of videos to YouTube

Last year, a client came to us looking to move their website from their old hosting provider and instead host with us. As we started working on moving their website, we found that they had a large number of video files stored on their server. The total storage space these videos required was huge. We proposed that they upload the videos to YouTube and then include links on the site to their YouTube channel and/or links to individual videos. The issue is uploading this many videos would be a huge chore. So, I was tasked with finding a solution to upload the videos simply and efficiently.

As some of my coworker will attest, my first thought was “script it with Python!” I was able to use some prewritten code from YouTube’s API documentation to get started, and after making some small modifications and sorting out some bugs, I started uploading videos. This solution worked marvelously and we had the videos uploaded within a matter of days.

I put together a quick GitHub repository with the code I used to accomplish this task. You can also take a look at the YouTube API documentation where I got the original code snippets for the YouTube uploader. I removed any personal information from the JSON configuration files which will need to be replaced if you use this tool.

The Takeaway

While these two tasks might have nothing to do with the kind of work that you do, they should serve as examples of how scripting repetitive tasks can save large amounts of time. Additionally, you don’t have to write scripts in Python, that was just the language I chose. Other languages that work well for tasks like these include Node JS, Go, and Ruby but the final decision of what language to use is up to you. Regardless of what language you choose, I do hope you will use scripts more to automate and simplify your development work.