Oakville Recreational Schedule


I recently published a web application that uses data provided by the Town of Oakville Open Data project. The application displays information about upcoming recreational activities around town.

After publishing the application, I had quite a few people contact me asking how I did it. So, I figured I’d write up an article on how it works. So here goes.

First, the platform is built on Amazon Web Services (AWS). Here’s a list of the AWS services it uses:

  • Simple Queueing Service (SQS)
  • Relational Database Service (RDS)
  • Elastic Compute Cloud (EC2), and
  • Simple Storage Service (S3)

Here’s how it works. Once a day, a Python script grabs the latest CSV schedules from the Town’s Open Data website. The script saves a copy of the schedules to an S3 bucket for processing. I’ve added a Life Cycle Management (LCM) rule to make sure that the bucket doesn’t keep all the CSV files. After the files are saved to S3 the script adds a job to one of the three SQS queues (one for each activity type – swimming, fitness, or skating).

Another Python script watches the SQS queues for jobs. When it sees that there is work to do it parses the CSV files and updates tables in a simple MySQL database running in RDS.

That’s pretty much it. ┬áThe project was more of an exploration of using application services from AWS and improving my Python scripts.