Automated snapshots using Python Flask Web API/Curl

I started to think about how I can make database service requests more agile and self served. As a Cloud Data platform architect I have access to full resources however, often times I overwhelmed with requests people think I am responsible for , but I am not really, the DBAs are. The point I am making here is, how well I can integrate db services into CI/CD pipelines and/or deployment work flows. During each release, devOps request a database snapshot be taken prior to deployment.

I developed a webApp written using python flask web framework to expose RESTful API calls using AWS python SDK. the flask application use API calls defined as routes

Below is the diagram depicts a simple architectural diagram

No alt text provided for this image

The web interface (HTML) or curl command can be used to request for a

  1. taking database backup
  2. checking the backup of an existing backup/snapshot
  3. deleting a manual on demand backup previously taken

API app is written using Python Flask Web Frame work with API endpoints called routes. The tool can be used in two forms

  1. HTTPS calls via browser. A basic interface for submitting a request through HTML interface.
Main backup tool page
Check backup
Create backup
Delete backup

  1. Examples:

webApp via HTTPS or CURL

#Main Page

#https://10.10.x.x:25443/backup

#create cluster db snapshot

curl -k https://10.10.x.x:25443/backup/create \
 --data "endpoint=mydB1-net1.us-east-1.rds.amazonaws.com"; \
 echo

#https://10.10.x.x:25443/backup/create

#check cluster db snapshot

curl -k https://10.10.x.x:25443/backup/status \
 --data "snapshotname=mydB1-2018-06-24-22-21-57" \
 --data "endpoint=mydB1.cluster-ro-net1.us-east-1.rds.amazonaws.com"; \
 echo

#https://10.10.x.x:25443/backup/status

#delete cluster db snapshot

curl -k https://10.10.x.x:25443/backup/delete \
 --data "snapshotname=mydB1-2018-06-24-22-21-57" \
 --data "endpoint=mydB1.cluster-ro-net1.us-east-1.rds.amazonaws.com"; \
 echo

#https://10.10.x.x:25443/backup/delete

Scheduling during or pre deployment:

#!/bin/bash#Author: Sudheer Kondla, 06/27/201
if [ $# -lt 1 ];then
echo "USAGE: bash $0 [require Fully Qualified db endpoint]"
exit 1
fi
dbEndPoint=$1 
/usr/bin/curl -k https://10.10.x.x:25443/backup/create \
--data "endpoint=${dbEndPoint}";  echo

Published by Sudheer Kondla

I am a software developer, devOps engineer, database and sys admin and software architect helping small to enterprise companies in many verticals. Have been in IT industry for more than 20+ years worked on very large platforms, dealing with battle tested software development, scaling, high availability , disaster recovery and fire-fighting scenarios.

Leave a comment

Design a site like this with WordPress.com
Get started