Mohammad's Blog

🌟 RDS AutoPilot 🌟

Md Nur Mohammad's photo
Md Nur Mohammad
·

8 min read

Description:
RDS AutoPilot is an intelligent AWS RDS management system that automates database start and stop schedules within a single VPC. Designed for cost optimization, it ensures that RDS instances run only during working hours, reducing unnecessary cloud expenses while maintaining operational efficiency.

With built-in AWS Lambda automation, EventBridge scheduling, and CloudWatch monitoring, RDS AutoPilot provides a seamless, hands-free solution for managing database availability in production and development environments.

🔹 Key Features:
✅ Automated start/stop scheduling for RDS
✅ Cost optimization by reducing idle runtime
✅ Secure deployment within a single VPC
✅ Serverless execution using AWS Lambda
✅ CloudWatch monitoring for real-time status

💡 Perfect for teams looking to:

  • Cut down AWS RDS costs

  • Automate DB uptime management

  • Enhance cloud efficiency with minimal manual intervention

"Smart RDS Scheduling for Cost-Optimized Cloud Operations"

📌 Project Overview

RDS AutoPilot is a smart, serverless solution designed to automate the scheduling of AWS RDS instances within a single VPC. It ensures that databases are stopped during non-working hours and restarted when needed, reducing unnecessary cloud expenses while maintaining seamless availability.

By leveraging AWS Lambda, EventBridge, CloudWatch, and IAM, RDS AutoPilot eliminates manual intervention in database management, making it a highly efficient tool for cloud-based applications.

🚀 Key Features

✅ Automated RDS Scheduling – Stop/start RDS based on pre-configured schedules
✅ Cost Optimization – Reduces AWS costs by preventing idle database usage
✅ Seamless Integration – Works with AWS EventBridge, Lambda, and CloudWatch
✅ Secure & Scalable – Operates inside a single VPC with least privilege IAM roles
✅ Real-Time Monitoring – Logs status updates and failures via CloudWatch
✅ Customizable – Supports multiple RDS instances with different schedules

💡 System Architecture

🛠 Tech Stack:

  • AWS RDS (MySQL/PostgreSQL)

  • AWS VPC (Private & Public Subnets)

  • AWS Lambda (Start/Stop Automation)

  • AWS EventBridge (Cron-based Scheduling)

  • AWS CloudWatch (Monitoring & Logs)

  • AWS IAM (Access Control & Security)

🗂 Architecture Flow

1️⃣ AWS EventBridge triggers a Lambda function at predefined times (e.g., stop RDS at 9 PM, start at 7 AM).
2️⃣ Lambda Function executes API calls to stop or start RDS instances securely.
3️⃣ CloudWatch Logs capture execution results and trigger alarms in case of failure.
4️⃣ IAM Policies restrict access to only required AWS resources within the VPC.
5️⃣ AWS RDS runs within the private subnet, ensuring a secure cloud environment.

🎯 Ideal Use Cases

✔ Development & Testing Environments – Stop RDS at night to cut costs.
✔ Production Workloads – Ensure RDS is running only when needed.
✔ Startup & Enterprise Teams – Save cloud costs automatically.
✔ Data-Intensive Applications – Automate RDS uptime for better efficiency.

💬 Tagline Ideas

💡 "Let Your Database Sleep When You Do!"
💡 "Smart Scheduling for Smarter Cloud Costs"
💡 "Automate RDS & Save Costs Effortlessly"

📊 Diagram Representation

I will generate a detailed architecture diagram to visually represent the setup. Give me a moment. 🚀

📌 AWS RDS AutoPilot Architecture

Here’s a visual representation of how RDS AutoPilot manages AWS RDS scheduling:

🛠 How It Works

1️⃣ AWS EventBridge triggers an AWS Lambda function based on a pre-defined schedule (e.g., stop RDS at 9 PM, start at 7 AM).
2️⃣ AWS Lambda executes API calls to start/stop RDS securely.
3️⃣ AWS CloudWatch monitors logs and triggers alerts in case of failure.
4️⃣ AWS IAM ensures security by granting limited access to necessary resources.
5️⃣ AWS RDS runs within a private subnet, ensuring a secure cloud environment.

🌟 RDS AutoPilot – Detailed Step-by-Step Implementation Guide

🔹 Step 1: Create a VPC

Since all resources will be inside a single VPC, follow these steps:

1️⃣ Create the VPC

  1. Go to AWS Console → VPC → Create VPC.

  1. Name: rds-autopilot-vpc

  2. CIDR Block: 10.0.0.0/16

  3. Tenancy: Default

  4. Click Create VPC.

2️⃣ Create Subnets

  1. Navigate to Subnets → Create Subnet.

  2. Select rds-autopilot-vpc.

  3. Create two subnets:

    • Private Subnet → 10.0.32.0/24 (For RDS)

  • Public Subnet → 10.0.64.0/24 (For NAT Gateway & Lambda)

3️⃣ Set Up Internet & NAT Gateway

Now that we have public and private subnets, we need to set up internet access for both.

  • Public Subnet (10.0.64.0/24) → Uses an Internet Gateway (IGW) to allow direct access to the internet.

  • Private Subnet (10.0.32.0/24) → Uses a NAT Gateway (in the public subnet) to allow outbound internet access but block inbound connections (for security).

  1. Internet Gateway:

    ✅ Steps to Create an Internet Gateway

    1. Go to AWS Console → VPC → Internet Gateways.

    2. Click Create Internet Gateway.

    3. Name it → rds-autopilot-igw.

    4. Click Create Internet Gateway.

1️⃣ Go to AWS Console → VPC → NAT Gateways

  1. Navigate to AWS Console → VPC.

  2. Click NAT Gateways in the left panel.

  3. Click Create NAT Gateway.

2️⃣ Configure NAT Gateway Settings

  1. Name → rds-autopilot-nat

  2. Subnet → Select the Public Subnet (10.0.64.0/24)

    • This ensures that the NAT Gateway is in a public-facing subnet.

  3. Elastic IP (EIP):

    • Click Allocate Elastic IP.

    • Click Select Elastic IP.

  4. Click Create NAT Gateway.

✅ Now, your private subnet will have controlled outbound internet access!

1️⃣ Configure the Public Route Table

The public subnet (10.0.64.0/24) needs to send traffic to the Internet Gateway.

  1. Go to AWS Console → VPC → Route Tables.

  2. Click Create Route Table.

    • Name → public-route-table

    • VPC → rds-autopilot-vpc

  3. Click Create.

✅ Steps to Add IGW to Public Route Table

  1. Select rds-autopilot-vpc-rtb-public.

  2. Click Edit Routes.

  3. Click Add Route.

  4. In Destination, enter:

     CopyEdit0.0.0.0/0

  5. In Target, select:

    • Internet Gateway (igw-0622193de7a37f4d5).

  1. Click Save Changes.

✅ Now, your public subnet should have internet access!

🎯 Step 2: Add NAT Gateway to the Private Route Table

Since private subnets cannot have direct internet access, we must route their internet traffic through the NAT Gateway.

✅ Steps to Add NAT Gateway

  1. Select RDS-Pvt-rt (or the correct private route table).

  2. Click Edit Routes.

  3. Click Add Route.

  4. In Destination, enter:

     CopyEdit0.0.0.0/0

  5. In Target, select:

    • NAT Gateway (nat-xxxxxxxx) (This should be the NAT Gateway created earlier in the public subnet).

  1. Click Save Changes.

  2. Associate with Private Subnet:

    • Click Subnet Associations.

    • Select private-subnet-rds (10.0.32.0/24).

  • Click Save Associations.

🔹 Step 2: Create an RDS Database

Now, we create an Amazon RDS instance inside the private subnet.

1️⃣ Go to AWS RDS Console

  1. Click Create Database.

  2. Choose Standard Create.

  3. Engine: Select MySQL or PostgreSQL.

  4. Version: Select latest stable version.

  5. Instance Class: db.t3.micro (for testing).

  6. Storage: Allocate at least 20 GB.

2️⃣ Networking Configuration

  1. VPC: Select rds-autopilot-vpc.

  2. Subnet Group: Select the private subnet (10.0.1.0/24).

  3. Public Access: DISABLED (for security).

  4. VPC Security Group:

    • Create a new security group allowing only EC2 or Lambda to access RDS.

3️⃣ Finalize & Create RDS

  1. Set Backup Retention Period to 7 days.

  2. Click Create Database.

🔹 Step 3: Configure AWS Lambda for Auto Start/Stop

We now create AWS Lambda functions to automate RDS start/stop operations.

1️⃣ Create an IAM Role for Lambda

  1. Go to IAM → Roles → Create Role.

  2. Trusted Entity: AWS Lambda.

  3. Attach Policies:

    • AmazonRDSFullAccess

    • AWSLambdaBasicExecutionRole

  4. Role Name: lambda-rds-scheduler

  5. Click Create Role.

2️⃣ Create AWS Lambda Function

Go to AWS Lambda → Create Function → Author from Scratch.

  • Function Name: rds-scheduler

  • Runtime: Python 3.9+

  • IAM Role: Select lambda-rds-scheduler

Lambda Code for Stopping RDS

import boto3

rds_client = boto3.client('rds', region_name='us-east-1')

def lambda_handler(event, context):
    db_instance = 'your-rds-instance-id'  # Replace with your RDS Instance ID
    try:
        response = rds_client.stop_db_instance(DBInstanceIdentifier=db_instance)
        print(f"Stopping RDS: {db_instance}")
        return response
    except Exception as e:
        print(f"Error stopping RDS: {str(e)}")
        return {"Error": str(e)}

Deploy this function.

Lambda Code for Starting RDS

import boto3

rds_client = boto3.client('rds', region_name='us-east-1')

def lambda_handler(event, context):
    db_instance = 'your-rds-instance-id'  # Replace with your RDS Instance ID
    try:
        response = rds_client.start_db_instance(DBInstanceIdentifier=db_instance)
        print(f"Starting RDS: {db_instance}")
        return response
    except Exception as e:
        print(f"Error starting RDS: {str(e)}")
        return {"Error": str(e)}

Deploy this function as well.

🔹 Step 4: Automate Scheduling with EventBridge

Now, we schedule:

  • Stop RDS at 9 PM (Mon-Fri)

  • Start RDS at 7 AM (Mon-Fri)

1️⃣ Stop RDS at Night

  1. Go to EventBridge → Rules → Create Rule.

  2. Name: stop-rds-schedule

  3. Rule Type: Schedule.

  4. Schedule Pattern:

     0 21 ? * MON-FRI *

    (9 PM UTC)

  5. Target: AWS Lambda → Select rds-scheduler-stop.

2️⃣ Start RDS in Morning

  1. Go to EventBridge → Rules → Create Rule.

  2. Name: start-rds-schedule

  3. Schedule Pattern:

     0 7 ? * MON-FRI *

    (7 AM UTC)

  4. Target: AWS Lambda → Select rds-scheduler-start.

🔹 Step 5: Configure Monitoring & Alerts

To track RDS start/stop status:

1️⃣ Enable CloudWatch Logging

  1. Go to AWS Lambda → Select rds-scheduler function.

  2. Enable CloudWatch Logs for tracking execution results.

2️⃣ Set CloudWatch Alarms

  • Create an Alarm for Failed Executions:

    1. Go to CloudWatch → Alarms.

    2. Click Create Alarm.

    3. Select Lambda Function Errors.

    4. Set notification for email or SNS.

lambdaDevopsrds