Skip to content

Instantly share code, notes, and snippets.

View aamir814's full-sized avatar

Aamir Haroon aamir814

0 followers · 2 following
View GitHub Profile
@polius
polius / mysql-locks-monitor.sql
Last active November 13, 2024 18:34
MySQL query to identify and retrieve detailed information about queries causing locks.
-- Requirement: performance_schema = ON
SELECT
lw.waiting_pid,
lw.waiting_query,
lw.waiting_trx_age AS 'waiting_time',
lw.blocking_pid,
(
SELECT h.SQL_TEXT
@ronaldbradford
ronaldbradford / README.md
Last active October 23, 2024 14:22
Minimalistic MySQL collection scripts

These bash scripts do the smallest amount of work possible to collect simple MySQL metrics and running SQL queries of a running system for subsequent analysis. These scripts include:

  • collect-mysql-status Snapshot of Status, ProcessList and supporting information for 60 seconds
  • sample-mysql-queries SQL sampling (brute force approach) for approximately 20 seconds

These scripts are designed to be executed on a Linux host, and require the mysql client to be installed, and MySQL credentials to run SQL placed in the $HOME/.my.cnf file.

NOTE: The output of information of SQL statements may include PII information. It is important data is reviewed and masked.

@polius
polius / rds-snapshots.sh
Last active May 8, 2024 14:27
Extract all AWS RDS manual snapshots from all regions into a CSV file.
account="123456789"
echo "Account, Region, SnapshotCreateTime, DBSnapshotIdentifier, DBInstanceIdentifier, Engine, AllocatedStorage" > "$account.csv"
for region in $(aws ec2 describe-regions --profile customer --query "Regions[].RegionName" --output text); do
echo "Region: $region"
snapshots=$(aws rds describe-db-snapshots --profile customer --region $region --query "DBSnapshots[?SnapshotType=='manual'].[SnapshotCreateTime, DBSnapshotIdentifier, DBInstanceIdentifier, Engine, AllocatedStorage]" --output json)
if [ -n "$snapshots" ]; then
echo "$snapshots" | jq -r --arg account "$account" --arg region "$region" '.[] | "\"\($account)\",\"\($region)\"," + @csv' >> "$account.csv"
fi
done
@polius
polius / dynamodb-scan.py
Last active October 18, 2024 20:45
A comprehensive tool for analyzing the workload of a DynamoDB table and extracting valuable insights.
# Install dependencies: python3 -m pip install boto3 numpy rich
import json
import boto3
import argparse
import numpy as np
from decimal import Decimal
from datetime import datetime, timedelta
from rich.console import Console
from rich.table import Table
@mikesparr
mikesparr / gcp-datastream-mysql-gcs.sh
Last active May 28, 2025 12:40
Experiment using Datastream to implement CDC between Cloud SQL (MySQL) database and Google Cloud Storage (GCS) bucket
#!/usr/bin/env bash
#####################################################################
# REFERENCES
# - https://cloud.google.com/vpc/docs/create-modify-vpc-networks
# - https://cloud.google.com/sql/docs/mysql/configure-private-services-access
# - https://cloud.google.com/sql/docs/mysql/configure-private-ip
# - https://cloud.google.com/iam/docs/service-accounts-create
# - https://cloud.google.com/compute/docs/instances/startup-scripts/linux
# - https://cloud.google.com/storage/docs/discover-object-storage-gcloud
@mikesparr
mikesparr / gcp-redis-twemproxy-benchmark.sh
Last active January 24, 2025 17:17
Experiment with Google Cloud Memorystore Redis instances fronted by internal load balancer and Twemproxy
#!/usr/bin/env bash
#####################################################################
# REFERENCES
# - https://cloud.google.com/blog/products/databases/running-redis-on-gcp-four-deployment-scenarios
# - https://cloud.google.com/memorystore/docs/redis/high-availability
# - https://redis.com/redis-enterprise/technology/highly-available-redis/
#####################################################################
export PROJECT_ID=$(gcloud config get-value project)