Skip to content

Instantly share code, notes, and snippets.

@schwesig

schwesig/s3_storage_observability.txt

Last active October 28, 2025 14:56
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save schwesig/0f25d77f25a73abd547f28688525a347 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save schwesig/0f25d77f25a73abd547f28688525a347 to your computer and use it in GitHub Desktop.
Download ZIP
S3_Storage_Observability
Raw
s3_storage_observability.txt
# S3 on OpenStack NERC, migrated from minio on 2025-06-13 (minio started Jan 15 2025)
## 2025-09-23 03:00pm
### Bucket: acm-metrics
Total Objects: 2512
Total Size: 477.3 GiB
### Bucket: acm-metrics-hypershift2
Total Objects: 1339
Total Size: 14.8 GiB
### Bucket: acm-metrics-test
Total Objects: 1057
Total Size: 20.0 GiB
### Bucket: loki-logs-test
Total Objects: 0
Total Size: 0 Bytes
### Bucket: open-telemetry
Total Objects: 0
Total Size: 0 Bytes
### Bucket: open-telemetry-test
Total Objects: 0
Total Size: 0 Bytes
@schwesig
Copy link
Author

schwesig commented Oct 8, 2025

Metric Value
Average over all 31 days 4.85 GiB/day
Total (31 days) 150.18 GiB
Date Bytes MiB GiB
2025-10-08 7497751099 7150.41 6.98
2025-10-07 29529351405 28161.38 27.50
2025-10-06 11332726783 10807.73 10.55
2025-10-05 9053864292 8634.44 8.43
2025-10-04 0 0.00 0.00
2025-10-03 7916092555 7549.37 7.37
2025-10-02 0 0.00 0.00
2025-10-01 8197106200 7817.37 7.63
2025-09-30 0 0.00 0.00
2025-09-29 8092382951 7717.50 7.54
2025-09-28 0 0.00 0.00
2025-09-27 8100073629 7724.83 7.54
2025-09-26 0 0.00 0.00
2025-09-25 1344009 1.28 0.00
2025-09-24 8158709682 7780.75 7.60
2025-09-23 7691775411 7335.45 7.16
2025-09-22 0 0.00 0.00
2025-09-21 1332099 1.27 0.00
2025-09-20 7814721048 7452.70 7.28
2025-09-19 7819468595 7457.23 7.28
2025-09-18 0 0.00 0.00
2025-09-17 7798931313 7437.64 7.26
2025-09-16 0 0.00 0.00
2025-09-15 1282086 1.22 0.00
2025-09-14 7772449357 7412.39 7.24
2025-09-13 8067753468 7694.01 7.51
2025-09-12 0 0.00 0.00
2025-09-11 1740468 1.66 0.00
2025-09-10 8527179815 8132.15 7.94
2025-09-09 1360669 1.30 0.00
2025-09-08 7915709723 7549.01 7.37 
echo "| Date       | Bytes     | MiB     | GiB     |"                     ─╯
echo "|------------|-----------|---------|---------|"

for i in {0..30}; do
  DAY=$(date -d "-$i day" +%Y-%m-%d)
  aws s3 ls s3://acm-metrics/ --recursive \
    --endpoint-url https://stack.nerc.mghpcc.org:13808 \
  | grep "$DAY" \
  | awk -v d="$DAY" 'BEGIN{sum=0}
       {sum += $3}
       END {
         mib = sum/1024/1024;
         gib = sum/1024/1024/1024;
         printf "| %s | %d | %.2f | %.2f |\n", d, sum, mib, gib
       }'
done

@schwesig
Copy link
Author

schwesig commented Oct 8, 2025

Metric Value
Average over all 30 days 3.11 GiB/day
Total (30 days) 93.22 GiB
Date Bytes MiB GiB
2025-09-07 1347687 1.29 0.00
2025-09-06 7817941954 7455.77 7.28
2025-09-05 7515077488 7166.94 7.00
2025-09-04 0 0.00 0.00
2025-09-03 7469928077 7123.88 6.96
2025-09-02 0 0.00 0.00
2025-09-01 7369493830 7028.10 6.86
2025-08-31 0 0.00 0.00
2025-08-30 6825113411 6508.94 6.36
2025-08-29 0 0.00 0.00
2025-08-28 1571072 1.50 0.00
2025-08-27 7043891557 6717.58 6.56
2025-08-26 7268164151 6931.46 6.77
2025-08-25 0 0.00 0.00
2025-08-24 7154521773 6823.08 6.66
2025-08-23 0 0.00 0.00
2025-08-22 7382739482 7040.73 6.88
2025-08-21 0 0.00 0.00
2025-08-20 7483204543 7136.54 6.97
2025-08-19 0 0.00 0.00
2025-08-18 7423028718 7079.15 6.91
2025-08-17 0 0.00 0.00
2025-08-16 7426459242 7082.42 6.92
2025-08-15 0 0.00 0.00
2025-08-14 2034804720 1940.54 1.90
2025-08-13 0 0.00 0.00
2025-08-12 262564722 250.40 0.24
2025-08-11 1632107933 1556.50 1.52
2025-08-10 1112547562 1061.01 1.04
2025-08-09 6862777600 6544.85 6.39

@schwesig
Copy link
Author

schwesig commented Oct 8, 2025

Metric Value
Average over all 30 days 2.87 GiB/day
Total (30 days) 86.20 GiB
Date Bytes MiB GiB
2025-08-08 6289871912 5998.49 5.86
2025-08-07 2196974 2.10 0.00
2025-08-06 6210674888 5922.96 5.78
2025-08-05 0 0.00 0.00
2025-08-04 6162319486 5876.85 5.74
2025-08-03 0 0.00 0.00
2025-08-02 6222776592 5934.50 5.80
2025-08-01 0 0.00 0.00
2025-07-31 34940195989 33321.57 32.54
2025-07-30 0 0.00 0.00
2025-07-29 0 0.00 0.00
2025-07-28 0 0.00 0.00
2025-07-27 0 0.00 0.00
2025-07-26 0 0.00 0.00
2025-07-25 0 0.00 0.00
2025-07-24 0 0.00 0.00
2025-07-23 0 0.00 0.00
2025-07-22 0 0.00 0.00
2025-07-21 0 0.00 0.00
2025-07-20 0 0.00 0.00
2025-07-19 0 0.00 0.00
2025-07-18 0 0.00 0.00
2025-07-17 32729023794 31212.83 30.48
2025-07-16 0 0.00 0.00
2025-07-15 0 0.00 0.00
2025-07-14 0 0.00 0.00
2025-07-13 0 0.00 0.00
2025-07-12 0 0.00 0.00
2025-07-11 0 0.00 0.00
2025-07-10 0 0.00 0.00

@schwesig
Copy link
Author

schwesig commented Oct 8, 2025 •
edited
Loading

2025-10-08 01-49 pm
data collection started Jan 15 2025

Bucket: acm-metrics

Total Objects: 2774
Total Size: 522.1 GiB

Kennzahl Wert
Zeitraum 277 Tage
Gesamtdatenmenge 522,1 GiB
Durchschnitt pro Tag ≈ 1,89 GiB/Tag

Bucket: acm-metrics-hypershift2

Total Objects: 1546
Total Size: 21.2 GiB

Bucket: acm-metrics-test

Total Objects: 1057
Total Size: 20.0 GiB

aws s3api list-objects-v2 \
  --bucket acm-metrics \
  --endpoint-url https://stack.nerc.mghpcc.org:13808 \
  --output json --query "sum(Contents[].Size)"
aws s3api list-objects-v2 \
  --bucket acm-metrics \
  --endpoint-url https://stack.nerc.mghpcc.org:13808 \
  --output json --query "sum(Contents[].Size)"
for b in acm-metrics acm-metrics-hypershift2 acm-metrics-test \
         loki-logs loki-logs-test \
         open-telemetry open-telemetry-test; do
  echo "### Bucket: $b"
  aws s3 ls s3://$b/ --recursive \
    --endpoint-url https://stack.nerc.mghpcc.org:13808 \
    --human-readable --summarize | tail -n 2
  echo
done

@schwesig
Copy link
Author

schwesig commented Oct 28, 2025

Bucket: acm-metrics

Total Objects: 3088
Total Size: 591.7 GiB

Bucket: acm-metrics-hypershift2

Total Objects: 1761
Total Size: 29.2 GiB

Bucket: acm-metrics-test

Total Objects: 1057
Total Size: 20.0 GiB

Bucket: loki-logs-test

Total Objects: 0
Total Size: 0 Bytes

Bucket: open-telemetry

Total Objects: 0
Total Size: 0 Bytes

Bucket: open-telemetry-test

Total Objects: 0
Total Size: 0 Bytes

@schwesig
Copy link
Author

schwesig commented Oct 28, 2025 •
edited
Loading 

echo "| Date       | Bytes     | MiB     | GiB     |"
echo "|------------|-----------|---------|---------|"

for i in {0..30}; do
  DAY=$(date -d "-$i day" +%Y-%m-%d)
  aws s3 ls s3://acm-metrics/ --recursive \
    --endpoint-url https://stack.nerc.mghpcc.org:13808 \
  | grep "$DAY" \
  | awk -v d="$DAY" 'BEGIN{sum=0}
       {sum += $3}
       END {
         mib = sum/1024/1024;
         gib = sum/1024/1024/1024;
         printf "| %s | %d | %.2f | %.2f |\n", d, sum, mib, gib
       }'
done
Date Bytes MiB GiB
2025-10-28 7857077874 7493.09 7.32
2025-10-27 32079892518 30593.77 29.88
2025-10-26 11677347734 11136.39 10.88
2025-10-25 10311231925 9833.56 9.60
2025-10-24 0 0.00 0.00
2025-10-23 9644908685 9198.10 8.98
2025-10-22 0 0.00 0.00
2025-10-21 8084740273 7710.21 7.53
2025-10-20 0 0.00 0.00
2025-10-19 8007399505 7636.45 7.46
2025-10-18 0 0.00 0.00
2025-10-17 8184415091 7805.27 7.62
2025-10-16 0 0.00 0.00
2025-10-15 8205295384 7825.18 7.64
2025-10-14 0 0.00 0.00
2025-10-13 8090056668 7715.28 7.53
2025-10-12 0 0.00 0.00
2025-10-11 8154914832 7777.13 7.59
2025-10-10 0 0.00 0.00
2025-10-09 7911879338 7545.36 7.37
2025-10-08 0 0.00 0.00
2025-10-07 7748094652 7389.16 7.22
2025-10-06 0 0.00 0.00
2025-10-05 7765105884 7405.38 7.23
2025-10-04 0 0.00 0.00
2025-10-03 7916092555 7549.37 7.37
2025-10-02 0 0.00 0.00
2025-10-01 8197106200 7817.37 7.63
2025-09-30 0 0.00 0.00
2025-09-29 8092382951 7717.50 7.54
2025-09-28 0 0.00 0.00

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment