Skip to content

Instantly share code, notes, and snippets.

@ozrabal
Created September 2, 2025 19:01
Show Gist options
  • Select an option

  • Save ozrabal/968f33814f27bbb6740ad8f0d592025a to your computer and use it in GitHub Desktop.

Select an option

Save ozrabal/968f33814f27bbb6740ad8f0d592025a to your computer and use it in GitHub Desktop.
#!/bin/bash
# Check if two arguments are provided
if [ "$#" -ne 2 ]; then
echo "Usage: $0 urls_file target_directory"
exit 1
fi
URLS_FILE="$1"
TARGET_DIR="$2"
# If path is relative (does not start with / or ~), make it relative to current working dir
if [[ "$TARGET_DIR" != /* && "$TARGET_DIR" != ~* ]]; then
TARGET_DIR="$(pwd)/$TARGET_DIR"
fi
# Check if wget is installed
if ! command -v wget >/dev/null 2>&1; then
echo "wget is not installed. Trying to install..."
if command -v apt-get >/dev/null 2>&1; then
sudo apt-get update && sudo apt-get install -y wget
elif command -v yum >/dev/null 2>&1; then
sudo yum install -y wget
elif command -v dnf >/dev/null 2>&1; then
sudo dnf install -y wget
elif command -v brew >/dev/null 2>&1; then
brew install wget
else
echo "Could not detect a supported package manager. Please install wget manually."
exit 1
fi
if ! command -v wget >/dev/null 2>&1; then
echo "wget installation failed. Please install it manually."
exit 1
fi
fi
# Try to create target directory
if ! mkdir -p "$TARGET_DIR" 2>/dev/null; then
echo "Error: Cannot create or access directory: $TARGET_DIR"
exit 1
fi
echo "Using directory: $TARGET_DIR"
SUCCESS=0
FAILED=0
# Download each URL
while IFS= read -r URL; do
# skip empty lines and comments
if [[ -z "$URL" || "$URL" =~ ^# ]]; then
continue
fi
echo "Downloading: $URL"
if wget -q -P "$TARGET_DIR" "$URL"; then
((SUCCESS++))
else
echo "Failed to download: $URL"
((FAILED++))
fi
done < "$URLS_FILE"
TOTAL=$((SUCCESS + FAILED))
echo "-----------------------------------"
echo "Download summary:"
echo " Total URLs processed: $TOTAL"
echo " Successfully downloaded: $SUCCESS"
echo " Failed: $FAILED"
echo "Files saved in: $TARGET_DIR"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment