-
-
Save tuxfight3r/7ccbd5abc4ded37ecdbc8fa46966b7e8 to your computer and use it in GitHub Desktop.
| #!/bin/bash -x | |
| #Date: 21/7/2017 | |
| #Author: Mohan | |
| #Purpose: To upload files to AWS S3 via Curl | |
| #Uploads file at the top level folder by default | |
| #S3 parameters | |
| S3KEY="XXXXXXXXXXX" | |
| S3SECRET="XXXXXXXXXXXXXXXX" | |
| S3BUCKET="storage-backup" | |
| S3STORAGETYPE="STANDARD" #REDUCED_REDUNDANCY or STANDARD etc. | |
| AWSREGION="s3-eu-west-1" | |
| putS3(){ | |
| file_path=$1 | |
| aws_path=$2 | |
| bucket="${S3BUCKET}" | |
| date=$(date -R) | |
| acl="x-amz-acl:private" | |
| content_type="application/x-compressed-tar" | |
| storage_type="x-amz-storage-class:${S3STORAGETYPE}" | |
| string="PUT\n\n$content_type\n$date\n$acl\n$storage_type\n/$bucket$aws_path${file_path##/*/}" | |
| signature=$(echo -en "${string}" | openssl sha1 -hmac "${S3SECRET}" -binary | base64) | |
| curl -s --retry 3 --retry-delay 10 -X PUT -T "$file_path" \ | |
| -H "Host: $bucket.${AWSREGION}.amazonaws.com" \ | |
| -H "Date: $date" \ | |
| -H "Content-Type: $content_type" \ | |
| -H "$storage_type" \ | |
| -H "$acl" \ | |
| -H "Authorization: AWS ${S3KEY}:$signature" \ | |
| "https://$bucket.${AWSREGION}.amazonaws.com$aws_path${file_path##/*/}" | |
| } | |
| usage(){ | |
| echo "Usage: $0 <absolutepath_to_file> <s3_folder_path>" | |
| echo "$0 '/tmp/storage_backup/storage-backup_07192017_112945.tar.gz' '/'" | |
| } | |
| #validate positional parameters are present | |
| if [ $# -ne 2 ]; then | |
| usage | |
| echo "Exiting .." | |
| exit 2 | |
| fi | |
| putS3 $1 $2 |
tag
amazonaws.com$aws_path${file_path##/*/}
should be changed to
amazonaws.com$aws_path/${file_path##/*/}
If you have a curl version >= 7.75.0 you can add files to s3 like this
#!/bin/sh
FILE=$1
S3_ACCESS_KEY="<ACCESS_KEY>"
S3_SECRET_KEY="<SECRET_KEY>"
S3_BUCKET="<BUCKET_NAME>"
S3_REGION="<REGION>"
curl --progress-bar -X PUT \
--user "${S3_ACCESS_KEY}":"${S3_SECRET_KEY}" \
--aws-sigv4 "aws:amz:${S3_REGION}:s3" \
--upload-file ${FILE} \
https://${S3_BUCKET}.s3.${S3_REGION}.amazonaws.com
is working this script?
Hi,
I come across this question.
In ubuntu I had to change "function usage" for " usage()" for solve error
@jamontesg
is it an issue with Bash vs sh 'POSIX' syntax ? Were you using Bash ?
see Bash documentation functions or https://linuxize.com/post/bash-functions/ for example.
HTH
For me I had to change the host
-H "Host:
I have a local S3 bucket, if that makes any difference, but I also got the signature error, which I fixed by changing
string="PUT\n\n$content_type\n$date\n$acl\n$storage_type\n/$bucket$aws_path${file_path##/*/}"
to
string="PUT\n\n$content_type\n$date\n$acl\n$storage_type\n/$bucket/$aws_path${file_path##/*/}"
iam encounter same error，have you found the solution?