dassio/uplink_batch.md

Last active March 21, 2023 10:37

Star (2) You must be signed in to star a gist
Fork (1) You must be signed in to fork a gist

Select an option

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/dassio/47d8a3f0e3e8495b6755192912abb58d.js"></script>
Save dassio/47d8a3f0e3e8495b6755192912abb58d to your computer and use it in GitHub Desktop.

Download ZIP

storj uplink batch download upload

Raw

uplink_batch.md

delete files

uplink ls --recursive "sj://pictures/Photo Backup/tedxnanjing/" | sed -En "s/OBJ [[:digit:]]+-[[:digit:]]+-[[:digit:]]+ [[:digit:]]+:[[:digit:]]+:[[:digit:]]+\s+[[:digit:]]+ (.*)/\1/p" | parallel -j 4 -q uplink rm "sj://pictures/Photo Backup/tedxnanjing/{}"

upload files

cd to_upload_folder

ls -R ./ | awk ' /:$/&&f{s=$0;f=0} /:$/&&!f{sub(/:$/,"");s=$0;f=1;next} NF&&f{ print s"/"$0 }' | sed -En "s/\.\/(.*)/\1/p" | parallel -j 5 uplink cp './{}' 'sj://photosbackup/{}'

create folder

uplink ls --recursive "sj://pictures/Photo Backup/" | sed -En "s/OBJ [[:digit:]]+-[[:digit:]]+-[[:digit:]]+ [[:digit:]]+:[[:digit:]]+:[[:digit:]]+\s+[[:digit:]]+ (.*)\/.*/\1/p" | uniq | xargs -d '\n' mkdir -p

download files

uplink ls --recursive "sj://pictures/Photo Backup/" | sed -En "s/OBJ [[:digit:]]+-[[:digit:]]+-[[:digit:]]+ [[:digit:]]+:[[:digit:]]+:[[:digit:]]+\s+[[:digit:]]+ (.*)/\1/p" | parallel -j 10 -q uplink cp "sj://pictures/Photo Backup/{}" "temp/{}"

Author

dassio commented Mar 28, 2020

xargs -d '\n' is for folder name with spaces

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment