Skip to content

Instantly share code, notes, and snippets.

@jacobobryant

jacobobryant/howto.md

Last active December 9, 2023 00:24
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save jacobobryant/02de6c2b3a1dae7c86737a2610311a3a to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save jacobobryant/02de6c2b3a1dae7c86737a2610311a3a to your computer and use it in GitHub Desktop.
Download ZIP
How to let users upload images to S3 with Biff
Raw
howto.md

How to let users upload images to S3 with Biff

UPDATE: Biff now includes a com.biffweb/s3-request function that can be used with the examples below, with some slight modifications.

Add the provided s3.clj file to your project, and set the following keys in config.edn and secrets.env:

;; config.edn
:s3/origin "https://nyc3.digitaloceanspaces.com"
:s3/edge "https://example.nyc3.cdn.digitaloceanspaces.com"
:s3/bucket "your-bucket"
:s3/access-key "..."
:s3/secret-key "S3_SECRET_KEY"
# secrets.env
S3_SECRET_KEY=...

S3 examples

Upload a publicly accessible object:

(s3/request ctx
            {:bucket "your-bucket"
             :method "PUT"
             :key "some-key"
             :body "some-body"
             :headers {"x-amz-acl" "public-read"
                       "content-type" "text/plain"}})
  • :bucket is optional; if unset, (:s3/bucket ctx) will be used.
  • :key will be coerced to a string.
  • :body can be a string or a file.
  • The public url will be (str (:s3/edge ctx) "/some-key").
  • To make the object private, set "x-amz-acl" to "private".

Retrieve an object (mainly useful for private objects):

(:body (s3/request ctx
                   {:bucket "your-bucket"
                    :method "GET"
                    :key "..."}))

Upload an image from a file input

Add a file input to your UI. It will trigger a request as soon as the user selects a file. Don't forget :hx-encoding:

[:input {:type "file"
         :name "image"
         :accept "image/apng, image/avif, image/gif, image/jpeg, image/png, image/svg+xml, image/webp"
         :hx-encoding "multipart/form-data"
         :hx-post "/upload"
         :hx-swap "outerHTML"}]

(You may want to use hx-indicator while the file uploads.)

Add a backend route that receives the image and uploads it to S3:

(defn upload-image [{:keys [s3/edge multipart-params] :as ctx}]
  (let [image-id (random-uuid)
        {:keys [tempfile content-type]} (get multipart-params "image")
        url (str edge "/" image-id)]
    (s3/request ctx {:method "PUT"
                     :key image-id
                     :body tempfile
                     :headers {"x-amz-acl" "public-read"
                               "content-type" content-type}})
    [:img {:src url}]))

(def plugin
  {:routes [["/upload" {:post upload-image}]]})

After the upload completes, the input element will be replaced with an img element.

Note: It is possible for the backend to give a signed URL to the client which allows them to upload the image directly to S3. I haven't bothered figuring out how to do that.

Raw
s3.clj
(ns com.example.s3
(:require [com.biffweb :as biff]
[buddy.core.mac :as mac]
[clj-http.client :as http]
[clojure.string :as str]))
(defn hmac-sha1-base64 [secret s]
(-> (mac/hash s {:key secret :alg :hmac+sha1})
biff/base64-encode))
(defn md5-base64 [body]
(with-open [f (cond
(string? body) (java.io.ByteArrayInputStream. (.getBytes body))
:else (java.io.FileInputStream. body))]
(let [buffer (byte-array 1024)
md (java.security.MessageDigest/getInstance "MD5")]
(loop [nread (.read f buffer)]
(if (pos? nread)
(do (.update md buffer 0 nread)
(recur (.read f buffer)))
(biff/base64-encode (.digest md)))))))
(defn format-date [date & [format]]
(.format (doto (new java.text.SimpleDateFormat (or format biff/rfc3339))
(.setTimeZone (java.util.TimeZone/getTimeZone "UTC")))
date))
(defn body->bytes [body]
(cond
(string? body) (.getBytes body)
:else (let [out (byte-array (.length body))]
(with-open [in (java.io.FileInputStream. body)]
(.read in out)
out))))
(defn request [{:keys [biff/secret
s3/origin
s3/access-key]
default-bucket :s3/bucket}
{:keys [method
key
body
headers
bucket]}]
;; See https://docs.aws.amazon.com/AmazonS3/latest/userguide/RESTAuthentication.html
(let [bucket (or bucket default-bucket)
date (format-date (java.util.Date.) "EEE, dd MMM yyyy HH:mm:ss Z")
path (str "/" bucket "/" key)
md5 (some-> body md5-base64)
headers' (->> headers
(map (fn [[k v]]
[(str/trim (str/lower-case k)) (str/trim v)]))
(into {}))
content-type (get headers' "content-type")
headers' (->> headers'
(filter (fn [[k v]]
(str/starts-with? k "x-amz-")))
(sort-by first)
(map (fn [[k v]]
(str k ":" v "\n")))
(apply str))
string-to-sign (str method "\n" md5 "\n" content-type "\n" date "\n" headers' path)
signature (hmac-sha1-base64 (secret :s3/secret-key) string-to-sign)
auth (str "AWS " access-key ":" signature)
s3-opts {:method method
:url (str origin path)
:headers (merge {"Authorization" auth
"Date" date
"Content-MD5" md5}
headers)
:body (some-> body body->bytes)}]
(http/request s3-opts)))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment