Skip to content

Instantly share code, notes, and snippets.

@heymatthenry
Created June 20, 2012 16:19
Show Gist options
  • Select an option

  • Save heymatthenry/2960755 to your computer and use it in GitHub Desktop.

Select an option

Save heymatthenry/2960755 to your computer and use it in GitHub Desktop.

The coming S3pocalypse

  • We currently use one S3 bucket for each client. We have about 70 buckets in use. Amazon limits accounts to 100 buckets. Is there a need to use separate buckets for each client? Can we just create another S3 account?

One bucket per client or one bucket per environment

  • As I understand it, the reason we've used a bucket for each client in the past has been so we could keep track of how much bandwidth each client uses. I don't know if this anybody has done this or currently does this, but as we run up against our hundred-bucket limit, we'll need an alternative.
  • The way forward is probably to create a bucket for each environment (local dev, development, qa, staging, & prodction). Actually, we'll likely want to have a 'draft' bucket for each of those environments to hold pre-publication assets. Alternatively there could be a 'drafts' folder in each bucket instead of creating a whole separate bucket.
  • If we do implement 'drafts' feature, we'll have to add some workflow functionality to our asset management.

Do we (or should we) charge for bandwidth?

  • AFAIK, we don't currently charge for bandwidth. As mentioned above I'm not sure if anybody is even keeping up with bandwidth usage. From poking around our S3 dashboard, it seemed like kind of a pain to do so.
  • I'd suggest we revisit the question once we've had out custom logging/reporting system in place for a little while.

Is it possible to create a separate S3 account for each client?

  • Possible, but it doesn't really buy us anything, would decentralize the usage data, and would be yet another task for PMs to handle.
  • The one thing this approach has going for it is that if we decided to charge clients for S3 usage, we could easily just tack the Amazon invoices onto the bills we send to the client. That part of it would be simple, but I still think centralizing has more to offer.

Update existing URLs to use new schema or just leave them as they are and roll out new URLs for new assets?

  • We could update existing URLs to use the new schema as an OTA.
  • I'm inclined to leave existing URLs as is, however.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment