Skip to content

Instantly share code, notes, and snippets.

View andrewdhazlett's full-sized avatar

Andrew Hazlett andrewdhazlett

29 followers · 25 following
  • Chicago, IL
View GitHub Profile
@KyleAMathews
KyleAMathews / lambda.md
Last active May 13, 2022 00:49
Using Kafka and a Samza-like node.js architecture

Disclaimer

I'm still very new to Kafka, eventsourcing, stream processing, etc. I'm in the middle of building my first production system with this stuff and am writing this at the request of a few folks on Twitter. So if you do have experience, please do me and anyone else reading this a favor by pointing out things I get wrong :)

Inspirations

@twotwotwo
twotwotwo / sorts.md
Last active December 9, 2023 08:41
Sorting 5x faster with Go: how it's possible, what didn't work so well, and what I learned

github.com/twotwotwo/sorts is a Go package with parallel radix- and quicksorts. It can run up to 5x faster than stdlib sort on the right kind of large sort task, so it could be useful for analysis and indexing/database-y work in which you have to sort millions of items. (To be clear, I don't recommend most folks drop stdlib sort, which is great, and which sorts depends on.)

While the process of writing it's fresh on my mind, here are some technical details, some things that didn't make the cut, and some thoughts about the process:

Concretely, what this looks like inside:

  • Both number and string versions are in-place MSD radix sorts that look at a byte at a time and, once the range being sorted gets down to 128 items, call (essentially) the stdlib's quicksort.

  • The [parallelization code