Sofiane Laouini mks0ff

BigQuery tricks

Data

Click to see script

with
dta as (
select '+++bHyZZlp6KRHJBj5Cw8wd48QA=' Id, timestamp('2019-11-07 06:53:39.632 UTC') Ts union all

Default

Powerlevel10k

I've been working with Kafka for over 7 years. I inevitably find myself doing the same set of activities while I'm developing or working with someone else's system. Here's a set of Kafka productivity hacks for doing a few things way faster than you're probably doing them now. 🔥

Show me all my Kafka topics and their partitions, replicas, and consumers
Show me the contents of a topic
Create a Kafka topic
Produce messages to a Kafka topic
Validate the schema of messages before producing to a topic
Do all of this at a distance

Get the tools

Create a schema based on the default one

https://github.com/basho/yokozuna/blob/develop/priv/default_schema.xml

Upload your schema

curl -XPUT -H 'Content-Type: application/xml'  http://localhost:8098/search/schema/drew_schema --data-binary @"my_schema.xml"

Docker - How to cleanup (unused) resources

Cleanup resources (containers, volumes, images, networks) ...

delete volumes

// see: https://github.com/chadoe/docker-cleanup-volumes

$ docker volume rm $(docker volume ls -qf dangling=true)

$ docker volume ls -qf dangling=true | xargs -r docker volume rm

Category theory jargon cheat sheet

A primer/refresher on the category theory concepts that most commonly crop up in conversations about Scala or FP. (Because it's embarassing when I forget this stuff!)

I'll be assuming Scalaz imports in code samples, and some of the code may be pseudo-Scala.

Functor

A functor is something that supports map.

	/*
	* This query will run across an entire organization looking at tables across every project
	* and shows how they will compare on compressed and uncompressed storage.
	*
	* Region Notes:
	* This query will only read from a single region or multi-region at a time. It's
	* currently not possible to read this data from across all
	*
	* By default this reads from the US multi-region, so this might need to be changed if
	* your data lives elsewhere.

	# Pre-req: gcc, gcc-c++, git (to clone), openjdk with debug symbols (centos: java-1.8.0-openjdk-debug.x86_64, ubuntu: openjdk-8-dbg)
	# Git clone async-profiler https://github.com/jvm-profiling-tools/async-profiler
	# Make sure JAVA_HOME is set and you are logged as same user running the java process
	cd async-profiler
	make all
	# Also git clone FlameGraph in the same directory as you clone async-profiler

	# sysctl changes
	echo 1 > /proc/sys/kernel/perf_event_paranoid
	echo 0 > /proc/sys/kernel/kptr_restrict

	# Add the following 'help' target to your Makefile
	# And add help text after each target name starting with '\#\#'

	help: ## Show this help.
	@fgrep -h "##" $(MAKEFILE_LIST) \| fgrep -v fgrep \| sed -e 's/\\$$//' \| sed -e 's/##//'

	# Everything below is an example

	target00: ## This message will show up when typing 'make help'
	@echo does nothing

	#!/bin/bash
	# Pull this file down, make it executable and run it with sudo
	# wget https://gist.githubusercontent.com/bryanhunter/10380945/raw/build-erlang-17.0.sh
	# chmod u+x build-erlang-17.0.sh
	# sudo ./build-erlang-17.0.sh

	if [ $(id -u) != "0" ]; then
	echo "You must be the superuser to run this script" >&2
	exit 1
	fi