Skip to content

Instantly share code, notes, and snippets.

View andrwng's full-sized avatar

Andrew Wong andrwng

19 followers · 3 following
  • San Francisco
View GitHub Profile
@andrwng
andrwng / rrr crc parser issue 23.1.20
Created November 2, 2023 19:19
rrr crc parser issue 23.1.20
TRACE 2023-11-02 18:31:57,596 [shard 0] cluster - controller_backend.cc:848 - [{kafka/panda-topic/16}] (retry 0) executing operation: {type: addition, revision: 17, assignment: { id: 16, group_id: 17, replicas: {{node_id: 2, shard: 0}, {node_id: 3, shard: 0}, {node_id: 1, shard: 0}} }, previous assignment: {nullopt}}
DEBUG 2023-11-02 18:31:57,596 [shard 0] cloud_storage - partition_recovery_manager.cc:100 - No manifest override for {kafka/panda-topic/16} found, skipping
TRACE 2023-11-02 18:31:57,596 [shard 1] cluster - controller_backend.cc:848 - [{kafka/panda-topic/16}] (retry 0) executing operation: {type: addition, revision: 17, assignment: { id: 16, group_id: 17, replicas: {{node_id: 2, shard: 0}, {node_id: 3, shard: 0}, {node_id: 1, shard: 0}} }, previous assignment: {nullopt}}
DEBUG 2023-11-02 18:31:57,596 [shard 1] cluster - controller_backend.cc:768 - [{kafka/panda-topic/16}] (retry 0) finished operation: {type: addition, revision: 17, assignment: { id: 16, group_id: 17, replicas: {{node_id: 2, shard:
@andrwng
andrwng / bench_arrow.py
Created October 18, 2020 05:22
import kudu
import pandas as pd
import pyarrow as pa
import time
def main():
client = kudu.connect("0.0.0.0", 8764)
num_rows = 100000
num_rows_per_batch = 1000
num_non_pks = 100
@andrwng
andrwng / site.diff
Created July 15, 2020 18:41
commit 4d2ed5ca7b18c0883242e4596374483cd5a9e68c
Author: Andrew Wong <[email protected]>
Date: Tue Jul 7 21:10:49 2020 -0700
docker: build the website
Adds a docs_build image that includes most of the required pieces for
running make_site.sh.
Some manual steps are still needed, e.g. before running anything, I
@andrwng
andrwng / eviction_cb.diff
Created July 10, 2020 01:46
diff --git a/src/kudu/master/table_locations_cache.cc b/src/kudu/master/table_locations_cache.cc
index adf2a4a1a..a903889e3 100644
--- a/src/kudu/master/table_locations_cache.cc
+++ b/src/kudu/master/table_locations_cache.cc
@@ -39,7 +39,7 @@ namespace kudu {
namespace master {
TableLocationsCache::TableLocationsCache(size_t capacity_bytes)
- : eviction_cb_(new EvictionCallback(this)),
+ : eviction_cb_(this),
@andrwng
andrwng / KUDU-3154.diff
Created July 6, 2020 17:55
From d1d8e6d72209521cc884dde419725c657d86f4ec Mon Sep 17 00:00:00 2001
From: Andrew Wong <[email protected]>
Date: Thu, 02 Jul 2020 20:08:02 -0700
Subject: [PATCH] ranger: add some logging
Debugging KUDU-3154
Change-Id: I06502d182b88ab1d7c77e842b8986c9228ed8245
---
@andrwng
andrwng / master_authz-itest.diff
Created April 8, 2020 22:14
commit 3ab9ac2f8f98edf9fae393f5514af4a01000e1fa
Author: Andrew Wong <[email protected]>
Date: Wed Apr 8 15:08:14 2020 -0700
reuse Sentry/Ranger-agnostic code
Change-Id: I826d89b4289c3c6116b6a7a540ed12827e21f4b6
diff --git a/src/kudu/integration-tests/master_authz-itest.cc b/src/kudu/integration-tests/master_authz-itest.cc
index bfc1b0405..839b9bb86 100644
@andrwng
andrwng / master_authz-itest.diff
Created April 8, 2020 22:10
commit 6fee34dc46a6dd19112a42ab0f51046f23913c30
Author: Andrew Wong <[email protected]>
Date: Wed Apr 8 15:08:14 2020 -0700
reuse Sentry/Ranger-agnostic code
Change-Id: I826d89b4289c3c6116b6a7a540ed12827e21f4b6
diff --git a/src/kudu/integration-tests/master_authz-itest.cc b/src/kudu/integration-tests/master_authz-itest.cc
index bfc1b0405..8de02cacb 100644
@andrwng
andrwng / boss_notes.txt
Last active August 30, 2019 20:21
export KUDU_QUICKSTART_IP=$(ifconfig | grep "inet " | grep -Fv 127.0.0.1 | awk '{print $2}' | tail -1)
docker pull apache/kudu:latest
docker run -it -p 8050:8050 -p 7050:7050 -p 8051:8051 -p 7051:7051 apache/kudu:latest bash
mkdir master
mkdir tserver
# Run a tablet server and master.
./bin/kudu-tserver --fs_wal_dir=/opt/kudu/tserver/test-wal --logtostderr --use_hybrid_clock=false --rpc_bind_addresses=0.0.0.0:7050 --rpc_advertised_addresses=${KUDU_QUICKSTART_IP}:7050 --webserver_port=8050 --webserver_advertised_addresses=${KUDU_QUICKSTART_IP}:8050 --webserver_doc_root=/opt/kudu/www --stderrthreshold=0 &> tserver.log &
./bin/kudu-master --fs_wal_dir=/opt/kudu/master/test-wal --logtostderr --use_hybrid_clock=false --rpc_bind_addresses=0.0.0.0:7051 --rpc_advertised_addresses=${KUDU_QUICKSTART_IP}:7051 --webserver_port=8051 --webserver_advertised_addresses=${KUDU_QUICKSTART_IP}:8051 --webserver_doc_root=/opt/kudu/www --stderrthreshold=0 &> master.log &
@andrwng
andrwng / boss_notes.txt
Created August 30, 2019 17:44
docker pull apache/kudu:latest
docker run -it --network="docker_default" -p 8050:8050 -p 7050:7050 -p 8051:8051 -p 7051:7051 apache/kudu:latest bash
mkdir master
mkdir tserver
# Run a tablet server and master.
./bin/kudu-tserver --fs_wal_dir=/opt/kudu/tserver/test-wal --logtostderr --use_hybrid_clock=false --rpc_bind_addresses=0.0.0.0:7050 --rpc_advertised_addresses=${KUDU_QUICKSTART_IP}:7050 --webserver_port=8050 --webserver_advertised_addresses=${KUDU_QUICKSTART_IP}:8050 --webserver_doc_root=/opt/kudu/www --stderrthreshold=0 &> tserver.log &
./bin/kudu-master --fs_wal_dir=/opt/kudu/master/test-wal --logtostderr --use_hybrid_clock=false --rpc_bind_addresses=0.0.0.0:7051 --rpc_advertised_addresses=${KUDU_QUICKSTART_IP}:7051 --webserver_port=8051 --webserver_advertised_addresses=${KUDU_QUICKSTART_IP}:8051 --webserver_doc_root=/opt/kudu/www --stderrthreshold=0 &> master.log &
# Now create a table.
@andrwng
andrwng / BroadAuthzScopeBenchmark.diff
Created June 3, 2019 03:00
diff --git a/src/kudu/master/sentry_authz_provider-test.cc b/src/kudu/master/sentry_authz_provider-test.cc
index e7304ab..42ad8f0 100644
--- a/src/kudu/master/sentry_authz_provider-test.cc
+++ b/src/kudu/master/sentry_authz_provider-test.cc
@@ -59,6 +59,7 @@
#include "kudu/util/random.h"
#include "kudu/util/random_util.h"
#include "kudu/util/status.h"
+#include "kudu/util/stopwatch.h"
#include "kudu/util/test_macros.h"