Arthur Schreiber arthurschreiber

Deep Dive: Why PR #19620 Succeeds Where #15522 and #17135 Failed

Context

Three PRs have attempted to optimize Vitess streaming by reducing the overhead of MySQL-to-vtgate data transfer. PRs #15522 and #17135 showed no meaningful performance gains. PR #19620 achieves +5.6% QPS on OLTP-READONLY-OLAP and +35% QPS on TPCC-OLAP with statistical significance (p=0.000). This document analyzes why.

PR #15522 — "[wip] Raw MySQL Packets" (vmg, Mar 2024)

You can "force" commands to be run on a login for a specific user in ~/.ssh/authorized_keys

To always run the /home/deployuser/deploy.sh command:


command="DEPLOY_USER='arthur.schreiber@hp.com' /home/deploy/deploy.sh $SSH_ORIGINAL_COMMAND",no-agent-forwarding,no-user-rc,no-X11-forwarding,no-port-forwarding" ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA1JnkxQDucKWOAD8QsmAVlXT0trq/QmtKeGkL+a6QUvctKBpe8qLuYUyGEgn+GvGwP9IqDZPYXBP0andFvim+40QKworx9oALuIei5rIWM3fCIbY0lTdM6gZVgTdYBNryW5axeuwsXjuT/i5lE+lA1JxQ5LjTq7n5gbT8tus4FbHySSkmrWmoz9rh8m9bULnmGquDQw6+xCbPFteVUhEDRsCsmlUldLMatMDuQ5hnhz0mKU41sqs+EPtmYh6pG/ofpyURMsR7/C/Rf1/i6AB9Mb3WqQOGE2x3cj+sj0vILiuHsnEx32qpNDvqrE+scRjcRsKIiaTSD4jqO9Mws5Ob7w==

command="DEPLOY_USER='arthur.schreiber@hp.com' /home/deploy/deploy.sh $SSH_ORIGINAL_COMMAND",no-agent-forwarding,no-user-rc,no-X11-forwarding,no-port-forwarding" <jenkins pub>
command="DEPLOY_USER='arthur.schreiber@hp.com' /home/deploy/deploy.sh $SSH_ORIGINAL_COMMAND",no-agent-forwarding,no-user-rc,no-X11-forwarding,no-port

	'use strict';

	var net = require('net');
	var http = require("http");

	var count = 0;

	var server = http.createServer(function(req, res) {
	var id = res.id = count++;

	require 'rugged'

	repo = Rugged::Repository.init_at('./test-repo')

	index = repo.index

	base_commit_options = {
	author: { name: "Matt", email: "matt@test.com" },
	committer: { name: "Matt", email: "matt@test.com" },
	update_ref: 'HEAD'

	$:.unshift File.expand_path("../lib", __FILE__)

	require 'rugged'

	repo = Rugged::Repository.init_at('./test-repo')

	index = repo.index

	base_commit_options = {
	author: { name: "Matt", email: "matt@test.com" },

	exec { "download redis-${version}":
	command => "wget --output-document redis-${version}.tar.gz https://github.com/antirez/redis/archive/${version}.tar.gz",
	cwd => '/usr/local/src',
	path => ['/bin', '/usr/bin'],
	creates => "/usr/local/src/redis-${version}.tar.gz",
	} -> exec { "extract redis-${version}":
	command => "tar xfz redis-${version}.tar.gz",
	cwd => '/usr/local/src',
	path => ['/bin', '/usr/bin'],
	creates => "/usr/local/src/redis-${version}",

	package require "TclOO"

	oo::class create oss::class {
	superclass oo::class

	constructor { args } {
	oo::class create [self].Meta

	next {*}$args

	Arthurs-iMac-2:rugged arthur$ bin/rake cross native gem RUBY_CC_VERSION=1.9.3:2.0.0
	make -f Makefile.embed clean
	rm -f src/attr.o src/attr_file.o src/blob.o src/branch.o src/buf_text.o src/buffer.o src/cache.o src/checkout.o src/clone.o src/commit.o src/commit_list.o src/compress.o src/config.o src/config_cache.o src/config_file.o src/crlf.o src/date.o src/delta-apply.o src/delta.o src/diff.o src/diff_driver.o src/diff_file.o src/diff_patch.o src/diff_print.o src/diff_tform.o src/diff_xdiff.o src/errors.o src/fetch.o src/fetchhead.o src/filebuf.o src/fileops.o src/filter.o src/fnmatch.o src/global.o src/graph.o src/hash.o src/hashsig.o src/ignore.o src/index.o src/indexer.o src/iterator.o src/merge.o src/merge_file.o src/message.o src/mwindow.o src/netops.o src/notes.o src/object.o src/object_api.o src/odb.o src/odb_loose.o src/odb_pack.o src/oid.o src/pack-objects.o src/pack.o src/path.o src/pathspec.o src/pool.o src/posix.o src/pqueue.o src/push.o src/refdb.o src/refdb_fs.o src/reflog.o src/refs.o src/refsp

	$:.unshift File.join(File.dirname(__FILE__), "lib")
	require "rugged"

	repo = Rugged::Repository.new(".")

	10000000.times do
	begin
	Rugged::Reference.each(repo) do \|ref\|
	raise "fail"
	end