Matti Picus mattip

Here is some code implementing a kahan summation. It is not fast

import cffi
import numpy as np
import time

ffi = cffi.FFI()
ffi.cdef("""

Cloning the hg repo as a git repo

the tools

Get this repo https://github.com/mnauw/git-remote-hg which is a fork of the original and supports modern git/hg. Copy git-hg-helper and git-remote-hg to ~/bin and make them executable with chmod +x ~/bin/git*

clone the repo as both hg and git

git clone hg::https://foss.heptapod.net/pypy/cffi cffi-git

	# commit 6ccbd45e79cb
	# running runner.py --baseline ../pypy3.11-v7.3.19-linux64/bin/pypy --changed /tmp/venv311pypy/bin/pypy -b \
	# pickle,pickle_dict,pickle_list,cpython_doc,unpickle,unpickle_list
	{
	"revision": null,
	"results": [
	[
	"sphinx",
	"RawResult",
	{

	The problematic place in the code is in the compression, when the test does compression in many threads:
	```
	def testThreading(self):
	# Issue #7205: Using a BZ2File from several threads shouldn't deadlock.
	data = b"1" * 2**20
	nthreads = 2
	with BZ2File(self.filename, 'wb') as f:
	def comp():
	for i in range(5):
	# f.write(data) # original test does this, change for debugging

	Running `python -c "import numpy", I get a recursion error. Running a lldebug0 build under gdb, here is the backtrace.

	Searching for cpyext, I see there is a chain starting around frame 75 that maybe I can do something about.
	But why are there 40 recursive calls to `pypy_g_dispatch_bytecode__AccessDirect_None` ?

	#0 __pyx_pf_5numpy_6random_13bit_generator_12SeedSequence_4generate_state (__pyx_v_self=0xcfb1c0, __pyx_v_n_words=0xcfd7b0, __pyx_v_dtype=0x95de18)
	at numpy/random/bit_generator.c:6289
	#1 0x00007fffec5c0cc5 in __pyx_pw_5numpy_6random_13bit_generator_12SeedSequence_5generate_state (__pyx_v_self=0xcfb1c0, __pyx_args=0xcfd960,
	__pyx_kwds=0xcfd9b0) at numpy/random/bit_generator.c:6114
	#2 0x00007ffff2b4446f in pypy_g_cpy_call_external__funcPtr__objectPtr__objectPtr_1 (

	$ grep '^[A-Za-z]*Error' /tmp/stdout.txt \|sort \|uniq -c \|sort -n -r
	65 AssertionError: DeprecationWarning not triggered
	48 AssertionError: TypeError not raised by dumps
	42 AssertionError: events did not match expectation:
	16 AssertionError: Lists differ: [1, 1] != [2, 2]
	13 ValueError: array.index(x): x not in list
	13 AssertionError: UnicodeEncodeError not raised
	8 AssertionError: TypeError not raised
	7 SyntaxError: invalid syntax
	6 AssertionError: 4294967298 != 0

	Build aarch64, ppc pypy feedstock

	for each machine aarch64 (for instance gcc185.fsffrance.org), 140.211.169.173 (ppc64le)
	1a. Mambaforge is preferred.
	1b. screen
	2a. conda create -n build python=3.11
	2b. conda activate build
	2c. conda install -y git conda-build conda-smithy anaconda-client boa patch
	3a. git clone https://github.com/conda-forge/pypy3.6-feedstock.git
	3b. cd pypy3.6-feedstock

	This C function

	typedef struct {
	PyObject *in;
	PyObject *out;
	} ufunc_full_args;

	static int
	PyUFunc_GenericFunctionInternal(PyUFuncObject ufunc, PyArrayObject *op,
	ufunc_full_args full_args, PyObject type_tup, PyObject extobj,

	BEFORE AFTER
	[4e2f663bd9fe8] {jit-summary [4e2d5889878b8] {jit-summary
	Tracing: 86 0.100334 Tracing: 78 0.113238
	Backend: 74 0.031218 Backend: 66 0.032329
	TOTAL: 1.515222 TOTAL: 1.509000
	ops: 124400 ops: 123038
	heapcached ops: 78660 heapcached ops: 79768
	recorded ops: 39196 recorded ops: 40168
	calls: 6160 calls: 5844
	guards: 9656 guards: 10059

	import random
	import pyperf

	def create_data(fname='/tmp/lines.txt'):
	printable = tuple(c for c in (chr(i) for i in range(32, 0x110000)) if c.isprintable() and not c.isspace())

	with open(fname, "w", encoding="utf8") as f:
	for _ in range(10000):
	f.write("".join(random.choices(printable, k=random.randint(1000, 10000))))
	f.write("\n")