mrjsj

Recently created

Least recently created

Recently updated

Least recently updated

mrjsj / identity_column.py

Created October 30, 2025 20:44

PySpark gap-free identity column

	import pyspark.sql.functions as F
	from pyspark.sql import DataFrame
	from pyspark.sql.types import LongType, StructField, StructType
	from pyspark.sql.window import Window


	def identity_column_with_zip_with_index(df: DataFrame, offset: int = 0) -> DataFrame:

	rdd_with_index = df.rdd.zipWithIndex()

mrjsj / export-notebook-definition.sql

Created March 15, 2025 11:23

The DuckDB UI stores notebook content in an internal database called _duckdb_ui. You can query and export notebook content, as well as insert new definitions into the database. Warning: Modifying the internal database may lead to corruption and data loss. Be cautious and use it at your own risk!

	copy (
	select
	"json"
	from _duckdb_ui.notebook_versions
	where 1=1
	and title = 'MySingleNotebook'
	and expires is null
	) to 'exported-notebook.json';

mrjsj / FabricDuckDBConnection.py

Last active December 2, 2024 16:41

A utility class combining functionality from DuckDB, Delta Lake and Microsoft Fabric

	class FabricDuckDBConnection():

	def __init__(self, config: dict = {}):
	self._registered_tables = []
	self._connection = duckdb.connect(config=config)

	def __getattr__(self, name):
	return getattr(self._connection, name)

	def __dir__(self):