Baiju Zhang zbjdonald

Parallelized `pd.read_csv` with joblib

Reading a large CSV file via pandas and joblib. Probably degrades due to pd.concat usage. Tests and better function parameter definitions and documentation pending.

A very objective test on a 5GB CSV file (shape=()) resulted in a Kernel died message (it was run in a Jupyter notebook and repeated twice) when using pd.read_csv directly. In contrast, using read_csv_joblib with the following settings returned in 3h 4m: Concatenating the row chunks took the longest.

How To Install OpenSSL 1.1.1 on CentOS 7

This tutorial goes through how to install openssl 1.1.1 on CentOS 7, since the yum repo only installs up to openssl 1.0.

Requirements

Upgrade the system

yum -y update

optimising-unifi-performance

NOTE: Content below is written by Adrian Mace. Click here for an updated version.

Below are the key settings that I apply on any unifi installation for optimal performance.

Settings

Settings > Site

Ensure Enable Advanced Features is enabled
This allows you to follow along with the guide in it's entirety.

the following regex will validate all examples provided here: https://www.conventionalcommits.org/en/v1.0.0/

  ^(build|chore|ci|docs|feat|fix|perf|refactor|revert|style|test){1}(\([\w\-\.]+\))?(!)?: ([\w ])+([\s\S]*)

a grep/posix compatible variant

  ^(build|chore|ci|docs|feat|fix|perf|refactor|revert|style|test){1}(\([[:alnum:]._-]+\))?(!)?: ([[:alnum:]])+([[:space:][:print:]]*)

How to set up CI with Synology

Before you start

Install packages on your Synology with the Package Manager

Docker
Git

	import tqdm
	from joblib import Parallel, delayed


	class ParallelTqdm(Parallel):
	"""joblib.Parallel, but with a tqdm progressbar

	Additional parameters:
	----------------------
	n_jobs: int, default: 0

	#!/bin/bash
	# 获取第一个参数
	ARG1="$1"
	ROOT_FOLDER=""
	SCRIPT_NAME="$0"
	SCRIPT_VERSION="1.2.3"
	VERSION=""
	WEB_FOLDER=""
	ORG_INDEX_FILE="index.original.html"
	INDEX_FILE="index.html"

	import streamlit as st
	from persist import persist, load_widget_state


	def main():
	if "page" not in st.session_state:
	# Initialize session state.
	st.session_state.update({
	# Default page.
	"page": "home",

	import { EChartOption } from 'echarts';

	declare interface _Format extends EChartOption.Tooltip.Format {
	marker: string;
	axisValueLabel: string;
	}

	const formatter: EChartOption.Tooltip.Formatter = function(params: _Format\|Array<_Format>, _ticket, _callback) {
	if(params instanceof Array) {
	if(params.length) {

	# If you don‘t want to build it youself, you can try `docker pull killercai/postgres`.
	FROM healthcheck/postgres:latest

	# China debian mirror
	RUN sed -i s@/deb.debian.org/@/mirrors.aliyun.com/@g /etc/apt/sources.list
	RUN apt-get clean && apt-get update
	RUN apt-get install -y wget git build-essential libpq-dev python-dev postgresql-server-dev-all
	# SCWS (Simple Chinese Word Segmentation library)
	RUN cd /tmp && wget -q -O - http://www.xunsearch.com/scws/down/scws-1.2.1.tar.bz2 \| tar xjf - && cd scws-1.2.1 && ./configure && make install
	# zhpaser (postgres plugin)