Harshavardhana harshavardhana

MinIO AIStor vs MinIO OSS - Complete Technical Comparison (13,061 commits analyzed)

MinIO AIStor vs MinIO OSS - Complete Technical Comparison

Analysis based on full commit history review of 13,061 commits

MinIO AIStor vs MinIO OSS - Comprehensive Technical Comparison

Analysis based on full commit history review of 2800+ commits

MinIO: Synchronous vs Asynchronous Replication

Synchronous Replication

Client waits for replication to complete before receiving response.

┌────────┐         ┌─────────────────────────────┐         ┌────────────┐
│        │  PUT    │      SOURCE CLUSTER         │         │   REMOTE   │
│ Client ├────────►│                             │         │   TARGET   │

    Outbound
    ┌───────────────────┐             ┌────────────────┐               ┌───────────────┐                      ┌────────────────┐
    │                   │   Parr.     │                │  (http body)  │               │                      │                │
    │ Bitrot Hash       │     Write   │      Pipe      │      Read     │  HTTP buffer  │    Write (syscall)   │  TCP Buffer    │
    │ Erasure Shard     │ ──────────► │  (unbuffered)  │ ────────────► │   (64K Max)   │ ───────────────────► │    (4MB)       │
    │                   │             │                │               │  (io.Copy)    │                      │                │

Spark-shell with S3A based checkpointing

Spark context Web UI available at http://nirisvara:4040
Spark context available as 'sc' (master = local[*], app id = local-1677271782301).
Spark session available as 'spark'.
Welcome to
      ____              __
     / __/__  ___ _____/ /__
 _\ \/ _ \/ _ `/ __/ '_/

	## template:jinja
	{#
	This file (/etc/cloud/templates/hosts.debian.tmpl) is only utilized
	if enabled in cloud-config. Specifically, in order to enable it
	you need to add the following to config:
	manage_etc_hosts: True
	-#}
	# Your system has configured 'manage_etc_hosts' as True.
	# As a result, if you wish for changes to this file to persist
	# then you will need to either

	version: '3.7'

	# Settings and configurations that are common for all containers
	x-minio-common: &minio-common
	image: quay.io/minio/minio:${RELEASE}
	command: server http://site1-minio{1...4}/data{1...2}
	environment:
	- MINIO_PROMETHEUS_AUTH_TYPE=public
	- CI=true

	"""
	FTP benchmark.

	Usage:
	ftpbench --help
	ftpbench -h <host> -u <user> -p <password> [options] login
	ftpbench -h <host> -u <user> -p <password> [options] upload <workdir> [-s <size>]
	ftpbench -h <host> -u <user> -p <password> [options] download <workdir> [-s <size>] [--files <count>]

	Connection options:

	#!/bin/bash \

	i=1

	sudo rm -f /tmp/fstab
	for disk in $(lsblk -i -p -n -o NAME \| grep -v 'loop\\|nvme0\\|nvme1\\|nvme2\\|nvme3\\|ubuntu\\|md\\|sda' \| sort); do
	sudo mkdir -p /disk${i}
	sudo mkfs.xfs -f -L disk${i} $disk
	echo "LABEL=disk${i} /disk${i} xfs defaults,noatime 0 0" \| sudo tee -a /tmp/fstab
	i=$(( $i + 1 ))