Mat Kelly machawk1

Install maven and java with coordinating versions that work with your system and Heritrix (see https://github.com/internetarchive/heritrix3/blob/master/.github/workflows/maven.yml):

sudo dnf install maven-openjdk17

Building latest Heritrix (in this example, I put heritrix distributions at /usr/local/ and have a symlink named h3 that points to the one I currently use, i.e. /usr/local/h3 is a symlink pointing to /usr/local/heritrix-3.6.1-SNAPSHOT-20241205)

cd /tmp
git clone https://github.com/internetarchive/heritrix3.git
cd heritrix3

Learn Go in ~5mins

This is inspired by A half-hour to learn Rust and Zig in 30 minutes.

Basics

Your first Go program as a classical "Hello World" is pretty simple:

First we create a workspace for our project:

The subfile package is a much more efficient way to bring in the sub parts of a large latex document.

Each part of the complete document that is split up into its own text file brought is added to the document from the main tex file via

\subfile{file}

Each subfile has the following structure

\documentclass[main.tex]{subfiles}

install solr and create a core (books)

brew install solr
solr start
solr create -c books -d /usr/local/Cellar/solr/7.2.1/example/files/conf

index a pdf

post -c books /tmp/gabriella-giannachi-archive-everything-mapping-the-everyday.pdf

https://asciinema.org/a/FqmauknkDWf8eIXHbyd77aJct

install rat https://github.com/ericfreese/rat

go get github.com/ericfreese/rat

install warcio and warctools

InterPlanetary Version Control (call it `IPVC`?)

IPLD-based Version History

This is just a sketch of a possibility. If we just want a git-style toolchain with git version graph, it might be better to just put an ipfs storage adapter behind go-git -- basically putting IPFS unixfs trees where git usually creates git tree objects. In that case you would have regular git commit objects, not IPLD objects. That would be less reusable beyond the git context but it would fit better with existing git-based tooling.

Keep in mind: it will be really useful to be able to use IPFS files api to manipulate these trees, allowing you to do things like modify part of a large dataset and commit the changes without ever pulling the whole dataset -- you can just pull the parts that you're changing.

	# run the script on a set of URI-Ms:
	# python3 capture-requests.py < to-request.txt >> requests-log.txt

	# process the results and generate a new list of URI-Ms that were requested:
	# awk '{if ($1 ~ /cnn\.com(:80)?[\/]+$/ && $2 == "200") print $0}' requests-log.txt \| sort -t '/' -k 5 >! requests.txt

	# https://pypi.org/project/selenium-wire/#installation
	import sys
	import time
	from seleniumwire import webdriver # Import from seleniumwire

	// what substring in the URL indicates to us that we are editing the blog post
	let isPostPreview = 'post-preview'
	// the query selector to use for referencing the click trap element
	let querySelectorForClickTrap = '.blogger-clickTrap'
	// how long should the delay be before we attempt to remove the click trap element
	let wait = 3000
	if (window.location.href.indexOf(isPostPreview) !== -1) {
	setTimeout(() => {
	console.log('nukeClickTrap')
	let it = document.querySelector(querySelectorForClickTrap)

	#!/usr/bin/env bash

	####
	# Original version of this file was created by Dr. Chuck Cartledge
	# as a make file and can be found in his WS-DL blog post:
	# https://ws-dl.blogspot.com/2014/07/2014-0-7-2-ode-to-margin-police-or-how.html
	####

	#### Set these variables to match your setup
	CHECK="msThesis"

Mat Kelly machawk1

Learn Go in ~5mins

Basics

InterPlanetary Version Control (call it IPVC?)

Features

InterPlanetary Version Control (call it `IPVC`?)