Thibaut Lamadon tlamadon

AWS Batch

Ref. https://docs.aws.amazon.com/batch/latest/userguide/what-is-batch.html

The goal of this document is submitting a job to AWS Batch and confirming the result in CloudWatch Logs.

Since I've worked on ap-northeast-1 region, The following examples includes this region name.

Setting Up with AWS Batch

Ref. https://docs.aws.amazon.com/batch/latest/userguide/get-set-up-for-aws-batch.html

List

Requirements

Dynamic workflow
Environment encapsulation

There are packages for this now!

2017-08-03: Since I wrote this in 2014, the universe, specifically Kirill Müller (https://github.com/krlmlr), has provided better solutions to this problem. I now recommend that you use one of these two packages:

rprojroot: This is the main package with functions to help you express paths in a way that will "just work" when developing interactively in an RStudio Project and when you render your file.
here: A lightweight wrapper around rprojroot that anticipates the most likely scenario: you want to write paths relative to the top-level directory, defined as an RStudio project or Git repo. TRY THIS FIRST.

I love these packages so much I wrote an ode to here.

I use these packages now instead of what I describe below. I'll leave this gist up for historical interest. 😆

	###
	### [2023-06-19] UPDATE: Just tried to use my instructions again on a fresh install and it failed in a number of places.
	###. Not sure if I'll update this gist (though I realise it seems to still have some traffic), but here's a list of
	###. things to watch out for:
	### - Check out the `nix-darwin` instructions, as they have changed.
	### - There's a home manager gotcha https://github.com/nix-community/home-manager/issues/4026
	###

	# I found some good resources but they seem to do a bit too much (maybe from a time when there were more bugs).
	# So here's a minimal Gist which worked for me as an install on a new M1 Pro.

	fif() {
	rg \
	--column \
	--line-number \
	--no-column \
	--no-heading \
	--fixed-strings \
	--ignore-case \
	--hidden \
	--follow \

	FROM alpine

	# Install OpenSSH:
	RUN apk -U add openssh

	# Generate host keys:
	RUN ssh-keygen -A

	# Create users:
	RUN adduser -D user1

	#SingleInstance Force
	#NoTrayIcon ;If you want the tray icon to be visible; comment this line by adding a semicolon ; in front of the #. Example: ;#NoTrayIcon

	; ALTERNATIVE METHOD: Try the built-in Everything option "hotkey_explorer_path_search"
	; ^^^INFO: https://www.voidtools.com/forum/viewtopic.php?p=17390#p17390

	EverythingPath := "C:\Program Files\Everything\Everything.exe" ;Set this to your everything.exe path. Keep the quotes.

	;---Optional setup for special folders---
	MyRecycleBin := "Recycle Bin" ;If your OS is not English, go to your explorer's "Recycle Bin" (🚮) folder and change this to the title of that window. Keep the quotes.

	#!/bin/env nextflow

	Channel
	.from('abc')
	.map { tuple(0,it) }
	.set { foo }

	Channel.create().set { feedback }

	index = 0

	#!/usr/bin/env python

	# File: log_exec.py
	# Author: McKay Davis
	# Date: Jun 23, 2014
	# Non-blocking stdout and stderr read from a
	# Popen process

	import os
	import subprocess

	function laplace_idiomatic (del_x::Array{Float64, 3}, x::Array{Float64, 3})

	n1, n2, n3 = size (x)
	for i3 = 2:n3-1, i2 = 2:n2-1, i1 = 2:n1-1

	del_x[i1,i2,i3] =
	x[i1+1,i2,i3] + x[i1-1,i2,i3] +
	x[i1,i2+1,i3] + x[i1,i2-1,i3] +
	x[i1,i2,i3+1] + x[i1,i2,i3-1] -
	6.0 * x[i1,i2,i3]