Wang Zhiwei zhiweio

Adding the Microsoft ODBC Driver for SQL Server to an AWS Glue Python Shell Job for use with pyodbc

AWS Glue does not provide an easy way of adding the Microsoft ODBC Driver to your Python Shell Glue job. To allow pyodbc to recognize our driver, we need to upload the driver's shared library files to a location the Glue job can access.

Step 1

Compile the Microsoft ODBC Driver for SQL Server. I am using the Docker image for AWS Lambda since the environment for Glue is likely also Amazon Linux 2 based.
```
FROM public.ecr.aws/lambda/python:3.9 as builder

RUN yum update -y
```

Docker Hub 镜像加速器

国内从 Docker Hub 拉取镜像有时会遇到困难，此时可以配置镜像加速器。

Dockerized 实践 https://github.com/y0ngb1n/dockerized

1️⃣ Docker daemon 配置代理（推荐）

参考 Docker daemon 配置代理

Commands Reference

Start a new tmux named session tmux new -s terminal-capture
Split the screen using these commands,

vertical split <C-b>"
horizontal split <C-b>%

To navigate between the panes,

To goto Left pane <C-b> left-key
To goto Right pane <C-b> right-key
To goto Top pane up-key

How to publish Python apps for human beings

So, you've created a Python app (be it a graphical user interface with Qt or the like, or a simple command line interface). Great! But how are others going to use it? Python applications often have dependencies (e.g. from third-party modules), and they also need a Python interpreter to run them. For a developer, installing all the necessary bits and bobs to make things work is okay, but that's unacceptable for a normal user - they just want to download the thing and run it.

Below are simple instructions to publish your app on the three main operating systems: Windows, macOS and Linux.

	# -- coding: utf-8 --
	# 14-8-20
	# create by: snower

	import os
	import time
	import struct
	import fcntl
	from logging.handlers import RotatingFileHandler, TimedRotatingFileHandler

	#!/bin/bash

	CONTAINER_NAME=stepfunctions
	STATE_MACHINE_DEFINITION=/path/to/yourstatemachine.json
	STATE_MACHINE_NAME="Name of your process"
	ENV_FILE=path/to/your-stepfunctions-local-credentials.txt
	INPUT="{ \"myInput\": \"hello world\"}"

	function get_value_from_json(){
	VALUE="$(jq .$1 temporary.json)"

	import boto3
	import pandas as pd
	from io import StringIO

	class S3DataFrame(pd.DataFrame):

	"""
	# Make a dataframe and upload it as csv
	s3df = S3DataFrame({'h1':[1], 'h2':[2]})
	s3df.to_s3(Bucket='bucket-name',

	import java.io.InputStreamReader;
	import java.io.Reader;
	import java.net.URL;
	import java.net.URLConnection;

	import javax.net.ssl.HostnameVerifier;
	import javax.net.ssl.HttpsURLConnection;
	import javax.net.ssl.SSLContext;
	import javax.net.ssl.SSLSession;
	import javax.net.ssl.TrustManager;

	# Copyright (c) 2018 Bao Nguyen <[email protected]>
	#
	# Permission is hereby granted, free of charge, to any person obtaining a copy
	# of this software and associated documentation files (the "Software"), to deal
	# in the Software without restriction, including without limitation the rights
	# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
	# copies of the Software, and to permit persons to whom the Software is
	# furnished to do so, subject to the following conditions:
	#
	# The above copyright notice and this permission notice shall be included in all

	#!/bin/bash -e

	nullsha="0000000000000000000000000000000000000000"
	maxsize=5
	maxbytes=$(( $maxsize * 1024 * 1024 ))
	status=0

	# Read stdin for ref information
	while read oldref newref refname; do
	echo ref: $oldref $newref $refname