xescuder’s gists

xescuder / load_all_exchanges_etl.py

Created June 13, 2024 16:45

Spark Load CSV into DB

	import os
	from urllib.request import urlopen
	from pyspark.sql import SparkSession
	from pyspark.sql.types import StructType, StructField, StringType
	from config import AppConfig
	import findspark
	from pipeline_utils import rename_cols, snake_case
	from trading_data_pipeline.load_daily_bars_etl import JDBC_PROPERTIES

	"""

xescuder / pandas_utils.py

Last active June 12, 2024 21:05

Pandas Storing Only Changes to DB (SqlAlchemy 2.0, Pandas 2.0)

	import sqlalchemy as db
	import pandas as pd


	def create_upsert_method(meta: db.MetaData):
	"""
	Create upsert method that satisfied the pandas's to_sql API. Adapted to 2.0
	"""

	def method(table, conn, keys, data_iter):

xescuder / StockChart.py

Created March 28, 2024 08:59

Trading View Chart to show in a Streamlit App

	import json
	from streamlit_lightweight_charts import renderLightweightCharts
	import pandas_ta as ta


	class StockChart:

	@staticmethod
	def show_chart(df, symbol):
	COLOR_BULL = 'rgba(38,166,154,0.9)' # #26a69a

xescuder / ib_insync_streamlit.py

Last active March 15, 2024 20:32

Example of streamlit using ib_insync and streamlit

	import logging
	import pandas as pd
	from dateutil.relativedelta import relativedelta
	from dateutil.utils import today

	from app.charts import render_candlestick_chart
	import streamlit as st
	import asyncio
	import plotly.graph_objects as go

xescuder / base_strategy.py

Last active September 4, 2024 20:50

Backtrader Base Strategy (adapted from Esteban Thiliez for multiinstruments and some simplifications)

	from abc import abstractmethod
	import backtrader as bt


	class BaseStrategy(bt.Strategy):
	params = dict(
	verbose=True,
	longs_enabled=True,
	shorts_enabled=True
	)

xescuder / golden_cross_stop_trailer.py

Created March 3, 2024 11:04

A backtrader multidata example with golden cross strategy and stop trailer (based on strategy entries and exits)

	import backtrader as bt

	from trading_backtesting.stop_trailer import StopTrailer


	class GoldenCrossStrategy(bt.Strategy):
	SHORT, NONE, LONG = -1, 0, 1

	params = dict(
	fast_length=50,

xescuder / top_instruments.ipynb

Last active February 15, 2024 17:43

Top Instruments - DuckDB and Polars

Sorry, something went wrong. Reload?

Sorry, we cannot display this file.

Sorry, this file is invalid so it cannot be displayed.

xescuder / return_changes_1year.sql

Created January 21, 2024 13:13

Return changes between today and 1 year ago

	SELECT s.symbol, s.sector, first(close, date) AS first_close, last(close, date) AS last_close,
	MIN(date) AS first_date,
	MAX(date) AS last_date,
	100 * (last(close, date) - first(close, date)) / first(close, date) AS return_change
	FROM
	daily_bars db, stocks s
	WHERE
	db.date >= now() - interval '1 year' AND db.symbol = s.symbol
	GROUP BY
	s.symbol

xescuder / load_sectors_etfs.py

Created January 16, 2024 17:45

Load CSV into Table

	import os
	from pathlib import Path
	from sqlalchemy import create_engine
	import pandas as pd

	from trading_data_pipeline.config import AppConfig

	config = AppConfig(os.environ)

	connection_url = config.get_postgres_uri()

xescuder / docker-compose-airflow.yaml

Created December 31, 2023 13:32

Docker Compose Airflow

	version: "3"
	x-airflow-common: &airflow-common
	# In order to add custom dependencies or upgrade provider packages you can use your extended image.
	# Comment the image line, place your Dockerfile in the directory where you placed the docker-compose.yaml
	# and uncomment the "build" line below, Then run `docker-compose build` to build the images.
	image: trading-airflow #${AIRFLOW_IMAGE_NAME:-apache/airflow}
	# build: .
	environment: &airflow-common-env
	AIRFLOW__CORE__EXECUTOR: LocalExecutor
	AIRFLOW__DATABASE__SQL_ALCHEMY_CONN: postgresql+psycopg2://airflow:airflow@postgres/airflow