Skip to content

Instantly share code, notes, and snippets.

View parj's full-sized avatar
💭
I may be slow to respond.

Parjanya Mudunuri parj

💭
I may be slow to respond.
View GitHub Profile
"""
Excel Large Data Handler Tool for AGNO with Azure OpenAI
Handles Excel files up to 20MB by chunking and intelligent data sampling
"""
from typing import Any, Dict, List, Optional, Union
import pandas as pd
import openpyxl
from pathlib import Path
import json
@parj
parj / Calcite-extract-Table-Where.java
Last active January 22, 2024 12:03
Example of calcite
package io.github.parj.sql;
import org.apache.calcite.config.Lex;
import org.apache.calcite.sql.*;
import org.apache.calcite.sql.fun.SqlStdOperatorTable;
import org.apache.calcite.sql.parser.SqlParseException;
import org.apache.calcite.sql.parser.SqlParser;
import java.util.HashMap;
import java.util.List;
@parj
parj / spark-run.txt
Created January 19, 2024 06:58
Spark on Kuberentes
# (optional) if using minikube switch docker context
eval $(minikube docker-env)
# Build Spark
bin/docker-image-tool.sh -r docker.io/myrepo -t v3.5 -p kubernetes/dockerfiles/spark/bindings/python/Dockerfile build
# Expose K8S API server (minikube)
kubectl proxy --port=8080
#Create spark service account and give permissions
Have you ever wondered how computer programs talk to databases? Well, that's where JDBC comes to the rescue! JDBC, which stands for "Java Database Connectivity," is like a magical translator between Java programs and databases. It helps them understand each other's language!
Imagine your program wants to store or get some data from a database. JDBC is the friendly bridge that helps them shake hands and exchange information. It makes life easier for developers by handling all the complicated stuff, so they can focus on building awesome apps. Thanks to JDBC, your favorite apps can store and fetch data, making them super cool and smart!
import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
import java.io.BufferedWriter;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStreamWriter;
import java.sql.*;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.io.BufferedWriter;
import java.io.FileWriter;
import java.io.IOException;
import java.sql.*;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
public class DatabaseToJson {
private static final int PAGE_SIZE = 10000;
private static final String DB_URL = "jdbc:mysql://localhost:3306/mydatabase";
import java.io.BufferedWriter;
import java.io.FileWriter;
import java.sql.*;
import org.json.*;
public class DatabaseToJson {
private static final String DB_URL = "jdbc:mysql://localhost/mydatabase";
private static final String DB_USER = "myusername";
private static final String DB_PASSWORD = "mypassword";
private static final int FETCH_SIZE = 1000;
import java.io.BufferedWriter;
import java.io.FileWriter;
import java.io.IOException;
import java.io.Writer;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.util.concurrent.ExecutorService;
import java.io.FileWriter;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import org.json.simple.JSONArray;
import org.json.simple.JSONObject;
import com.fasterxml.jackson.databind.ObjectMapper;
import java.io.File;
import java.sql.*;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
public class DatabaseToJson {
private static final String DB_URL = "jdbc:mysql://localhost:3306/mydatabase";