Ali Pardhan alik604

OpenTelemetry Logging Flow - In Depth

This document explains the complete journey of a log from your Python application to Grafana through OpenTelemetry and Loki.

l*ink to code: https://github.com/grafana/loki-fundamentals/blob/microservice-otel-collector/greenhouse/loggingfw.py

1. Application Initialization

In your application (e.g., bug_service.py):

Lambda 1 (DB to SQS)

import boto3
from boto3.dynamodb.conditions import Key
from boto3.dynamodb.types import TypeDeserializer

import json

print('Loading function')
tableName = 'fizzbuzz'

FFMPEG to reencode (encode) entire windows directory with GPU or x265

CPU

for %i in (*.mp4) do ffmpeg -i "%i" -c:v libx265 -vtag hvc1 -preset fast "%~ni_265.mp4"

ffmpeg -i input.mp4 -c:v libx265 -vtag hvc1 -preset fast input_compressed.mp4

Nvidia GPU

for %i in (*.mp4) do ffmpeg -i "%i" -c:v hevc_nvenc -preset fast "%~ni_nvda.mp4"

Python & Machine Learning Hello world

Quickly get up to speed

Software setup

Use Python 3.8.*. No point in 3.9, however it should be fine.

Run these to test if install is successful, and install some important packages. Please follow the errors as they come, one will ask the user to install C++ build tools, if not already installed (see below). This will take several minutes, as the dependencies are a few GBs. You must be off VPN, or set-proxy, as shown below.

	Your task is to implement a simplified version of an in-memory database. Plan your design according to the level specifications below:

	Level 1: In-memory database should support basic operations to manipulate records, fields, and values within fields.
	Level 2: In-memory database should support displaying a specific record's fields based on a filter.
	Level 3: In-memory database should support TTL (Time-To-Live) configurations on database records.
	Level 4: In-memory database should support backup and restore functionality.

	I only got to level 3, got 750/750.
	0/?? for Level 4.

	import numpy as np
	import matplotlib.pyplot as plt

	from sklearn.datasets import load_digits
	from sklearn.decomposition import PCA, IncrementalPCA

	iris = load_iris()
	X = iris.data
	y = iris.target

	import requests
	import json
	import webbrowser

	USER = "alik604"
	chrome_path = 'C:/Program Files (x86)/Google/Chrome/Application/chrome.exe %s'
	# Change to your OS - https://stackoverflow.com/a/24353812/5728614

	req = requests.get('https://api.github.com/users/' + USER + '/starred?per_page=100')
	# ?page=2&per_page=100 # page is not working, first result is always the same

	import numpy as np
	import pylab as pl
	from numpy import fft


	def fourierExtrapolation(x, n_predict):
	n = x.size
	n_harm = 10 # number of harmonics in model
	t = np.arange(0, n)
	p = np.polyfit(t, x, 1) # find linear trend in x

	!pip install mnist
	import numpy as np
	import matplotlib.pyplot as plt
	import pandas as pd
	from sklearn import metrics
	import mnist

	# from hmmlearn.hmm import GaussianHMM, MultinomialHMM

	X_train = mnist.train_images()

	import urllib.parse as parse

	url = ""
	if 'url' not in globals():
	print("Enter URL to parse")
	url = input()


	def printDict(data):
	for x in data: