These files are associated with this blog post.
Note
These filesa are named *.tcl to get syntax highlighting in the gists, should be *.folk when copied to your Folk system.
Created
August 23, 2023 17:31
-
-
Save cwervo/456cc51984fced2533b8f10ea7eb60e7 to your computer and use it in GitHub Desktop.
Folk Data Manipulation Example
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| set fd [file tempfile py script.py] | |
| puts $fd { | |
| import pandas as pd | |
| file_path = '~/image-label/points.parquet' | |
| df = pd.read_parquet(file_path) | |
| json_data = df.to_json(orient='records') | |
| print(json_data) | |
| }; close $fd | |
| Retract $this claims $this has python output /anything/ | |
| On process [list apply {{this py} { | |
| set output [exec python3 $py] | |
| Assert $this claims $this has python output $output | |
| Assert $this claims $this has data $output | |
| Step | |
| }} $this $py] | |
| When $this has python output /o/ { | |
| Wish $this is labelled $o | |
| } | |
| Wish $this is outlined white | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| package require json | |
| proc cleanJSON {inputStr} { | |
| # Use the json::json2dict function from the Tcllib JSON package | |
| set dict [json::json2dict $inputStr] | |
| return $dict | |
| } | |
| When $this points down with length 0.4 at /target/ { | |
| When $target has data /data/ { | |
| set cleanedData [cleanJSON $data] | |
| Wish $this is labelled $cleanedData | |
| Claim $this has cleaned data $cleanedData | |
| } | |
| } | |
| Wish $this is outlined red | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| When $this points down with length 0.4 at /target/ & /target/ has cleaned data /data/ & $this has region /r/ { | |
| lassign [region top $r] x y | |
| foreach point $data { | |
| set pointX [expr {$x + [dict get $point x] * 5}] | |
| set pointY [expr {$y + [dict get $point y] * 5}] | |
| Display::circle $pointX $pointY 10 10 white | |
| Display::text [expr {$pointX + 100}] $pointY 2 [dict get $point label] 0 | |
| } | |
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Step 1: Import necessary libraries | |
| import pandas as pd | |
| import pyarrow as pa | |
| import pyarrow.parquet as pq | |
| # Step 2: Create a DataFrame | |
| df = pd.DataFrame({ | |
| 'x': [1, 2, 3, 4, 5, 6, 7, 8, 9, 10], | |
| 'y': [15, 2, 8, 10, 28, 56, 35, 88, 99, 82], | |
| 'label': ['A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J'] | |
| }) | |
| # Step 3: Convert the DataFrame to PyArrow Table | |
| table = pa.Table.from_pandas(df) | |
| # Step 4: Write the PyArrow Table to Parquet | |
| pq.write_table(table, 'points.parquet') |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment