One Paragraph of project description goes here
These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on how to deploy the project on a live system.
Alex Marvin amarvin
joshlk
/ faster_toPandas.py
Last active
September 19, 2025 16:11
PySpark faster toPandas using mapPartitions
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import pandas as pd | |
| def _map_to_pandas(rdds): | |
| """ Needs to be here due to pickling issues """ | |
| return [pd.DataFrame(list(rdds))] | |
| def toPandas(df, n_partitions=None): | |
| """ | |
| Returns the contents of `df` as a local `pandas.DataFrame` in a speedy fashion. The DataFrame is | |
| repartitioned if `n_partitions` is passed. |
PurpleBooth
/ README-Template.md
Last active
December 3, 2025 00:19
A template to make good README.md
vladignatyev
/ progress.py
Last active
November 9, 2025 21:37
Python command line progress bar in less than 10 lines of code.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # The MIT License (MIT) | |
| # Copyright (c) 2016 Vladimir Ignatev | |
| # | |
| # Permission is hereby granted, free of charge, to any person obtaining | |
| # a copy of this software and associated documentation files (the "Software"), | |
| # to deal in the Software without restriction, including without limitation | |
| # the rights to use, copy, modify, merge, publish, distribute, sublicense, | |
| # and/or sell copies of the Software, and to permit persons to whom the Software | |
| # is furnished to do so, subject to the following conditions: | |
| # |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| select.form-control + .chosen-container.chosen-container-single .chosen-single { | |
| display: block; | |
| width: 100%; | |
| height: 34px; | |
| padding: 6px 12px; | |
| font-size: 14px; | |
| line-height: 1.428571429; | |
| color: #555; | |
| vertical-align: middle; | |
| background-color: #fff; |