John McCrary ouachitalabs
ouachitalabs
/ analyze_whitespace_tokens.py
Created
October 21, 2025 12:18
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| """ | |
| Total vocabulary size: 200,019 | |
| WHITESPACE TOKEN ANALYSIS | |
| Found 434 whitespace-only tokens | |
| Percentage of vocab: 0.217% | |
| SUMMARY BY WHITESPACE TYPE: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # Generated by Powerlevel10k configuration wizard on 2025-06-13 at 07:41 CDT. | |
| # Based on romkatv/powerlevel10k/config/p10k-pure.zsh, checksum 7533. | |
| # Wizard options: powerline, pure, original, 1 line, compact, instant_prompt=verbose. | |
| # Type `p10k configure` to generate another config. | |
| # | |
| # Config file for Powerlevel10k with the style of Pure (https://github.com/sindresorhus/pure). | |
| # | |
| # Differences from Pure: | |
| # | |
| # - Git: |