This is a script to convert the csv of crawling at home to parquet
See the result at https://colab.research.google.com/drive/14Hc_fUUOrG9260VzD_XsTxWX7f5cptyL?usp=sharing
名無しＫさん koke2c95
kinoc
/ j6b_train_hf_ds.py
Last active
September 17, 2024 18:53
So now you want to finetune that GPT-J-6B on a 3090/TITAN GPU ... okay, using HF and DeepSpeed too
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # So now you want to finetune that GPT-J-6B on a 3090/TITAN GPU ... okay | |
| # More exploratory coding. It uses the Huggingface model port, deepspeed and reads all text/md files from a target directory | |
| # It is a fragment of a larger system with remote editing, but that's another story | |
| # This is the raw, training tester. Items to look out for: | |
| # - uses DeepSpeed and has a DS config | |
| # - to save space uses SGD instead of ADAM | |
| # - uses gradient checkpointing | |
| # - freezes 25% of the layers to fit | |
| # Assumes you can already run https://gist.github.com/kinoc/2d636a68876cd3de7b6e9c9452b61089 |
fredgido
/ LITD.user.js
Last active
April 8, 2023 19:42
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| // ==UserScript== | |
| // @name Look Into The Deep | |
| // @namespace LITD | |
| // @version 10.0 | |
| // @description ... | |
| // @author made by fredgy-kun | |
| // @match https://*.donmai.us/uploads/* | |
| // @match https://*.donmai.us/posts/* | |
| // @connect deepdanbooru.donmai.us | |
| // @grant GM.xmlHttpRequest |
r7kamura
/ 複数の役職に分割できる可能性のあるアニメスタッフの役職の表記一覧.txt
Created
February 12, 2017 07:38
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| 2Dデザイン・特殊効果 | |
| 2Dワークス・特殊効果 | |
| 2D・モニターデザイン | |
| 2D・特殊効果 | |
| 3Dディレクター・モデリング | |
| 3Dディレクター・モデルデザイン | |
| 3Dメカニックデザイン・3D監修 | |
| 3Dヴィジュアル・ディレクター | |
| 3D・CG制作・モニターグラフィックス | |
| CAD・メカニックデザイン |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [ | |
| { | |
| "num": "n01440764", | |
| "en": "tench", | |
| "ja": "テンチ" | |
| }, | |
| { | |
| "num": "n01443537", | |
| "en": "goldfish", | |
| "ja": "金魚" |
yrevar
/ imagenet1000_clsidx_to_labels.txt
Last active
November 21, 2025 06:36
text: imagenet 1000 class idx to human readable labels (Fox, E., & Guestrin, C. (n.d.). Coursera Machine Learning Specialization.)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| {0: 'tench, Tinca tinca', | |
| 1: 'goldfish, Carassius auratus', | |
| 2: 'great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias', | |
| 3: 'tiger shark, Galeocerdo cuvieri', | |
| 4: 'hammerhead, hammerhead shark', | |
| 5: 'electric ray, crampfish, numbfish, torpedo', | |
| 6: 'stingray', | |
| 7: 'cock', | |
| 8: 'hen', | |
| 9: 'ostrich, Struthio camelus', |