Created
November 24, 2025 14:24
-
-
Save raditotev/70b913fa027c2f67f0b6d89550c45d47 to your computer and use it in GitHub Desktop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| FROM ./gguf/Llama-3.2-1B-Instruct.gguf | |
| TEMPLATE """{{ if .System }}<|start_header_id|>system<|end_header_id|> | |
| {{ .System }}<|eot_id|>{{ end }}<|start_header_id|>user<|end_header_id|> | |
| {{ .Prompt }}<|eot_id|><|start_header_id|>assistant<|end_header_id|> | |
| {{ .Response }}<|eot_id|>""" | |
| # Stop tokens — prevent garbage output | |
| PARAMETER stop "<|start_header_id|>" | |
| PARAMETER stop "<|end_header_id|>" | |
| PARAMETER stop "<|eot_id|>" | |
| PARAMETER stop "<|reserved_special_token" | |
| PARAMETER stop "[INST]" | |
| PARAMETER stop "[/INST]" | |
| # Inference settings — reduce hallucinations | |
| PARAMETER temperature 0.3 | |
| PARAMETER top_p 0.95 | |
| PARAMETER top_k 20 | |
| PARAMETER min_p 0.05 | |
| PARAMETER repeat_penalty 1.15 | |
| PARAMETER presence_penalty 0.1 | |
| PARAMETER frequency_penalty 0.1 | |
| PARAMETER num_ctx 4096 | |
| PARAMETER num_predict 256 | |
| SYSTEM """You are a truthful, precise, and grounded assistant based on Llama-3.2-3B-Instruct. | |
| - Only answer using information explicitly in the user query or your training. | |
| - If you are uncertain or don't know, say: "I don't have enough information to answer accurately." | |
| - Never invent facts, names, dates, numbers, or citations. | |
| - Never assume or infer beyond what is stated. | |
| - Keep answers concise and directly relevant. | |
| - Do not use external knowledge unless explicitly provided in the prompt. | |
| - If the question is vague, ask for clarification instead of guessing. | |
| """ |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment