Skip to content

Instantly share code, notes, and snippets.

@jeffehobbs

jeffehobbs/results.txt

Created July 19, 2025 18:01
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save jeffehobbs/02cd73e69472fac71fec2eae2e9a083e to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save jeffehobbs/02cd73e69472fac71fec2eae2e9a083e to your computer and use it in GitHub Desktop.
Download ZIP
Local LLM Testing (16GB, M1 MacBook Pro)
Raw
results.txt
(venv) jeffwork@MYR6NHR63N0 ollama_tests % python structure.py
---
Testing: llama3.2:latest ...
Using prompt of: Tell me about Canada.
name='Canada' capital='Ottawa' languages=['English', 'French']
Execution time: 4.7 seconds
---
---
Testing: llama3.2:latest ...
Using prompt of: Tell me about Australia.
name='Australia' capital='Canberra' languages=['English', 'Indigenous Australian languages']
Execution time: 0.8 seconds
---
---
Testing: llama3.2:latest ...
Using prompt of: Tell me about China.
name='China' capital='Beijing' languages=['Mandarin Chinese', 'Cantonese', 'Min Nan', 'Wu', 'Xia', 'Zhuang', 'Yi', 'Tibetan', 'Mongolian', 'Uyghur', 'Kazakh', 'Miao', 'Bouyei', 'Dong', 'Hui', 'Manchu']
Execution time: 2.1 seconds
---
TOTAL llama3.2:latest MODEL EXECUTION TIME: 7.5 seconds
---
Testing: phi3:latest ...
Using prompt of: Tell me about Canada.
name='Canada' capital='Ottawa' languages=['English', 'French']
Execution time: 2.9 seconds
---
---
Testing: phi3:latest ...
Using prompt of: Tell me about Australia.
name='Australia' capital='Canberra' languages=['English', 'Aboriginal languages (over 250)']
Execution time: 1.0 seconds
---
---
Testing: phi3:latest ...
Using prompt of: Tell me about China.
name='China' capital='Beijing' languages=['Mandarin Chinese (Putonghua)', 'Cantonese', 'Min Nan or Hokkien']
Execution time: 1.1 seconds
---
TOTAL phi3:latest MODEL EXECUTION TIME: 5.0 seconds
---
Testing: gemma3:latest ...
Using prompt of: Tell me about Canada.
name='Canada' capital='Ottawa' languages=['English', 'French (official in Quebec)']
Execution time: 4.6 seconds
---
---
Testing: gemma3:latest ...
Using prompt of: Tell me about Australia.
name='Australia' capital='Canberra' languages=['English', 'Australian Indigenous languages (various)']
Execution time: 2.2 seconds
---
---
Testing: gemma3:latest ...
Using prompt of: Tell me about China.
name='China' capital='Beijing' languages=['Mandarin Chinese (official)', 'Cantonese, Wu, Min, Xiang, Gan, Hakka, Gan, Yi, Tibetan, Mongolian, Uyghur, etc.']
Execution time: 2.9 seconds
---
TOTAL gemma3:latest MODEL EXECUTION TIME: 9.7 seconds
---
Testing: deepseek-r1:latest ...
Using prompt of: Tell me about Canada.
name='Canada' capital='Ottawa' languages=['English', 'French']
Execution time: 4.8 seconds
---
---
Testing: deepseek-r1:latest ...
Using prompt of: Tell me about Australia.
name='Australia' capital='Canberra' languages=['English', 'Aboriginal languages', 'Australian English dialects like Australian English and Kiwi English, but note: New Zealand English is from New Zealand, not Australia.']
Execution time: 2.8 seconds
---
---
Testing: deepseek-r1:latest ...
Using prompt of: Tell me about China.
name='China' capital='Beijing' languages=['Mandarin (Putonghua)', 'Cantonese', 'Wu (Shanghainese)', 'Jinjiang Min, etc.']
Execution time: 2.6 seconds
---
TOTAL deepseek-r1:latest MODEL EXECUTION TIME: 10.2 seconds
---
Testing: gemma3n:latest ...
Using prompt of: Tell me about Canada.
name='Canada' capital='Ottawa' languages=['English', 'French']
Execution time: 7.9 seconds
---
---
Testing: gemma3n:latest ...
Using prompt of: Tell me about Australia.
name='Australia' capital='Canberra' languages=['English']
Execution time: 2.0 seconds
---
---
Testing: gemma3n:latest ...
Using prompt of: Tell me about China.
name='China' capital='Beijing' languages=['Mandarin Chinese', 'Wu', 'Yue', 'Min', 'Xiang', 'Gan', 'Hakka', 'Monghu', 'Korean', 'Japanese', 'English (increasingly)']
Execution time: 4.2 seconds
---
TOTAL gemma3n:latest MODEL EXECUTION TIME: 14.1 seconds
---
Testing: qwen3:latest ...
Using prompt of: Tell me about Canada.
name='Canada' capital='Ottawa' languages=['English', 'French', 'Official Languages Act of 1969 (1982) enshrined both English and French as official languages in the Constitution Act, 1982, ensuring their use in federal government and services.']
Execution time: 6.0 seconds
---
---
Testing: qwen3:latest ...
Using prompt of: Tell me about Australia.
name='Australia' capital='Canberra' languages=['English', 'Aboriginal languages', 'other languages such as Chinese, Japanese, Korean, Vietnamese, Greek, Italian, and Spanish, etc.']
Execution time: 2.5 seconds
---
---
Testing: qwen3:latest ...
Using prompt of: Tell me about China.
name='China' capital='Beijing' languages=['Mandarin', 'Cantonese', 'other regional languages such as Shanghainese, Hakka, etc.']
Execution time: 2.4 seconds
---
TOTAL qwen3:latest MODEL EXECUTION TIME: 10.9 seconds
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment