Skip to content

Instantly share code, notes, and snippets.

@bpmooch

bpmooch/models.md

Last active April 30, 2023 22:52
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save bpmooch/f25cb04d26cf6f6561a612a0c2d5228f to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save bpmooch/f25cb04d26cf6f6561a612a0c2d5228f to your computer and use it in GitHub Desktop.
Download ZIP
Detecting Bugs in C++ Code
Raw
models.md

Detecting Bugs in C++ Code

These are interesting but ultimately throwaway numbers. However, I think benchmarks like these suggest that the value provided by giving your data to openai can outweigh the costs. Prompt engineering provides an avenue to prevent leaking sensitive data into the system while still creating value with openai projects

llm model false alarms detections
LLaMa 65B (4-bit GPTQ) 1 / 15 0 / 13
Baize 30B (8-bit) 0 / 15 1 / 13
Galpaca 30B (8-bit) 0 / 15 1 / 13
Koala 13B (8-bit) 0 / 15 0 / 13
Vicuna 13B (8-bit) 2 / 15 1 / 13
Vicuna 7B (FP16) 1 / 15 0 / 13
GPT 3.5 0 / 15 7 / 13
GPT 4 0 / 15 13 / 13

from https://github.com/catid/supercharger/tree/main/airate

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment