Skip to content

Instantly share code, notes, and snippets.

@thecatfix
Last active February 4, 2026 15:50
Show Gist options
  • Select an option

  • Save thecatfix/fbabe1f4ec6e36b43209c0b042a41775 to your computer and use it in GitHub Desktop.

Select an option

Save thecatfix/fbabe1f4ec6e36b43209c0b042a41775 to your computer and use it in GitHub Desktop.
V2 Model Improvements

V2 Model Improvements

V2 models fix critical bugs in the original training code:

Issue V1 (Broken) V2 (Fixed)
Double-softmax Loss stuck at 0.3133, no learning Loss decreases, models learn
Regularization None (instant overfit) BatchNorm + Dropout(0.3)
'both' spread Used only log features (90) Uses both features (180)
Probability calibration 0.99-1.0 (overconfident) 0.50-0.90 (realistic)

Key change: V2 probabilities are properly calibrated. A 0.75 probability now means something real, unlike V1's meaningless 0.99.


Probability Distribution (V2 vs V1)

V1 Models (BROKEN):
  Probabilities: 0.99-1.0 for everything
  Interpretation: Model not learning, outputting constant predictions

V2 Models (FIXED):
  both_100ep_v2: 0.50-0.87, mean=0.67
  both_50ep_v2:  0.50-0.90, mean=0.66
  log_75ep_v2:   0.50-0.92, mean=0.65
  log_50ep_v2:   0.50-0.88, mean=0.63
  coint_25ep_v2: 0.50-0.90, mean=0.62
  coint_10ep_v2: 0.50-0.81, mean=0.59

V2 probabilities are properly calibrated - use p>0.70 threshold (not p>0.85).


Model Performance Summary

Model Cointegrated Pairs Prob Range Best For
both_100ep_v2 9 0.71-0.82 Most signals, balanced
both_50ep_v2 7 0.72-0.83 Highest individual probs
log_50ep_v2 6 0.71-0.78 Traditional pairs
log_75ep_v2 4 0.71-0.79 Conservative
coint_25ep_v2 3 0.70-0.79 Extreme z-scores
coint_10ep_v2 3 0.71-0.74 Quick validation

Recommendation: Use both_100ep_v2 as primary model - most cointegrated pairs detected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment