Created
December 29, 2021 07:40
-
-
Save onnyyonn/1d2f0f90f5e26e5ac93c9fd92003e999 to your computer and use it in GitHub Desktop.
Quantile Bias Correction with Empirical CDF
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import numpy as np | |
| from statsmodels.distributions.empirical_distribution import ECDF | |
| def bias_correct(obs_data_train, model_data_train, model_data_test): | |
| """ | |
| Function for bias correcting model_data_test, based on obs_data_train and madel_data_train CDF. | |
| All three should be numpy array. Can be multi-dimensional. | |
| obs_data_train = observation / ground truth | |
| model_data_train = biased model ouput. Its CDF should match observation CDF when bias corrected | |
| model_data_test = biased model output for a different period / experiment. | |
| Its CDF has a shift from model_data_train. When bias corrected, it should have same shift from observation CDF | |
| """ | |
| cdf = ECDF(model_data_train.ravel()) | |
| q = cdf(model_data_test.ravel()) | |
| model_data_test_corrected = model_data_test.ravel() + np.quantile(obs_data_train, q) - np.quantile(model_data_train, q) | |
| return np.reshape(model_data_test_corrected, np.shape(model_data_test)) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment