Skip to content

Instantly share code, notes, and snippets.

@kn1kn1

kn1kn1/synthetic.ipynb

Last active June 11, 2025 11:49
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save kn1kn1/e3a8092a25290d283eb3e176d9cf7252 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save kn1kn1/e3a8092a25290d283eb3e176d9cf7252 to your computer and use it in GitHub Desktop.
Download ZIP
Display the source blob
Display the rendered blob
Raw
synthetic.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/gist/kn1kn1/e3a8092a25290d283eb3e176d9cf7252/synthetic.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "PGAJmAMZoGgL"
},
"source": [
"# Quickstart Example with Synthetic Bandit Data\n",
"---\n",
"This notebook provides an example of conducting OPE of several different evaluation policies with synthetic logged bandit data.\n",
"\n",
"The example consists of the following four major steps:\n",
"- (1) Generating Synthetic Data\n",
"- (2) Off-Policy Learning\n",
"- (3) Off-Policy Evaluation\n",
"- (4) Evaluation of OPE Estimators\n",
"\n",
"Please see [../examples/synthetic](../synthetic) for a more sophisticated example of the evaluation of OPE with synthetic bandit data."
]
},
{
"cell_type": "code",
"source": [
"!python3 --version"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "KsEC_wltsO22",
"outputId": "e46ebeba-e518-4019-cd52-fef61258bfe3"
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Python 3.11.13\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"Run the first cell, reload(CTRL + R), then run the second cell\n",
"\n",
"cf. https://colab.research.google.com/drive/12Uy1fTJy4XQDMdXVM_5XJ0iGEdCDOgYf"
],
"metadata": {
"id": "D1AcSaRwyLKC"
}
},
{
"cell_type": "code",
"source": [
"!wget https://github.com/korakot/kora/releases/download/v0.10/py310.sh\n",
"!bash ./py310.sh -b -f -p /usr/local\n",
"!python -m ipykernel install --name \"py310\" --user"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "aa-AUzfEqk4N",
"outputId": "7ff8d582-fb20-4d35-d50d-9e893df94adf"
},
"execution_count": null,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"--2025-06-11 11:40:46-- https://github.com/korakot/kora/releases/download/v0.10/py310.sh\n",
"Resolving github.com (github.com)... 140.82.114.3\n",
"Connecting to github.com (github.com)|140.82.114.3|:443... connected.\n",
"HTTP request sent, awaiting response... 302 Found\n",
"Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/266951884/0d0623be-3dec-4820-9e7b-69a3a5a75ef7?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20250611%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20250611T114046Z&X-Amz-Expires=300&X-Amz-Signature=9436aee48cb0b848eb498b04cd7423463e197b73f97191bc7c9175eb210167d9&X-Amz-SignedHeaders=host&response-content-disposition=attachment%3B%20filename%3Dpy310.sh&response-content-type=application%2Foctet-stream [following]\n",
"--2025-06-11 11:40:46-- https://objects.githubusercontent.com/github-production-release-asset-2e65be/266951884/0d0623be-3dec-4820-9e7b-69a3a5a75ef7?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20250611%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20250611T114046Z&X-Amz-Expires=300&X-Amz-Signature=9436aee48cb0b848eb498b04cd7423463e197b73f97191bc7c9175eb210167d9&X-Amz-SignedHeaders=host&response-content-disposition=attachment%3B%20filename%3Dpy310.sh&response-content-type=application%2Foctet-stream\n",
"Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.108.133, 185.199.111.133, 185.199.110.133, ...\n",
"Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.108.133|:443... connected.\n",
"HTTP request sent, awaiting response... 200 OK\n",
"Length: 364632383 (348M) [application/octet-stream]\n",
"Saving to: ‘py310.sh’\n",
"\n",
"py310.sh 100%[===================>] 347.74M 87.3MB/s in 4.0s \n",
"\n",
"2025-06-11 11:40:51 (87.2 MB/s) - ‘py310.sh’ saved [364632383/364632383]\n",
"\n",
"PREFIX=/usr/local\n",
"Unpacking payload ...\n",
"Collecting package metadata (current_repodata.json): - \b\b\\ \b\b| \b\bdone\n",
"Solving environment: - \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\bdone\n",
"\n",
"## Package Plan ##\n",
"\n",
" environment location: /usr/local\n",
"\n",
" added / updated specs:\n",
" - _libgcc_mutex==0.1=conda_forge\n",
" - _openmp_mutex==4.5=2_gnu\n",
" - aiohttp==3.8.3=py310h5764c6d_1\n",
" - aiosignal==1.2.0=pyhd8ed1ab_0\n",
" - alsa-lib==1.2.7.2=h166bdaf_0\n",
" - argon2-cffi-bindings==21.2.0=py310h5764c6d_3\n",
" - argon2-cffi==21.3.0=pyhd8ed1ab_0\n",
" - asttokens==2.0.8=pyhd8ed1ab_0\n",
" - async-timeout==4.0.2=pyhd8ed1ab_0\n",
" - attr==2.5.1=h166bdaf_1\n",
" - attrs==22.1.0=pyh71513ae_1\n",
" - backcall==0.2.0=pyh9f0ad1d_0\n",
" - backports.functools_lru_cache==1.6.4=pyhd8ed1ab_0\n",
" - backports==1.0=py_2\n",
" - beautifulsoup4==4.11.1=pyha770c72_0\n",
" - bleach==5.0.1=pyhd8ed1ab_0\n",
" - brotlipy==0.7.0=py310h5764c6d_1005\n",
" - bzip2==1.0.8=h7f98852_4\n",
" - ca-certificates==2022.9.24=ha878542_0\n",
" - cachetools==5.2.0=pyhd8ed1ab_0\n",
" - certifi==2022.9.24=pyhd8ed1ab_0\n",
" - cffi==1.15.1=py310h255011f_2\n",
" - charset-normalizer==2.1.1=pyhd8ed1ab_0\n",
" - colorama==0.4.6=pyhd8ed1ab_0\n",
" - conda-package-handling==1.9.0=py310h5764c6d_1\n",
" - conda==22.9.0=py310hff52083_1\n",
" - cryptography==38.0.2=py310h597c629_2\n",
" - dbus==1.13.6=h5008d03_3\n",
" - debugpy==1.6.3=py310hd8f1fbe_1\n",
" - decorator==5.1.1=pyhd8ed1ab_0\n",
" - defusedxml==0.7.1=pyhd8ed1ab_0\n",
" - entrypoints==0.4=pyhd8ed1ab_0\n",
" - executing==1.1.1=pyhd8ed1ab_0\n",
" - expat==2.5.0=h27087fc_0\n",
" - fftw==3.3.10=nompi_hf0379b8_105\n",
" - flit-core==3.7.1=pyhd8ed1ab_0\n",
" - font-ttf-dejavu-sans-mono==2.37=hab24e00_0\n",
" - font-ttf-inconsolata==3.000=h77eed37_0\n",
" - font-ttf-source-code-pro==2.038=h77eed37_0\n",
" - font-ttf-ubuntu==0.83=hab24e00_0\n",
" - fontconfig==2.14.1=hc2a2eb6_0\n",
" - fonts-conda-ecosystem==1=0\n",
" - fonts-conda-forge==1=0\n",
" - freetype==2.12.1=hca18f0e_0\n",
" - frozenlist==1.3.1=py310h5764c6d_1\n",
" - gettext==0.21.1=h27087fc_0\n",
" - glib-tools==2.74.1=h6239696_0\n",
" - glib==2.74.1=h6239696_0\n",
" - google-auth==2.13.0=pyh1a96a4e_0\n",
" - google-colab==1.0.0=pyh44b312d_0\n",
" - gst-plugins-base==1.20.3=h57caac4_2\n",
" - gstreamer==1.20.3=hd4edc92_2\n",
" - icu==70.1=h27087fc_0\n",
" - idna==3.4=pyhd8ed1ab_0\n",
" - importlib-metadata==5.0.0=pyha770c72_1\n",
" - importlib_resources==5.10.0=pyhd8ed1ab_0\n",
" - ipykernel==6.16.2=pyh210e3f2_0\n",
" - ipython==8.5.0=pyh41d4057_1\n",
" - ipython_genutils==0.2.0=py_1\n",
" - ipywidgets==8.0.2=pyhd8ed1ab_1\n",
" - jack==1.9.21=h2a1e645_0\n",
" - jedi==0.18.1=pyhd8ed1ab_2\n",
" - jinja2==3.1.2=pyhd8ed1ab_1\n",
" - jpeg==9e=h166bdaf_2\n",
" - jsonschema==4.16.0=pyhd8ed1ab_0\n",
" - jupyter==1.0.0=py310hff52083_7\n",
" - jupyter_client==7.4.4=pyhd8ed1ab_0\n",
" - jupyter_console==6.4.4=pyhd8ed1ab_0\n",
" - jupyter_core==4.11.1=py310hff52083_1\n",
" - jupyterlab_pygments==0.2.2=pyhd8ed1ab_0\n",
" - jupyterlab_widgets==3.0.3=pyhd8ed1ab_0\n",
" - keyutils==1.6.1=h166bdaf_0\n",
" - krb5==1.19.3=h3790be6_0\n",
" - lame==3.100=h166bdaf_1003\n",
" - ld_impl_linux-64==2.39=hc81fddc_0\n",
" - libblas==3.9.0=16_linux64_openblas\n",
" - libcap==2.66=ha37c62d_0\n",
" - libcblas==3.9.0=16_linux64_openblas\n",
" - libclang13==14.0.6=default_h3a83d3e_0\n",
" - libclang==14.0.6=default_h2e3cab8_0\n",
" - libcups==2.3.3=h3e49a29_2\n",
" - libdb==6.2.32=h9c3ff4c_0\n",
" - libedit==3.1.20191231=he28a2e2_2\n",
" - libevent==2.1.10=h9b69904_4\n",
" - libffi==3.4.2=h7f98852_5\n",
" - libflac==1.4.2=h27087fc_0\n",
" - libgcc-ng==12.2.0=h65d4601_19\n",
" - libgfortran-ng==12.2.0=h69a702a_19\n",
" - libgfortran5==12.2.0=h337968e_19\n",
" - libglib==2.74.1=h7a41b64_0\n",
" - libgomp==12.2.0=h65d4601_19\n",
" - libiconv==1.17=h166bdaf_0\n",
" - liblapack==3.9.0=16_linux64_openblas\n",
" - libllvm14==14.0.6=he0ac6c6_0\n",
" - libnsl==2.0.0=h7f98852_0\n",
" - libogg==1.3.4=h7f98852_1\n",
" - libopenblas==0.3.21=pthreads_h78a6416_3\n",
" - libopus==1.3.1=h7f98852_1\n",
" - libpng==1.6.38=h753d276_0\n",
" - libpq==14.5=hd77ab85_1\n",
" - libsndfile==1.1.0=h27087fc_0\n",
" - libsodium==1.0.18=h36c2ea0_1\n",
" - libsqlite==3.39.4=h753d276_0\n",
" - libstdcxx-ng==12.2.0=h46fd767_19\n",
" - libtool==2.4.6=h9c3ff4c_1008\n",
" - libudev1==251=h166bdaf_0\n",
" - libuuid==2.32.1=h7f98852_1000\n",
" - libvorbis==1.3.7=h9c3ff4c_0\n",
" - libxcb==1.13=h7f98852_1004\n",
" - libxkbcommon==1.0.3=he3ba5ed_0\n",
" - libxml2==2.10.3=h7463322_0\n",
" - libzlib==1.2.13=h166bdaf_4\n",
" - markupsafe==2.1.1=py310h5764c6d_2\n",
" - matplotlib-inline==0.1.6=pyhd8ed1ab_0\n",
" - mistune==2.0.4=pyhd8ed1ab_0\n",
" - mpg123==1.30.2=h27087fc_1\n",
" - multidict==6.0.2=py310h5764c6d_2\n",
" - mysql-common==8.0.31=haf5c9bc_0\n",
" - mysql-libs==8.0.31=h28c427c_0\n",
" - nbclient==0.7.0=pyhd8ed1ab_0\n",
" - nbconvert-core==7.2.3=pyhd8ed1ab_0\n",
" - nbconvert-pandoc==7.2.3=pyhd8ed1ab_0\n",
" - nbconvert==7.2.3=pyhd8ed1ab_0\n",
" - nbformat==5.7.0=pyhd8ed1ab_0\n",
" - ncurses==6.3=h27087fc_1\n",
" - nest-asyncio==1.5.6=pyhd8ed1ab_0\n",
" - notebook==6.4.12=pyha770c72_0\n",
" - nspr==4.32=h9c3ff4c_1\n",
" - nss==3.78=h2350873_0\n",
" - numpy==1.23.4=py310h53a5b5f_1\n",
" - openssl==1.1.1q=h166bdaf_1\n",
" - packaging==21.3=pyhd8ed1ab_0\n",
" - pandas==1.5.1=py310h769672d_1\n",
" - pandoc==2.19.2=h32600fe_1\n",
" - pandocfilters==1.5.0=pyhd8ed1ab_0\n",
" - parso==0.8.3=pyhd8ed1ab_0\n",
" - pcre2==10.37=hc3806b6_1\n",
" - pexpect==4.8.0=pyh9f0ad1d_2\n",
" - pickleshare==0.7.5=py_1003\n",
" - pip==22.3=pyhd8ed1ab_0\n",
" - pkgutil-resolve-name==1.3.10=pyhd8ed1ab_0\n",
" - ply==3.11=py_1\n",
" - portpicker==1.5.2=pyhd8ed1ab_0\n",
" - prometheus_client==0.15.0=pyhd8ed1ab_0\n",
" - prompt-toolkit==3.0.31=pyha770c72_0\n",
" - prompt_toolkit==3.0.31=hd8ed1ab_0\n",
" - psutil==5.9.3=py310h5764c6d_1\n",
" - pthread-stubs==0.4=h36c2ea0_1001\n",
" - ptyprocess==0.7.0=pyhd3deb0d_0\n",
" - pulseaudio==14.0=habe0971_10\n",
" - pure_eval==0.2.2=pyhd8ed1ab_0\n",
" - pyasn1-modules==0.2.7=py_0\n",
" - pyasn1==0.4.8=py_0\n",
" - pycosat==0.6.4=py310h5764c6d_1\n",
" - pycparser==2.21=pyhd8ed1ab_0\n",
" - pygments==2.13.0=pyhd8ed1ab_0\n",
" - pyopenssl==22.1.0=pyhd8ed1ab_0\n",
" - pyparsing==3.0.9=pyhd8ed1ab_0\n",
" - pyqt5-sip==12.11.0=py310hd8f1fbe_2\n",
" - pyqt==5.15.7=py310h29803b5_2\n",
" - pyrsistent==0.18.1=py310h5764c6d_2\n",
" - pysocks==1.7.1=pyha2e5f31_6\n",
" - python-dateutil==2.8.2=pyhd8ed1ab_0\n",
" - python-fastjsonschema==2.16.2=pyhd8ed1ab_0\n",
" - python==3.10.6=h582c2e5_0_cpython\n",
" - python_abi==3.10=2_cp310\n",
" - pytz==2022.5=pyhd8ed1ab_0\n",
" - pyu2f==0.1.5=pyhd8ed1ab_0\n",
" - pyzmq==24.0.1=py310h330234f_1\n",
" - qt-main==5.15.6=hc525480_0\n",
" - qtconsole-base==5.3.2=pyha770c72_0\n",
" - qtconsole==5.3.2=pyhd8ed1ab_0\n",
" - qtpy==2.2.1=pyhd8ed1ab_0\n",
" - readline==8.1.2=h0f457ee_0\n",
" - requests==2.28.1=pyhd8ed1ab_1\n",
" - rsa==4.9=pyhd8ed1ab_0\n",
" - ruamel_yaml==0.15.80=py310h5764c6d_1008\n",
" - send2trash==1.8.0=pyhd8ed1ab_0\n",
" - setuptools==65.5.0=pyhd8ed1ab_0\n",
" - sip==6.7.2=py310hd8f1fbe_1\n",
" - six==1.16.0=pyh6c4a22f_0\n",
" - soupsieve==2.3.2.post1=pyhd8ed1ab_0\n",
" - sqlite==3.39.4=h4ff8645_0\n",
" - stack_data==0.5.1=pyhd8ed1ab_0\n",
" - terminado==0.17.0=pyh41d4057_0\n",
" - tinycss2==1.2.1=pyhd8ed1ab_0\n",
" - tk==8.6.12=h27826a3_0\n",
" - toml==0.10.2=pyhd8ed1ab_0\n",
" - toolz==0.12.0=pyhd8ed1ab_0\n",
" - tornado==6.2=py310h5764c6d_1\n",
" - tqdm==4.64.1=pyhd8ed1ab_0\n",
" - traitlets==5.5.0=pyhd8ed1ab_0\n",
" - typing-extensions==4.4.0=hd8ed1ab_0\n",
" - typing_extensions==4.4.0=pyha770c72_0\n",
" - tzdata==2022e=h191b570_0\n",
" - urllib3==1.26.11=pyhd8ed1ab_0\n",
" - wcwidth==0.2.5=pyh9f0ad1d_2\n",
" - webencodings==0.5.1=py_1\n",
" - wheel==0.37.1=pyhd8ed1ab_0\n",
" - widgetsnbextension==4.0.3=pyhd8ed1ab_0\n",
" - xcb-util-image==0.4.0=h166bdaf_0\n",
" - xcb-util-keysyms==0.4.0=h166bdaf_0\n",
" - xcb-util-renderutil==0.3.9=h166bdaf_0\n",
" - xcb-util-wm==0.4.1=h166bdaf_0\n",
" - xcb-util==0.4.0=h166bdaf_0\n",
" - xorg-libxau==1.0.9=h7f98852_0\n",
" - xorg-libxdmcp==1.1.3=h7f98852_0\n",
" - xz==5.2.6=h166bdaf_0\n",
" - yaml==0.2.5=h7f98852_2\n",
" - yarl==1.8.1=py310h5764c6d_0\n",
" - zeromq==4.3.4=h9c3ff4c_1\n",
" - zipp==3.10.0=pyhd8ed1ab_0\n",
" - zstd==1.5.2=h6239696_4\n",
"\n",
"\n",
"The following NEW packages will be INSTALLED:\n",
"\n",
" _libgcc_mutex conda-forge/linux-64::_libgcc_mutex-0.1-conda_forge\n",
" _openmp_mutex conda-forge/linux-64::_openmp_mutex-4.5-2_gnu\n",
" aiohttp conda-forge/linux-64::aiohttp-3.8.3-py310h5764c6d_1\n",
" aiosignal conda-forge/noarch::aiosignal-1.2.0-pyhd8ed1ab_0\n",
" alsa-lib conda-forge/linux-64::alsa-lib-1.2.7.2-h166bdaf_0\n",
" argon2-cffi conda-forge/noarch::argon2-cffi-21.3.0-pyhd8ed1ab_0\n",
" argon2-cffi-bindi~ conda-forge/linux-64::argon2-cffi-bindings-21.2.0-py310h5764c6d_3\n",
" asttokens conda-forge/noarch::asttokens-2.0.8-pyhd8ed1ab_0\n",
" async-timeout conda-forge/noarch::async-timeout-4.0.2-pyhd8ed1ab_0\n",
" attr conda-forge/linux-64::attr-2.5.1-h166bdaf_1\n",
" attrs conda-forge/noarch::attrs-22.1.0-pyh71513ae_1\n",
" backcall conda-forge/noarch::backcall-0.2.0-pyh9f0ad1d_0\n",
" backports conda-forge/noarch::backports-1.0-py_2\n",
" backports.functoo~ conda-forge/noarch::backports.functools_lru_cache-1.6.4-pyhd8ed1ab_0\n",
" beautifulsoup4 conda-forge/noarch::beautifulsoup4-4.11.1-pyha770c72_0\n",
" bleach conda-forge/noarch::bleach-5.0.1-pyhd8ed1ab_0\n",
" brotlipy conda-forge/linux-64::brotlipy-0.7.0-py310h5764c6d_1005\n",
" bzip2 conda-forge/linux-64::bzip2-1.0.8-h7f98852_4\n",
" ca-certificates conda-forge/linux-64::ca-certificates-2022.9.24-ha878542_0\n",
" cachetools conda-forge/noarch::cachetools-5.2.0-pyhd8ed1ab_0\n",
" certifi conda-forge/noarch::certifi-2022.9.24-pyhd8ed1ab_0\n",
" cffi conda-forge/linux-64::cffi-1.15.1-py310h255011f_2\n",
" charset-normalizer conda-forge/noarch::charset-normalizer-2.1.1-pyhd8ed1ab_0\n",
" colorama conda-forge/noarch::colorama-0.4.6-pyhd8ed1ab_0\n",
" conda conda-forge/linux-64::conda-22.9.0-py310hff52083_1\n",
" conda-package-han~ conda-forge/linux-64::conda-package-handling-1.9.0-py310h5764c6d_1\n",
" cryptography conda-forge/linux-64::cryptography-38.0.2-py310h597c629_2\n",
" dbus conda-forge/linux-64::dbus-1.13.6-h5008d03_3\n",
" debugpy conda-forge/linux-64::debugpy-1.6.3-py310hd8f1fbe_1\n",
" decorator conda-forge/noarch::decorator-5.1.1-pyhd8ed1ab_0\n",
" defusedxml conda-forge/noarch::defusedxml-0.7.1-pyhd8ed1ab_0\n",
" entrypoints conda-forge/noarch::entrypoints-0.4-pyhd8ed1ab_0\n",
" executing conda-forge/noarch::executing-1.1.1-pyhd8ed1ab_0\n",
" expat conda-forge/linux-64::expat-2.5.0-h27087fc_0\n",
" fftw conda-forge/linux-64::fftw-3.3.10-nompi_hf0379b8_105\n",
" flit-core conda-forge/noarch::flit-core-3.7.1-pyhd8ed1ab_0\n",
" font-ttf-dejavu-s~ conda-forge/noarch::font-ttf-dejavu-sans-mono-2.37-hab24e00_0\n",
" font-ttf-inconsol~ conda-forge/noarch::font-ttf-inconsolata-3.000-h77eed37_0\n",
" font-ttf-source-c~ conda-forge/noarch::font-ttf-source-code-pro-2.038-h77eed37_0\n",
" font-ttf-ubuntu conda-forge/noarch::font-ttf-ubuntu-0.83-hab24e00_0\n",
" fontconfig conda-forge/linux-64::fontconfig-2.14.1-hc2a2eb6_0\n",
" fonts-conda-ecosy~ conda-forge/noarch::fonts-conda-ecosystem-1-0\n",
" fonts-conda-forge conda-forge/noarch::fonts-conda-forge-1-0\n",
" freetype conda-forge/linux-64::freetype-2.12.1-hca18f0e_0\n",
" frozenlist conda-forge/linux-64::frozenlist-1.3.1-py310h5764c6d_1\n",
" gettext conda-forge/linux-64::gettext-0.21.1-h27087fc_0\n",
" glib conda-forge/linux-64::glib-2.74.1-h6239696_0\n",
" glib-tools conda-forge/linux-64::glib-tools-2.74.1-h6239696_0\n",
" google-auth conda-forge/noarch::google-auth-2.13.0-pyh1a96a4e_0\n",
" google-colab conda-forge/noarch::google-colab-1.0.0-pyh44b312d_0\n",
" gst-plugins-base conda-forge/linux-64::gst-plugins-base-1.20.3-h57caac4_2\n",
" gstreamer conda-forge/linux-64::gstreamer-1.20.3-hd4edc92_2\n",
" icu conda-forge/linux-64::icu-70.1-h27087fc_0\n",
" idna conda-forge/noarch::idna-3.4-pyhd8ed1ab_0\n",
" importlib-metadata conda-forge/noarch::importlib-metadata-5.0.0-pyha770c72_1\n",
" importlib_resourc~ conda-forge/noarch::importlib_resources-5.10.0-pyhd8ed1ab_0\n",
" ipykernel conda-forge/noarch::ipykernel-6.16.2-pyh210e3f2_0\n",
" ipython conda-forge/noarch::ipython-8.5.0-pyh41d4057_1\n",
" ipython_genutils conda-forge/noarch::ipython_genutils-0.2.0-py_1\n",
" ipywidgets conda-forge/noarch::ipywidgets-8.0.2-pyhd8ed1ab_1\n",
" jack conda-forge/linux-64::jack-1.9.21-h2a1e645_0\n",
" jedi conda-forge/noarch::jedi-0.18.1-pyhd8ed1ab_2\n",
" jinja2 conda-forge/noarch::jinja2-3.1.2-pyhd8ed1ab_1\n",
" jpeg conda-forge/linux-64::jpeg-9e-h166bdaf_2\n",
" jsonschema conda-forge/noarch::jsonschema-4.16.0-pyhd8ed1ab_0\n",
" jupyter conda-forge/linux-64::jupyter-1.0.0-py310hff52083_7\n",
" jupyter_client conda-forge/noarch::jupyter_client-7.4.4-pyhd8ed1ab_0\n",
" jupyter_console conda-forge/noarch::jupyter_console-6.4.4-pyhd8ed1ab_0\n",
" jupyter_core conda-forge/linux-64::jupyter_core-4.11.1-py310hff52083_1\n",
" jupyterlab_pygmen~ conda-forge/noarch::jupyterlab_pygments-0.2.2-pyhd8ed1ab_0\n",
" jupyterlab_widgets conda-forge/noarch::jupyterlab_widgets-3.0.3-pyhd8ed1ab_0\n",
" keyutils conda-forge/linux-64::keyutils-1.6.1-h166bdaf_0\n",
" krb5 conda-forge/linux-64::krb5-1.19.3-h3790be6_0\n",
" lame conda-forge/linux-64::lame-3.100-h166bdaf_1003\n",
" ld_impl_linux-64 conda-forge/linux-64::ld_impl_linux-64-2.39-hc81fddc_0\n",
" libblas conda-forge/linux-64::libblas-3.9.0-16_linux64_openblas\n",
" libcap conda-forge/linux-64::libcap-2.66-ha37c62d_0\n",
" libcblas conda-forge/linux-64::libcblas-3.9.0-16_linux64_openblas\n",
" libclang conda-forge/linux-64::libclang-14.0.6-default_h2e3cab8_0\n",
" libclang13 conda-forge/linux-64::libclang13-14.0.6-default_h3a83d3e_0\n",
" libcups conda-forge/linux-64::libcups-2.3.3-h3e49a29_2\n",
" libdb conda-forge/linux-64::libdb-6.2.32-h9c3ff4c_0\n",
" libedit conda-forge/linux-64::libedit-3.1.20191231-he28a2e2_2\n",
" libevent conda-forge/linux-64::libevent-2.1.10-h9b69904_4\n",
" libffi conda-forge/linux-64::libffi-3.4.2-h7f98852_5\n",
" libflac conda-forge/linux-64::libflac-1.4.2-h27087fc_0\n",
" libgcc-ng conda-forge/linux-64::libgcc-ng-12.2.0-h65d4601_19\n",
" libgfortran-ng conda-forge/linux-64::libgfortran-ng-12.2.0-h69a702a_19\n",
" libgfortran5 conda-forge/linux-64::libgfortran5-12.2.0-h337968e_19\n",
" libglib conda-forge/linux-64::libglib-2.74.1-h7a41b64_0\n",
" libgomp conda-forge/linux-64::libgomp-12.2.0-h65d4601_19\n",
" libiconv conda-forge/linux-64::libiconv-1.17-h166bdaf_0\n",
" liblapack conda-forge/linux-64::liblapack-3.9.0-16_linux64_openblas\n",
" libllvm14 conda-forge/linux-64::libllvm14-14.0.6-he0ac6c6_0\n",
" libnsl conda-forge/linux-64::libnsl-2.0.0-h7f98852_0\n",
" libogg conda-forge/linux-64::libogg-1.3.4-h7f98852_1\n",
" libopenblas conda-forge/linux-64::libopenblas-0.3.21-pthreads_h78a6416_3\n",
" libopus conda-forge/linux-64::libopus-1.3.1-h7f98852_1\n",
" libpng conda-forge/linux-64::libpng-1.6.38-h753d276_0\n",
" libpq conda-forge/linux-64::libpq-14.5-hd77ab85_1\n",
" libsndfile conda-forge/linux-64::libsndfile-1.1.0-h27087fc_0\n",
" libsodium conda-forge/linux-64::libsodium-1.0.18-h36c2ea0_1\n",
" libsqlite conda-forge/linux-64::libsqlite-3.39.4-h753d276_0\n",
" libstdcxx-ng conda-forge/linux-64::libstdcxx-ng-12.2.0-h46fd767_19\n",
" libtool conda-forge/linux-64::libtool-2.4.6-h9c3ff4c_1008\n",
" libudev1 conda-forge/linux-64::libudev1-251-h166bdaf_0\n",
" libuuid conda-forge/linux-64::libuuid-2.32.1-h7f98852_1000\n",
" libvorbis conda-forge/linux-64::libvorbis-1.3.7-h9c3ff4c_0\n",
" libxcb conda-forge/linux-64::libxcb-1.13-h7f98852_1004\n",
" libxkbcommon conda-forge/linux-64::libxkbcommon-1.0.3-he3ba5ed_0\n",
" libxml2 conda-forge/linux-64::libxml2-2.10.3-h7463322_0\n",
" libzlib conda-forge/linux-64::libzlib-1.2.13-h166bdaf_4\n",
" markupsafe conda-forge/linux-64::markupsafe-2.1.1-py310h5764c6d_2\n",
" matplotlib-inline conda-forge/noarch::matplotlib-inline-0.1.6-pyhd8ed1ab_0\n",
" mistune conda-forge/noarch::mistune-2.0.4-pyhd8ed1ab_0\n",
" mpg123 conda-forge/linux-64::mpg123-1.30.2-h27087fc_1\n",
" multidict conda-forge/linux-64::multidict-6.0.2-py310h5764c6d_2\n",
" mysql-common conda-forge/linux-64::mysql-common-8.0.31-haf5c9bc_0\n",
" mysql-libs conda-forge/linux-64::mysql-libs-8.0.31-h28c427c_0\n",
" nbclient conda-forge/noarch::nbclient-0.7.0-pyhd8ed1ab_0\n",
" nbconvert conda-forge/noarch::nbconvert-7.2.3-pyhd8ed1ab_0\n",
" nbconvert-core conda-forge/noarch::nbconvert-core-7.2.3-pyhd8ed1ab_0\n",
" nbconvert-pandoc conda-forge/noarch::nbconvert-pandoc-7.2.3-pyhd8ed1ab_0\n",
" nbformat conda-forge/noarch::nbformat-5.7.0-pyhd8ed1ab_0\n",
" ncurses conda-forge/linux-64::ncurses-6.3-h27087fc_1\n",
" nest-asyncio conda-forge/noarch::nest-asyncio-1.5.6-pyhd8ed1ab_0\n",
" notebook conda-forge/noarch::notebook-6.4.12-pyha770c72_0\n",
" nspr conda-forge/linux-64::nspr-4.32-h9c3ff4c_1\n",
" nss conda-forge/linux-64::nss-3.78-h2350873_0\n",
" numpy conda-forge/linux-64::numpy-1.23.4-py310h53a5b5f_1\n",
" openssl conda-forge/linux-64::openssl-1.1.1q-h166bdaf_1\n",
" packaging conda-forge/noarch::packaging-21.3-pyhd8ed1ab_0\n",
" pandas conda-forge/linux-64::pandas-1.5.1-py310h769672d_1\n",
" pandoc conda-forge/linux-64::pandoc-2.19.2-h32600fe_1\n",
" pandocfilters conda-forge/noarch::pandocfilters-1.5.0-pyhd8ed1ab_0\n",
" parso conda-forge/noarch::parso-0.8.3-pyhd8ed1ab_0\n",
" pcre2 conda-forge/linux-64::pcre2-10.37-hc3806b6_1\n",
" pexpect conda-forge/noarch::pexpect-4.8.0-pyh9f0ad1d_2\n",
" pickleshare conda-forge/noarch::pickleshare-0.7.5-py_1003\n",
" pip conda-forge/noarch::pip-22.3-pyhd8ed1ab_0\n",
" pkgutil-resolve-n~ conda-forge/noarch::pkgutil-resolve-name-1.3.10-pyhd8ed1ab_0\n",
" ply conda-forge/noarch::ply-3.11-py_1\n",
" portpicker conda-forge/noarch::portpicker-1.5.2-pyhd8ed1ab_0\n",
" prometheus_client conda-forge/noarch::prometheus_client-0.15.0-pyhd8ed1ab_0\n",
" prompt-toolkit conda-forge/noarch::prompt-toolkit-3.0.31-pyha770c72_0\n",
" prompt_toolkit conda-forge/noarch::prompt_toolkit-3.0.31-hd8ed1ab_0\n",
" psutil conda-forge/linux-64::psutil-5.9.3-py310h5764c6d_1\n",
" pthread-stubs conda-forge/linux-64::pthread-stubs-0.4-h36c2ea0_1001\n",
" ptyprocess conda-forge/noarch::ptyprocess-0.7.0-pyhd3deb0d_0\n",
" pulseaudio conda-forge/linux-64::pulseaudio-14.0-habe0971_10\n",
" pure_eval conda-forge/noarch::pure_eval-0.2.2-pyhd8ed1ab_0\n",
" pyasn1 conda-forge/noarch::pyasn1-0.4.8-py_0\n",
" pyasn1-modules conda-forge/noarch::pyasn1-modules-0.2.7-py_0\n",
" pycosat conda-forge/linux-64::pycosat-0.6.4-py310h5764c6d_1\n",
" pycparser conda-forge/noarch::pycparser-2.21-pyhd8ed1ab_0\n",
" pygments conda-forge/noarch::pygments-2.13.0-pyhd8ed1ab_0\n",
" pyopenssl conda-forge/noarch::pyopenssl-22.1.0-pyhd8ed1ab_0\n",
" pyparsing conda-forge/noarch::pyparsing-3.0.9-pyhd8ed1ab_0\n",
" pyqt conda-forge/linux-64::pyqt-5.15.7-py310h29803b5_2\n",
" pyqt5-sip conda-forge/linux-64::pyqt5-sip-12.11.0-py310hd8f1fbe_2\n",
" pyrsistent conda-forge/linux-64::pyrsistent-0.18.1-py310h5764c6d_2\n",
" pysocks conda-forge/noarch::pysocks-1.7.1-pyha2e5f31_6\n",
" python conda-forge/linux-64::python-3.10.6-h582c2e5_0_cpython\n",
" python-dateutil conda-forge/noarch::python-dateutil-2.8.2-pyhd8ed1ab_0\n",
" python-fastjsonsc~ conda-forge/noarch::python-fastjsonschema-2.16.2-pyhd8ed1ab_0\n",
" python_abi conda-forge/linux-64::python_abi-3.10-2_cp310\n",
" pytz conda-forge/noarch::pytz-2022.5-pyhd8ed1ab_0\n",
" pyu2f conda-forge/noarch::pyu2f-0.1.5-pyhd8ed1ab_0\n",
" pyzmq conda-forge/linux-64::pyzmq-24.0.1-py310h330234f_1\n",
" qt-main conda-forge/linux-64::qt-main-5.15.6-hc525480_0\n",
" qtconsole conda-forge/noarch::qtconsole-5.3.2-pyhd8ed1ab_0\n",
" qtconsole-base conda-forge/noarch::qtconsole-base-5.3.2-pyha770c72_0\n",
" qtpy conda-forge/noarch::qtpy-2.2.1-pyhd8ed1ab_0\n",
" readline conda-forge/linux-64::readline-8.1.2-h0f457ee_0\n",
" requests conda-forge/noarch::requests-2.28.1-pyhd8ed1ab_1\n",
" rsa conda-forge/noarch::rsa-4.9-pyhd8ed1ab_0\n",
" ruamel_yaml conda-forge/linux-64::ruamel_yaml-0.15.80-py310h5764c6d_1008\n",
" send2trash conda-forge/noarch::send2trash-1.8.0-pyhd8ed1ab_0\n",
" setuptools conda-forge/noarch::setuptools-65.5.0-pyhd8ed1ab_0\n",
" sip conda-forge/linux-64::sip-6.7.2-py310hd8f1fbe_1\n",
" six conda-forge/noarch::six-1.16.0-pyh6c4a22f_0\n",
" soupsieve conda-forge/noarch::soupsieve-2.3.2.post1-pyhd8ed1ab_0\n",
" sqlite conda-forge/linux-64::sqlite-3.39.4-h4ff8645_0\n",
" stack_data conda-forge/noarch::stack_data-0.5.1-pyhd8ed1ab_0\n",
" terminado conda-forge/noarch::terminado-0.17.0-pyh41d4057_0\n",
" tinycss2 conda-forge/noarch::tinycss2-1.2.1-pyhd8ed1ab_0\n",
" tk conda-forge/linux-64::tk-8.6.12-h27826a3_0\n",
" toml conda-forge/noarch::toml-0.10.2-pyhd8ed1ab_0\n",
" toolz conda-forge/noarch::toolz-0.12.0-pyhd8ed1ab_0\n",
" tornado conda-forge/linux-64::tornado-6.2-py310h5764c6d_1\n",
" tqdm conda-forge/noarch::tqdm-4.64.1-pyhd8ed1ab_0\n",
" traitlets conda-forge/noarch::traitlets-5.5.0-pyhd8ed1ab_0\n",
" typing-extensions conda-forge/noarch::typing-extensions-4.4.0-hd8ed1ab_0\n",
" typing_extensions conda-forge/noarch::typing_extensions-4.4.0-pyha770c72_0\n",
" tzdata conda-forge/noarch::tzdata-2022e-h191b570_0\n",
" urllib3 conda-forge/noarch::urllib3-1.26.11-pyhd8ed1ab_0\n",
" wcwidth conda-forge/noarch::wcwidth-0.2.5-pyh9f0ad1d_2\n",
" webencodings conda-forge/noarch::webencodings-0.5.1-py_1\n",
" wheel conda-forge/noarch::wheel-0.37.1-pyhd8ed1ab_0\n",
" widgetsnbextension conda-forge/noarch::widgetsnbextension-4.0.3-pyhd8ed1ab_0\n",
" xcb-util conda-forge/linux-64::xcb-util-0.4.0-h166bdaf_0\n",
" xcb-util-image conda-forge/linux-64::xcb-util-image-0.4.0-h166bdaf_0\n",
" xcb-util-keysyms conda-forge/linux-64::xcb-util-keysyms-0.4.0-h166bdaf_0\n",
" xcb-util-renderut~ conda-forge/linux-64::xcb-util-renderutil-0.3.9-h166bdaf_0\n",
" xcb-util-wm conda-forge/linux-64::xcb-util-wm-0.4.1-h166bdaf_0\n",
" xorg-libxau conda-forge/linux-64::xorg-libxau-1.0.9-h7f98852_0\n",
" xorg-libxdmcp conda-forge/linux-64::xorg-libxdmcp-1.1.3-h7f98852_0\n",
" xz conda-forge/linux-64::xz-5.2.6-h166bdaf_0\n",
" yaml conda-forge/linux-64::yaml-0.2.5-h7f98852_2\n",
" yarl conda-forge/linux-64::yarl-1.8.1-py310h5764c6d_0\n",
" zeromq conda-forge/linux-64::zeromq-4.3.4-h9c3ff4c_1\n",
" zipp conda-forge/noarch::zipp-3.10.0-pyhd8ed1ab_0\n",
" zstd conda-forge/linux-64::zstd-1.5.2-h6239696_4\n",
"\n",
"\n",
"Preparing transaction: | \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\bdone\n",
"Executing transaction: / \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\b\\ \b\b| \b\b/ \b\b- \b\bdone\n",
"installation finished.\n",
"WARNING:\n",
" You currently have a PYTHONPATH environment variable set. This may cause\n",
" unexpected behavior when running the Python interpreter in Colab.\n",
" For best results, please verify that your PYTHONPATH only points to\n",
" directories of packages that are compatible with the Python interpreter\n",
" in Colab: /usr/local\n",
"Installed kernelspec py310 in /root/.local/share/jupyter/kernels/py310\n"
]
}
]
},
{
"cell_type": "code",
"source": [
"import sys\n",
"print(\"version:\", sys.version)"
],
"metadata": {
"id": "zSHH_uzvyhS0",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "3b8cbd8f-6360-4cc5-a854-d3a57f5f750c"
},
"execution_count": 1,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"version: 3.10.6 | packaged by conda-forge | (main, Aug 22 2022, 20:35:26) [GCC 10.4.0]\n"
]
}
]
},
{
"cell_type": "code",
"source": [
"!python3 --version"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "ssJjOsmJsTOb",
"outputId": "986a346a-ec12-407e-ae5c-037e672662b6"
},
"execution_count": 2,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Python 3.10.6\n"
]
}
]
},
{
"cell_type": "code",
"source": [
"# needed when using Google Colab\n",
"!pip install obp==0.5.7 matplotlib==3.5.2"
],
"metadata": {
"id": "g3rPDTAtyNhW",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
},
"outputId": "2493ee94-60cd-4da3-a43b-89f0961dcd9e"
},
"execution_count": 3,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Collecting obp==0.5.7\n",
" Downloading obp-0.5.7.tar.gz (1.3 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m15.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25h Installing build dependencies ... \u001b[?25l\u001b[?25hdone\n",
" Getting requirements to build wheel ... \u001b[?25l\u001b[?25hdone\n",
" Preparing metadata (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
"Collecting matplotlib==3.5.2\n",
" Downloading matplotlib-3.5.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.9 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m11.9/11.9 MB\u001b[0m \u001b[31m90.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting seaborn<0.12.0,>=0.11.2\n",
" Downloading seaborn-0.11.2-py3-none-any.whl (292 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m292.8/292.8 kB\u001b[0m \u001b[31m32.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting pandas==1.5.2\n",
" Downloading pandas-1.5.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.1 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m12.1/12.1 MB\u001b[0m \u001b[31m68.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting scikit-learn==1.1.3\n",
" Downloading scikit_learn-1.1.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (30.5 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m30.5/30.5 MB\u001b[0m \u001b[31m14.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting numpy==1.23.5\n",
" Downloading numpy-1.23.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.1 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m17.1/17.1 MB\u001b[0m \u001b[31m91.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting torch==1.12\n",
" Downloading torch-1.12.0-cp310-cp310-manylinux1_x86_64.whl (776.3 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m776.3/776.3 MB\u001b[0m \u001b[31m1.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: tqdm<5.0.0,>=4.62.2 in /usr/local/lib/python3.10/site-packages (from obp==0.5.7) (4.64.1)\n",
"Collecting Pillow==9.1.1\n",
" Downloading Pillow-9.1.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m3.1/3.1 MB\u001b[0m \u001b[31m96.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting scipy==1.9.3\n",
" Downloading scipy-1.9.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (33.7 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m33.7/33.7 MB\u001b[0m \u001b[31m36.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting pyieoe<0.2.0,>=0.1.1\n",
" Downloading pyieoe-0.1.1.tar.gz (15 kB)\n",
" Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
"Collecting mypy-extensions<0.5.0,>=0.4.3\n",
" Downloading mypy_extensions-0.4.4.tar.gz (4.2 kB)\n",
" Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
"Collecting PyYAML==6.0.0\n",
" Downloading PyYAML-6.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (682 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m682.2/682.2 kB\u001b[0m \u001b[31m59.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting pingouin<0.5.0,>=0.4.0\n",
" Downloading pingouin-0.4.0.tar.gz (206 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m206.9/206.9 kB\u001b[0m \u001b[31m24.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25h Preparing metadata (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
"Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.10/site-packages (from matplotlib==3.5.2) (2.8.2)\n",
"Collecting cycler>=0.10\n",
" Downloading cycler-0.12.1-py3-none-any.whl (8.3 kB)\n",
"Collecting kiwisolver>=1.0.1\n",
" Downloading kiwisolver-1.4.8-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.6 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.6/1.6 MB\u001b[0m \u001b[31m80.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/site-packages (from matplotlib==3.5.2) (21.3)\n",
"Collecting fonttools>=4.22.0\n",
" Downloading fonttools-4.58.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.8 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m4.8/4.8 MB\u001b[0m \u001b[31m105.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: pyparsing>=2.2.1 in /usr/local/lib/python3.10/site-packages (from matplotlib==3.5.2) (3.0.9)\n",
"Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/site-packages (from pandas==1.5.2->obp==0.5.7) (2022.5)\n",
"Collecting threadpoolctl>=2.0.0\n",
" Downloading threadpoolctl-3.6.0-py3-none-any.whl (18 kB)\n",
"Collecting joblib>=1.0.0\n",
" Downloading joblib-1.5.1-py3-none-any.whl (307 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m307.7/307.7 kB\u001b[0m \u001b[31m33.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: typing-extensions in /usr/local/lib/python3.10/site-packages (from torch==1.12->obp==0.5.7) (4.4.0)\n",
"Collecting statsmodels>=0.12.0\n",
" Downloading statsmodels-0.14.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (10.8 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m10.8/10.8 MB\u001b[0m \u001b[31m113.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting pandas_flavor>=0.2.0\n",
" Downloading pandas_flavor-0.7.0-py3-none-any.whl (8.4 kB)\n",
"Collecting outdated\n",
" Downloading outdated-0.2.2-py2.py3-none-any.whl (7.5 kB)\n",
"Collecting tabulate\n",
" Downloading tabulate-0.9.0-py3-none-any.whl (35 kB)\n",
"Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/site-packages (from python-dateutil>=2.7->matplotlib==3.5.2) (1.16.0)\n",
"Collecting xarray\n",
" Downloading xarray-2025.6.0-py3-none-any.whl (1.3 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m82.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting patsy>=0.5.6\n",
" Downloading patsy-1.0.1-py2.py3-none-any.whl (232 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m232.9/232.9 kB\u001b[0m \u001b[31m24.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: requests in /usr/local/lib/python3.10/site-packages (from outdated->pingouin<0.5.0,>=0.4.0->obp==0.5.7) (2.28.1)\n",
"Requirement already satisfied: setuptools>=44 in /usr/local/lib/python3.10/site-packages (from outdated->pingouin<0.5.0,>=0.4.0->obp==0.5.7) (65.5.0)\n",
"Collecting littleutils\n",
" Downloading littleutils-0.2.4-py3-none-any.whl (8.1 kB)\n",
"Requirement already satisfied: charset-normalizer<3,>=2 in /usr/local/lib/python3.10/site-packages (from requests->outdated->pingouin<0.5.0,>=0.4.0->obp==0.5.7) (2.1.1)\n",
"Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/site-packages (from requests->outdated->pingouin<0.5.0,>=0.4.0->obp==0.5.7) (3.4)\n",
"Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.10/site-packages (from requests->outdated->pingouin<0.5.0,>=0.4.0->obp==0.5.7) (2022.9.24)\n",
"Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/site-packages (from requests->outdated->pingouin<0.5.0,>=0.4.0->obp==0.5.7) (1.26.11)\n",
"Collecting packaging>=20.0\n",
" Downloading packaging-25.0-py3-none-any.whl (66 kB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m66.5/66.5 kB\u001b[0m \u001b[31m8.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hCollecting xarray\n",
" Downloading xarray-2025.4.0-py3-none-any.whl (1.3 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m56.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25h Downloading xarray-2025.3.1-py3-none-any.whl (1.3 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m60.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25h Downloading xarray-2025.3.0-py3-none-any.whl (1.3 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.3/1.3 MB\u001b[0m \u001b[31m58.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25h Downloading xarray-2025.1.2-py3-none-any.whl (1.2 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.2/1.2 MB\u001b[0m \u001b[31m56.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25h Downloading xarray-2025.1.1-py3-none-any.whl (1.2 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.2/1.2 MB\u001b[0m \u001b[31m64.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25h Downloading xarray-2025.1.0-py3-none-any.whl (1.2 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.2/1.2 MB\u001b[0m \u001b[31m58.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25h Downloading xarray-2024.11.0-py3-none-any.whl (1.2 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.2/1.2 MB\u001b[0m \u001b[31m65.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25h Downloading xarray-2024.10.0-py3-none-any.whl (1.2 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.2/1.2 MB\u001b[0m \u001b[31m65.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25h Downloading xarray-2024.9.0-py3-none-any.whl (1.2 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.2/1.2 MB\u001b[0m \u001b[31m58.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25h Downloading xarray-2024.7.0-py3-none-any.whl (1.2 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.2/1.2 MB\u001b[0m \u001b[31m51.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25h Downloading xarray-2024.6.0-py3-none-any.whl (1.2 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.2/1.2 MB\u001b[0m \u001b[31m62.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25h Downloading xarray-2024.5.0-py3-none-any.whl (1.2 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.2/1.2 MB\u001b[0m \u001b[31m63.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25h Downloading xarray-2024.3.0-py3-none-any.whl (1.1 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.1/1.1 MB\u001b[0m \u001b[31m52.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25hBuilding wheels for collected packages: obp, mypy-extensions, pingouin, pyieoe\n",
" Building wheel for obp (pyproject.toml) ... \u001b[?25l\u001b[?25hdone\n",
" Created wheel for obp: filename=obp-0.5.7-py3-none-any.whl size=1406297 sha256=83492e829a62f7deaaabe07055ed19942fd69f2776bbc99f8c49d97017775c2b\n",
" Stored in directory: /root/.cache/pip/wheels/a2/1b/c0/3166edeead80bf41198d2d925a9965b0f0457b25780ac502fd\n",
" Building wheel for mypy-extensions (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
" Created wheel for mypy-extensions: filename=mypy_extensions-0.4.4-py2.py3-none-any.whl size=4479 sha256=fdfd6b133b2c3afae9daf880ed70934a1fba3a46a3fdf4458d9756e56d3c66b5\n",
" Stored in directory: /root/.cache/pip/wheels/cc/6e/ef/235563c33f993f2785ea0f83ccf91935738401c5590fc72cf7\n",
" Building wheel for pingouin (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
" Created wheel for pingouin: filename=pingouin-0.4.0-py3-none-any.whl size=206872 sha256=ca04780a0628bf3dc145bfaff38e43d815e564691c9f433dbbf8b9894ad3b294\n",
" Stored in directory: /root/.cache/pip/wheels/30/5c/e8/03f179405922592dea5b33fa428bb5d91391630cb79e5debb6\n",
" Building wheel for pyieoe (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
" Created wheel for pyieoe: filename=pyieoe-0.1.1-py3-none-any.whl size=12687 sha256=c4a765ee9795538d9d5eb63b3cd32c4cc47b290bdd5a0c82453db55099710784\n",
" Stored in directory: /root/.cache/pip/wheels/23/c3/6b/2c01cd27cec1fa38fe31e74981fdb530e122413439c843916e\n",
"Successfully built obp mypy-extensions pingouin pyieoe\n",
"Installing collected packages: torch, threadpoolctl, tabulate, PyYAML, Pillow, packaging, numpy, mypy-extensions, littleutils, kiwisolver, joblib, fonttools, cycler, scipy, patsy, pandas, outdated, matplotlib, xarray, statsmodels, seaborn, scikit-learn, pandas_flavor, pingouin, pyieoe, obp\n",
" Attempting uninstall: packaging\n",
" Found existing installation: packaging 21.3\n",
" Uninstalling packaging-21.3:\n",
" Successfully uninstalled packaging-21.3\n",
" Attempting uninstall: numpy\n",
" Found existing installation: numpy 1.23.4\n",
" Uninstalling numpy-1.23.4:\n",
" Successfully uninstalled numpy-1.23.4\n",
" Attempting uninstall: pandas\n",
" Found existing installation: pandas 1.5.1\n",
" Uninstalling pandas-1.5.1:\n",
" Successfully uninstalled pandas-1.5.1\n",
"Successfully installed Pillow-9.1.1 PyYAML-6.0 cycler-0.12.1 fonttools-4.58.2 joblib-1.5.1 kiwisolver-1.4.8 littleutils-0.2.4 matplotlib-3.5.2 mypy-extensions-0.4.4 numpy-1.23.5 obp-0.5.7 outdated-0.2.2 packaging-25.0 pandas-1.5.2 pandas_flavor-0.7.0 patsy-1.0.1 pingouin-0.4.0 pyieoe-0.1.1 scikit-learn-1.1.3 scipy-1.9.3 seaborn-0.11.2 statsmodels-0.14.4 tabulate-0.9.0 threadpoolctl-3.6.0 torch-1.12.0 xarray-2024.3.0\n"
]
},
{
"output_type": "display_data",
"data": {
"application/vnd.colab-display-data+json": {
"pip_warning": {
"packages": [
"numpy",
"pandas"
]
}
}
},
"metadata": {}
}
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"id": "VZGm4hh6oGgO"
},
"outputs": [],
"source": [
"from sklearn.ensemble import RandomForestClassifier as RandomForest\n",
"from sklearn.linear_model import LogisticRegression\n",
"\n",
"# import open bandit pipeline (obp)\n",
"import obp\n",
"from obp.dataset import (\n",
" SyntheticBanditDataset,\n",
" logistic_reward_function,\n",
" linear_reward_function\n",
")\n",
"from obp.policy import IPWLearner, Random\n",
"from obp.ope import (\n",
" OffPolicyEvaluation,\n",
" RegressionModel,\n",
" InverseProbabilityWeighting,\n",
" DirectMethod,\n",
" DoublyRobust\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"id": "ejfoEIlMoGgP",
"outputId": "62847e8e-51be-4e20-905b-78e193278970",
"colab": {
"base_uri": "https://localhost:8080/"
}
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"0.5.7\n"
]
}
],
"source": [
"# obp version\n",
"print(obp.__version__)"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"id": "RSi8ntuxoGgP"
},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "fq0qu4iKoGgQ"
},
"source": [
"## (1) Generating Synthetic Data\n",
"`obp.dataset.SyntheticBanditDataset` is an easy-to-use synthetic data generator.\n",
"\n",
"It takes\n",
"- number of actions (`n_actions`, $|\\mathcal{A}|$)\n",
"- dimension of context vectors (`dim_context`, $d$)\n",
"- reward function (`reward_function`, $q(x,a)=\\mathbb{E}[r|x,a]$)\n",
"\n",
"as inputs and generates synthetic logged bandit data that can be used to evaluate the performance of decision making policies (obtained by `off-policy learning`) and OPE estimators."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"tags": [],
"id": "OQYsclpCoGgQ"
},
"outputs": [],
"source": [
"# generate synthetic logged bandit data with 10 actions\n",
"# we use `logistic function` as the reward function and control the behavior policy with `beta`\n",
"# one can define their own reward function and behavior policy function such as nonlinear ones.\n",
"dataset = SyntheticBanditDataset(\n",
" n_actions=10,\n",
" dim_context=5,\n",
" beta=1.0, # inverse temperature parameter to control the optimality and entropy of the behavior policy\n",
" reward_type=\"binary\", # \"binary\" or \"continuous\"\n",
" reward_function=logistic_reward_function,\n",
" random_state=12345,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"id": "0LORFTL3oGgQ"
},
"outputs": [],
"source": [
"# obtain training and test sets of synthetic logged bandit data\n",
"n_rounds_train, n_rounds_test = 100000, 100000\n",
"bandit_feedback_train = dataset.obtain_batch_bandit_feedback(n_rounds=n_rounds_train)\n",
"bandit_feedback_test = dataset.obtain_batch_bandit_feedback(n_rounds=n_rounds_test)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "14xrbNneoGgQ"
},
"source": [
"Note that a logged bandit dataset is collected by the behavior policy as follows.\n",
"\n",
"$ \\mathcal{D}_b := \\{(x_i,a_i,r_i)\\}$ where $(x,a,r) \\sim p(x)\\pi_b(a \\mid x)p(r \\mid x,a) $"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"id": "cZq0qjjHoGgR",
"outputId": "30817c1b-1af0-4e81-8f0d-717ab43d43e7",
"colab": {
"base_uri": "https://localhost:8080/"
}
},
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"{'n_rounds': 100000,\n",
" 'n_actions': 10,\n",
" 'context': array([[-0.20470766, 0.47894334, -0.51943872, -0.5557303 , 1.96578057],\n",
" [ 1.39340583, 0.09290788, 0.28174615, 0.76902257, 1.24643474],\n",
" [ 1.00718936, -1.29622111, 0.27499163, 0.22891288, 1.35291684],\n",
" ...,\n",
" [ 1.36946256, 0.58727761, -0.69296769, -0.27519988, -2.10289159],\n",
" [-0.27428715, 0.52635353, 1.02572168, -0.18486381, 0.72464834],\n",
" [-1.25579833, -1.42455203, -0.26361242, 0.27928604, 1.21015571]]),\n",
" 'action_context': array([[1, 0, 0, 0, 0, 0, 0, 0, 0, 0],\n",
" [0, 1, 0, 0, 0, 0, 0, 0, 0, 0],\n",
" [0, 0, 1, 0, 0, 0, 0, 0, 0, 0],\n",
" [0, 0, 0, 1, 0, 0, 0, 0, 0, 0],\n",
" [0, 0, 0, 0, 1, 0, 0, 0, 0, 0],\n",
" [0, 0, 0, 0, 0, 1, 0, 0, 0, 0],\n",
" [0, 0, 0, 0, 0, 0, 1, 0, 0, 0],\n",
" [0, 0, 0, 0, 0, 0, 0, 1, 0, 0],\n",
" [0, 0, 0, 0, 0, 0, 0, 0, 1, 0],\n",
" [0, 0, 0, 0, 0, 0, 0, 0, 0, 1]]),\n",
" 'action': array([9, 3, 2, ..., 9, 1, 6]),\n",
" 'position': None,\n",
" 'reward': array([0, 1, 1, ..., 1, 0, 1]),\n",
" 'expected_reward': array([[0.97096929, 0.87495197, 0.59430992, ..., 0.87451356, 0.84290861,\n",
" 0.59365568],\n",
" [0.78700232, 0.4464541 , 0.72028615, ..., 0.90952832, 0.81461862,\n",
" 0.85843021],\n",
" [0.71759218, 0.68220215, 0.61577871, ..., 0.92899054, 0.75136178,\n",
" 0.93245495],\n",
" ...,\n",
" [0.84913066, 0.71707452, 0.98276248, ..., 0.34089248, 0.70975667,\n",
" 0.98762405],\n",
" [0.78305787, 0.85890577, 0.78610525, ..., 0.76868184, 0.92010434,\n",
" 0.43809616],\n",
" [0.92193732, 0.91745325, 0.17322462, ..., 0.81479071, 0.92356546,\n",
" 0.71907434]]),\n",
" 'pi_b': array([[[0.12633552],\n",
" [0.11476929],\n",
" [0.08668509],\n",
" ...,\n",
" [0.11471898],\n",
" [0.11114999],\n",
" [0.08662839]],\n",
" \n",
" [[0.10467048],\n",
" [0.07446051],\n",
" [0.09791512],\n",
" ...,\n",
" [0.11831413],\n",
" [0.10760138],\n",
" [0.11242036]],\n",
" \n",
" [[0.09335391],\n",
" [0.09010789],\n",
" [0.08431706],\n",
" ...,\n",
" [0.11532983],\n",
" [0.09656027],\n",
" [0.11573007]],\n",
" \n",
" ...,\n",
" \n",
" [[0.11534189],\n",
" [0.10107316],\n",
" [0.13183255],\n",
" ...,\n",
" [0.06938443],\n",
" [0.10033622],\n",
" [0.13247502]],\n",
" \n",
" [[0.11566126],\n",
" [0.12477519],\n",
" [0.11601426],\n",
" ...,\n",
" [0.1140104 ],\n",
" [0.13264975],\n",
" [0.08191679]],\n",
" \n",
" [[0.11655941],\n",
" [0.11603792],\n",
" [0.05512969],\n",
" ...,\n",
" [0.10471627],\n",
" [0.11674934],\n",
" [0.09515795]]]),\n",
" 'pscore': array([0.08662839, 0.11692449, 0.08431706, ..., 0.13247502, 0.12477519,\n",
" 0.0969212 ])}"
]
},
"metadata": {},
"execution_count": 5
}
],
"source": [
"# `bandit_feedback` is a dictionary storing synthetic logged bandit data\n",
"bandit_feedback_train"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"id": "t4kbbMLBoGgR"
},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "6jO72EOWoGgR"
},
"source": [
"## (2) Off-Policy Learning\n",
"After generating synthetic data, we now train some candidate evaluation policies using the training bandit dataset. <br>\n",
"\n",
"We use `obp.ope.IPWLearner` to train evaluation policies.\n",
"We also use `RandomForestClassifier` and `LogisticRegression` implemented in scikit-learn for base ML methods."
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"tags": [],
"id": "nOQko4IOoGgR"
},
"outputs": [],
"source": [
"# define IPWLearner with Logistic Regression as its base ML model\n",
"ipw_lr = IPWLearner(\n",
" n_actions=dataset.n_actions,\n",
" base_classifier=LogisticRegression(C=100, random_state=12345)\n",
")\n",
"\n",
"# train IPWLearner on the training set of logged bandit data\n",
"ipw_lr.fit(\n",
" context=bandit_feedback_train[\"context\"],\n",
" action=bandit_feedback_train[\"action\"],\n",
" reward=bandit_feedback_train[\"reward\"],\n",
" pscore=bandit_feedback_train[\"pscore\"]\n",
")\n",
"\n",
"# obtains action choice probabilities for the test set\n",
"action_dist_ipw_lr = ipw_lr.predict(context=bandit_feedback_test[\"context\"])"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"tags": [],
"id": "VbmW-KwWoGgR"
},
"outputs": [],
"source": [
"# define IPWLearner with Random Forest as its base ML model\n",
"ipw_rf = IPWLearner(\n",
" n_actions=dataset.n_actions,\n",
" base_classifier=RandomForest(n_estimators=30, min_samples_leaf=10, random_state=12345)\n",
")\n",
"\n",
"# train IPWLearner on the training set of logged bandit data\n",
"ipw_rf.fit(\n",
" context=bandit_feedback_train[\"context\"],\n",
" action=bandit_feedback_train[\"action\"],\n",
" reward=bandit_feedback_train[\"reward\"],\n",
" pscore=bandit_feedback_train[\"pscore\"]\n",
")\n",
"\n",
"# obtains action choice probabilities for the test set\n",
"action_dist_ipw_rf = ipw_rf.predict(context=bandit_feedback_test[\"context\"])"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"id": "Oo0iU7NkoGgR"
},
"outputs": [],
"source": [
"# define Uniform Random Policy as a baseline evaluation policy\n",
"random = Random(n_actions=dataset.n_actions,)\n",
"\n",
"# obtains action choice probabilities for the test set\n",
"action_dist_random = random.compute_batch_action_dist(\n",
" n_rounds=bandit_feedback_test[\"n_rounds\"]\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"id": "r8O52nvGoGgR",
"outputId": "7f4094a1-31f0-4acd-b24a-b07d6f8f0204",
"colab": {
"base_uri": "https://localhost:8080/"
}
},
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"array([[0., 0., 1., ..., 0., 0., 0.],\n",
" [0., 0., 0., ..., 0., 1., 0.],\n",
" [1., 0., 0., ..., 0., 0., 0.],\n",
" ...,\n",
" [0., 0., 0., ..., 0., 1., 0.],\n",
" [0., 0., 0., ..., 0., 0., 0.],\n",
" [0., 1., 0., ..., 0., 0., 0.]])"
]
},
"metadata": {},
"execution_count": 9
}
],
"source": [
"# action_dist is a probability distribution over actions (can be deterministic)\n",
"action_dist_ipw_lr[:, :, 0]"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"id": "YNUr481SoGgR"
},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "Sm9KoySOoGgR"
},
"source": [
"## (3) Off-Policy Evaluation (OPE)\n",
"Our next step is OPE, which aims to estimate the performance of evaluation policies using logged bandit data and OPE estimators.\n",
"\n",
"Here, we use\n",
"- `obp.ope.InverseProbabilityWeighting` (IPW)\n",
"- `obp.ope.DirectMethod` (DM)\n",
"- `obp.ope.DoublyRobust` (DR)\n",
"\n",
"as OPE estimators and visualize the OPE results."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "d8EuA072oGgS"
},
"source": [
"### (3-1) Obtaining a reward estimator\n",
"A reward estimator $\\hat{q}(x,a)$ is needed for model dependent estimators such as DM or DR.\n",
"\n",
"$\\hat{q}(x,a) \\approx \\mathbb{E} [r \\mid x,a]$"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"id": "nKQ588AnoGgS"
},
"outputs": [],
"source": [
"# estimate the expected rewards by using an ML model (Logistic Regression here)\n",
"# the estimated rewards are used by model-dependent estimators such as DM and DR\n",
"regression_model = RegressionModel(\n",
" n_actions=dataset.n_actions,\n",
" action_context=dataset.action_context,\n",
" base_model=LogisticRegression(random_state=12345),\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"id": "T2FoRSsyoGgS"
},
"outputs": [],
"source": [
"estimated_rewards_by_reg_model = regression_model.fit_predict(\n",
" context=bandit_feedback_test[\"context\"],\n",
" action=bandit_feedback_test[\"action\"],\n",
" reward=bandit_feedback_test[\"reward\"],\n",
" n_folds=3, # use 3-fold cross-fitting\n",
" random_state=12345,\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "RzUVLKtwoGgS"
},
"source": [
"please refer to https://arxiv.org/abs/2002.08536 about the details of the cross-fitting procedure."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "2ZAAPVA6oGgS"
},
"source": [
"### (3-2) Off-Policy Evaluation\n",
"$V(\\pi_e) \\approx \\hat{V} (\\pi_e; \\mathcal{D}_b, \\theta)$ using DM, IPW, and DR"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"tags": [],
"id": "0DP5_ODLoGgS"
},
"outputs": [],
"source": [
"# estimate the policy value of the evaluation policies based on their action choice probabilities\n",
"# it is possible to set multiple OPE estimators to the `ope_estimators` argument\n",
"ope = OffPolicyEvaluation(\n",
" bandit_feedback=bandit_feedback_test,\n",
" ope_estimators=[InverseProbabilityWeighting(), DirectMethod(), DoublyRobust()]\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"tags": [],
"id": "zVQaWBk6oGgS",
"outputId": "62c41d1e-8e87-4788-9895-aeed225dd678",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 709
}
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
" mean 95.0% CI (lower) 95.0% CI (upper)\n",
"ipw 0.959324 0.947568 0.973285\n",
"dm 0.800633 0.800135 0.801075\n",
"dr 0.958316 0.954360 0.962206 \n",
"\n"
]
},
{
"output_type": "stream",
"name": "stderr",
"text": [
"/usr/local/lib/python3.10/site-packages/seaborn/categorical.py:82: FutureWarning: iteritems is deprecated and will be removed in a future version. Use .items instead.\n",
" plot_data = [np.asarray(s, float) for k, s in iter_data]\n"
]
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"<Figure size 800x600 with 1 Axes>"
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAswAAAIzCAYAAAAUFU/WAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAA9hAAAPYQGoP6dpAABs4UlEQVR4nO3dd3xUVeL///eE9EYSQkIAgUCA0EEQpIYiTZEFKeoull3167Lo2nf1g4rYUVfdtX3WtYF+kAUUUZq41IRepYSSUBJKCCmEJIQ0Zn5/5DezGZJMJrmTBq/n47GPx8y9555zhs017zk59xyTxWKxCAAAAEC53Oq6AwAAAEB9RmAGAAAAHCAwAwAAAA4QmAEAAAAHCMwAAACAAwRmAAAAwAECMwAAAOAAgRkAAABwgMAMAAAAOEBgBgAAABwgMAMAAAAOEJgBAAAAB9xdUUlxcbFOnjyp8+fPKysrS/n5+XJ3d5evr69CQ0N1ww03qEmTJq5oCgAAAKhV1Q7M58+fV1xcnPbu3avExERduXLFYfng4GB1795dffv2Va9evdSoUaPqNg0AAADUGpPFYrFU5YLt27dr5cqVio+Pr3aj/v7+GjFihEaPHs3IMwAAAOo1pwPz9u3btXDhQp06dcp2zMPDQ23atFFUVJTatm2rwMBA+fv7y9/fX4WFhcrNzVVubq5SUlJ07NgxJSYmKiMjw3a9u7u7hg8frsmTJ6tx48au/3QAAACAQU4F5tmzZ9tGlD08PNSrVy8NHjxYN954o9zdqzar49y5c4qNjdWmTZuUkpIiSfL19dUjjzyi3r17V+Mj1G8XLlxQcXFxXXcDAAAAV3F3d1dwcHCl5ZwKzHfeeaf8/f01btw4jR49Wr6+vi7p5OHDh/Xdd99p3759mjJliiZPnuySeuuTtLQ0FRUV1XU3AAAAcBUPDw81bdq00nJOBeYff/xRo0aNkre3t0s6d7XExETl5OSoV69eTl9z/Phx7du3T4mJiUpMTFRmZqYkaeHChdXqQ25urhYtWqQdO3YoKytLQUFB6tu3r6ZMmSI/P79q1SkRmAEAAOorlwbm+uitt97Szp07yxyvTmDOzs7W888/r3Pnzik8PFxt27bV6dOnderUKUVEROi1116Tv79/tfpJYAYAAKifnA3MLlmHuS506NBBrVu3Vrt27RQVFaUZM2ZUO5h+9dVXOnfunPr27asnnnjCtuTdF198oVWrVmnu3LmaMWOGK7sPAACABqLBBuYJEya4pJ4LFy5o06ZNcnd314MPPmi3PvQ999yjzZs3KzY2VtOmTWMlDwAAgOvQdb819t69e2WxWNSpUycFBQXZnfPw8FDv3r1lNpu1Z8+euukgAAAA6pTTI8wbNmxweeMxMTEur7OqkpKSJEmRkZHlno+MjNS6dets5QAAAHB9cTowf/zxxy5t2GQy1YvAnJ6eLkkKCQkp97x1J8K0tDSH9RQVFdnNoTaZTPLx8bG9BgAAQMNUZ3OY68viHPn5+ZIkLy+vcs9bj1vLVWTJkiVavHix7X1kZKTmzJnj1JOXAAAAqL+cDszTp0+vyX40eBMnTtS4ceNs762jymlpaez0BwAAUA+5u7u7dlm5oUOHGulPvWXdjKWgoKDc89bjlW3a4uHhIQ8Pj3LP1ZfRdAAAAFTddb9KRmhoqCTZdgq8WkZGhiQxtQIAAOA6dd0H5tatW0uSTpw4Ue5563FrOQAAAFxfqvTQ36VLl3To0CFJJatHVLQUW3mOHz9uG8Xt0qWLbQWJutazZ0+ZTCYdOnRIFy9etNucpKioSLt27ZKbm5t69epVh70EAABAXanSCPO3336rt99+Wx988EGF83Ur4unpqQ8++EBvv/22Fi5cWKVrXWHVqlV6/PHHNX/+fLvjwcHBGjhwoIqLi/XZZ5/pypUrtnPffPONsrOzNXjwYHb5AwAAuE45PcKclZWlNWvWSJLuuusutWzZskoNtWzZUnfffbe+/PJLrV69WhMnTlRgYGDVelvK7t279d1339neW1eimDlzpu3YpEmTdOONN0qSsrOzdfbsWV24cKFMXffff78SEhK0bds2Pf7442rXrp1OnTqlU6dOKSIiQvfdd1+1+wkAAICGzekR5ri4OJnNZoWEhGjUqFHVamzkyJEKDQ1VcXGx4uLiqlWHVXZ2thISEmz/s65EUfpYdna2U3UFBgbqjTfe0JgxY1RcXKzt27crLy9PY8eO1euvvy5/f39DfQUAAEDDZbI4uebZnDlztHv3bo0dO1b3339/tRucN2+eli9frt69e+svf/lLtetpKNLS0ux2AAQAAED94OHh4dp1mJOTkyWVPLBnRKdOnbR8+XIlJSUZqgeojldeecX28GlISIheeOGFOu4RAACo75wOzDk5OZJKHpIzwnq9tT6gNmVmZio9Pb2uuwEAABoQp+cwm81mlzbs6voAAACAmuB0YA4ICJCkcleZqIqsrCy7+gAAAID6zOnAHBYWJkm2jUuqKz4+3q4+AAAAoD5zeg5zt27ddPjwYcXFxWnq1Kny9vaucmP5+fmKjY211QcAAHCt4kHza4fTI8z9+/eXyWTSxYsXNW/evGo1NnfuXGVnZ8vNzU0DBgyoVh0AAAANgfVB8/T0dFtwRsPkdGBu0aKFBg0aJElas2aNPv30UxUWFjp1bWFhoT799FOtXbtWkjRo0CA1b968Gt0FAAAAapfTUzKk/24hfe7cOa1Zs0a7d+/WiBEj1KtXL7Vp00bu7v+trri4WCdPntTu3bu1du1a28OCzZs3N7TxCQAAAFCbnN7pzyo1NVVz5szRmTNn7I67ubnJ19dXXl5eKigoUF5eXpml41q0aKFnn332unrgz5U7/aU886BL6rmevXG+QBf+/x/LYDfpuTCvuu3QNSDi7c/qugsAasjPP6bUdRcatCXL39ClvJIBQz/fYE287bk67lHDN3p8hEvrc3anP6enZFiFh4frzTff1JgxY+Tp6Wk7bjablZubq4yMDOXm5tqFZU9PT40dO1ZvvPHGdRWWAQAA0PBVaUqGlaenp37/+99r0qRJ2rx5s+Lj45WUlKScnBxdvnxZPj4+CggIUOvWrdW5c2cNGDBAgYGBru47AAAAUOOqFZitAgMDNWbMGI0ZM8ZV/QEAALgm+Po0Lvc1Gh5DgRkAAADlGz38T3XdBbhIlecwAwAAANcTAjMAAADgAIEZAAAAcIA5zLiuNG5kkmQp9RoAAMAxAjOuK39q4ll5IQAAgFKYkgEAAAA4QGAGAAAAHCAwAwAAAA4QmAEAAAAHajww5+fn68KFC8rPz6/ppgAAAACXq5FVMoqLi/Xjjz9qw4YNOnfunO14eHi4YmJiNH78eHl4eNRE0wAAAIBLuTww5+fn6+WXX9axY8fKnEtNTdXChQu1c+dOvfjii/Lx8XF18wAAAIBLuTwwL168WMeOHVPz5s111113qWPHjvLz81NGRoZiY2P1/fff6/jx41q8eLHuueceVzcPAAAAuJTL5zBv2bJF7u7umjVrlvr166egoCB5eHioWbNmmjJliu666y5J0tatW13dNAAAAOByTgfmDz/8UNnZ2ZWWy8rKUrNmzRQUFFTu+U6dOtnKAQAAAPWd04E5NjZWjz/+uFatWiWz2VxhuaCgIJ07d67CQBwfH28rBwAAANR3Tgfm22+/Xfn5+fryyy/17LPP6siRI+WW69evn4qLi/XKK69ox44dys7OVnFxsVJTU7V48WItXLhQknTzzTe75hMAAAAANcjph/6mTZumoUOH6osvvtDBgwf14osvasiQIZo2bZoaN25sKzdlyhQdPHhQJ0+e1DvvvFNuXW3atNHkyZON9x4AAACoYSaLxWKp6kWbNm3S119/rQsXLsjX11dTp07V6NGj5eZWMmBdWFiopUuXauPGjTp//rzturCwMNs6zJ6enq77FPVYWlqaioqKXFJXyjMPuqQewJUi3v6srrsAoIb8/GNKXXcBsDN6fIRL6/Pw8FDTpk0rLVetwCyVrLe8cOFCrVy5UmazWa1bt9YDDzygjh07limXl5cnHx+f63LdZQIzrnUEZuDaRWBGfVNXgbnay8p5e3vr3nvv1dtvv63OnTsrKSlJL774oj766CNdvHjRrlxISMh1GZYBAADQ8Bleh7lly5aaNWuWHn30UQUFBWnjxo16/PHHbSPPAAAAQEPmso1LBg0apL///e+69dZblZ+fr6+++krPPvusDh8+7KomAAAAgFpXrcBsNpt1+vRpHT16VKdPn9aVK1cklUy/uO+++zRnzhxFR0crKSlJs2bN0ocffmg3TQMAAABoKJxeVk6ScnNzNX/+fMXGxqqwsNB23NPTU4MGDdLvfvc7+fv7q1WrVpo9e7Y2btyob775RrGxsdq1a5emTJmiMWPG2FbTAAAAAOo7p5NrTk6OZs6cqTVr1qiwsFBubm5q3Lix3NzcVFhYqLVr12rmzJl222cPGTJE77//vsaMGaP8/HzNnTtXf/3rX5mmAQAAgAbD6RHmhQsX6ty5c/Lx8dFDDz2k/v37y83NTWazWVu2bNFnn32mc+fOadGiRXrggQds1/n6+ur3v/+9hg8frs8//1xHjhzRrFmzNHjwYD3yyCM18qEAAAAAV3F6hHnnzp2SpN/97ncaOHCgbVqFm5ubBg4cqN/+9rd25a7WunVrvfzyy5o+fboCAwMVGxtrtO8AAABAjXN6hNk61aJNmzblnrceLz0lozxDhw5V3759tWDBAmebBgAAAOqM0yPMwcHBkqRDhw6Ve956PCgoqNK6fH199Yc//MHZpgEAAIA643RgvvnmmyVJ//73v7V06VKlp6eruLhY6enpWrZsmRYuXChJ6t+/f830FAAAAKgDTk/JmDx5sg4ePKjjx49r/vz5mj9/fpkybdq00aRJk1zaQQAAAKAuOR2Yvb299fLLL2vp0qXasGGDzp8/bzsXFhamwYMHa8KECfL09KyRjgIAAAB1oUobl3h4eGjy5MmaPHmy8vPzlZeXJx8fH/n4+NRU/wAAAIA6VaXAXJq3t7e8vb1d2RcAAACg3mGPagAAAMABAjMAAADgQLWnZFQkOTlZq1atUnx8vDIzMyWVrOHcuXNnjRo1SpGRka5uEgAAAKgxTgfmxYsXS5JGjRqlwMDAcsssWLBAP/zwgywWi93xc+fO6dy5c1q3bp3Gjx9v20YbAAAAqO+cDsyLFi2SVLKBSXmBedGiRVqyZIntvY+PjyIiImQymZSSkqK8vDxZLBYtXbpUXl5erNcMAACABsElUzJSUlL03XffSfrvttcDBw6Um1vJFGmz2axNmzbpiy++UF5enhYvXqyYmBiFhoa6onkAAACgxrjkob/Vq1fLYrHI3d1dzz//vAYPHmwLy5Lk5uamwYMHa+bMmXJ3d5fZbNaaNWtc0TQAAABQo1wSmOPj4yVJQ4YMUbt27SosFxUVpcGDB9tdAwAAANRnLgnM1m2ye/fuXWlZa5mzZ8+6omkAAACgRrkkMBcWFkqSU3OSrWXy8vJc0TQAAABQo1wSmIOCgiRJV65cqbSs2WyWJHl4eLiiaQAAAKBGVXmVjMTERGVnZ9sdCwkJUXp6ulJTUx3OYZakCxcuSJICAgKq2jQAAABQ66ocmD/55JMKzx05ckQDBgxweP3x48cllYRsAAAAoL5zyZQMq23bttmmXFRkx44dkkpWzAAAAADqO6dHmGfNmuVUueLiYnl6epZ7Lj4+XsnJyZKkDh06ONs0AAAAUGecDsydO3c23Fjr1q314YcfSpKCg4MN1wcAAADUNJdsje0sPz8/+fn51WaTAAAAgCEuncMMAAAAXGsIzAAAAIADhqZkpKWl6dChQzp9+rTS09OVn5+vwsJCeXt7y9/fXy1btlRUVJQ6duwok8nkqj4DAAAAtaZagXnbtm364YcfbGsqVyY0NFQjR47UuHHj5O5eq9OmAQAAAEOqlF4LCgr0/vvva/fu3VVqJD09Xd9++602bNigJ554Qq1atarS9QAAAEBdqVJgfuedd7Rv3z5JJaPG3bp1U1BQkPLy8nTo0CElJyfL3d1d9913n9q2bauUlBQdP35c27dvV3p6us6ePavZs2dr1qxZhGYAAAA0CE4H5s2bN9vC8uTJkzVp0iS5udk/MxgXF6ePP/5Y33zzjd58800NHjxYgwcP1j333KO1a9dq7ty5ys3N1QcffKA5c+aUuR4AAACob5wOzBs2bJAk9e/fX1OmTCm3zKBBg5SamqqFCxdq/vz5evrppyVJbm5uuuWWW9SsWTO9+uqrSk5OVmxsrGJiYgx1vrCwUEuWLNHmzZuVnp4uf39/9ejRQ3fddZdCQkKqVNe+ffu0fPlyJSYmKi8vTz4+Pmrbtq1GjRqlvn37GuonAAAAGi6nh3hPnjwpSZWGXOv53bt3q7i42O5c165dNWjQIEnSli1bqtLPMgoLCzV79mx99913ys/PV58+fdSkSROtX79ef/nLX5Samup0XcuXL9err76qvXv3qnnz5urXr59atGih/fv365133tG3335rqK8AAABouJweYc7JyZFU+ZbW1vNXrlxRZmamwsLC7M737dtXsbGxSkpKqmpf7Xz//fdKSEhQhw4d9Pzzz8vb21uStGzZMs2bN0+ffPKJXnrppUrryc7O1vz589WoUSO98MILdluAx8fH67XXXtMPP/yg4cOHKzw83FCfAQAA0PA4PcLs6+srSbpw4YLDcqXPl7eEXGhoqKSSoFpdxcXFWrVqlSTpgQcesIVlSRo3bpxat26t+Ph4p5a9S0hIUFFRkbp27WoXliWpc+fO6tGjhywWi44dO1bt/gIAAKDhcjowt27dWtJ/5zJXZOPGjZIkb29vBQUFlTlvnaZROuRW1eHDh5WXl6fw8HBFRkaWOd+vXz9J0s6dOyuty8PDw6k2AwICqtZJAAAAXBOcnpLRt29fHThwQFu2bFHLli01adKkMrv3bd68WYsXL5Ykde/evdxVME6dOiWp8qkdjlinc5QXliWpbdu2kqTk5ORK64qKipKfn58OHDig+Pj4MlMyfv31V0VERKhTp07V7i8ANHSvvPKKMjMzJUkhISF64YUX6rhHAFB7nA7Mw4cP18qVK5WSkqJFixZp/fr16t69uxo3bmxbh9kaZBs1aqRJkyaVW49105OoqKhqdzo9PV2S1KRJk3LPW1fISEtLq7QuX19f/fGPf9Q//vEPzZ49Wx06dFCTJk2UkZGho0ePqmPHjnrkkUfYoRDAdS0zM9P2314AuN44nQI9PDz0l7/8xTbKkJaWpjVr1pQp5+bmpoceekht2rQpcy4zM1Px8fHy8vJS9+7dq93p/Px8SZKXl1e5563TPazlKtOvXz8999xzeu+993TkyBHbcR8fH3Xv3t2pJeqKiopUVFRke28ymeTj42N7DVyrGsLP931fba7rLjR4V3ILbK/Tcwt0/1xjKx1d7+beP6CuuwA0SHX1O6dKw6bNmzfXnDlztGjRIsXFxSkvL892rlGjRurataumTJmi9u3bl3t9SEiIvvzyS2M9rgE//fSTvvnmG910002aOnWqwsLCdP78ef373//WwoULlZiYqGeffdZhHUuWLLFNR5FKpovMmTNHTZs2dVk/z7qsJsB1IiIi6roLQIPTcO4bfvOgfqmre6fK8wwCAwP1wAMP6Pe//73OnTunnJwceXt7q1mzZhWO+LqadQS5oKCg3PPWkWVnHiw8ePCgvv76a0VGRurJJ5+0zbtu1aqVnnrqKT377LPavXu39uzZo169elVYz8SJEzVu3Djbe+s3oLS0tDLrUQPXkpSUlLruAmqDl1/5r1Et3DdA9bj63nF3d3dqcLPaE3Pd3NzUvHnz6l5uiHVpuoyMjHLPWx9MceYfwLqqR9++fcs8pOjm5qa+ffvq5MmTOnTokMPA7OHhUeGKGxaLpdJ+AA0VP9/Xh0Y33VHXXbimcN8A1VNX947Ty8rVJ9Yl7k6cOFHueev6y61ataq0Lmu4tq4zfTXr8dzc3Cr3EwAAAA1fgwzM0dHR8vX1VWpqqm3L7tK2bdsmSerTp0+ldTVu3FiSKtyYxHr86h0LAQAAcH1okIHZ3d1dY8aMkSR9/vnndqthLFu2TElJSercubNtPWZJWrVqlR5//HHNnz/frq6+fftKkuLi4rRr1y67czt27FBcXJxMJpOtHAAAAK4vDXZx4TvuuEP79+/XkSNH9Nhjjyk6Olrp6elKSEhQYGCgpk+fblc+OztbZ8+eLbO190033aSbb75ZW7du1Zw5c9SuXTs1bdpUaWlpttHlu+66q87mawMAAKBuNdjA7OnpqVmzZmnJkiWKi4vTjh075O/vr6FDh+rOO++scFOTq5lMJj3xxBNat26dNmzYoOTkZJ08eVK+vr7q1auXxo4dq549e9bshwEAAEC91WADs1QSmu+8807deeedlZadOnWqpk6dWu45k8mk4cOHa/jw4a7uIgAAABq4BjmHGQAAAKgtBGYAAADAAQIzAAAA4ACBGQAAAHCAwAwAAAA44LJVMsxms22b6dDQUMPlAAAAgPrAZYH5zJkzevrpp2UymbRgwQLD5QAAAID6wOVTMiwWi0vLAQAAAHWJOcwAAACAAwRmAAAAwAECMwAAAOAAgRkAAABwgMAMAAAAOFCtZeUWL15c5tjFixcdnp88eXJ1mgIAAADqVLUC86JFi6p8nsAMAACAhqhagblTp04ymUx2x/Lz83X8+HFJUufOnY33DAAAAKgHqhWYX3rppTLHTp06paefflqSNGvWLEOdAgAAAOoLHvoDAAAAHCAwAwAAAA4QmAEAAAAHCMwAAACAAwRmAAAAwAECMwAAAOBAtZaVK09YWJhTy8k5Ww4AAACoD1wWmL28vJzasMTZcgAAAEB9wJQMAAAAwAECMwAAAOAAgRkAAABwwKnAvG3bthrtRGZmpo4ePVqjbQAAAADV4VRgfvfdd/XMM89o69atLm08PT1dn332mR599FHt27fPpXUDAAAAruDUKhnh4eFKTk7We++9p9DQUA0aNEiDBg3SDTfcUOUG8/PztX37dm3atEn79u2T2WyWm5ubwsPDq1wXAAAAUNOcCszvvvuuVqxYoaVLlyo9PV0//PCDfvjhB0VERKh9+/Zq166dIiMj1bhxY/n5+cnPz0+FhYXKzc3VpUuXlJKSosTERB07dkyJiYkqLCy01d23b1/dfffdat68eY19SAAAAKC6nArM7u7uGj9+vEaNGqWff/5ZP//8szIyMpSSkqKUlBRt3Lixao26u6tfv3669dZbFRUVVa2OAwAAALWhShuXeHt76ze/+Y1uv/127du3T1u2bNHBgweVlpZW6bUeHh5q3769+vTpo5iYGPn7+1e70wAAAEBtqdZOf25uburZs6d69uwpqWSViyNHjigjI0PZ2dnKzc2Vp6enAgMDFRgYqFatWqlt27Zyd3fZxoIAAABArXBJgg0JCVH//v1dURUAAABQr7BxCQAAAOAAgRkAAABwgMAMAAAAOEBgBgAAABwgMAMAAAAOEJgBAAAABwjMAAAAgAMEZgAAAMABAjMAAADgAIEZAAAAcIDADAAAADjg7qqKzGazDh48qKNHjyorK0sFBQW6++67FRwcbCtTXFysK1euyM3NTR4eHq5qGgAAAKgxLgnMu3bt0pdffqm0tDS74+PHj7cLzGvWrNEXX3whb29v/fOf/5S3t7crmgcAAABqjOEpGf/5z3/01ltv2cJyQEBAhWVHjBghX19f5efna/v27UabBgAAAGqcocCckpKizz//XJLUtWtXvfvuu/rss88qLO/u7q5+/fpJkvbt22ekaQAAAKBWGArMy5cvl9ls1g033KDnnntOLVq0qPSaTp06SZJOnDhhpGkAAACgVhgKzAcOHJAk3XrrrXJ3d246dLNmzSRJGRkZRpoGAAAAaoWhwGwNvW3atHH6Gi8vL0lSQUGBkaYBAACAWmEoMJtMJklVC7+5ubmSJF9fXyNNAwAAALXCUGAOCQmRJKWmpjp9zeHDhyVJ4eHhRpoGAAAAaoWhwNy5c2dJ0oYNG5wqn5eXp19++UVSyaoaAAAAQH1nKDCPHDlSkhQfH6/169c7LJuTk6O3335bWVlZatSoke1aAAAAoD4ztNNfZGSkbr31Vq1YsUKffPKJ9uzZY1tnWZKOHDmikydP6vDhw9q0aZPy8vIkSZMmTVLTpk2N9RwAAACoBYa3xr733ntVVFSkX375RVu3btXWrVtt5z799NMy5W+99VZNmjTJaLMAAABArTAcmE0mkx588EH17dtXP/zwg+Lj42WxWMqU69Chg+644w716tXLaJMAAABArTEcmK26d++u7t276/Llyzpx4oSys7NlNpvl7++vNm3aKDAw0FVNAQAAALXGZYHZysfHx7Z6BgAAANDQGVolAwAAALjWEZgBAAAABwxNyfj444+rfa3JZNL06dONNA8AAADUOEOB2dkd/ipCYAYAAEB9Zygwh4aGVlqmoKBAOTk5tveBgYHy9PQ00iwAAABQawwF5o8++sipcrm5udq0aZMWLlwoX19f/fWvf1Xz5s2NNA0AAADUilp56M/f31+jR4/WK6+8ouzsbL3++uvKzc2tjaYBAAAAQ2p1lYzmzZtr7NixSktL07Jly2qzaQAAAKBaan1ZuW7dukmStm3bVttNAwAAAFVW64HZ29tbkpSenl7bTQMAAABVVuuB+cSJE5Ikd3eX78oNAAAAuFytptbz589r0aJFkqQ2bdoYrq+wsFBLlizR5s2blZ6eLn9/f/Xo0UN33XWXQkJCqtW/H374Qb/++qsuXLggHx8fNWvWTP369dP48eMN9xcAAAANT41vXGKxWJSbm6vjx49rx44dKiwslCSNHDnSSNMqLCzU7NmzlZCQoODgYPXp00dpaWlav369du/erddee03h4eFO17dnzx69++67KiwsVGRkpNq3b6/c3FwlJyfrl19+ITADAABcp+pka+yxY8dqwIABRprW999/r4SEBHXo0EHPP/+8bW70smXLNG/ePH3yySd66aWXnKrrzJkzeuedd+Tj46Pnn39eHTt2tJ0zm822aSQAAAC4/tTalAxfX1916tRJo0ePVo8ePQzVVVxcrFWrVkmSHnjgAVtYlqRx48Zpw4YNio+P1/Hjx9W2bdtK65s3b56Kior01FNP2YVlSXJzc1O7du0M9RcAAAANl6HA/OGHH1ZaxmQyycfHR35+fkaasnP48GHl5eUpPDxckZGRZc7369dPSUlJ2rlzZ6WBOT09XXv37lV4eLhuvPFGl/URAAAA1wZDgblp06au6keVJCUlSVK5YVmSLSQnJydXWld8fLwsFos6dOigK1euaNu2bTpy5IjMZrNuuOEGDRgwQP7+/q7rPAAAABqUBrm2m3UN5yZNmpR73rpCRlpaWqV1nT59WlLJ+tAvvviiEhIS7M4vWLBATz75pLp27eqwnqKiIhUVFdneW0fWra+BaxU/30DVcd8A1VNX906DDMz5+fmSJC8vr3LPW+c0W8s5kpubK0lau3atvL299ec//1k9e/ZUdna2vvvuO8XGxuqdd97Ru+++63CpuiVLlmjx4sW295GRkZozZ45LR+HPuqwmwHUiIiLqugtAg9Nw7ht+86B+qat7p0EGZleyWCySpCtXruihhx6yrd7h7++vRx99VGfPntWxY8f0888/6+67766wnokTJ2rcuHG299ZvQGlpaSouLq7BTwDUrZSUlLruAtDgcN8A1ePqe8fd3d2pwU2nAvMjjzxiuENXM5lM+uCDD6p1rXUEuaCgoNzz1pHl0qtnVFaXt7e3+vfvX+b8sGHDdOzYMcXHxzusx8PDQx4eHuWes4Zy4FrEzzdQddw3QPXU1b3jVGB2Zi5wbQoNDZUkZWRklHs+MzNTknMPJVrLhIaGljsvxno+Ozu7Wn0FAABAw+ZUYI6JianpflRJ69atJanCDUWOHz8uSWrVqlWldVm36L506VK5561znJ0ZrQYAAMC1x6nA/Kc//amm+1El0dHR8vX1VWpqqk6ePGkLvVbbtm2TJPXp06fSujp27KiAgABlZWXp7Nmzat68ud1561SMq9sAAADA9cGtrjtQHe7u7hozZowk6fPPP7dbDWPZsmVKSkpS586d7TYtWbVqlR5//HHNnz/frq5GjRrptttuk8Vi0eeff668vDzbuX379mn9+vUymUwaOXJkDX8qAAAA1EcNdpWMO+64Q/v379eRI0f02GOPKTo6Wunp6UpISFBgYKCmT59uVz47O1tnz57VhQsXytQ1fvx4HTx4UPv379djjz2m9u3bKycnRwkJCTKbzbrrrrsUFRVVWx8NAAAA9UiDDcyenp6aNWuWlixZori4OO3YsUP+/v4aOnSo7rzzzgo3NSmPu7u7nnvuOS1fvlwbN27Ur7/+Knd3d3Xu3Fm33XabevfuXYOfBAAAAPWZyeLi9TnOnz+vnJwcFRYWVrr0R+fOnV3ZdL2UlpZmtwOgESnPPOiSegBXinj7s7ruQqXun7ulrrsA2PnqvrLLmNZHP//IetGoX0aPd+3GJR4eHq5bh7kyZ8+e1ffff69du3bZzQF2xGQyacGCBa5oHgAAAKgxhgPz9u3b9cEHH6iwsLBK17FoOwAAABoCQ4E5PT3dFpZDQkJ0++23y8vLS59++qkk6YUXXlBubq6OHz+ujRs36sKFC4qOjtaUKVPk5tYgF+gAAADAdcZQYF65cqUKCwvl7e2t1157TSEhITp16pTtfNeuXSVJN998syZPnqxPPvlEmzdv1tq1a/XnP//ZWM8BAACAWmBomHf//v2SpNGjRyskJMRhWU9PTz366KOKjIzUpk2btHXrViNNAwAAALXCUGBOS0uTJHXo0MF2zGQy2V5fuXLFvjE3N40dO1aStG7dOiNNAwAAALXCUGC27rAXGhpqO+bp6Wl7Xd6KGTfccIMkKSkpyUjTAAAAQK0wFJh9fX0lyW6FjICAANvr1NTUMtdYQ3ROTo6RpgEAAIBaYSgwN2/eXFLJZiVWPj4+thHnX3/9tcw1+/btk/TfsA0AAADUZ4YCs3Xu8tGjR+2OW7eS/vHHH3XgwAHb8c2bN2vFihWSpI4dOxppGgAAAKgVhpaV69Wrl5YtW6bt27fr/vvvt62tPH78eK1bt075+fl65ZVX5O/vr6KiIhUUFEgqefhv/PjxxnsPAAAA1DBDI8xdunTR5MmTNXToUGVmZtqOh4aG6sknn7RNu8jNzbWFZXd3dz388MN2K2sAAAAA9ZXTI8wrVqzQoEGDFBgYaDtmMpk0ZcqUcsv36tVL//jHP7R161adOnVKZrNZzZo104ABAypdsxkAAACoL5wOzHPnztU333yj7t27KyYmRjfddJPc3R1fHhAQoJEjRxruJAAAAFBXqjSH+cqVK9qzZ4/27NkjX19f9e/fX0OGDFF0dHRN9Q8AAACoU04H5oceekixsbE6fPiwpJL1lNesWaM1a9YoLCxMMTExGjJkiMLCwmqsswAAAEBtczow33LLLbrlllt0/vx5bdy4UXFxcUpJSZFUsg7zokWLtGjRInXs2FFDhgzRgAEDWGsZAAAADV6Vl5ULCwvT5MmTNXnyZCUmJmrjxo3avHmzbee+I0eO6MiRI/rqq6904403KiYmRr169bItOQcAAAA0JIbWYY6KilJUVJTuu+8+7dmzRxs3btSuXbtUXFysoqIibdu2Tdu2bVNgYKAGDhyoIUOGqG3btq7qOwAAAFDjDAVmq0aNGqlPnz7q06eP8vLytGXLFm3cuNE23zk7O1srV67UypUr1bJlS8XExGjQoEEsLwcAAIB6zyWBuTRfX1+NGDFCI0aMUFpamjZu3KjY2FjbfOfTp0/r//7v//Ttt9/q22+/dXXzAAAAgEu5PDCX1rRpU02aNEmTJk3S0aNH9c9//lOnT5+WJJnN5ppsGgAAAHCJGg3MknT8+HHFxsZq06ZNunjxYk03BwAAALhUjQTm9PR0xcXFaePGjTpz5kyZ8126dFFMTExNNA0AAAC4lMsC8+XLl7VlyxbFxsbq0KFDslgsduebN2+uIUOGaPDgwQoNDXVVswAAAECNMhSYzWazbTm53bt3q7Cw0O68v7+/bTm5qKgoQx0FAAAA6kK1ArN1w5ItW7YoOzvbvkJ3d/Xq1UsxMTG68cYb1ahRI5d0FAAAAKgLTgfm8+fPKzY21m6JuNKioqI0ZMgQDRw4UP7+/i7tJAAAAFBXnA7Mjz76aJljoaGhGjx4sIYMGaLmzZu7tGMAAABAfVDlKRne3t7q16+fYmJi1KVLl5roEwAAAFBvOB2Yu3fvrpiYGPXt21eenp412ScAAACg3nA6MM+cObMm+wEAAADUS2513QEAAACgPiMwAwAAAA4QmAEAAAAHCMwAAACAAwRmAAAAwAECMwAAAOAAgRkAAABwgMAMAAAAOGAoMGdlZbmoGwAAAED95PROf+WZPn26evXqpWHDhql3795yc2PAGgAAANcWQ4HZbDZr165d2rVrlwIDAzVkyBANGzZMLVu2dFX/AAAAgDplKDDfdtttio2NVXZ2trKzs7Vs2TItW7ZMUVFRGjZsmAYOHCgfHx9X9RUAAACodYYC87333qtp06Zp9+7dWrt2rfbs2SOz2azExEQlJiZq7ty56tevn4YNG6YuXbq4qs8AAABArTEUmCXJzc1Nffr0UZ8+fXTx4kVt3LhR69ev1+nTp1VYWKjY2FjFxsYqLCxMw4YNU0xMjJo0aeKKvgMAAAA1znBgLq1x48a6/fbbdfvttysxMVHr1q3T5s2blZeXp/Pnz+vf//63Fi1apK5du2r48OG66aab5O7u0i4AAAAALlVjaTUqKkpRUVG6//77tW3bNq1fv14HDhyQ2WzWvn37tG/fPvn7+2vw4MG65ZZbeFAQAAAA9VKNrwPn4eGhDh06qEOHDgoICLA7l5ubq5UrV+qpp57SO++8o/Pnz9d0dwAAAIAqqbER5sLCQm3dulXr1q3ToUOHZLFYbOdatmypwYMHKzk5WTt27FBhYaF27Nih+Ph4vfzyy4w2AwAAoN5weWA+cuSI1q1bpy1btig/P9923MvLS/3799eIESPUoUMH2/G8vDytWLFCS5Ys0aVLl7RgwQI9/fTTru4WAAAAUC0uCcyZmZm21TFSUlLszrVr107Dhw/XoEGD5O3tXeZaX19fTZ48Wf7+/vryyy919OhRV3QJAAAAcAlDgXnz5s3asGGD9u3bJ7PZbDvu5+enwYMHa8SIEWrVqpVTdXXt2lWSdPHiRSNdAgAAAFzKUGD++9//bve+S5cuGj58uPr16ycPD4+qdYTl5QAAAFAPGU6pQUFBGjp0qIYPH67w8PBq1xMWFqYPP/zQaHcAAAAAlzIUmJ955hndeOONcnMzvjqdm5ubmjZtargeAAAAwJUMBeY+ffq4qh8AAABAvVTjG5cAAAAADZmhwJycnKxHHnlEf/7zn5WZmVlp+czMTD366KN69NFHdfbsWSNNAwAAALXCUGDeuHGj0tLS1KxZM4WEhFRaPiQkRM2bN9f58+cVGxtrpGkAAACgVhgKzIcOHZJUtbnM1rIHDhww0jQAAABQKwwFZuu0Cmc3J5GkG264we5aAAAAoD4zFJjz8/MlqdwtrytiLZuXl2ekaQAAAKBWGArM/v7+kqSsrCynr7GW9fHxMdI0AAAAUCsMBeZmzZpJkvbu3ev0NdayRnYFBAAAAGqLocDco0cPSdKaNWt0+vTpSsufOnVKa9askST17NnTSNMAAABArTAUmEeNGiUvLy8VFhbq5Zdf1q5duyosu3PnTr3yyisqLCyUp6enRo8ebaRpAAAAoFYY2ho7MDBQDz30kD788ENdvHhRb731lsLDwxUdHa2goCBJJXOWDx06pPPnz9uue+ihh2znAQAAgPrMUGCWpMGDB8tisehf//qXCgsLlZqaqtTU1HLLenl56cEHH9SQIUOMNgsAAADUCsOBWZKGDBmi7t27a8WKFdqzZ4+Sk5Nt50wmk2644Qb17t1bY8aMYWQZAAAADYpLArMkBQUF6be//a1++9vf6sqVK8rNzZVUsvRco0aNXNUMAAAAUKtcFphLa9SokRo3blwTVQMAAAC1ytAqGQAAAMC1rkZGmGtLYWGhlixZos2bNys9PV3+/v7q0aOH7rrrLoWEhFS73pSUFD399NMqKipSt27d9MILL7iw1wAAAGhInArMs2fPllTyAN+LL75Y5nh1XF1XVRUWFmr27NlKSEhQcHCw+vTpo7S0NK1fv167d+/Wa6+9Vu3dBD/99FMVFxdXu28AAAC4djgVmOPj46t0vDZ8//33SkhIUIcOHfT888/L29tbkrRs2TLNmzdPn3zyiV566aUq17t27VodPHhQt9xyi/7zn/+4uNcAAABoaJwKzJ06dZLJZHL6eE0rLi7WqlWrJEkPPPCALSxL0rhx47RhwwbFx8fr+PHjatu2rdP1ZmVl6euvv1b37t01cOBAAjMAAACcC8wVjdRWZwTXFQ4fPqy8vDyFh4crMjKyzPl+/fopKSlJO3furFJg/uqrr1RYWKgHH3xQGRkZruwyAAAAGqgGuUpGUlKSJJUbliXZQnLpDVQqs3v3bm3evFkTJ05Us2bNjHcSAAAA14QGuUpGenq6JKlJkyblnreukJGWluZUffn5+fr888/VvHlzTZgwoVp9KioqUlFRke29yWSSj4+P7TVwreLnG6g67hugeurq3mmQgTk/P1+S5OXlVe5565xma7nKLFiwQGlpaZo1a5bc3av3T7JkyRItXrzY9j4yMlJz5sxR06ZNq1Vfec66rCbAdSIiIuq6C0CD03DuG37zoH6pq3unQQZmVzp27JhWrlypIUOGqEuXLtWuZ+LEiRo3bpztvfUbUFpaGkvU4ZqWkpJS110AGhzuG6B6XH3vuLu7OzW46VRgvvPOOw136Gomk0kLFiyo1rXWEeSCgoJyz1tHlkuvnlGeK1eu6J///Kf8/Px07733VqsvVh4eHvLw8Cj3nMViMVQ3UJ/x8w1UHfcNUD11de/U2QizkQ8cGhoqSRWuZJGZmSlJlX5jyMjI0MmTJxUUFKR3333X7tylS5ckScePH7etBlJXq4IAAACg7jgVmCdPnlzT/aiS1q1bS5JOnDhR7vnjx49Lklq1auVUfVlZWcrKyir33KVLl+p0gxYAAADULacC85QpU2q6H1USHR0tX19fpaam6uTJk2rTpo3d+W3btkmS+vTp47CesLAwLVy4sNxzBw8e1OzZs9WtWze98MILLuk3AAAAGp4GuQ6zu7u7xowZI0n6/PPP7VbDWLZsmZKSktS5c2e7TUtWrVqlxx9/XPPnz6/1/gIAAKDharCrZNxxxx3av3+/jhw5oscee0zR0dFKT09XQkKCAgMDNX36dLvy2dnZOnv2rC5cuFBHPQYAAEBD5PLAnJWVpVOnTik3N1eS5O/vrxtuuEFBQUEubcfT01OzZs3SkiVLFBcXpx07dsjf319Dhw7VnXfeWeGmJgAAAEBVuCQwWywW/ec//9GqVat0+vTpcsu0bNlSY8eO1YgRI1y2S4unp6fuvPNOp5a9mzp1qqZOnep03V26dKlwfjMAAACuH4YDc25urt566y0dOXLEYbnTp0/rX//6lzZu3Ki//vWv8vPzM9o0AAAAUOMMBWaLxaK3337bFpb9/f3Vv39/tW/f3jYFIysrS4mJidqyZYtycnJ05MgRvfXWW5o9e7bhzgMAAAA1zVBgjouL0+HDhyVJgwYN0oMPPigfH58y5WJiYvS73/1On332mWJjY3X48GHFxcVp0KBBRpoHAAAAapyhZeXi4uIkSZ07d9ajjz5abli28vb21iOPPKLOnTtLkmJjY400DQAAANQKQ4HZutOedU1kZ1jLnjx50kjTAAAAQK0wFJitS8eFhYU5fY21rPVaAAAAoD4zFJh9fX0lqUqbgVjLOpq+AQAAANQXhgLzDTfcIElat26d09esX7/e7loAAACgPjMUmG+++WZJ0vbt27Vw4UJZLBaH5RcvXqxt27ZJkvr372+kaQAAAKBWGFpWbsSIEVq1apXOnj2r7777Ttu3b9fQoUMVFRWlxo0by2Qy2dZh3rBhg5KTkyVJLVq00IgRI1zyAQAAAICaZCgwu7u767nnntMrr7yi8+fP69SpU/r6668dXhMWFqb/+Z//UaNGjYw0DQAAANQKQ1MypJIA/Pbbb2vcuHG2hwDL4+vrq9tvv11vv/22QkNDjTYLAAAA1ApDI8xW3t7euueee3T33Xfr+PHjSk5Oti0b5+/vr1atWqlt27Zyd3dJcwAAAECtcWmCdXd3V4cOHdShQwdXVgsAAADUmSoH5gsXLmjZsmXau3ev0tPTZTabFRwcrC5duui2225Ty5Yta6KfAAAAQJ2oUmA+fPiw3nrrLV26dMnueGpqqlJTU7Vhwwb98Y9/1JAhQ1zaSQAAAKCuOP3Q36VLl/Tee+/ZhWV/f38FBQXZ3l+5ckX//Oc/bcvHAQAAAA2d0yPMa9euVVZWlqSSDUt+97vfKSwsTJKUnZ2tpUuXatmyZSouLtZPP/2kGTNm1EiHAQAAgNrkdGDeu3evJKlTp0564okn7M4FBgbqnnvu0eXLl7VmzRpbWQAAAKChc3pKhnWaxahRoyosM3bsWEklI87Z2dkGuwYAAADUvSrNYZak5s2bV1gmIiKiTHkAAACgIXM6MF+5ckWSHG4+UvqctTwAAADQkBneGhsAAAC4lhGYAQAAAAeqvNPfxx9/LC8vL8PlTCaTXnzxxao2DwAAANSqKgfmY8eOubQcAAAAUJ8xJQMAAABwwOkR5n//+9812Q8AAACgXmKEGQAAAHCAwAwAAAA4QGAGAAAAHCAwAwAAAA4QmAEAAAAHCMwAAACAAwRmAAAAwAECMwAAAOAAgRkAAABwgMAMAAAAOEBgBgAAABwgMAMAAAAOEJgBAAAAB9ydKfTII4+4vGGTyaQPPvjA5fUCAAAAruRUYE5LS6vpfgAAAAD1klOBOSYmxuH5kydPKikpSZLk6+uryMhINW7cWJJ08eJFnTx5UpcuXZIktWnTRq1btzbSZwAAAKDWOBWY//SnP1V4bu3atYqLi1NISIjuvfde9e3bV40aNbIrYzabtW3bNn3zzTc6ffq0Ro8ereHDhxvrOQAAAFALDD30d+zYMf3rX/9SQECAXnvtNfXv379MWJYkNzc39e/fX6+++qr8/f312Wef6dixY0aaBgAAAGqFocC8fPlymc1mTZw4USEhIZWWDw4O1sSJE3XlyhUtW7bMSNMAAABArTAUmA8dOiRJat++vdPXWMsePnzYSNMAAABArTAUmLOzsyVJRUVFTl9jLWu9FgAAAKjPDAXmwMBASdLevXudvmbPnj121wIAAAD1maHA3LVrV0nSsmXLnJpiceTIES1fvtzuWgAAAKA+c2pZuYpMmDBBmzdvVlFRkV555RWNHDlSQ4cOVevWrWUymSRJFotFSUlJWr9+vX755RcVFxfL3d1dEyZMcEX/AQAAgBplKDC3aNFCM2bM0AcffKDi4mKtXLlSK1eulLu7u/z9/WUymZSTk6Pi4mLbNW5ubvrTn/6kFi1aGO48AAAAUNMMBWZJGjBggMLCwvT555/r+PHjkqTi4mJlZWWVKRsZGakHH3xQUVFRRpsFAAAAaoXhwCxJUVFReuONN3Ts2DHt379fycnJys3NlST5+fmpVatW6tatG0EZAAAADY5LArNVu3bt1K5dO1dWCQAAANQpQ6tkAAAAANc6l44wS1JGRoaysrJUUFCgqKgoeXp6uroJAAAAoNa4JDBfvnxZS5cu1YYNG5SZmWk7/re//U0tW7a0vd+0aZO2bdsmX19f/fGPf3RF0wAAAECNMhyYU1JS9MYbbyg1NbXSsu3bt9cHH3wgi8WioUOHKjo62mjzAAAAQI0yNIe5sLBQb775plJTU+Xl5aXx48frr3/9a4Xlw8LC1KVLF0nSzp07jTQNAAAA1ApDI8yrV6/WuXPn5OXlpZdffllt2rSp9JpevXrpwIEDOnr0qJGmAQAAgFphaIR5+/btkqRbb73VqbAsSa1bt5ZUMpUDAAAAqO8MBeYzZ85Ikrp37+70NQEBAZKkvLw8I00DAAAAtcJQYM7Pz5ckeXt7O31NUVGRJKlRo0ZGmgYAAABqhaHA7O/vL0lKS0tz+ppTp05JkoKCgow0DQAAANQKQ4E5MjJSknTo0CGnr9m4caMkqUOHDkaaBgAAAGqFocB88803S5L+85//KD09vdLyy5cvt4XrgQMHGmkaAAAAqBWGAvOQIUPUqlUrFRUV6aWXXtKePXtksVjsylgsFiUmJuof//iH5s2bJ0nq1KmTevXqZaRpAAAAoFYYWofZzc1Nf/3rX/XCCy8oLS1Nb775pry8vGznZ8+ercuXL9se9JOk8PBwPfHEE0aaBQAAAGqNoRFmSQoNDdXbb7+tgQMHys3NTQUFBbZz2dnZdmG5f//+ev3119W4cWOjzQIAAAC1wtAIs5W/v7/+/Oc/6+6779bu3bt17NgxZWdny2w2y9/fX5GRkerdu7eaN2/uiuYAAACAWuOSwGzVtGlTjR492pVVAgAAAHXKUGC2rowREhIiNzfnZneYzWZlZmZKKpnOYURhYaGWLFmizZs3Kz09Xf7+/urRo4fuuusuhYSEOFXHpUuXtGfPHu3cuVMJCQnKzMyUh4eHWrZsqUGDBmnUqFFyd3fp9woAAAA0IIaS4IwZM2QymfTOO++oZcuWTl1z/vx5PfbYYzKZTFqwYEG12y4sLNTs2bOVkJCg4OBg9enTR2lpaVq/fr12796t1157TeHh4ZXW89NPP+n777+XyWRSmzZt1L59e2VnZ+vIkSNKTEzU1q1bNXPmTLuHGQEAAHD9MDx0evUycjV9ndX333+vhIQEdejQQc8//7xte+5ly5Zp3rx5+uSTT/TSSy9VWo+Xl5fGjx+vMWPG2I14p6Sk6JVXXtHhw4f13Xff6be//a2h/gIAAKBhMrxKRrUbdnIKR3mKi4u1atUqSdIDDzxgC8uSNG7cOLVu3Vrx8fE6fvx4pXVNnDhR06ZNKzM9JCIiwhaSN23aVO2+AgAAoGGr9cB84cIFSbILuVV1+PBh5eXlKTw83LY9d2n9+vWTJO3cubPabUhSmzZtJP23zwAAALj+1NrTbMXFxUpNTdX3338vSYaWmEtKSpKkcsOyJLVt21aSlJycXO02JCk1NVWSFBQUZKgeAAAANFxVCsx33nlnucefeuqpKjd88803V/kaK+vqHE2aNCn3vHWFjLS0tGq3IUkrVqyQJPXp06fSskVFRXabtJhMJvn4+NheA9cqfr6BquO+Aaqnru6dOlkvrX///rrtttuqfX1+fr4kVbhyhXW6h7VcdaxevVr79++Xn5+fJkyYUGn5JUuWaPHixbb3kZGRmjNnjpo2bVrtPlztrMtqAlwnIiKirrsANDgN577hNw/ql7q6d6oUmCdPnmz33hoQR44c6XC7a5PJJA8PDwUHB6tDhw5q1qxZNbpaew4dOqSvvvpKJpNJ06dPd2pN54kTJ2rcuHG299ZvQGlpaSouLq6xvgJ1LSUlpa67ADQ43DdA9bj63nF3d3dqcLNKgXnKlCl2762BecyYMU6vw+wK1hHkgoKCcs9bR5ar82BhcnKy3nrrLRUXF+v3v/+9+vbt69R1Hh4e8vDwKPec0SX0gPqMn2+g6rhvgOqpq3vH0JSM6dOnS5LTu+q5inUJuIyMjHLPW3cSrOp0iPPnz+u1117TpUuXNGXKFI0dO9ZYRwEAANDgGQrMQ4cOdVE3qqZ169aSpBMnTpR73rr+cqtWrZyu88KFC3rllVd04cIF3XrrrWVG0wEAAHB9qrONS4yIjo6Wr6+vUlNTdfLkyTLnt23bJsm51S0kKTc3V6+99ppSU1M1dOhQ3Xfffa7sLgAAABowl66SkZubq6SkJOXk5KiwsLDSeSYxMTHVasfd3V1jxozR999/r88//1wzZ8602xo7KSlJnTt3tq3HLEmrVq3SqlWr1LdvX7ttrgsKCvTmm28qOTlZ/fv31x//+EeW+wEAAICNSwLzwYMHtXDhQh0+fNjpa0wmU7UDsyTdcccd2r9/v44cOaLHHntM0dHRSk9PV0JCggIDA23zq62ys7N19uzZMrv2ffvttzp69Kjc3NzUqFEjffLJJ+W2N2PGjGr3FQAAAA2X4cC8evVqffHFF1V+atHoU46enp6aNWuWlixZori4OO3YsUP+/v4aOnSo7rzzzgo3NbnapUuXJElms1lxcXEVliMwAwAAXJ9MFgPJ9fTp03rmmWdkNpvVqlUr3XnnnWrUqJHefPNNSdIHH3yg3NxcHTt2TGvWrNGJEycUHR2thx56SF5eXi7d1KO+SktLs9sB0IiUZx50ST2AK0W8/Vldd6FS98/dUtddAOx8dV//uu6CU37+kfWiUb+MHu/ajUs8PDycyqOGHvpbtWqVzGazAgMD9fLLL6tPnz62Jd8kKSwsTG3bttXIkSP1xhtv6Pbbb9fhw4f15ZdfXhdhGQAAAA2focAcHx8vSRo7dqx8fHwcljWZTJo2bZq6du2qAwcOaO3atUaaBgAAAGqFocBs3TgkMjLSdqz0ChPlbQk9YsQISVJsbKyRpgEAAIBaYSgwW+fmBgcH2455eXnZXlsfqCutWbNmkkrmPwMAAAD1naHA7O/vL6lkLWOrwMBA2+uUlLIPC+Tk5EiS8vLyjDQNAAAA1ApDgblFixaS7IOxl5eXIiJKnmDcuXNnmWu2b98uyT5YAwAAAPWVocDcsWNHSSqzYUnfvn0lSStXrtS6deuUn5+vixcvaunSpbaH/bp06WKkaQAAAKBWGArMvXv3liTt2LFDhYWFtuO33367/P39VVxcrP/93//Vfffdp//3//6f5s+fL7PZLE9PT02YMMFQxwEAAIDaYCgwt2/fXtOnT9fvfvc7uwf8AgICNHPmTIWFhZW5JjAwUE8//bRatmxppGkAAACgVhjeGnvo0KHlHm/btq3ee+89HThwQKdPn9aVK1cUERGhHj162K2kAQAAANRnhgOzw8rd3dWzZ0/17NmzJpsBAAAAaoyhKRkAAADAtY7ADAAAADjgsikZOTk5Onr0qFJTU5Wfny+z2VzpNZMnT3ZV8wAAAECNMByYs7KyNHfuXG3btk1Xrlyp0rUEZgAAANR3hgJzdna2nn/+eaWlpbmqPwAAAEC9YigwL1y40BaWb775Zo0aNUqtW7eWn5+fTCaTSzoIAAAA1CVDgXnXrl2SpCFDhmjGjBku6RAAAABQnxhaJSM7O1uSNGzYMJd0BgAAAKhvDAXmkJAQSZK3t7dLOgMAAADUN4YCc6dOnSRJycnJLukMAAAAUN8YCsy333673N3d9dNPP6mwsNBVfQIAAADqDUOB+YYbbtD06dN19uxZvfbaazp79qyr+gUAAADUC4Y3Lhk0aJAiIiL05ptv6sknn1Tr1q0VEREhT09Ph9eZTCZNnz7daPMAAABAjTIcmM+ePat58+bZVsw4efKkTp486dS1BGYAAADUd4YCc3p6umbNmmULy1LJihlsXAIAAIBrhaHAvHjxYmVnZ8tkMmncuHEaNWqUwsLCXNU3AAAAoM4ZCsz79++XJN16662aNm2aSzoEAAAA1CeGVsm4ePGiJKlfv34u6QwAAABQ3xgKzMHBwZIkd3fDzw4CAAAA9ZKhwNytWzdJ0rFjx1zSGQAAAKC+MRSYx48fL29vby1dulS5ubmu6hMAAABQbxgKzM2aNdPTTz+ty5cv64UXXtC+fftc1S8AAACgXjA0+Xj27NmSpICAANv22H5+fk7v9Pfiiy8aaR4AAACocYYCc3x8fJljly5dUmJiopFqAQAAgHrDUGDu1KkTO/oBAADgmmYoML/00ksu6gYAAABQPxl66A8AAAC41hGYAQAAAAcIzAAAAIADTs1hTk9Pt70ODQ0t93h1lK4LAAAAqI+cCswzZsyQVLJ28oIFC8ocr46r6wIAAADqoypNybBYLC5r2JV1AQAAADXFqRHm6dOnV+k4AAAAcK1wKjAPHTq0SscBAACAawWrZAAAAAAOGNrpb8OGDZKkm266Sb6+vk5dk5+fr23btkmSYmJijDQPAAAA1DhDgfnjjz+WJP3tb39zOjBnZWXp448/lslkIjADAACg3quzKRmskgEAAICGoNYDs9lsliQ1atSotpsGAAAAqqzWA/PZs2clSf7+/rXdNAAAAFBlVZrDHB8fX+7xxMREZWdnO7y2uLhY586d008//SRJatOmTVWaBgAAAOpElQLz7Nmzyz3+ySefVLnhW265pcrXAAAAALXN0CoZ1RESEqKJEyeqb9++td00AAAAUGVVCsyzZs2yvbZYLHr55ZcllWyRHRYW5vBaT09PBQUFKTQ0tBrdBAAAAOpGlQJz586dyz0eFRWlli1buqRDAAAAQH1iaErGhx9+KKlkmgUAAABwLTIUmJs2beqqfgAAAAD1Uo0/9Ldz505t2bJFOTk5CgsL0/Dhw9W2bduabhYAAABwCUOB+cCBA3r//ffl4eGhd955R35+fnbnFyxYoCVLltgdW7NmjaZPn64hQ4YYaRoAAACoFYZ2+tuzZ49ycnLUrl27MmE5KSnJLixbz5vNZn366ac6f/68kaYBAACAWmEoMB8+fFiS1L179zLnVq9eLakkKL/55pv64osv9Prrr8vf319FRUX65ZdfjDQNAAAA1ApDgTkrK0uSyl1Sbvfu3ZKkMWPGKDIyUpLUrl07jR49WpK0f/9+I00DAAAAtcJQYM7OzpakMtMxzp07p8zMTEkqs6Nfp06dJEmpqalGmgYAAABqhaHAbLFYJEl5eXl2x61TNXx9fdWmTRu7cwEBAZKkgoICI00DAAAAtcJQYA4KCpIknTlzxu743r17JUkdO3Ysc01+fr6ksqPSAAAAQH1kKDC3b99eUskDftYR49TUVO3cuVNS+Q8DpqSkSPpv2AYAAADqM0PrMI8YMUKbN29WUlKSnnrqKUVGRurQoUMqKiqSp6enBg0aVOaaQ4cOSZIiIiKMNA0AAADUCkMjzF27dtXYsWMlSWlpadq+fbtycnIkSffcc48CAwPtyhcWFmrHjh2SpM6dOxtpGgAAAKgVhrfGvv/++9WtWzdt2bJFFy9eVFBQkGJiYtS1a9cyZXfu3ClfX1/5+vqqd+/eRpsGAAAAapzhwCxJvXv3dioADxgwQAMGDHBFkwAAAECtMDQlAwAAALjWEZgBAAAAB5yekjF37lxJ0oQJE9S4ceMy581ms213v9DQ0ArrSU1N1bvvvitJmjNnTpU6e7XCwkItWbJEmzdvVnp6uvz9/dWjRw/dddddCgkJqVJdubm5WrRokXbs2KGsrCwFBQWpb9++mjJlCmtGAwAAXMecHmFesWKFVqxYYVsF42pnzpzRjBkz9Mgjjzisp7CwUCdPntTJkyer1NHy6pk9e7a+++475efnq0+fPmrSpInWr1+vv/zlL1Xaejs7O1v/8z//o5UrV6pRo0a66aab5OPjoxUrVuh//ud/lJuba6ivAAAAaLhc8tBfadbtsmva999/r4SEBHXo0EHPP/+8vL29JUnLli3TvHnz9Mknn+ill15yqq6vvvpK586dU9++ffXEE0+oUaNGkqQvvvhCq1at0ty5czVjxoya+igAAACoxxrkHObi4mKtWrVKkvTAAw/YwrIkjRs3Tq1bt1Z8fLyOHz9eaV0XLlzQpk2b5O7urgcffNAWlqX/riUdGxurixcvuv6DAAAAoN5rkIH58OHDysvLU3h4uCIjI8uc79evnyTZtuh2ZO/evbJYLOrUqVOZ7bo9PDzUu3dvmc1m7dmzxyV9BwAAQMPSIANzUlKSJJUbliWpbdu2kqTk5GTDdVmPW8sBAADg+tIgA3N6erokqUmTJuWet66QkZaW5nRdFa2qYW3DmboAAABw7XH5Q3+1IT8/X5Lk5eVV7nnrnGZrOSN1WY9XVldRUZGKiops700mk3x8fOTu7rp/Yp827VxWF+AqHh4edd2FSkWFB9V1FwA7DeG+kaQmoT513QXAjqvvHWdzWoMMzPXRkiVLtHjxYtv7gQMH6rHHHlNwcLDL2mj62gcuqwu4nnz8hxF13QWgQRo/uWlddwGoFxrklAzrCHJBQUG5562jwaVXz6huXdbjldU1ceJEffXVV7b/PfTQQ3Yjzqg/Ll++rL/+9a+6fPlyXXcFaFC4d4Cq4765NlR5hPnnn38ud6e/0suulR5pdVSuuqw7CWZkZJR73rrjYNOmlX8zttZlveZq1jYqq8vDw6PB/IntemexWHTixIlaWzMcuFZw7wBVx31zbahyYF69enWlZRYtWlStzjirdevWkqQTJ06Ue966/nKrVq0M12U9bi0HAACA60uDnJIRHR0tX19fpaamlrvF9rZt2yRJffr0qbSunj17ymQy6dChQ2VGv4uKirRr1y65ubmpV69eLuk7AAAAGhanR5hnzZpVk/2oEnd3d40ZM0bff/+9Pv/8c82cOdNua+ykpCR17tzZth6zJK1atUqrVq1S37599dvf/tZ2PDg4WAMHDlRcXJw+++wzPf7447bd/r755htlZ2crJiam3GkoaJg8PDw0efJkptAAVcS9A1Qd9821wWRpoJNqCgsLNXv2bCUkJCg4OFjR0dFKT09XQkKCAgMD9dprryk8PNxWfuHChVq8eLFiYmI0Y8YMu7qys7M1c+ZMpaamKjw8XO3atdOpU6d06tQpRURE6LXXXpO/v39tf0QAAADUAw1ySoYkeXp6atasWZo0aZI8PT21Y8cOpaWlaejQoZozZ45dWK5MYGCg3njjDY0ZM0bFxcXavn278vLyNHbsWL3++uuEZQAAgOtYgx1hBgAAAGpDgx1hBgAAAGoDO/2hXlu/fr0+/vhjSSUPnnbp0sV2zjov/Wpubm7y8/NTixYt1KtXL40cOdJuWs2BAwf08ssvS5JmzJihmJiYCtt/7rnndOzYsSqVjYiI0N///veqfVCgBpW+j0rz8PCQr6+vAgIC1KpVK0VFRenmm2+2rU9fnvPnz+uRRx6xvQ8KCtL//u//ys3N8fjL/Pnz9cMPP9jeT548WVOnTq36hwHqQE3eQ6V5eXkpODhYUVFRGjZsmLp16+ayzwBjGGHGNcdsNisnJ0eHDx/Wt99+qyeffFKJiYm28x06dLCthHLo0KEK68nPz7dbn9vZsp06dTL6EYBaUVRUpIsXL+r06dPavHmz5s2bp0ceeUTvvPNOhZs5XS0rK0v79u1zWMZisSg2NtYVXQbqFVfcQ6UVFBTo3LlziouL0yuvvKJPPvmEDU/qCUaYcU149913bd/or1y5ovPnz2vVqlVat26dsrKyNGfOHP3973+Xr6+vPD091a5dOx09elSHDx+usM6jR4/KbDbLw8NDRUVFTpWVpM6dO7v2wwEu9Nxzz9m+1JnNZl26dEkZGRk6fPiwNmzYoDNnzmj79u2Kj4/Xs88+qw4dOlRYl4+Pjy5fvqyNGzeqZ8+eFZY7ePCgMjIybOWBhsyV99CECRN0xx13SCr5YpmRkaH4+HgtWLBAubm5WrdunVq2bKnbb7+9Vj4bKsYIM64Jnp6e8vb2lre3t/z8/BQZGanp06dr6NChkkq2ZP/Pf/5jKx8dHS1JOnv2bIXbtVtHlPv37y9PT0+nykqMMKN+K32v+Pr6qmnTpoqOjtaECRP07rvv6t5775Wbm5tyc3P19ttvKyMjo8K6br75ZknSjh07lJ+fX2G5jRs3Siq5l4CGzpX3kLu7u60uHx8ftWzZUqNGjdLzzz8vk8kkqWR/CdQ9AjOuaZMnT7a93r9/v+116VHgiqZaWEeUu3btqnbt2jlVNjQ0VE2bNjXWaaCOmEwmjRs3zra508WLF7Vo0aIKy0dHRyssLEwFBQW2HVavVvrckCFDXN9poB6p6j1UkbZt26p79+6SpAsXLujcuXMu7SeqjsCMa1pYWJhtF8jS3/Kjo6Nt397Lm2pRXFyshIQEW9mOHTs6XRZo6G6//XZFRERIKhkdzs7OLrecyWTS4MGDbeXKs2PHDl2+fFlNmzblry+4bjh7DznSvHlz2+vqXA/XIjDjmmcNxqUfnPD19VWrVq0klT9qfPz4cRUWFio4OFjNmjWzBWFHZSXmL+PaYDKZNGzYMEklXwjj4+MrLGsNzAcOHCj3IacNGzbYylnvReBaV5V7qCKlf2f5+fm5rG+oHgIzrmmZmZm2h4xCQkLszllHu5KSkpSXl2d3zhqMrSPLHTt2lMlkcli2dJ1AQ2f92ZdKHmqtSPPmzdW+fftyV8LIysqyTYViOgauN87eQxU5c+aMpJKl5pjqV/cIzLimff/997bXpddwlmT3lPPV/zGzhmDryLKfn59atmzpsGzjxo3VokUL134AoI5Y/5wslQRfRyqalhEbGyuz2ax27drZ/XkZuB5U5R662vHjx3XgwAFJ0rBhw+Tp6enKrqEaCMy45pjNZp07d05z587VL7/8Iqkk8N5yyy125UqPBpceJbZYLDpy5EiZMuVNyyhdlvnLuJaU/hNwbm6uw7IDBw5Uo0aNdOrUKZ08edJ23BqgHW34A1yrnLmHiouLlZ+fr/z8fF2+fFmnT5/W6tWr9eqrr8pisSg6Otr2ACHqFusw45pQ0a5JUsl85SeffFKBgYF2x4OCghQREaGUlBS7EHzq1CldunRJPj4+at26te14dHS0fvnll3LLSkzHwLWl9PzJyuYeBwQEqFevXtq5c6c2btyoNm3aKDk5WUlJSWrUqJEGDBhQ090F6h1n7qEffvjBbgfM0u6++26NHz/ettEW6hYjzLgmubu7q1WrVho/frzefffdCrcXtYbcY8eOqaioSNJ/R5A7dOhgt92vdT5aeWVL1wVcC0rP1XfmgSPrHOW4uDiZzWbbw349e/Ys82UVuB5U9R662g8//GBbgQl1jxFmXBNK7/Tn5ubm9HyvTp06ae3atSoqKlJiYqI6depUZv6yVVhYmEJCQpSZmVmmrK+vr91oNNDQpaSk2F4HBQVVWr53797y8/NTVlaW9u7dq7i4OElMx8D1y5l7aPLkyZo6daqkkm2209LStHbtWv3000+6fPmy/va3v+mdd95R48aNa6PLcIARZlwTSu+8VJWHI8qbx2xda7m8OclXz2O2lu3YsaPdaDTQ0Fnn5kv2T/tXxMPDw7aT35dffqkLFy7Iz89PvXv3rrE+AvVZde6h5s2ba9q0abrvvvsklWx88n//93811kc4j9/wuK6FhYWpSZMmkkpC8Pnz55WZmalGjRqpffv2Zcpb/6NXuqzEdAxcWywWi9atWyepZHqTsz/f1mkZqampkkq2zvbw8KiZTgL1WHXvIauxY8faft9s3LhRZ8+edXkfUTUEZlz3rP8hO3r0qA4ePCipZFvS8kaqrSPMpcuWrgO4Fixfvtz25+SYmBin5yBHR0crPDzc9p7pGLheVfcesjKZTLr77rsllaz8VHqJVNQN5jDjuhcdHa24uDhdvnxZq1atsh0rT+vWreXj42NX1tPTU+3atau1/gI1xWKxaOXKlbY/AQcFBWnKlClVqmP27Nm2zYJYexnXG1fcQ1adO3dWly5ddPDgQW3atElTp05VWFiYK7uLKiAw47pXejvrEydOSKp4xNjNzU3t27fXvn37bGXbt28vd3duJTQMhYWFys/Pl1QycpWXl6f09HQdPXpU69ats+0uFhAQoGeeeabMDpmVqWp5oKGp6XuotKlTp2rWrFm6cuWKlixZoocfftglnwFVx295XPdatGihgIAA5eTkSCr5U5ijBzSio6O1b98+23umY6AheeONNxyed3Nz00033aQ//OEPCg4OrqVeAQ1Hbd5DnTp1UteuXXXgwAFt2LBBkydPtj13g9pFYMZ1z2QyKTo6Wjt27JBU8mfkgICACstfPV2j9Ag10JB4eHjIx8dHAQEBat26taKionTzzTfblmgE4Fht3ENTp07VgQMHVFxcrKVLl+oPf/iDy+qG80yW0lvRAAAAALDDKhkAAACAAwRmAAAAwAECMwAAAOAAgRkAAABwgMAMAAAAOEBgBgAAABwgMAMAAAAOEJgBAAAABwjMAAAAgAMEZgAAAMABAjMAAADggHtddwAA0HC99NJLio+PlyT96U9/0tChQ+u2QwBQAwjMAKrFbDbr4MGD2rVrl44cOaKsrCxlZ2fL09NTQUFBCg8PV69evdSnTx81adLE6XrPnz+vRx55pMLzJpNJ3t7e8vf3V5s2bdS1a1cNGTJEfn5+1a6zMk2bNtVHH31U7esl6eDBg5o9e3a1r+/cubNeeuklQ30AAFQPgRlAle3Zs0dff/21Tp8+XeZcUVGRLl26pDNnzmj37t366quvNGrUKE2ZMkX+/v6G27ZYLLp8+bIuX76stLQ07dixQ99++63uuecejRw50nD916uFCxdq8eLFkqSYmBjNmDGjjnvUMJT+IuSKL1YA6icCMwCnWSwWffXVV1q5cqXd8UaNGiksLEzBwcHKz89Xenq6srOzJUlXrlzRypUrtXXrVs2cOVOtWrWqUpvt2rUrE7Rzc3OVnJysoqIiSVJ+fr7+9a9/6eLFi5o8eXK16nSkcePGVeqzMzp16iRPT0+ny7du3drlfQAAOIfADMApFotFH3zwgeLi4mzHAgICNGXKFA0YMECBgYF2ZRMSEvTjjz9q+/btkqQLFy5o1qxZmjlzpqKiopxud9q0aerSpUuZ4wUFBVq1apUWLFigK1euSJIWLVqknj17Vlp/RXXWphkzZigsLKxO++AKTBMBcD1glQwATlmxYoVdWI6KitJ7772nMWPG2IVlqWSecYcOHfT000/rkUcekZtbyX9qLl26pPfee095eXmG++Pl5aXf/OY3+tOf/mQ7ZrFYtGTJEsN1AwBQGoEZQKXOnDmj+fPn2963aNFCM2fOLBOUyzNkyBA9+OCDtvdpaWn66quvXNa3wYMHq23btrb3+/fvV3FxscvqBwCAKRkAKvXjjz/a5gubTCY9/PDDDleluNott9yibdu26ddff5Ukbdy4UVOmTFHTpk1d0r+ePXvq+PHjkkrmM6elpSkiIsIldTcU+/btU1xcnI4dO6aMjAzl5+fLw8ND/v7+Cg8PV2RkpLp06aIePXrIw8PDdl3pZeGsNmzYoA0bNpTbzocffmg3lcSZZeUqejDu1KlT+uWXX7R//35lZGTIYrGoefPmGjRokMaMGWPXT0nKzs7WqlWrtGvXLqWkpOjKlSsKDQ1Vr169NGHCBAUFBTn1b3X8+HHt27dPhw8f1pkzZ5SVlaXi4mL5+/srNDRUnTp10rBhw3TDDTdUWEfphySt0tLSNHXq1HLLO1pyr7CwUBs2bNDOnTt16tQpXbx4UR4eHgoKClKnTp00YMAAdevWrdLPtX79en388ceS7FdVSUxM1Pr163Xo0CFlZmbq0qVL5a66cvnyZcXGxmrXrl1KTk5WTk6Orly5Ii8vLwUHBysiIkLt27dX7969q/wsAtDQEZgBOJSdnW03FaNnz56Kjo6ucj133323LTCbzWatXLlS9957r0v6GBoaavc+JyfnugnMly5d0vvvv2/7ty2toKBABQUFysjIUHx8vJYvX65Ro0bZjfjXleXLl+ubb76xzT+3OnHihE6cOKGtW7fq+eefl4+PjyRp7969+sc//qHc3Fy78ikpKUpJSdGGDRsqnR+fm5urmTNnKiUlpdzzFy9e1MWLF3Xs2DEtX75cI0eO1P333y9395r7Vfnrr7/qn//8p9LT0+2OFxUVKS8vT2fPntWaNWvUs2dPzZgxo0oPoBYXF+ubb77RihUrKi174MABffDBB7pw4UKZc3l5ecrLy9OZM2e0c+dOffvtt3r99der9CwC0NARmAE4tHfvXtvosiQNGzasWvW0bdtWrVq1UnJysiRp586dLgvMV0/BqMmAU5+YzWbNmTNHhw8fth3z8PBQixYtFBAQoOLiYuXk5NhGY63XlBYVFSUPDw+dO3dOqampkqTg4OAKRxCrsrJHRVavXq25c+dKknx9fdWyZUs1atRISUlJtvntCQkJevfddzVz5kwdPHhQb731loqLi9WoUSO1atVKvr6+OnfunDIyMiSVfHGYM2eO3n///Qr/+lFQUGAXlj09PRURESE/Pz+ZTCZlZmbq3LlzslgsslgsWr16tbKzs/Xkk0+WqatZs2bq0aOHcnNzdezYMUkl//adO3cut+2QkJAyx7Zt26b333/f7ktD48aN1bx5cxUWFtqtBLN37169+OKLmjVrVrl1leerr77S6tWrJUne3t5q2bKlPDw8dP78ebtySUlJeuONN+zu88DAQEVERMjT01P5+fnKyMhQZmam7fzVP0fAte76+K0CoNpKhzGTyaTu3btXu67u3bvbAvO5c+d08eJFlyzZdvbsWbv3NbEMXH20fft22/8/7u7umjZtmkaMGCEvLy+7csXFxYqPj1dcXFyZwDtt2jRJ9lMMunfvXmPrMGdnZ2vu3Lny8vLS/fffr5iYGNsXnKKiIn377bdatmyZpJLR1x07duizzz5TcXGxbrvtNk2aNMluScDY2Fh9/PHHunLlii5evKiffvpJd911V4XtBwUFafjw4erbt6/atGljeyDVKjMzU8uWLdPy5ctlsVi0detWxcXFadCgQXblhgwZoiFDhthNNwkKCtLMmTOd+nc4f/68PvroI1tYDgoK0oMPPqg+ffrY+pSXl6cffvhBS5culcViUUpKij766CM9//zzMplMDus/ceKE4uPj5e3trWnTpmnYsGF2U1ysX44k6dtvv7WF5RYtWujhhx8u969IFy9e1K5du7RmzRqnPiNwLSEwA3DoxIkTttfNmjWTr69vtesq/XCeJB07dkw33nhjteuTSsKgdek6qWQkryo7CzZke/bssb0eP368br311nLLubu7q3v37urevXudjwwWFBTIzc1NM2fOLDMa6+HhoXvvvVcnT57UgQMHJEnvvfeeiouLddddd+mOO+4oU9/gwYOVkpJiC/sbN26sMDA3btxYH3/8scO/QISEhOjee+9VaGio7eHU5cuXlwnMRs2bN0/5+fmSSkbZZ82apRYtWtiV8fX11W9/+1sFBATo66+/llTyUOuWLVs0YMAAh/VfvnxZbm5uevbZZ8sd9Q4PD5dUsk76/v37JZV8If7LX/5S4XSmxo0ba/jw4Ro+fHid/xwBtY3ADMChixcv2l4bfUjv6rnG1s1NjJg3b57dn4r79u1b6TVV3aLa0QNb1VXVrbpnzZpVZu3o0p/b2XnlV4+o1oURI0ZUOHXBet4amIuLi3XDDTdowoQJFZa/5ZZb9N1338lisSg9PV0ZGRnlfmmqylSdsWPHatmyZUpPT9exY8d04cIFBQcHO329I+np6dq5c6ft/dSpU8uE5dLGjRunrVu3KiEhQZK0atWqSgOzJI0cOdLhv7NUcg9aR5et0zCcUR9+joDaxE88AIdKP2RlZHS5vOuvfoDLGRaLRbm5udq7d69effVVrVq1ynbOx8fHYbC61pT+E3tSUlId9qRqhg8f7vD81Q+TxcTEOAxoISEhdvN6z5w5Y6yDKhltLd2PxMREw3Va7d692zZC6+XlVem/h8lksvvrweHDh536snnLLbdUWqb0z1B2draysrIqvQa4HjHCDMCh0g8CGX2Y7uplwgoLCyu9xtnRYE9PTz311FNOPRBV1a2xnX3IqiqqujV2ef1t27atbaRy8eLFCg4O1sCBA9WoUSOX9dPV3N3d1aZNG4dlrl4erkOHDpXWGxQUZPcAYGUKCgr066+/6sSJE0pLS1NeXp6Ki4tlsVhsZazz7SX70XyjrCPFUsnyb97e3pVe06tXL5lMJlv/EhMTHU5n8vX1dWrpN39/f4WFhen8+fOyWCx6++239fDDD7NsHHAVAjMAh/z8/GyjWZcvXzZU19XXVyW0OtKtWzf9/ve/V8uWLZ0qf61sjT18+HD9+OOPys/PV0FBgT788EPNnTtXvXr1UufOnRUdHa3mzZu7qMeuERAQUGmgv/qhRWc2yCl9TUFBQYXlCgsLtXjxYv38889V+nl2xe6UVufOnbO9drTWc2m+vr4KDQ1VWlpamTrKExYWVumDgVa33nqrbb52QkKCnn76abVu3Vo9evRQp06d1LFjR5fdq0BDRWAG4FDpwFydKRSlXX29M7+Erx4NdnNzk5eXlwICAtSmTRt17dr1ullz+WohISF66qmn7LYbz8nJ0caNG7Vx40ZJUpMmTdSnTx+NGDGi0pHd2lCdv1K4apnAy5cv69VXX7Ub4XVW6b+0GFU6fAcEBDh9XWBgoC0wVzaKbl2/2hljx47V2bNnbUvQSSVTfJKSkvTjjz/KZDKpXbt2GjBggIYPH254ahbQEBGYATgUHh5uW7v21KlTslgsTo9cXa30n7glOTXCWh9Gg+uzHj166P3339eyZcsUGxtbZuOJjIwM/fzzz1q9erViYmL0wAMPlBnBvV58/fXXdmG5Z8+eGjBggCIjIxUSEiJvb2+7aUMfffRRhTseGlHdaU6ly1YW4Ktyj5pMJj344IMaMGCAVqxYoT179tjVb7FYlJiYqMTERH333Xd64IEHXL5qCFDfEZgBONSxY0ft3btXkmy7fTk79eFqpR+c8vLyUmRkpCu6eN0LCgrStGnTNG3aNJ0+fVoHDx5UfHy8Dhw4oJycHEkloWf9+vXKzc3VX/7ylzruce3LycnR2rVrbe+nTZum8ePHO7zG6BSkipQeoa1KG6XLVmVremd17txZnTt3VmFhoY4cOaL4+HjFx8fr6NGjtvWiL126pH/84x/y8PBQv379XN4HoL4iMANw6OplqTZt2qQ777yzyvXk5+dr165dtvcdOnSo1w+nNVQtW7ZUy5YtNXr0aJnNZv3666+aP3++bRWNnTt36tChQ+rUqVMd97R2HThwwLYyRVhYmG6//fZKrylvm2hXKD0n++pd9ypisVjsyjozr7u6PD091a1bN3Xr1k1SyVSqNWvWaPHixbb54d988w2BGdcVlpUD4FB0dLTdHOF169Y5tbrF1davX2/3MNaIESNc0j9UzM3NTb169dILL7xgN1f2119/LbesVemVIq4V6enpttdt27atdMpCYWGhTp48WWm9petx9t+t9F9WrNtqV+bUqVO2jU6urqOm+fv76ze/+Y0eeOAB27HU1NRKHzwEriUEZgAOmUwm3Xbbbbb3mZmZWrRoUZXquHjxov7973/b3oeGhjI6VYsCAwPVsWNH2/vSm9FYlZ7XXJ0vRPWddUqBszZt2uTUg36ll4Rz9t+t9Oj+qVOnysztL09sbKzttZ+fX50s+3bTTTfZvS/v5wi4VhGYAVRq+PDhdtta//jjj4qLi3Pq2vz8fL399tt2T/X/4Q9/YDqGC1RlJLj06GR5q5OUXvs4NTXVUL/qo9KfLyEhweHWzpcuXbL7gudsvTk5OU4tP9e9e3e7nQjnz5/vsHx6errdBj2VbeRSFVX5GaqpZSGBhoDADKBS7u7ueuyxx2yjaRaLRR9++KEWLVqk4uLiCq87efKkZs2apaNHj9qOjRo1Sn369KnxPl8PXn75Za1evbrSkLZ7924dPHjQ9r68+cul/8R/8uRJ29bU14rSc/EzMjL03XfflVsuOztbb7zxhtMblYSEhKhx48aSSu6LFStWVHqNm5ubfvOb39je7969W//3f/9XbojPysrSnDlzbNOZvL297f7iY9ShQ4f05ptv6uDBgw7Ds9lstgv2QUFB1+1yjrg+8dAfAKdERERo5syZevPNN3Xp0iWZzWYtWrRIa9asUf/+/dW+fXsFBQUpPz9faWlp2rVrl/bv32/3p/CYmBj9/ve/r8NPUeKbb76p8ujYgw8+qPDwcJf14aOPPqrSTn+S9NRTT9lNATh//rw+++wzzZ07Vz169FCHDh3UokUL+fv7y2w2Ky0tTbt379a2bdtsYahdu3bq2bNnmbpbtmypNm3a6OTJk7JYLHr55ZfVunVrNWnSxG408+GHH7YFxIYkLCxMffr0se2MuGjRIiUmJmrw4MFq0qSJ8vLydPjwYa1Zs0a5ubkKDg5WmzZttGfPnkrrHjhwoC0oL1y4UGvXrlWLFi3slqi79dZb1bVrV9v70aNHa/v27bYvJkuXLtWBAwc0bNgwNW/eXEVFRTpy5Ih++eUX20onknTvvfeqadOmLvk3kUpC/u7du7V79241adJEvXr1Utu2bRUaGipvb2/l5+crOTlZGzZs0KlTp2zXTZgwwWWj3EBDQGAG4LSOHTvqlVde0UcffWR7WCkzM1PLly93eJ2np6cmTpyoO+64o9prOLuSsw9alebqJcYOHTpU5WsqmodbVFSknTt32sJgRSIiIvTUU09VGHQefvhhvfrqq7bpM9bNK0q7//77q9zv+uKhhx7SiRMnbFto79mzp9xA7OvrqyeeeEJr1qxxqt6pU6dq//79tkCZnp5u95ChJPXt29fuvclk0jPPPKO33nrLNvp/7Ngxhz+b06ZN0y233OJUn6ojIyND//nPfyotN3LkSI0dO7bG+gHURwRmAFXSsmVLvf7664qLi9Pq1at19OjRCv+U27hxY91000264447FBoaWss9vfbdfffd2rJliw4ePOhwWkZAQIBGjBihO+64w26E+mrt2rXT3/72N61evVr79+9XSkqKLl++XOUH5uqr4OBgvf766/r000/tlji0MplM6t69ux566CGFhYU5HZh9fX31xhtvaO3atdq5c6dOnTql3NzcSh8a9PHx0fPPP6+ff/5ZS5curXAZu44dO+p3v/udoqOjnepPVbRq1UpTpkzR7t27deLECYdzuyMjIzVp0qQy4R+4Hpgs1+L6QQBqTU5Ojo4ePaqsrCzl5OTIw8NDjRs3VrNmzdS2bVv+bFsLzGazTp8+rbNnzyozM1OXL1+Wu7u7AgIC1KpVK7Vp08Zl20tfK1JTU3Xo0CFduHBBnp6eCgkJUceOHRUSElIn/bFYLDp27JiSk5OVnZ0td3d3BQUFqVOnTnYPCNak/Px8nTx5UufOnVN2draKiork7e2tkJAQtWvXzqmdOYFrFYEZAAAAcIChHwAAAMABAjMAAADgAIEZAAAAcIDADAAAADhAYAYAAAAcIDADAAAADhCYAQAAAAcIzAAAAIADBGYAAADAAQIzAAAA4ACBGQAAAHCAwAwAAAA4QGAGAAAAHCAwAwAAAA78f+oxovNJ31V+AAAAAElFTkSuQmCC\n"
},
"metadata": {}
}
],
"source": [
"# estimate the policy value of IPWLearner with Logistic Regression\n",
"estimated_policy_value_a, estimated_interval_a = ope.summarize_off_policy_estimates(\n",
" action_dist=action_dist_ipw_lr,\n",
" estimated_rewards_by_reg_model=estimated_rewards_by_reg_model\n",
")\n",
"print(estimated_interval_a, '\\n')\n",
"\n",
"# visualize the estimated policy values of IPWLearner with Logistic Regression\n",
"ope.visualize_off_policy_estimates(\n",
" action_dist=action_dist_ipw_lr,\n",
" estimated_rewards_by_reg_model=estimated_rewards_by_reg_model,\n",
" n_bootstrap_samples=1000, # number of resampling performed in bootstrap sampling\n",
" random_state=12345,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"tags": [],
"id": "3fjNHzFzoGgS",
"outputId": "94bf6491-7e77-48d2-c7eb-cf86f6bf4cca",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 709
}
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
" mean 95.0% CI (lower) 95.0% CI (upper)\n",
"ipw 0.898878 0.881897 0.915042\n",
"dm 0.793182 0.792633 0.793699\n",
"dr 0.888986 0.882527 0.895118 \n",
"\n"
]
},
{
"output_type": "stream",
"name": "stderr",
"text": [
"/usr/local/lib/python3.10/site-packages/seaborn/categorical.py:82: FutureWarning: iteritems is deprecated and will be removed in a future version. Use .items instead.\n",
" plot_data = [np.asarray(s, float) for k, s in iter_data]\n"
]
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"<Figure size 800x600 with 1 Axes>"
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAswAAAIzCAYAAAAUFU/WAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAA9hAAAPYQGoP6dpAABqIUlEQVR4nO3dd3xUVeL///ekd5IQQpUaIIS+IEgNRZqLrEoRFduqX5dF17WsZVERXAvqqru2z7q6irrKUkSUEnFpSQDpSAmhJ6GEVEISQhqZ3x/5zWyGJJNJ7qTyej4e+/hk7j33nDP55Mp7zpx7jslsNpsFAAAAoEIu9d0BAAAAoCEjMAMAAAB2EJgBAAAAOwjMAAAAgB0EZgAAAMAOAjMAAABgB4EZAAAAsIPADAAAANhBYAYAAADsIDADAAAAdhCYAQAAADsIzAAAAIAdbs6opLi4WAkJCUpNTVVWVpby8/Pl5uYmHx8fhYSE6LrrrlPz5s2d0RQAAABQp2ocmFNTUxUbG6t9+/bp+PHjunLlit3yQUFB6tOnjwYNGqT+/fvL1dW1pk0DAAAAdcZkNpvN1blgx44dWrt2reLi4mrcqJ+fn8aOHasJEyYw8gwAAIAGzeHAvGPHDi1ZskSnT5+2HnN3d1fHjh0VFhamzp07KyAgQH5+fvLz81NhYaFyc3OVm5ur5ORknThxQsePH1dGRob1ejc3N40ZM0bTpk1Ts2bNnP/uAAAAAIMcCszz58+3jii7u7urf//+GjFihH71q1/Jza16szrOnz+vmJgYbdmyRcnJyZIkHx8fPfLIIxowYEAN3kLDduHCBRUXF9d3NwAAAHAVNzc3BQUFVVnOocB8++23y8/PT5MnT9aECRPk4+PjlE7Gx8dr+fLl2r9/v6ZPn65p06Y5pd6GJC0tTUVFRfXdDQAAAFzF3d1dLVq0qLKcQ4H5+++/1/jx4+Xl5eWUzl3t+PHjysnJUf/+/Wul/vpEYAYAAGiYnBqYUXMEZgAAgIbJ0cDMxiUAAACAHQRmAAAAwA4CMwAAAGCHw2vCbd682emNR0ZGOr1OAAAAwJkcDswffvihUxs2mUwEZgAAADR49TYlg8U5AAAA0Bg4PMI8e/bs2uwHAAAA0CCxDnMtYx1mAACAhol1mAEAAAAnIDADAAAAdjg8h1mSLl26pMOHD0uSmjdvrk6dOjl87cmTJ5WZmSlJ6tmzp7y9vavTNAAAAFAvqhWYv/nmG/3000/y8vLSK6+8Uq2GPDw89N577yk/P1833XST7r333mpdDwAAANQHh6dkZGVlaf369ZKkmTNnql27dtVqqF27drrjjjskSevWrVN2dna1rgcAAADqg8OBOTY2ViUlJQoODtb48eNr1Ni4ceMUEhKi4uJixcbG1qgOAAAAoC45HJgPHTokSRo8eLBcXV1r1Jirq6sGDx4sSTp48GCN6gAAAADqksNzmJOSkiSVPrBnRI8ePbR69WolJiYaqgeoiZdfftn68GlwcLBeeOGFeu4RAABo6BwOzDk5OZKkoKAgQw1arrfUB9SlzMxMpaen13c3AABAI+LwlIySkhKnNuzs+gAAAIDa4HBg9vf3lyRduHDBUINZWVk29QEAAAANmcOBOTQ0VJKsG5fUVFxcnE19AAAAQEPmcGDu3bu3pNLl5fLz82vUWH5+vmJiYmzqAwAAABoyhwPzkCFDZDKZdPHiRX3xxRc1amzRokXKzs6Wi4uLhg4dWqM6AAAAgLrkcGBu27athg8fLklav369Pv74YxUWFjp0bWFhoT7++GNt2LBBkjR8+HC1adOmBt0FAAAA6pbDy8pJ0n333adjx47p/PnzWr9+vfbs2aOxY8eqf//+6tixo9zc/lddcXGxEhIStGfPHm3YsMH6sGCbNm103333OfVNAAAAALXFZDabzdW5ICUlRQsXLtTZs2dtjru4uMjHx0eenp4qKChQXl5euaXj2rZtq2efffaaeuAvLS1NRUVFTqkr+U8POqWea9lrqQW68P//WQa5SM+FetZvh5qA1m9+Ut9dAACgRtzd3dWiRYsqy1VrhFmSWrZsqddff13//ve/tWHDBuu0jJKSEuXm5io3N7fcNR4eHho7dqzuuOMOeXoSUAAAQNPH7rJNR7UDs1QagO+//35NnTpVW7duVVxcnBITE5WTk6PLly/L29tb/v7+6tChgyIiIjR06FAFBAQ4u+8AAAANFrvLNh01CswWAQEBmjhxoiZOnOis/gAAAAANisOrZAAAAADXIkMjzAAAoOn68fvk+u5Co3Y574rNz/w+jZswpXW9tMsIMwAAAGAHgRkAAACwg8AMAAAA2MEcZlxTmrmaJJnL/AwAQO3w8W5W4c9ofAjMuKb8vrlHfXcBAHCNmDDm9/XdBTgJUzIAAAAAOwjMAAAAgB0EZgAAAMCOWg/M+fn5unDhgvLz82u7KQAAAMDpauWhv+LiYn3//ffavHmzzp8/bz3esmVLRUZGasqUKXJ3d6+NpgEAAACncnpgzs/P14IFC3TixIly51JSUrRkyRLt2rVLL774ory9vZ3dPAAAAOBUTg/My5Yt04kTJ9SmTRvNnDlT3bt3l6+vrzIyMhQTE6Nvv/1WJ0+e1LJly3T33Xc7u3kAAADAqZw+h3nbtm1yc3PTvHnzNHjwYAUGBsrd3V2tWrXS9OnTNXPmTEnSzz//7OymAQAAAKdzODC///77ys7OrrJcVlaWWrVqpcDAwArP9+jRw1oOAAAAaOgcDswxMTH64x//qKioKJWUlFRaLjAwUOfPn680EMfFxVnLAQAAAA2dw4H55ptvVn5+vj777DM9++yzOnLkSIXlBg8erOLiYr388svauXOnsrOzVVxcrJSUFC1btkxLliyRJN1www3OeQcAAABALXL4ob9Zs2Zp1KhR+te//qVDhw7pxRdf1MiRIzVr1iw1a9bMWm769Ok6dOiQEhIS9NZbb1VYV8eOHTVt2jTjvQcAAABqmclsNpure9GWLVv05Zdf6sKFC/Lx8dGMGTM0YcIEubiUDlgXFhZq5cqVio6OVmpqqvW60NBQ6zrMHh4eznsXDVhaWpqKioqcUlfynx50Sj2AM7V+85P67gKAWvLj98n13QXAxoQprZ1an7u7u1q0aFFluRoFZql0veUlS5Zo7dq1KikpUYcOHfTAAw+oe/fu5crl5eXJ29v7mlx3mcCMpo7ADDRdBGY0NPUVmGu8rJyXl5fuuecevfnmm4qIiFBiYqJefPFFffDBB7p48aJNueDg4GsyLAMAAKDxM7wOc7t27TRv3jw9+uijCgwMVHR0tP74xz9aR54BAACAxsxpG5cMHz5cf/vb33TTTTcpPz9fn3/+uZ599lnFx8c7qwkAAACgztUoMJeUlOjMmTM6evSozpw5oytXrkgqnX5x7733auHChQoPD1diYqLmzZun999/32aaBgAAANBYOLysnCTl5ubq66+/VkxMjAoLC63HPTw8NHz4cN11113y8/NT+/btNX/+fEVHR+urr75STEyMdu/erenTp2vixInW1TQAAACAhs7h5JqTk6O5c+dq/fr1KiwslIuLi5o1ayYXFxcVFhZqw4YNmjt3rs322SNHjtS7776riRMnKj8/X4sWLdIzzzzDNA0AAAA0Gg6PMC9ZskTnz5+Xt7e3HnroIQ0ZMkQuLi4qKSnRtm3b9Mknn+j8+fNaunSpHnjgAet1Pj4+uv/++zVmzBh9+umnOnLkiObNm6cRI0bokUceqZU3BQAAADiLwyPMu3btkiTdddddGjZsmHVahYuLi4YNG6Y777zTptzVOnTooAULFmj27NkKCAhQTEyM0b4DAAAAtc7hEWbLVIuOHTtWeN5yvOyUjIqMGjVKgwYN0uLFix1tGgAAAKg3Do8wBwUFSZIOHz5c4XnL8cDAwCrr8vHx0W9/+1tHmwYAAADqjcOB+YYbbpAk/ec//9HKlSuVnp6u4uJipaena9WqVVqyZIkkaciQIbXTUwAAAKAeODwlY9q0aTp06JBOnjypr7/+Wl9//XW5Mh07dtTUqVOd2kEAAACgPjkcmL28vLRgwQKtXLlSmzdvVmpqqvVcaGioRowYoVtuuUUeHh610lEAAACgPlRr4xJ3d3dNmzZN06ZNU35+vvLy8uTt7S1vb+/a6h8AAABQr6oVmMvy8vKSl5eXM/sCAAAANDjsUQ0AAADYUeMR5oagsLBQK1as0NatW5Weni4/Pz/17dtXM2fOVHBwcLXq2r9/v1avXq3jx49bp5p07txZ48eP16BBg2rpHQAAAKChc3pgTkpKUlRUlOLi4pSZmSmpdA3niIgIjR8/Xp06dXJKO4WFhZo/f76OHTumoKAgDRw4UGlpadq0aZP27NmjV155RS1btnSortWrV2vRokUymUzq1q2bmjdvroyMDB04cED79+/XrbfeqjvuuMMp/QYAAEDj4nBgXrZsmSRp/PjxCggIqLDM4sWL9d1338lsNtscP3/+vM6fP6+NGzdqypQp1m20jfj222917NgxdevWTc8//7x1PvWqVav0xRdf6KOPPtJLL71UZT3Z2dn6+uuv5erqqhdeeEERERHWc3FxcXrllVf03XffacyYMQ4HcAAAADQdDs9hXrp0qZYuXVrp1tdLly7VihUrrGHZMqWhS5cu8vHxkSSZzWatXLlSy5cvN9Tp4uJiRUVFSZIeeOABm4cPJ0+erA4dOiguLk4nT56ssq5jx46pqKhIvXr1sgnLkhQREaG+ffvKbDbrxIkThvoMAACAxskpUzKSk5OtIdiy7fWwYcPk4lKax0tKSrRlyxb961//Ul5enpYtW6bIyEiFhITUqL34+Hjl5eWpZcuWFU7xGDx4sBITE7Vr1y517tzZbl3u7u4Otenv71+jvgIAAKBxc8oqGevWrZPZbJabm5uef/55jRgxwhqWJcnFxUUjRozQ3Llz5ebmppKSEq1fv77G7SUmJkpSpfOhLSE5KSmpyrrCwsLk6+urgwcPKi4uzuZcXFycfvnlF7Vu3Vo9evSocX8BAADQeDllhNkSNEeOHKkuXbpUWi4sLEwjRozQxo0by4XT6khPT5ckNW/evMLzlhUy0tLSqqzLx8dHv/vd7/T3v/9d8+fPt3no7+jRo+revbseeeQRubnZ/1UVFRWpqKjI+tpkMlk3dDGZTA69L6Ax4u8bAFBX6uvfHKcEZss22QMGDKiy7IABA7Rx40adO3euxu3l5+dLkjw9PSs8b5nTbClXlcGDB+u5557TO++8oyNHjliPe3t7q0+fPg4tUbdixQrrg5FS6ej3woUL1aJFC4f64Iia/8aA2tO6dev67gKAWsO/PGhY6uvfHKcE5sLCQklyaE6ypUxeXp4zmnaKH374QV999ZWuv/56zZgxQ6GhoUpNTdV//vMfLVmyRMePH9ezzz5rt45bb71VkydPtr62fAJKS0tTcXFxrfYfqE/Jycn13QXUgQULFliXCg0ODtaLL75Yzz0CcC1y9r85bm5uDg1uOiUwBwYGKj09XVeuXKmybElJiSTHH7ariGUEuaCgoMLzlpFlR7buPnTokL788kt16tRJTzzxhHXudfv27fXkk0/q2Wef1Z49e7R3717179+/0nrc3d0rfU9XL7MHNCX8fV8bMjMzrdPhJP7/DqB+1Nd/e6odmI8fP15uabng4GClp6crJSXF7hxmSbpw4YIkY6tOWEapMzIyKjxvGQVx5BNDdHS0JGnQoEE2DypKpQ8rDho0SAkJCTp8+LDdwAwAAICmqdqB+aOPPqr03JEjRzR06FC711vWRq7u1tVldejQQZJ06tQpu220b9++yros4dqyVvTVLMdzc3Or3U8ADcN9i7bVdxcavSu5//tGLz23gN+pQZ/fO6S+uwCgGpyyrJzF9u3brVMuKrNz505JpStm1FR4eLh8fHyUkpKihISECvshSQMHDqyyrmbNmklSpRuTWI6HhobWsLcA0AR4+kpe/qX/8/St794AQJ1yeIR53rx5DpUrLi6Wh4dHhefi4uKsayN369bN0abLcXNz08SJE/Xtt9/q008/1dy5c222xk5MTFRERITNpiVRUVGKiorSoEGDbLbmHjRokKKjoxUbG6shQ4bYrPSxc+dOxcbGymQyadCgQTXuLwA0dq7X31bfXQCAeuNwYL562+ia6NChg95//31JUlBQkKG6brvtNh04cEBHjhzRY489pvDwcKWnp+vYsWMKCAjQ7NmzbcpnZ2fr3Llz1jnUFtdff71uuOEG/fzzz1q4cKG6dOmiFi1aKC0tzTq6PHPmTLVp08ZQfwEAANA4OWWVDEf5+vrK19c5X+V5eHho3rx5WrFihWJjY7Vz5075+flp1KhRuv322yvd1ORqJpNJjz/+uDZu3KjNmzcrKSlJCQkJ8vHxUf/+/TVp0iT169fPKX0GAABA41OngdnZPDw8dPvtt+v222+vsuyMGTM0Y8aMCs+ZTCaNGTNGY8aMcXYXAQAA0Mg59aE/AAAAoKkxNMKclpamw4cP68yZM0pPT1d+fr4KCwvl5eUlPz8/tWvXTmFhYerevXu97f0NAAAAGFGjwLx9+3Z999131vWOqxISEqJx48Zp8uTJcnNr1LNAAAAAcI2pVnotKCjQu+++qz179lSrkfT0dH3zzTfavHmzHn/8cYc2FAEAAAAagmoF5rfeekv79++XVDpq3Lt3bwUGBiovL0+HDx9WUlKS3NzcdO+996pz585KTk7WyZMntWPHDqWnp+vcuXOaP3++5s2bR2gGAABAo+BwYN66das1LE+bNk1Tp06Vi4vtM4OxsbH68MMP9dVXX+n111/XiBEjNGLECN19993asGGDFi1apNzcXL333ntauHBhuesBAACAhsbhxLp582ZJ0pAhQzR9+vQKw+7w4cM1depUFRQU6Ouvv/5fIy4uuvHGG/XMM8/IZDIpKSlJMTExTug+AAAAULscDswJCQmSpMjISLvlLOf37Nmj4uJim3O9evXS8OHDJUnbtm2rTj8BAACAeuFwYM7JyZFU9ZbWlvNXrlxRZmZmufODBg2SJCUmJjrcSQAAAKC+OByYfXx8JEkXLlywW67s+YqWkAsJCZEkZWdnO9o0AAAAUG8cDswdOnSQ9L+5zJWJjo6WJHl5eSkwMLDcecs0DS8vL0ebBgAAAOqNw4HZMpVi27ZtWrZsmcxmc7kyW7du1bJlyyRJffr0qfDBwNOnT0uqemoHAAAA0BA4vKzcmDFjtHbtWiUnJ2vp0qXatGmT+vTpo2bNmlnXYbbMS3Z1ddXUqVMrrMey6UlYWJgTug8AAADULocDs7u7u55++mm9/PLLyszMVFpamtavX1+unIuLix566CF17Nix3LnMzEzFxcXJ09NTffr0MdRxAAAAoC5Ua6e/Nm3aaOHChVq6dKliY2OVl5dnPefq6qpevXpp+vTp6tq1a4XXBwcH67PPPjPWYwAAAKAOVSswS1JAQIAeeOAB3X///Tp//rxycnLk5eWlVq1aydPTszb6CAAAANSbagdmCxcXF7Vp08aZfQEAAAAaHIdXyQAAAACuRQRmAAAAwA4CMwAAAGAHgRkAAACwg8AMAAAA2EFgBgAAAOwgMAMAAAB2EJgBAAAAOwjMAAAAgB0EZgAAAMAOAjMAAABgh5uzKiopKVFmZqYkKSQkxHA5AAAAoCFwWmA+e/asnnrqKZlMJi1evNhwOQAAAKAhcPqUDLPZ7NRyAAAAQH1iDjMAAABgB4EZAAAAsIPADAAAANhBYAYAAADsIDADAAAAdtRoWblly5aVO3bx4kW756dNm1aTpgAAAIB6VaPAvHTp0mqfJzADAACgMapRYO7Ro4dMJpPNsfz8fJ08eVKSFBERYbxnAAAAQANQo8D80ksvlTt2+vRpPfXUU5KkefPmGeoUAAAA0FDw0B8AAABgB4EZAAAAsIPADAAAANhBYAYAAADsIDADAAAAdhCYAQAAADtqtKxcRUJDQx1aTs7RcgAAAEBD4LTA7Onp6dCGJY6WAwAAABoCpmQAAAAAdhCYAQAAADsIzAAAAIAdDgXm7du312onMjMzdfTo0VptAwAAAKgJhwLz22+/rT/96U/6+eefndp4enq6PvnkEz366KPav3+/U+sGAAAAnMGhVTJatmyppKQkvfPOOwoJCdHw4cM1fPhwXXfdddVuMD8/Xzt27NCWLVu0f/9+lZSUyMXFRS1btqx2XQAAAEBtcygwv/3221qzZo1Wrlyp9PR0fffdd/ruu+/UunVrde3aVV26dFGnTp3UrFkz+fr6ytfXV4WFhcrNzdWlS5eUnJys48eP68SJEzp+/LgKCwutdQ8aNEh33HGH2rRpU2tvEgAAAKgphwKzm5ubpkyZovHjx+vHH3/Ujz/+qIyMDCUnJys5OVnR0dHVa9TNTYMHD9ZNN92ksLCwGnUcAAAAqAvV2rjEy8tLv/nNb3TzzTdr//792rZtmw4dOqS0tLQqr3V3d1fXrl01cOBARUZGys/Pr8adBgAAAOpKjXb6c3FxUb9+/dSvXz9JpatcHDlyRBkZGcrOzlZubq48PDwUEBCggIAAtW/fXp07d5abm9M2FgQAAADqhFMSbHBwsIYMGeKMqgAAAIAGhY1LAAAAADsIzAAAAIAdBGYAAADADgIzAAAAYAeBGQAAALCDwAwAAADYQWAGAAAA7CAwAwAAAHYQmAEAAAA7CMwAAACAHQRmAAAAwA43Z1VUUlKiQ4cO6ejRo8rKylJBQYHuuOMOBQUFWcsUFxfrypUrcnFxkbu7u7OaBgAAAGqNUwLz7t279dlnnyktLc3m+JQpU2wC8/r16/Wvf/1LXl5e+sc//iEvLy9nNA8AAADUGsNTMv773//qjTfesIZlf3//SsuOHTtWPj4+ys/P144dO4w2DQAAANQ6Q4E5OTlZn376qSSpV69eevvtt/XJJ59UWt7NzU2DBw+WJO3fv99I0wAAAECdMBSYV69erZKSEl133XV67rnn1LZt2yqv6dGjhyTp1KlTRpoGAAAA6oShwHzw4EFJ0k033SQ3N8emQ7dq1UqSlJGRYaRpAAAAoE4YCsyW0NuxY0eHr/H09JQkFRQUGGkaAAAAqBOGArPJZJJUvfCbm5srSfLx8THSNAAAAFAnDAXm4OBgSVJKSorD18THx0uSWrZsaaRpAAAAoE4YWoc5IiJCycnJ2rx5s0aNGlVl+by8PP3000+SSlfVMKqwsFArVqzQ1q1blZ6eLj8/P/Xt21czZ860hvnqSE1N1XfffadffvlFFy5ckLe3t1q1aqXBgwdrypQphvsLAACAxsfQCPO4ceMkSXFxcdq0aZPdsjk5OXrzzTeVlZUlV1dX67U1VVhYqPnz52v58uXKz8/XwIED1bx5c23atElPP/10tUa9JWnv3r168skntX79evn7+2vQoEHq1KmT0tLSrCEfAAAA1x5DI8ydOnXSTTfdpDVr1uijjz7S3r17ressS9KRI0eUkJCg+Ph4bdmyRXl5eZKkqVOnqkWLFoY6/u233+rYsWPq1q2bnn/+eeuugatWrdIXX3yhjz76SC+99JJDdZ09e1ZvvfWWvL299fzzz6t79+7WcyUlJSyBBwAAcA0zvDX2Pffco6KiIv3000/6+eef9fPPP1vPffzxx+XK33TTTZo6daqhNouLixUVFSVJeuCBB2y22J48ebI2b96suLg4nTx5Up07d66yvi+++EJFRUV68sknbcKyJLm4uKhLly6G+gsAAIDGy3BgNplMevDBBzVo0CB99913iouLk9lsLleuW7duuu2229S/f3+jTSo+Pl55eXlq2bKlOnXqVO784MGDlZiYqF27dlUZmNPT07Vv3z61bNlSv/rVrwz3DQAAAE2L4cBs0adPH/Xp00eXL1/WqVOnlJ2drZKSEvn5+aljx44KCAhwVlNKTEyUpArDsiRrSE5KSqqyLkvA79atm65cuaLt27fryJEj1h0Mhw4dKj8/P6f1HQAAAI2L0wKzhbe3tyIiIpxdrY309HRJUvPmzSs8b1khIy0trcq6zpw5I0ny8vLSiy++qGPHjtmcX7x4sZ544okqV/UoKipSUVGR9bXJZJK3t7f1Z6Cp4u8bqD7uG6Bm6uvecXpgrgv5+fmS/rdr4NUsc5ot5eyxbKSyYcMGeXl56Q9/+IP69eun7OxsLV++XDExMXrrrbf09ttv212qbsWKFVq2bJn1dadOnbRw4ULDDzeWdc5pNQHO07p16/ruAtDoNJ77hn950LDU173TKAOzM1nmW1+5ckUPPfSQhg4dKkny8/PTo48+qnPnzunEiRP68ccfdccdd1Raz6233qrJkydbX1s+AaWlpam4uLgW3wFQv5KTk+u7C0Cjw30D1Iyz7x03NzeHBjcNBeYPP/ywxteaTCbNnj27RtdaRpAr25LbMrJcdvWMqury8vLSkCFDyp0fPXq0Tpw4obi4OLv1uLu7y93dvcJzFT0ECTQV/H0D1cd9A9RMfd07hgLz5s2bDTVe08AcEhIiScrIyKjwfGZmpiQ59InBUiYkJKTCeTGW89nZ2TXqKwAAABo3Q4HZElztKSgoUE5OjvV1QECAPDw8jDSrDh06SFKlG4qcPHlSktS+ffsq6+rYsaMk6dKlSxWet8xxdmS0GgAAAE2PocD8wQcfOFQuNzdXW7Zs0ZIlS+Tj46NnnnlGbdq0qXG74eHh8vHxUUpKihISEqyh12L79u2SpIEDB1ZZV/fu3eXv76+srCydO3euXL8sUzGubgMAAADXBpe6aMTPz08TJkzQyy+/rOzsbL366qvWkduacHNz08SJEyVJn376qc1qGKtWrVJiYqIiIiJsNi2JiorSH//4R3399dc2dbm6uurXv/61zGazPv30U+v23ZK0f/9+bdq0SSaTSePGjatxfwEAANB41ekqGW3atNGkSZO0fPlyrVq1SjNnzqxxXbfddpsOHDigI0eO6LHHHlN4eLjS09N17NgxBQQElJsfnZ2drXPnzunChQvl6poyZYoOHTqkAwcO6LHHHlPXrl2Vk5OjY8eOqaSkRDNnzlRYWFiN+woAAIDGq05GmMvq3bu3pP9Nm6gpDw8PzZs3T1OnTpWHh4d27typtLQ0jRo1SgsXLlTLli0drsvNzU3PPfec7rrrLgUEBOiXX35RUlKSIiIi9Mwzz+i2224z1FcAAAA0XnW+DrPl4TnLbn1GeHh46Pbbb9ftt99eZdkZM2ZoxowZlZ53c3PTb37zG/3mN78x3C8AAAA0HXU+wmxZ2cLN7ZrfMwUAAACNQJ0G5tTUVC1dulQSq04AAACgcaj1jUvMZrNyc3N18uRJ7dy5U4WFhZLEqhMAAABoFOpla+xJkyZp6NChRpoGAAAA6kSdTST28fFRjx49NGHCBPXt27eumgUAAAAMMRSY33///SrLmEwmeXt7y9fX10hTAAAAQL0wFJhbtGjhrH4AAAAADVKdLysHAAAANCYEZgAAAMAOAjMAAABgh0NzmB955BGnN2wymfTee+85vV4AAADAmRwKzGlpabXdDwAAAKBBcigwR0ZG1nY/AAAAgAbJocD8+9//vrb7AQAAADRIPPQHAAAA2EFgBgAAAOwgMAMAAAB2GNoauyKpqanKyclRYWGhzGaz3bIRERHObh4AAABwKqcE5nPnzunbb7/V7t27lZeX59A1JpNJixcvdkbzAAAAQK0xHJh37Nih9957T4WFhdW6rqrRZwAAAKAhMBSY09PTrWE5ODhYN998szw9PfXxxx9Lkl544QXl5ubq5MmTio6O1oULFxQeHq7p06fLxYXp0wAAAGj4DAXmtWvXqrCwUF5eXnrllVcUHBys06dPW8/36tVLknTDDTdo2rRp+uijj7R161Zt2LBBf/jDH4z1HAAAAKgDhoZ5Dxw4IEmaMGGCgoOD7Zb18PDQo48+qk6dOmnLli36+eefjTQNAAAA1AlDgTktLU2S1K1bN+sxk8lk/fnKlSu2jbm4aNKkSZKkjRs3GmkaAAAAqBOGAnN+fr4kKSQkxHrMw8PD+nNFK2Zcd911kqTExEQjTQMAAAB1wlBg9vHxkSSbFTL8/f2tP6ekpJS7xhKic3JyjDQNAAAA1AlDgblNmzaSSjcrsfD29raOOP/yyy/lrtm/f7+k/4VtAAAAoCEzFJgtc5ePHj1qc3zAgAGSpO+//14HDx60Ht+6davWrFkjSerevbuRpgEAAIA6YWhZuf79+2vVqlXasWOH7rvvPuvaylOmTNHGjRuVn5+vl19+WX5+fioqKlJBQYGk0of/pkyZYrz3AAAAQC0zNMLcs2dPTZs2TaNGjVJmZqb1eEhIiJ544gnrtIvc3FxrWHZzc9PDDz9ss7IGAAAA0FA5PMK8Zs0aDR8+XAEBAdZjJpNJ06dPr7B8//799fe//10///yzTp8+rZKSErVq1UpDhw6tcs1mAAAAoKFwODAvWrRIX331lfr06aPIyEhdf/31cnOzf7m/v7/GjRtnuJMAAABAfanWHOYrV65o79692rt3r3x8fDRkyBCNHDlS4eHhtdU/AAAAoF45HJgfeughxcTEKD4+XlLpesrr16/X+vXrFRoaqsjISI0cOVKhoaG11lkAAACgrjkcmG+88UbdeOONSk1NVXR0tGJjY5WcnCypdB3mpUuXaunSperevbtGjhypoUOHstYyAAAAGr1qLysXGhqqadOmadq0aTp+/Liio6O1detW6859R44c0ZEjR/T555/rV7/6lSIjI9W/f3/rknMAAABAY2JoHeawsDCFhYXp3nvv1d69exUdHa3du3eruLhYRUVF2r59u7Zv366AgAANGzZMI0eOVOfOnZ3VdwAAAKDWGQrMFq6urho4cKAGDhyovLw8bdu2TdHR0db5ztnZ2Vq7dq3Wrl2rdu3aKTIyUsOHD2d5OQAAADR4TgnMZfn4+Gjs2LEaO3as0tLSFB0drZiYGOt85zNnzujf//63vvnmG33zzTfObh4AAABwKqcH5rJatGihqVOnaurUqTp69Kj+8Y9/6MyZM5KkkpKS2mwaAAAAcIpaDcySdPLkScXExGjLli26ePFibTcHAAAAOFWtBOb09HTFxsYqOjpaZ8+eLXe+Z8+eioyMrI2mAQAAAKdyWmC+fPmytm3bppiYGB0+fFhms9nmfJs2bTRy5EiNGDFCISEhzmoWAAAAqFWGAnNJSYl1Obk9e/aosLDQ5ryfn591ObmwsDBDHQUAAADqQ40Cs2XDkm3btik7O9u2Qjc39e/fX5GRkfrVr34lV1dXp3QUAAAAqA8OB+bU1FTFxMTYLBFXVlhYmEaOHKlhw4bJz8/PqZ0EAAAA6ovDgfnRRx8tdywkJEQjRozQyJEj1aZNG6d2DAAAAGgIqj0lw8vLS4MHD1ZkZKR69uxZG30CAAAAGgyHA3OfPn0UGRmpQYMGycPDozb7BAAAADQYDgfmuXPn1mY/AAAAgAbJpb47AAAAADRkBGYAAADADgIzAAAAYAeBGQAAALCDwAwAAADYQWAGAAAA7CAwAwAAAHYQmAEAAAA7DAXmrKwsJ3UDAAAAaJgc3umvIrNnz1b//v01evRoDRgwQC4uDFgDAACgaTEUmEtKSrR7927t3r1bAQEBGjlypEaPHq127do5q38AAABAvTIUmH/9618rJiZG2dnZys7O1qpVq7Rq1SqFhYVp9OjRGjZsmLy9vZ3VVwAAAKDOGQrM99xzj2bNmqU9e/Zow4YN2rt3r0pKSnT8+HEdP35cixYt0uDBgzV69Gj17NnTWX0GAAAA6oyhwCxJLi4uGjhwoAYOHKiLFy8qOjpamzZt0pkzZ1RYWKiYmBjFxMQoNDRUo0ePVmRkpJo3b+6MvgMAAAC1znBgLqtZs2a6+eabdfPNN+v48ePauHGjtm7dqry8PKWmpuo///mPli5dql69emnMmDG6/vrr5ebm1C4AAAAATlVraTUsLExhYWG67777tH37dm3atEkHDx5USUmJ9u/fr/3798vPz08jRozQjTfeyIOCAAAAaJBqfR04d3d3devWTd26dZO/v7/NudzcXK1du1ZPPvmk3nrrLaWmptZ2dwAAAIBqqbUR5sLCQv3888/auHGjDh8+LLPZbD3Xrl07jRgxQklJSdq5c6cKCwu1c+dOxcXFacGCBYw2AwAAoMFwemA+cuSINm7cqG3btik/P9963NPTU0OGDNHYsWPVrVs36/G8vDytWbNGK1as0KVLl7R48WI99dRTzu4WAAAAUCNOCcyZmZnW1TGSk5NtznXp0kVjxozR8OHD5eXlVe5aHx8fTZs2TX5+fvrss8909OhRZ3QJAAAAcApDgXnr1q3avHmz9u/fr5KSEutxX19fjRgxQmPHjlX79u0dqqtXr16SpIsXLxrpEgAAAOBUhgLz3/72N5vXPXv21JgxYzR48GC5u7tXryMsLwcAAIAGyHBKDQwM1KhRozRmzBi1bNmyxvWEhobq/fffN9odAAAAwKkMBeY//elP+tWvfiUXF+Or07m4uKhFixaG6wEAAACcyVBgHjhwoLP6AQAAADRItb5xCQAAANCYGQrMSUlJeuSRR/SHP/xBmZmZVZbPzMzUo48+qkcffVTnzp0z0jQAAABQJwwF5ujoaKWlpalVq1YKDg6usnxwcLDatGmj1NRUxcTEGGkaAAAAqBOGAvPhw4clVW8us6XswYMHjTQNAAAA1AlDD/1ZplU4ujmJJF133XU21xpRWFioFStWaOvWrUpPT5efn5/69u2rmTNnOjTiXZnk5GQ99dRTKioqUu/evfXCCy8Y7isAAAAaJ0MjzPn5+ZJU4ZbXlbGUzcvLM9K0CgsLNX/+fC1fvlz5+fkaOHCgmjdvrk2bNunpp59WSkpKjev++OOPVVxcbKh/AAAAaBoMBWY/Pz9JUlZWlsPXWMp6e3sbaVrffvutjh07pm7duulvf/ubHn/8cb366qu65557lJ2drY8++qhG9W7YsEGHDh3S2LFjDfUPAAAATYOhwNyqVStJ0r59+xy+xlLWyK6AxcXFioqKkiQ98MADNiPckydPVocOHRQXF6eTJ09Wq96srCx9+eWX6tOnj4YNG1bj/gEAAKDpMBSY+/btK0lav369zpw5U2X506dPa/369ZKkfv361bjd+Ph45eXlqWXLlurUqVO584MHD5Yk7dq1q1r1fv755yosLNSDDz5Y474BAACgaTEUmMePHy9PT08VFhZqwYIF2r17d6Vld+3apZdfflmFhYXy8PDQhAkTatxuYmKiJFUYliWpc+fOkkrXiXbUnj17tHXrVt16663WkXMAAADA0CoZAQEBeuihh/T+++/r4sWLeuONN9SyZUuFh4crMDBQUuk0h8OHDys1NdV63UMPPWQ9XxPp6emSpObNm1d43rJCRlpamkP15efn69NPP1WbNm10yy231KhPRUVFKioqsr42mUzWedomk6lGdQKNAX/fQPVx3wA1U1/3jqHALEkjRoyQ2WzWP//5TxUWFiolJaXSFSo8PT314IMPauTIkYbatKzO4enpWeF5y5xmS7mqLF68WGlpaZo3b57c3Gr2K1mxYoWWLVtmfd2pUyctXLhQLVq0qFF9FWFvRDRErVu3ru8uAI1O47lv+JcHDUt93TuGA7MkjRw5Un369NGaNWu0d+9em6kQJpNJ1113nQYMGKCJEycaGlmuDSdOnNDatWs1cuRI9ezZs8b13HrrrZo8ebL1teUTUFpaGkvUoUlLTk6u7y4AjQ73DVAzzr533NzcHBrcdEpglqTAwEDdeeeduvPOO3XlyhXl5uZKKl16ztXV1VnNSPrfCHJBQUGF5x1dH/rKlSv6xz/+IV9fX91zzz2G+uTu7i53d/cKz5nNZkN1Aw0Zf99A9XHfADVTX/eO0wJzWa6urmrWrFltVC1JCgkJkSRlZGRUeD4zM1OSqvzEkJGRoYSEBAUGBurtt9+2OXfp0iVJ0smTJ/XSSy9JkvX/AgAA4NpRK4G5tnXo0EGSdOrUqQrPW9ZfdnTL7qysrEo3X7l06ZLi4uKq30kAAAA0CY0yMIeHh8vHx0cpKSlKSEhQx44dbc5v375dkjRw4EC79YSGhmrJkiUVnjt06JDmz5+v3r1764UXXnBKvwEAAND4OBSY58+fL6n0QbYXX3yx3PGauLqu6nBzc9PEiRP17bff6tNPP9XcuXOt85VXrVqlxMRERUREWNdjlqSoqChFRUVp0KBBuvPOO2vcbwAAAFxbHArMlU1JqM+pCrfddpsOHDigI0eO6LHHHlN4eLjS09N17NgxBQQEaPbs2Tbls7Ozde7cOV24cKGeegwAAIDGyKHA3KNHjwoXiq7seF3w8PDQvHnztGLFCsXGxmrnzp3y8/PTqFGjdPvtt1e6qQkAAABQHSYza9vUqrS0NJsdAI1I/tODTqkHcKbWb35S312o0n2LttV3FwAbn987pL674JAfv2e9aDQsE6Y4d+MSd3d3h9ZhdnFqqwAAAEATQ2AGAAAA7CAwAwAAAHYQmAEAAAA7HFol4/bbb3d6wyaTSYsXL3Z6vQAAAIAz1dsIM4tzAAAAoDFwaIR52rRptd0PAAAAoEFyKDBPnz69tvsBAAAANEg89AcAAADYQWAGAAAA7HBoSkZ1ZGVl6fTp08rNzZUk+fn56brrrlNgYKCzmwIAAABqnVMCs9ls1n//+19FRUXpzJkzFZZp166dJk2apLFjx8pkMjmjWQAAAKDWGQ7Mubm5euONN3TkyBG75c6cOaN//vOfio6O1jPPPCNfX1+jTQMAAAC1zlBgNpvNevPNN61h2c/PT0OGDFHXrl2tUzCysrJ0/Phxbdu2TTk5OTpy5IjeeOMNzZ8/33DnAQAAgNpmKDDHxsYqPj5ekjR8+HA9+OCD8vb2LlcuMjJSd911lz755BPFxMQoPj5esbGxGj58uJHmAQAAgFpnaJWM2NhYSVJERIQeffTRCsOyhZeXlx555BFFRERIkmJiYow0DQAAANQJQ4H51KlTkqSJEyc6fI2lbEJCgpGmAQAAgDphKDBblo4LDQ11+BpLWcu1AAAAQENmKDD7+PhIki5cuODwNZay9qZvAAAAAA2FocB83XXXSZI2btzo8DWbNm2yuRYAAABoyAwF5htuuEGStGPHDi1ZskRms9lu+WXLlmn79u2SpCFDhhhpGgAAAKgThpaVGzt2rKKionTu3DktX75cO3bs0KhRoxQWFqZmzZrJZDJZ12HevHmzkpKSJElt27bV2LFjnfIGAAAAgNpkKDC7ubnpueee08svv6zU1FSdPn1aX375pd1rQkND9ec//1murq5GmgYAAADqhKEpGVJpAH7zzTc1efJk60OAFfHx8dHNN9+sN998UyEhIUabBQAAAOqEoRFmCy8vL91999264447dPLkSSUlJVmXjfPz81P79u3VuXNnubk5pTkAAACgzjg1wbq5ualbt27q1q2bM6sFAAAA6k21A/OFCxe0atUq7du3T+np6SopKVFQUJB69uypX//612rXrl1t9BMAAACoF9UKzPHx8XrjjTd06dIlm+MpKSlKSUnR5s2b9bvf/U4jR450aicBAACA+uLwQ3+XLl3SO++8YxOW/fz8FBgYaH195coV/eMf/7AuHwcAAAA0dg6PMG/YsEFZWVmSSjcsueuuuxQaGipJys7O1sqVK7Vq1SoVFxfrhx9+0Jw5c2qlwwAAAEBdcjgw79u3T5LUo0cPPf744zbnAgICdPfdd+vy5ctav369tSwAAADQ2Dk8JcMyzWL8+PGVlpk0aZKk0hHn7Oxsg10DAAAA6l+15jBLUps2bSot07p163LlAQAAgMbM4cB85coVSbK7+UjZc5byAAAAQGNmeGtsAAAAoCkjMAMAAAB2VHunvw8//FCenp6Gy5lMJr344ovVbR4AAACoU9UOzCdOnHBqOQAAAKAhY0oGAAAAYIfDI8z/+c9/arMfAAAAQIPECDMAAABgB4EZAAAAsIPADAAAANhBYAYAAADsIDADAAAAdhCYAQAAADsIzAAAAIAdBGYAAADADgIzAAAAYAeBGQAAALCDwAwAAADYQWAGAAAA7CAwAwAAAHa4OVLokUcecXrDJpNJ7733ntPrBQAAAJzJocCclpZW2/0AAAAAGiSHAnNkZKTd8wkJCUpMTJQk+fj4qFOnTmrWrJkk6eLFi0pISNClS5ckSR07dlSHDh2M9BkAAACoMw4F5t///veVntuwYYNiY2MVHByse+65R4MGDZKrq6tNmZKSEm3fvl1fffWVzpw5owkTJmjMmDHGeg4AAADUAUMP/Z04cUL//Oc/5e/vr1deeUVDhgwpF5YlycXFRUOGDNFf/vIX+fn56ZNPPtGJEyeMNA0AAADUCUOBefXq1SopKdGtt96q4ODgKssHBQXp1ltv1ZUrV7Rq1SojTQMAAAB1wlBgPnz4sCSpa9euDl9jKRsfH2+kaQAAAKBOGArM2dnZkqSioiKHr7GUtVwLAAAANGSGAnNAQIAkad++fQ5fs3fvXptrAQAAgIbMUGDu1auXJGnVqlUOTbE4cuSIVq9ebXMtAAAA0JA5tKxcZW655RZt3bpVRUVFevnllzVu3DiNGjVKHTp0kMlkkiSZzWYlJiZq06ZN+umnn1RcXCw3Nzfdcsstzug/AAAAUKsMBea2bdtqzpw5eu+991RcXKy1a9dq7dq1cnNzk5+fn0wmk3JyclRcXGy9xsXFRb///e/Vtm1bw50HAAAAapuhwCxJQ4cOVWhoqD799FOdPHlSklRcXKysrKxyZTt16qQHH3xQYWFhRpsFAAAA6oThwCxJYWFheu2113TixAkdOHBASUlJys3NlST5+vqqffv26t27N0EZAAAAjY5TArNFly5d1KVLF2dWCQAAANQrQ6tkAAAAAE2dU0eYJSkjI0NZWVkqKChQWFiYPDw8nN0EAAAAUGecEpgvX76slStXavPmzcrMzLQe/+tf/6p27dpZX2/ZskXbt2+Xj4+Pfve73zmjaQAAAKBWGQ7MycnJeu2115SSklJl2a5du+q9996T2WzWqFGjFB4ebrR5AAAAoFYZmsNcWFio119/XSkpKfL09NSUKVP0zDPPVFo+NDRUPXv2lCTt2rXLSNMAAABAnTA0wrxu3TqdP39enp6eWrBggTp27FjlNf3799fBgwd19OhRI00DAAAAdcLQCPOOHTskSTfddJNDYVmSOnToIKl0KgcAAADQ0BkKzGfPnpUk9enTx+Fr/P39JUl5eXlGmgYAAADqhKHAnJ+fL0ny8vJy+JqioiJJkqurq5GmAQAAgDphKDD7+flJktLS0hy+5vTp05KkwMBAI00DAAAAdcJQYO7UqZMk6fDhww5fEx0dLUnq1q2bkaYBAACAOmFolYwbbrhBe/fu1X//+19NnjxZISEhdsuvXr3aGq6HDRtmpGlJpcvarVixQlu3blV6err8/PzUt29fzZw5U8HBwQ7VcenSJe3du1e7du3SsWPHlJmZKXd3d7Vr107Dhw/X+PHj5ebm9A0RAQAA0EgYGmEeOXKk2rdvr6KiIr300kvau3evzGazTRmz2azjx4/r73//u7744gtJUo8ePdS/f38jTauwsFDz58/X8uXLlZ+fr4EDB6p58+batGmTnn76aYc2UpGkH374QX//+9+1bds2+fn5afDgwQoLC1NiYqI+//xzLViwQAUFBYb6CgAAgMbL0NCpi4uLnnnmGb3wwgtKS0vT66+/Lk9PT+v5+fPn6/Lly9YH/SSpZcuWevzxx400K0n69ttvdezYMXXr1k3PP/+89cHDVatW6YsvvtBHH32kl156qcp6LBuuTJw40WaEPDk5WS+//LLi4+O1fPly3XnnnYb7DAAAgMbH0AizJIWEhOjNN9/UsGHD5OLiYjMam52dbROWhwwZoldffVXNmjUz1GZxcbGioqIkSQ888IDNKh2TJ09Whw4dFBcXp5MnT1ZZ16233qpZs2aVm07SunVra0jesmWLof4CAACg8XLK5Fw/Pz/94Q9/0B133KE9e/boxIkTys7OVklJifz8/NSpUycNGDBAbdq0cUZzio+PV15enlq2bGl98LCswYMHKzExUbt27VLnzp1r3I5lM5YLFy7UuA4AAAA0bk59mq1FixaaMGGCM6usUGJioiRVGJYlWUNyUlKSoXYs86BZAg8AAODaZSgwp6enS5KCg4Pl4uLY7I6SkhJlZmZKUpWralTVbvPmzSs8b1khozrrQ1dkzZo1kqSBAwcaqgcAAACNl6HAPGfOHJlMJr311ltq166dQ9ekpqbqsccek8lk0uLFi2vUrmWHwbIPGJZlmdNsKVcT69at04EDB+Tr66tbbrmlyvJFRUU287VNJpO8vb2tPwNNFX/fQPVx3wA1U1/3juEpGVcvI1fb19WFw4cP6/PPP5fJZNLs2bMdWtN5xYoVWrZsmfV1p06dtHDhQrVo0cJp/TrntJoA52ndunV9dwFodBrPfcO/PGhY6uveqbcdORydwlERywhyZesjW0aWy66e4aikpCS98cYbKi4u1v33369BgwY5dN2tt96qyZMnW19bPgGlpaWpuLi42v0AGovk5OT67gLQ6HDfADXj7HvHzc3NocHNOg/MlhUnahJmLSxznzMyMio8b5kjXd3R3dTUVL3yyiu6dOmSpk+frkmTJjl8rbu7u9zd3Ss815BH0wGj+PsGqo/7BqiZ+rp36iwwFxcXKyUlRd9++60kGVpirkOHDpKkU6dOVXjesv5y+/btHa7zwoULevnll3XhwgXddNNNmj59eo37BwAAgKajWoH59ttvr/D4k08+We2Gb7jhhmpfYxEeHi4fHx+lpKQoISHBul6yxfbt2yU5vrpFbm6uXnnlFaWkpGjUqFG69957a9w3AAAANC2Gd/qriSFDhujXv/51ja93c3PTxIkTJUmffvqpzWoYq1atUmJioiIiImw2LYmKitIf//hHff311zZ1FRQU6PXXX1dSUpKGDBmi3/3udzy9DAAAAKtqjTBPmzbN5rVlVYhx48bZ3e7aZDLJ3d1dQUFB6tatm1q1alWDrtq67bbbdODAAR05ckSPPfaYwsPDlZ6ermPHjikgIECzZ8+2KZ+dna1z586V27Xvm2++0dGjR+Xi4iJXV1d99NFHFbY3Z84cw30GAABA41OtwHz1vF5LYJ44caLD6zA7i4eHh+bNm6cVK1YoNjZWO3fulJ+fn0aNGqXbb7+90k1Nrnbp0iVJpRuqxMbGVlqOwAwAAHBtMpkNPG64adMmSdKgQYPk4+PjrD41KWlpaTYbmhiR/KcHnVIP4Eyt3/ykvrtQpfsWbavvLgA2Pr93SH13wSE/fs/yd2hYJkxx7jrM7u7utb+s3KhRo4xcDgAAADR49fLQHwAAANBYOHUd5tzcXCUmJionJ0eFhYVVLi4dGRnpzOYBAAAAp3NKYD506JCWLFmi+Ph4h68xmUwEZgAAADR4hgPzunXr9K9//avaWxWyLSgAAAAaA0OB+cyZM/rss89kNpvVvn173X777XJ1ddXrr78uSXrvvfeUm5urEydOaP369Tp16pTCw8P10EMPydPT0ylvAAAAAKhNhh76i4qKUklJiQICArRgwQINHDhQISEh1vOhoaHq3Lmzxo0bp9dee00333yz4uPj9dlnnzm0hAcAAABQ3wwF5ri4OEnSpEmT5O3tbbesyWTSrFmz1KtXLx08eFAbNmww0jQAAABQJwwF5oyMDElSp06drMdMJpP15+Li4nLXjB07VpIUExNjpGkAAACgThgKzJYd7IKCgqzHys5Ntmw7XVarVq0klc5/BgAAABo6Q4HZz89PklRQUGA9FhAQYP05Obn8lpo5OTmSpLy8PCNNAwAAAHXCUGBu27atJNtg7OnpqdatS/f53rVrV7lrduzYIck2WAMAAAANlaHA3L17d0kqt2HJoEGDJElr167Vxo0blZ+fr4sXL2rlypXWh/169uxppGkAAACgThgKzAMGDJAk7dy5U4WFhdbjN998s/z8/FRcXKz/+7//07333qv/9//+n77++muVlJTIw8NDt9xyi6GOAwAAAHXBUGDu2rWrZs+erbvuusvmAT9/f3/NnTtXoaGh5a4JCAjQU089pXbt2hlpGgAAAKgThrfGHjVqVIXHO3furHfeeUcHDx7UmTNndOXKFbVu3Vp9+/Zllz8AAAA0GoYDs93K3dzUr18/9evXrzabAQAAAGqNoSkZAAAAQFNHYAYAAADscNqUjJycHB09elQpKSnKz89XSUlJlddMmzbNWc0DAAAAtcJwYM7KytKiRYu0fft2XblypVrXEpgBAADQ0BkKzNnZ2Xr++eeVlpbmrP4AAAAADYqhwLxkyRJrWL7hhhs0fvx4dejQQb6+vjKZTE7pIAAAAFCfDAXm3bt3S5JGjhypOXPmOKVDAAAAQENiaJWM7OxsSdLo0aOd0hkAAACgoTEUmIODgyVJXl5eTukMAAAA0NAYCsw9evSQJCUlJTmlMwAAAEBDYygw33zzzXJzc9MPP/ygwsJCZ/UJAAAAaDAMBebrrrtOs2fP1rlz5/TKK6/o3LlzzuoXAAAA0CAY3rhk+PDhat26tV5//XU98cQT6tChg1q3bi0PDw+715lMJs2ePdto8wAAAECtMhyYz507py+++MK6YkZCQoISEhIcupbADAAAgIbOUGBOT0/XvHnzrGFZKl0xg41LAAAA0FQYCszLli1Tdna2TCaTJk+erPHjxys0NNRZfQMAAADqnaHAfODAAUnSTTfdpFmzZjmlQwAAAEBDYmiVjIsXL0qSBg8e7JTOAAAAAA2NocAcFBQkSXJzM/zsIAAAANAgGQrMvXv3liSdOHHCKZ0BAAAAGhpDgXnKlCny8vLSypUrlZub66w+AQAAAA2GocDcqlUrPfXUU7p8+bJeeOEF7d+/31n9AgAAABoEQ5OP58+fL0ny9/e3bo/t6+vr8E5/L774opHmAQAAgFpnKDDHxcWVO3bp0iUdP37cSLUAAABAg2EoMPfo0YMd/QAAANCkGQrML730kpO6AQAAADRMhh76AwAAAJo6AjMAAABgB4EZAAAAsMOhOczp6enWn0NCQio8XhNl6wIAAAAaIocC85w5cySVrp28ePHicsdr4uq6AAAAgIaoWlMyzGaz0xp2Zl0AAABAbXFohHn27NnVOg4AAAA0FQ4F5lGjRlXrOAAAANBUsEoGAAAAYIehnf42b94sSbr++uvl4+Pj0DX5+fnavn27JCkyMtJI8wAAAECtMxSYP/zwQ0nSX//6V4cDc1ZWlj788EOZTCYCMwAAABq8epuSwSoZAAAAaAzqPDCXlJRIklxdXeu6aQAAAKDa6jwwnzt3TpLk5+dX100DAAAA1VatOcxxcXEVHj9+/Liys7PtXltcXKzz58/rhx9+kCR17NixOk0DAAAA9aJagXn+/PkVHv/oo4+q3fCNN95Y7WsAAACAumZolYyaCA4O1q233qpBgwbVddMAAABAtVUrMM+bN8/6s9ls1oIFCySVbpEdGhpq91oPDw8FBgYqJCSkBt0EAAAA6ke1AnNERESFx8PCwtSuXTundAgAAABoSAxNyXj//fcllU6zAAAAAJoiQ4G5RYsWzuoHAAAA0CDV+kN/u3bt0rZt25STk6PQ0FCNGTNGnTt3ru1mAQAAAKcwFJgPHjyod999V+7u7nrrrbfk6+trc37x4sVasWKFzbH169dr9uzZGjlypJGmAQAAgDphaKe/vXv3KicnR126dCkXlhMTE23CsuV8SUmJPv74Y6WmphppGgAAAKgThgJzfHy8JKlPnz7lzq1bt05SaVB+/fXX9a9//Uuvvvqq/Pz8VFRUpJ9++slI0wAAAECdMBSYs7KyJKnCJeX27NkjSZo4caI6deokSerSpYsmTJggSTpw4ICRpgEAAIA6YSgwZ2dnS1K56Rjnz59XZmamJJXb0a9Hjx6SpJSUFCNNAwAAAHXCUGA2m82SpLy8PJvjlqkaPj4+6tixo805f39/SVJBQYGRpgEAAIA6YSgwBwYGSpLOnj1rc3zfvn2SpO7du5e7Jj8/X1L5UWkAAACgITIUmLt27Sqp9AE/y4hxSkqKdu3aJanihwGTk5Ml/S9sAwAAAA2ZoXWYx44dq61btyoxMVFPPvmkOnXqpMOHD6uoqEgeHh4aPnx4uWsOHz4sSWrdurWRpgEAAIA6YWiEuVevXpo0aZIkKS0tTTt27FBOTo4k6e6771ZAQIBN+cLCQu3cuVOSFBERYaRpAAAAoE4Y3hr7vvvuU+/evbVt2zZdvHhRgYGBioyMVK9evcqV3bVrl3x8fOTj46MBAwYYbRoAAACodYYDsyQNGDDAoQA8dOhQDR061BlNAgAAAHXC0JQMAAAAoKkjMAMAAAB2ODwlY9GiRZKkW265Rc2aNSt3vqSkxLq7X0hISKX1pKSk6O2335YkLVy4sFqdvVphYaFWrFihrVu3Kj09XX5+furbt69mzpyp4ODgatWVm5urpUuXaufOncrKylJgYKAGDRqk6dOns2Y0AADANczhEeY1a9ZozZo11lUwrnb27FnNmTNHjzzyiN16CgsLlZCQoISEhGp1tKJ65s+fr+XLlys/P18DBw5U8+bNtWnTJj399NPV2no7Oztbf/7zn7V27Vq5urrq+uuvl7e3t9asWaM///nPys3NNdRXAAAANF5OeeivLMt22bXt22+/1bFjx9StWzc9//zz8vLykiStWrVKX3zxhT766CO99NJLDtX1+eef6/z58xo0aJAef/xxubq6SpL+9a9/KSoqSosWLdKcOXNq660AAACgAWuUc5iLi4sVFRUlSXrggQesYVmSJk+erA4dOiguLk4nT56ssq4LFy5oy5YtcnNz04MPPmgNy9L/1pKOiYnRxYsXnf9GAAAA0OA1ysAcHx+vvLw8tWzZUp06dSp3fvDgwZJk3aLbnn379slsNqtHjx7ltut2d3fXgAEDVFJSor179zql7wAAAGhcGmVgTkxMlKQKw7Ikde7cWZKUlJRkuC7LcUs5AAAAXFsaZWBOT0+XJDVv3rzC85YVMtLS0hyuq7JVNSxtOFIXAAAAmh6nP/RXF/Lz8yVJnp6eFZ63zGm2lDNSl+V4VXUVFRWpqKjI+tpkMsnb21tubs77FXt37OK0ugBncXd3r+8uVCmsZWB9dwGw0RjuG0lqHuJd310AbDj73nE0pzXKwNwQrVixQsuWLbO+HjZsmB577DEFBQU5rY0Wr7zntLqAa8mHvx1b310AGqUp01rUdxeABqFRTsmwjCAXFBRUeN4yGlx29Yya1mU5XlVdt956qz7//HPr/x566CGbEWc0HJcvX9Yzzzyjy5cv13dXgEaFeweoPu6bpqHaI8w//vhjhTv9lV12rexIq71yNWXZSTAjI6PC85YdB1u0qPqTsaUuyzVXs7RRVV3u7u6N5iu2a53ZbNapU6fqbM1woKng3gGqj/umaah2YF63bl2VZZYuXVqjzjiqQ4cOkqRTp05VeN6y/nL79u0N12U5bikHAACAa0ujnJIRHh4uHx8fpaSkVLjF9vbt2yVJAwcOrLKufv36yWQy6fDhw+VGv4uKirR79265uLiof//+Tuk7AAAAGheHR5jnzZtXm/2oFjc3N02cOFHffvutPv30U82dO9dma+zExERFRERY12OWpKioKEVFRWnQoEG68847rceDgoI0bNgwxcbG6pNPPtEf//hH625/X331lbKzsxUZGVnhNBQ0Tu7u7po2bRpTaIBq4t4Bqo/7pmkwmRvppJrCwkLNnz9fx44dU1BQkMLDw5Wenq5jx44pICBAr7zyilq2bGktv2TJEi1btkyRkZGaM2eOTV3Z2dmaO3euUlJS1LJlS3Xp0kWnT5/W6dOn1bp1a73yyivy8/Or67cIAACABqBRTsmQJA8PD82bN09Tp06Vh4eHdu7cqbS0NI0aNUoLFy60CctVCQgI0GuvvaaJEyequLhYO3bsUF5eniZNmqRXX32VsAwAAHANa7QjzAAAAEBdaLQjzAAAAEBdYKc/NGibNm3Shx9+KKn0wdOePXtaz1nmpV/NxcVFvr6+atu2rfr3769x48bZTKs5ePCgFixYIEmaM2eOIiMjK23/ueee04kTJ6pVtnXr1vrb3/5WvTcK1KKy91FZ7u7u8vHxkb+/v9q3b6+wsDDdcMMN1vXpK5KamqpHHnnE+jowMFD/93//JxcX++MvX3/9tb777jvr62nTpmnGjBnVfzNAPajNe6gsT09PBQUFKSwsTKNHj1bv3r2d9h5gDCPMaHJKSkqUk5Oj+Ph4ffPNN3riiSd0/Phx6/lu3bpZV0I5fPhwpfXk5+fbrM/taNkePXoYfQtAnSgqKtLFixd15swZbd26VV988YUeeeQRvfXWW5Vu5nS1rKws7d+/324Zs9msmJgYZ3QZaFCccQ+VVVBQoPPnzys2NlYvv/yyPvroIzY8aSAYYUaT8Pbbb1s/0V+5ckWpqamKiorSxo0blZWVpYULF+pvf/ubfHx85OHhoS5duujo0aOKj4+vtM6jR4+qpKRE7u7uKioqcqisJEVERDj3zQFO9Nxzz1k/1JWUlOjSpUvKyMhQfHy8Nm/erLNnz2rHjh2Ki4vTs88+q27dulVal7e3ty5fvqzo6Gj169ev0nKHDh1SRkaGtTzQmDnzHrrlllt02223SSr9YJmRkaG4uDgtXrxYubm52rhxo9q1a6ebb765Tt4bKscIM5oEDw8PeXl5ycvLS76+vurUqZNmz56tUaNGSSrdkv2///2vtXx4eLgk6dy5c5Vu124ZUR4yZIg8PDwcKisxwoyGrey94uPjoxYtWig8PFy33HKL3n77bd1zzz1ycXFRbm6u3nzzTWVkZFRa1w033CBJ2rlzp/Lz8ystFx0dLan0XgIaO2feQ25ubta6vL291a5dO40fP17PP/+8TCaTpNL9JVD/CMxo0qZNm2b9+cCBA9afy44CVzbVwjKi3KtXL3Xp0sWhsiEhIWrRooWxTgP1xGQyafLkydbNnS5evKilS5dWWj48PFyhoaEqKCiw7rB6tbLnRo4c6fxOAw1Ide+hynTu3Fl9+vSRJF24cEHnz593aj9RfQRmNGmhoaHWXSDLfsoPDw+3fnqvaKpFcXGxjh07Zi3bvXt3h8sCjd3NN9+s1q1bSyodHc7Ozq6wnMlk0ogRI6zlKrJz505dvnxZLVq04NsXXDMcvYfsadOmjfXnmlwP5yIwo8mzBOOyD074+Pioffv2kioeNT558qQKCwsVFBSkVq1aWYOwvbIS85fRNJhMJo0ePVpS6QfCuLi4SstaAvPBgwcrfMhp8+bN1nKWexFo6qpzD1Wm7L9Zvr6+TusbaobAjCYtMzPT+pBRcHCwzTnLaFdiYqLy8vJszlmCsWVkuXv37jKZTHbLlq0TaOwsf/tS6UOtlWnTpo26du1a4UoYWVlZ1qlQTMfAtcbRe6gyZ8+elVS61BxT/eofgRlN2rfffmv9uewazpJsnnK++j9mlhBsGVn29fVVu3bt7JZt1qyZ2rZt69w3ANQTy9fJUmnwtaeyaRkxMTEqKSlRly5dbL5eBq4F1bmHrnby5EkdPHhQkjR69Gh5eHg4s2uoAQIzmpySkhKdP39eixYt0k8//SSpNPDeeOONNuXKjgaXHSU2m806cuRIuTIVTcsoW5b5y2hKyn4FnJuba7fssGHD5OrqqtOnTyshIcF63BKg7W34AzRVjtxDxcXFys/PV35+vi5fvqwzZ85o3bp1+stf/iKz2azw8HDrA4SoX6zDjCahsl2TpNL5yk888YQCAgJsjgcGBqp169ZKTk62CcGnT5/WpUuX5O3trQ4dOliPh4eH66effqqwrMR0DDQtZedPVjX32N/fX/3799euXbsUHR2tjh07KikpSYmJiXJ1ddXQoUNru7tAg+PIPfTdd9/Z7IBZ1h133KEpU6ZYN9pC/WKEGU2Sm5ub2rdvrylTpujtt9+udHtRS8g9ceKEioqKJP1vBLlbt2422/1a5qNVVLZsXUBTUHauviMPHFnmKMfGxqqkpMT6sF+/fv3KfVgFrgXVvYeu9t1331lXYEL9Y4QZTULZnf5cXFwcnu/Vo0cPbdiwQUVFRTp+/Lh69OhRbv6yRWhoqIKDg5WZmVmurI+Pj81oNNDYJScnW38ODAyssvyAAQPk6+urrKws7du3T7GxsZKYjoFrlyP30LRp0zRjxgxJpdtsp6WlacOGDfrhhx90+fJl/fWvf9Vbb72lZs2a1UWXYQcjzGgSyu68VJ2HIyqax2xZa7miOclXz2O2lO3evbvNaDTQ2Fnm5ku2T/tXxt3d3bqT32effaYLFy7I19dXAwYMqLU+Ag1ZTe6hNm3aaNasWbr33nsllW588u9//7vW+gjH8S88rmmhoaFq3ry5pNIQnJqaqszMTLm6uqpr167lylv+o1e2rMR0DDQtZrNZGzdulFQ6vcnRv2/LtIyUlBRJpVtnu7u7104ngQaspveQxaRJk6z/3kRHR+vcuXNO7yOqh8CMa57lP2RHjx7VoUOHJJVuS1rRSLVlhLls2bJ1AE3B6tWrrV8nR0ZGOjwHOTw8XC1btrS+ZjoGrlU1vYcsTCaT7rjjDkmlKz+VXSIV9YM5zLjmhYeHKzY2VpcvX1ZUVJT1WEU6dOggb29vm7IeHh7q0qVLnfUXqC1ms1lr1661fgUcGBio6dOnV6uO+fPnWzcLYu1lXGuccQ9ZREREqGfPnjp06JC2bNmiGTNmKDQ01JndRTUQmHHNK7ud9alTpyRVPmLs4uKirl27av/+/dayXbt2lZsbtxIah8LCQuXn50sqHbnKy8tTenq6jh49qo0bN1p3F/P399ef/vSncjtkVqW65YHGprbvobJmzJihefPm6cqVK1qxYoUefvhhp7wHVB//yuOa17ZtW/n7+ysnJ0dS6Vdh9h7QCA8P1/79+62vmY6BxuS1116ze97FxUXXX3+9fvvb3yooKKiOegU0HnV5D/Xo0UO9evXSwYMHtXnzZk2bNs363A3qFoEZ1zyTyaTw8HDt3LlTUunXyP7+/pWWv3q6RtkRaqAxcXd3l7e3t/z9/dWhQweFhYXphhtusC7RCMC+uriHZsyYoYMHD6q4uFgrV67Ub3/7W6fVDceZzGW3ogEAAABgg1UyAAAAADsIzAAAAIAdBGYAAADADgIzAAAAYAeBGQAAALCDwAwAAADYQWAGAAAA7CAwAwAAAHYQmAEAAAA7CMwAAACAHQRmAAAAwA63+u4AAKDxeumllxQXFydJ+v3vf69Ro0bVb4cAoBYQmAHUSElJiQ4dOqTdu3fryJEjysrKUnZ2tjw8PBQYGKiWLVuqf//+GjhwoJo3b+5wvampqXrkkUcqPW8ymeTl5SU/Pz917NhRvXr10siRI+Xr61vjOqvSokULffDBBzW+XpIOHTqk+fPn1/j6iIgIvfTSS4b6AACoGQIzgGrbu3evvvzyS505c6bcuaKiIl26dElnz57Vnj179Pnnn2v8+PGaPn26/Pz8DLdtNpt1+fJlXb58WWlpadq5c6e++eYb3X333Ro3bpzh+q9VS5Ys0bJlyyRJkZGRmjNnTj33qHEo+0HIGR+sADRMBGYADjObzfr888+1du1am+Ourq4KDQ1VUFCQ8vPzlZ6eruzsbEnSlStXtHbtWv3888+aO3eu2rdvX602u3TpUi5o5+bmKikpSUVFRZKk/Px8/fOf/9TFixc1bdq0GtVpT7NmzarVZ0f06NFDHh4eDpfv0KGD0/sAAHAMgRmAQ8xms9577z3FxsZaj/n7+2v69OkaOnSoAgICbMoeO3ZM33//vXbs2CFJunDhgubNm6e5c+cqLCzM4XZnzZqlnj17ljteUFCgqKgoLV68WFeuXJEkLV26VP369auy/srqrEtz5sxRaGhovfbBGZgmAuBawCoZAByyZs0am7AcFhamd955RxMnTrQJy1LpPONu3brpqaee0iOPPCIXl9L/1Fy6dEnvvPOO8vLyDPfH09NTv/nNb/T73//eesxsNmvFihWG6wYAoCwCM4AqnT17Vl9//bX1ddu2bTV37txyQbkiI0eO1IMPPmh9nZaWps8//9xpfRsxYoQ6d+5sfX3gwAEVFxc7rX4AAJiSAaBK33//vXW+sMlk0sMPP2x3VYqr3Xjjjdq+fbt++eUXSVJ0dLSmT5+uFi1aOKV//fr108mTJyWVzmdOS0tT69atnVJ3Y7F//37FxsbqxIkTysjIUH5+vtzd3eXn56eWLVuqU6dO6tmzp/r27St3d3frdWWXhbPYvHmzNm/eXGE777//vs1UEkeWlavswbjTp0/rp59+0oEDB5SRkSGz2aw2bdpo+PDhmjhxok0/JSk7O1tRUVHavXu3kpOTdeXKFYWEhKh///665ZZbFBgY6NDv6uTJk9q/f7/i4+N19uxZZWVlqbi4WH5+fgoJCVGPHj00evRoXXfddZXWUfYhSYu0tDTNmDGjwvL2ltwrLCzU5s2btWvXLp0+fVoXL16Uu7u7AgMD1aNHDw0dOlS9e/eu8n1t2rRJH374oSTbVVWOHz+uTZs26fDhw8rMzNSlS5cqXHXl8uXLiomJ0e7du5WUlKScnBxduXJFnp6eCgoKUuvWrdW1a1cNGDCg2s8iAI0dgRmAXdnZ2TZTMfr166fw8PBq13PHHXdYA3NJSYnWrl2re+65xyl9DAkJsXmdk5NzzQTmS5cu6d1337X+bssqKChQQUGBMjIyFBcXp9WrV2v8+PE2I/71ZfXq1frqq6+s888tTp06pVOnTunnn3/W888/L29vb0nSvn379Pe//125ubk25ZOTk5WcnKzNmzdXOT8+NzdXc+fOVXJycoXnL168qIsXL+rEiRNavXq1xo0bp/vuu09ubrX3T+Uvv/yif/zjH0pPT7c5XlRUpLy8PJ07d07r169Xv379NGfOnGo9gFpcXKyvvvpKa9asqbLswYMH9d577+nChQvlzuXl5SkvL09nz57Vrl279M033+jVV1+t1rMIQGNHYAZg1759+6yjy5I0evToGtXTuXNntW/fXklJSZKkXbt2OS0wXz0FozYDTkNSUlKihQsXKj4+3nrM3d1dbdu2lb+/v4qLi5WTk2MdjbVcU1ZYWJjc3d11/vx5paSkSJKCgoIqHUGszsoelVm3bp0WLVokSfLx8VG7du3k6uqqxMRE6/z2Y8eO6e2339bcuXN16NAhvfHGGyouLparq6vat28vHx8fnT9/XhkZGZJKPzgsXLhQ7777bqXffhQUFNiEZQ8PD7Vu3Vq+vr4ymUzKzMzU+fPnZTabZTabtW7dOmVnZ+uJJ54oV1erVq3Ut29f5ebm6sSJE5JKf/cREREVth0cHFzu2Pbt2/Xuu+/afGho1qyZ2rRpo8LCQpuVYPbt26cXX3xR8+bNq7Cuinz++edat26dJMnLy0vt2rWTu7u7UlNTbcolJibqtddes7nPAwIC1Lp1a3l4eCg/P18ZGRnKzMy0nr/67who6q6Nf1UA1FjZMGYymdSnT58a19WnTx9rYD5//rwuXrzolCXbzp07Z/O6NpaBa4h27Nhh/f+Pm5ubZs2apbFjx8rT09OmXHFxseLi4hQbG1su8M6aNUuS7RSDPn361No6zNnZ2Vq0aJE8PT113333KTIy0voBp6ioSN98841WrVolqXT0defOnfrkk09UXFysX//615o6darNkoAxMTH68MMPdeXKFV28eFE//PCDZs6cWWn7gYGBGjNmjAYNGqSOHTtaH0i1yMzM1KpVq7R69WqZzWb9/PPPio2N1fDhw23KjRw5UiNHjrSZbhIYGKi5c+c69HtITU3VBx98YA3LgYGBevDBBzVw4EBrn/Ly8vTdd99p5cqVMpvNSk5O1gcffKDnn39eJpPJbv2nTp1SXFycvLy8NGvWLI0ePdpmiovlw5EkffPNN9aw3LZtWz388MMVfot08eJF7d69W+vXr3foPQJNCYEZgF2nTp2y/tyqVSv5+PjUuK6yD+dJ0okTJ/SrX/2qxvVJpWHQsnSdVDqSV52dBRuzvXv3Wn+eMmWKbrrppgrLubm5qU+fPurTp0+9jwwWFBTIxcVFc+fOLTca6+7urnvuuUcJCQk6ePCgJOmdd95RcXGxZs6cqdtuu61cfSNGjFBycrI17EdHR1camJs1a6YPP/zQ7jcQwcHBuueeexQSEmJ9OHX16tXlArNRX3zxhfLz8yWVjrLPmzdPbdu2tSnj4+OjO++8U/7+/vryyy8llT7Uum3bNg0dOtRu/ZcvX5aLi4ueffbZCke9W7ZsKal0nfQDBw5IKv1A/PTTT1c6nalZs2YaM2aMxowZU+9/R0BdIzADsOvixYvWn40+pHf1XGPL5iZGfPHFFzZfFQ8aNKjKa6q7RbW9B7Zqqrpbdc+bN6/c2tFl37ej88qvHlGtD2PHjq106oLlvCUwFxcX67rrrtMtt9xSafkbb7xRy5cvl9lsVnp6ujIyMir80FSdqTqTJk3SqlWrlJ6erhMnTujChQsKCgpy+Hp70tPTtWvXLuvrGTNmlAvLZU2ePFk///yzjh07JkmKioqqMjBL0rhx4+z+nqXSe9AyumyZhuGIhvB3BNQl/uIB2FX2ISsjo8sVXX/1A1yOMJvNys3N1b59+/SXv/xFUVFR1nPe3t52g1VTU/Yr9sTExHrsSfWMGTPG7vmrHyaLjIy0G9CCg4Nt5vWePXvWWAdVOtpath/Hjx83XKfFnj17rCO0np6eVf4+TCaTzbcH8fHxDn3YvPHGG6ssU/ZvKDs7W1lZWVVeA1yLGGEGYFfZB4GMPkx39TJhhYWFVV7j6Giwh4eHnnzySYceiKru1tiOPmRVHdXdGrui/nbu3Nk6Urls2TIFBQVp2LBhcnV1dVo/nc3NzU0dO3a0W+bq5eG6detWZb2BgYE2DwBWpaCgQL/88otOnTqltLQ05eXlqbi4WGaz2VrGMt9esh3NN8oyUiyVLv/m5eVV5TX9+/eXyWSy9u/48eN2pzP5+Pg4tPSbn5+fQkNDlZqaKrPZrDfffFMPP/wwy8YBVyEwA7DL19fXOpp1+fJlQ3VdfX11Qqs9vXv31v3336927do5VL6pbI09ZswYff/998rPz1dBQYHef/99LVq0SP3791dERITCw8PVpk0bJ/XYOfz9/asM9Fc/tOjIBjllrykoKKi0XGFhoZYtW6Yff/yxWn/Pztid0uL8+fPWn+2t9VyWj4+PQkJClJaWVq6OioSGhlb5YKDFTTfdZJ2vfezYMT311FPq0KGD+vbtqx49eqh79+5Ou1eBxorADMCusoG5JlMoyrr6ekf+Eb56NNjFxUWenp7y9/dXx44d1atXr2tmzeWrBQcH68knn7TZbjwnJ0fR0dGKjo6WJDVv3lwDBw7U2LFjqxzZrQs1+ZbCWcsEXr58WX/5y19sRngdVfabFqPKhm9/f3+HrwsICLAG5qpG0S3rVzti0qRJOnfunHUJOql0ik9iYqK+//57mUwmdenSRUOHDtWYMWMMT80CGiMCMwC7WrZsaV279vTp0zKbzQ6PXF2t7FfckhwaYW0Io8ENWd++ffXuu+9q1apViomJKbfxREZGhn788UetW7dOkZGReuCBB8qN4F4rvvzyS5uw3K9fPw0dOlSdOnVScHCwvLy8bKYNffDBB5XueGhETac5lS1bVYCvzj1qMpn04IMPaujQoVqzZo327t1rU7/ZbNbx48d1/PhxLV++XA888IDTVw0BGjoCMwC7unfvrn379kmSdbcvR6c+XK3sg1Oenp7q1KmTM7p4zQsMDNSsWbM0a9YsnTlzRocOHVJcXJwOHjyonJwcSaWhZ9OmTcrNzdXTTz9dzz2uezk5OdqwYYP19axZszRlyhS71xidglSZsiO01WmjbNnqbE3vqIiICEVERKiwsFBHjhxRXFyc4uLidPToUet60ZcuXdLf//53ubu7a/DgwU7vA9BQEZgB2HX1slRbtmzR7bffXu168vPztXv3buvrbt26NeiH0xqrdu3aqV27dpowYYJKSkr0yy+/6Ouvv7auorFr1y4dPnxYPXr0qOee1q2DBw9aV6YIDQ3VzTffXOU1FW0T7Qxl52RfveteZcxms01ZR+Z115SHh4d69+6t3r17SyqdSrV+/XotW7bMOj/8q6++IjDjmsKycgDsCg8Pt5kjvHHjRodWt7japk2bbB7GGjt2rFP6h8q5uLiof//+euGFF2zmyv7yyy8VlrUou1JEU5Genm79uXPnzlVOWSgsLFRCQkKV9Zatx9HfW9lvVizbalfl9OnT1o1Orq6jtvn5+ek3v/mNHnjgAeuxlJSUKh88BJoSAjMAu0wmk379619bX2dmZmrp0qXVquPixYv6z3/+Y30dEhLC6FQdCggIUPfu3a2vy25GY1F2XnNNPhA1dJYpBY7asmWLQw/6lV0SztHfW9nR/dOnT5eb21+RmJgY68++vr71suzb9ddfb/O6or8joKkiMAOo0pgxY2y2tf7+++8VGxvr0LX5+fl68803bZ7q/+1vf8t0DCeozkhw2dHJilYnKbv2cUpKiqF+NURl39+xY8fsbu186dIlmw94jtabk5Pj0PJzffr0sdmJ8Ouvv7ZbPj093WaDnqo2cqmO6vwN1daykEBjQGAGUCU3Nzc99thj1tE0s9ms999/X0uXLlVxcXGl1yUkJGjevHk6evSo9dj48eM1cODAWu/ztWDBggVat25dlSFtz549OnTokPV1RfOXy37Fn5CQYN2auqkoOxc/IyNDy5cvr7Bcdna2XnvtNYc3KgkODlazZs0kld4Xa9asqfIaFxcX/eY3v7G+3rNnj/79739XGOKzsrK0cOFC63QmLy8vm298jDp8+LBef/11HTp0yG54LikpsQn2gYGB1+xyjrg28dAfAIe0bt1ac+fO1euvv65Lly6ppKRES5cu1fr16zVkyBB17dpVgYGBys/PV1pamnbv3q0DBw7YfBUeGRmp+++/vx7fRamvvvqq2qNjDz74oFq2bOm0PnzwwQfV2ulPkp588kmbKQCpqan65JNPtGjRIvXt21fdunVT27Zt5efnp5KSEqWlpWnPnj3avn27NQx16dJF/fr1K1d3u3bt1LFjRyUkJMhsNmvBggXq0KGDmjdvbjOa+fDDD1sDYmMSGhqqgQMHWndGXLp0qY4fP64RI0aoefPmysvLU3x8vNavX6/c3FwFBQWpY8eO2rt3b5V1Dxs2zBqUlyxZog0bNqht27Y2S9TddNNN6tWrl/X1hAkTtGPHDusHk5UrV+rgwYMaPXq02rRpo6KiIh05ckQ//fSTdaUTSbrnnnvUokULp/xOpNKQv2fPHu3Zs0fNmzdX//791blzZ4WEhMjLy0v5+flKSkrS5s2bdfr0aet1t9xyi9NGuYHGgMAMwGHdu3fXyy+/rA8++MD6sFJmZqZWr15t9zoPDw/deuutuu2222q8hrMzOfqgVVnOXmLs8OHD1b6msnm4RUVF2rVrlzUMVqZ169Z68sknKw06Dz/8sP7yl79Yp89YNq8o67777qt2vxuKhx56SKdOnbJuob13794KA7GPj48ef/xxrV+/3qF6Z8yYoQMHDlgDZXp6us1DhpI0aNAgm9cmk0l/+tOf9MYbb1hH/0+cOGH3b3PWrFm68cYbHepTTWRkZOi///1vleXGjRunSZMm1Vo/gIaIwAygWtq1a6dXX31VsbGxWrdunY4ePVrpV7nNmjXT9ddfr9tuu00hISF13NOm74477tC2bdt06NAhu9My/P39NXbsWN122202I9RX69Kli/76179q3bp1OnDggJKTk3X58uVqPzDXUAUFBenVV1/Vxx9/bLPEoYXJZFKfPn300EMPKTQ01OHA7OPjo9dee00bNmzQrl27dPr0aeXm5lb50KC3t7eef/55/fjjj1q5cmWly9h1795dd911l8LDwx3qT3W0b99e06dP1549e3Tq1Cm7c7s7deqkqVOnlgv/wLXAZG6K6wcBqDM5OTk6evSosrKylJOTI3d3dzVr1kytWrVS586d+dq2DpSUlOjMmTM6d+6cMjMzdfnyZbm5ucnf31/t27dXx44dnba9dFORkpKiw4cP68KFC/Lw8FBwcLC6d++u4ODgeumP2WzWiRMnlJSUpOzsbLm5uSkwMFA9evSweUCwNuXn5yshIUHnz59Xdna2ioqK5OXlpeDgYHXp0sWhnTmBporADAAAANjB0A8AAABgB4EZAAAAsIPADAAAANhBYAYAAADsIDADAAAAdhCYAQAAADsIzAAAAIAdBGYAAADADgIzAAAAYAeBGQAAALCDwAwAAADYQWAGAAAA7CAwAwAAAHYQmAEAAAA7/j/Ja5XWRHGNfAAAAABJRU5ErkJggg==\n"
},
"metadata": {}
}
],
"source": [
"# estimate the policy value of IPWLearner with Random Forest\n",
"estimated_policy_value_b, estimated_interval_b = ope.summarize_off_policy_estimates(\n",
" action_dist=action_dist_ipw_rf,\n",
" estimated_rewards_by_reg_model=estimated_rewards_by_reg_model\n",
")\n",
"print(estimated_interval_b, '\\n')\n",
"\n",
"# visualize the estimated policy values of IPWLearner with Random Forest\n",
"ope.visualize_off_policy_estimates(\n",
" action_dist=action_dist_ipw_rf,\n",
" estimated_rewards_by_reg_model=estimated_rewards_by_reg_model,\n",
" n_bootstrap_samples=1000, # number of resampling performed in bootstrap sampling\n",
" random_state=12345,\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"tags": [],
"id": "mtE7nu7loGgS",
"outputId": "1cb9dd3f-99d0-4f7a-fa74-13dd1b2c0933",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 709
}
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
" mean 95.0% CI (lower) 95.0% CI (upper)\n",
"ipw 0.683341 0.680651 0.685275\n",
"dm 0.717023 0.716593 0.717411\n",
"dr 0.683142 0.679919 0.685859 \n",
"\n"
]
},
{
"output_type": "stream",
"name": "stderr",
"text": [
"/usr/local/lib/python3.10/site-packages/seaborn/categorical.py:82: FutureWarning: iteritems is deprecated and will be removed in a future version. Use .items instead.\n",
" plot_data = [np.asarray(s, float) for k, s in iter_data]\n"
]
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"<Figure size 800x600 with 1 Axes>"
],
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAswAAAIzCAYAAAAUFU/WAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAA9hAAAPYQGoP6dpAABzG0lEQVR4nO3deXxU1eH///dkT0hCEkLCJhCWEBZZJIKsYREERerC4kLVFv1ailarVttCRaQuqFVbFz61blCLFCJRixCxrAkgO7KEQEJIWBKyEpIQss/vD34zzZBkMslMVl7Px6OPx8y9555zhuaa95yce47BaDQaBQAAAKBaTk3dAQAAAKA5IzADAAAAVhCYAQAAACsIzAAAAIAVBGYAAADACgIzAAAAYAWBGQAAALCCwAwAAABYQWAGAAAArCAwAwAAAFYQmAEAAAArCMwAAACAFS6OqKSsrEzJycnKyMhQbm6uioqK5OLiIi8vLwUGBuqGG25Qu3btHNEUAAAA0KjqHZgzMjIUGxurQ4cOKTExUeXl5VbL+/v7a+DAgRo2bJiGDBkiZ2fn+jYNAAAANBqD0Wg01uWCPXv2aMOGDYqLi6t3o97e3po4caJuu+02Rp4BAADQrNkcmPfs2aPVq1fr7Nmz5mOurq7q3r27evXqpR49esjX11fe3t7y9vZWSUmJCgoKVFBQoLS0NJ06dUqJiYnKzs42X+/i4qIJEyZoxowZatu2reM/HQAAAGAnmwLz4sWLzSPKrq6uGjJkiMaMGaObbrpJLi51m9Vx4cIFxcTEaMeOHUpLS5MkeXl56YknntDQoUPr8RGat4sXL6qsrKypuwEAAIBruLi4yN/fv9ZyNgXm2bNny9vbW9OmTdNtt90mLy8vh3QyPj5eX331lQ4fPqyZM2dqxowZDqm3OcnMzFRpaWlTdwMAAADXcHV1Vfv27WstZ1Ng/vbbbzV58mR5eHg4pHPXSkxMVH5+voYMGdIg9TclAjMAAEDz5NDAjPojMAMAADRPtgZmNi4BAAAArCAwAwAAAFYQmAEAAAArbF4Tbtu2bQ5vPCIiwuF1AgAAAI5kc2D+8MMPHdqwwWAgMAMAAKDZa7IpGSzOAQAAgJbA5hHmefPmNWQ/AAAAgGaJdZgbGOswAwAANE+swwwAAAA4AIEZAAAAsMLmOcySdPnyZR0/flyS1K5dO4WEhNh8bVJSknJyciRJ/fv3l6enZ12aBgAAAJpEnQLzl19+qR9++EEeHh565ZVX6tSQm5ub3nvvPRUVFen222/Xww8/XKfrAQAAgKZg85SM3Nxcbdq0SZJ03333qUuXLnVqqEuXLrr//vslSRs3blReXl6drgcAAACags2BOTY2VhUVFQoICNDkyZPr1dikSZMUGBiosrIyxcbG1qsOAAAAoDHZHJiPHTsmSRo+fLicnZ3r1Zizs7OGDx8uSTp69Gi96gAAAAAak82B+cyZM5KuPrBnj759+0qSUlJS7KoHAAAAaAw2B+b8/HxJkr+/v10Nmq431QcAAAA0ZzavklFRUeHQhh1dHwDU5JHlu5q6C4CFzx8e0dRdAFAHNo8w+/j4SJIuXrxoV4O5ubkW9QEAAADNmc2BOSgoSJLMG5fUV1xcnEV9AAAAQHNm85SMG2+8UfHx8YqNjdWsWbPk4eFR58aKiooUExNjrs9eJSUlioqK0s6dO5WVlSVvb28NGjRI9913nwICAmyqY+vWrfrwww9rLTd//nxFRETY22UAAAC0MDYH5hEjRigyMlKXLl3SihUr9P/+3/+rc2PLly9XXl6enJycNHLkyDpfX1lJSYkWL16shIQE+fv7Kzw8XJmZmdq6dasOHDigV155RcHBwbXW06FDhxqDcGFhofbu3StJCgsLs6u/AAAAaJlsDsydO3fW6NGjFRMTY97x75FHHpGbm1ut15aUlOjzzz/X5s2bJUmjR49Wp06d6tnlq9auXauEhASFhoZq4cKF5hHvdevWacWKFVq2bJleeumlWusJCwurMQxv3LhRe/fuVZ8+fWwK3wAAAGh9bA7M0tWAnJCQoAsXLmjTpk06cOCAJk6cqCFDhqh79+5ycflfdWVlZUpOTtaBAwe0efNm88OCnTp10iOPPGJXp8vKyhQdHS1Jmjt3rsX0kGnTpmnbtm2Ki4tTUlKSevToUe92TNNHxo4da1d/AQAA0HLVKTB7e3vrj3/8o5YuXarz58/r4sWLioyMVGRkpJycnOTl5SV3d3cVFxersLCwytJxnTt31u9//3u1adPGrk7Hx8ersLBQwcHBCgkJqXJ++PDhSklJ0b59++odmDMyMnTixAm5uLhoxAiW/wEAALhe2bxKhklwcLBef/11TZkyxWI6RkVFhQoKCpSdna2CggKLsOzm5qapU6fqtddec8jqGKZdAqsLy5LMIdm0O2F9bN++XZJ00003ydvbu971AAAAoGWr0wiziZubm37xi1/o3nvv1c6dOxUXF6eUlBTl5+frypUr8vT0lI+Pj7p166Z+/fpp5MiR8vX1dVins7KyJEnt2rWr9rxphYzMzMx6t1HX6RilpaUqLS01vzcYDPL09DS/BgDAhN8LQMtSr8Bs4uvrqylTpmjKlCmO6o9NioqKJEnu7u7VnjfNaTaVq6vExESlpaXJ29tbN910k03XREVFKTIy0vw+JCRES5cuVfv27evVBwBA69WxY8em7gKAOrArMLdWpukYI0aMsHiQ0Zq7775b06ZNM783jR5kZmaqrKzM8Z0EALRYaWlpTd0FAJJcXFxsGtxskYHZNIJcXFxc7XnTyHJ9NlcpLy/Xrl27JNVtdQxXV1e5urpWe85oNNa5HwCA1ovfC0DLUueH/pqDwMBASVJ2dna153NyciSpXtMhfvrpJ126dEnBwcHq06dP/TsJAACAVqFFBuZu3bpJkk6fPl3t+aSkJElS165d61y36WG/MWPG1LN3AAAAaE1aZGAOCwuTl5eX0tPTlZycXOX87t27JUnh4eF1qreoqEj79u2TRGAGAADAVS0yMLu4uJhX5vjkk08sVsNYt26dUlJS1K9fP4tNS6Kjo/X0009r5cqVNda7e/duFRcXq3fv3jzBDAAAAEkt9KE/Sbrnnnt05MgRnThxQk899ZTCwsKUlZWlhIQE+fr6at68eRbl8/LylJqaat6iuzpshd36LVmyxDzHPSAgQH/605+auEcAgNaK3zmtR4sNzG5ublq0aJGioqIUGxurvXv3ytvbW+PGjdPs2bNr3NSkJhcvXtTRo0fl7OyskSNHNlCv0dRycnLMG98AANCQ+J3TerTYwCxdDc2zZ8/W7Nmzay07a9YszZo1q8bz/v7+WrVqlSO7BwAAgFagRc5hBgAAABpLg48wFxUV6cqVK/L09KzXRiL4n7TfPdrUXWjxynOKK73O4t/UATq++XFTdwFAA/n+W3YktMeVwnKL1/x72u+26U2zKEODBOaysjJ9++232rZtmy5cuGA+HhwcrIiICE2fPr3GXfEAAACA5sThgbmoqEgvv/yyTp06VeVcenq6Vq9erX379unFF1+Up6eno5sHrGrrbJBkrPQaAADAOocH5sjISJ06dUqdOnXSfffdpz59+qhNmzbKzs5WTEyM1q5dq6SkJEVGRurnP/+5o5sHrPp1O7em7gIA4Drh5dm22tdoeRwemHft2iUXFxctWrRIfn5+5uMdOnTQzJkz5ebmppUrV+rHH38kMAMAgFbrtgm/buouwEFsXiXj/fffV15eXq3lcnNz1aFDB4uwXFnfvn3N5QAAAIDmzubAHBMTo6efflrR0dGqqKiosZyfn58uXLhQYyCOi4szlwMAAACaO5sD85133qmioiJ99tln+v3vf68TJ05UW2748OEqKyvTkiVLtHfvXuXl5amsrEzp6emKjIzU6tWrJUm33HKLYz4BAAAA0IBsnsM8Z84cjRs3Tp9++qmOHTumF198UWPHjtWcOXPUtu3/JrLPnDlTx44dU3Jyst56661q6+revbtmzJhhf+8BAACABlannf66dOmiF198Ub/5zW/k7++v7du36+mnn9aGDRvM0zQ8PT21ZMkSzZgxQ0FBQRbXBwUFaebMmVqyZAlLygEAAKBFqNcqGaNGjdLQoUO1evVqbdiwQZ9//rm2bNmiuXPnqk+fPnJzc9PMmTM1c+ZMFRUVqbCwUJ6enoRkAAAAtDh1GmGuzMPDQw899JDefPNN9evXTykpKXrxxRf1wQcf6NKlSxblAgICCMsAAABokeodmE26dOmiRYsW6cknn5Sfn1+10zQAAACAlsruwGwyevRo/fWvf9Xtt9+uoqIiff755/r973+v+Ph4RzUBAAAANLp6BeaKigqdO3dOJ0+e1Llz51ReXi7p6vSLhx9+WEuXLlVYWJhSUlK0aNEivf/++xbTNAAAAICWok4P/RUUFGjlypWKiYlRSUmJ+bibm5tGjx6tBx98UN7e3uratasWL16s7du364svvlBMTIz279+vmTNnasqUKXJyctjANgAAANCgbE6u+fn5WrBggTZt2qSSkhI5OTmpbdu2cnJyUklJiTZv3qwFCxZYbJ89duxYvfvuu5oyZYqKioq0fPlyvfDCC0zTAAAAQIth8wjz6tWrdeHCBXl6euqxxx7TiBEj5OTkpIqKCu3atUsff/yxLly4oDVr1mju3Lnm67y8vPSLX/xCEyZM0CeffKITJ05o0aJFGjNmjJ544okG+VAAAACAo9g8wrxv3z5J0oMPPqhRo0aZp1U4OTlp1KhReuCBByzKXatbt256+eWXNW/ePPn6+iomJsbevgMAAAANzuYRZtNUi+7du1d73nS88pSM6owbN07Dhg3TqlWrbG0aAAAAaDI2jzD7+/tLko4fP17tedNxPz+/Wuvy8vLSL3/5S1ubBgAAAJqMzYH5lltukST9+9//1jfffKOsrCyVlZUpKytL69at0+rVqyVJI0aMaJieAgAAAE3A5ikZM2bM0LFjx5SUlKSVK1dq5cqVVcp0795d9957r0M7CAAAADQlmwOzh4eHXn75ZX3zzTfatm2bMjIyzOeCgoI0ZswY3XXXXXJzc2uQjgIAAABNoU4bl7i6umrGjBmaMWOGioqKVFhYKE9PT3l6ejZU/wAAAIAmVafAXJmHh4c8PDwc2RcAAACg2WGPagAAAMAKAjMAAABgRb2nZNTkzJkzio6OVlxcnHJyciRdXcO5X79+mjx5skJCQhzdJAAAANBgbA7MkZGRkqTJkyfL19e32jKrVq3S119/LaPRaHH8woULunDhgrZs2aLp06ebt9EGAAAAmjubA/OaNWskXd3ApLrAvGbNGkVFRZnfe3p6qmPHjjIYDEpLS1NhYaGMRqO++eYbubu7s14zAAAAWgSHTMlIS0vTV199Jel/216PGjVKTk5Xp0hXVFRox44d+vTTT1VYWKjIyEhFREQoMDDQEc0DAAAADcYhD/1t3LhRRqNRLi4uWrhwocaMGWMOy5Lk5OSkMWPGaMGCBXJxcVFFRYU2bdrkiKYBAACABuWQwBwXFydJGjt2rHr27FljuV69emnMmDEW1wAAAADNmUMCs2mb7KFDh9Za1lQmNTXVEU0DAAAADcohgbmkpESSbJqTbCpTWFjoiKYBAACABuWQwOzn5ydJKi8vr7VsRUWFJMnV1dURTQMAAAANqs6rZCQmJiovL8/iWEBAgLKyspSenm51DrMkXbx4UZLk4+NT16YBAACARlfnwLxs2bIaz504cUIjR460en1SUpKkqyEbAAAAaO4cMiXDZPfu3eYpFzXZu3evpKsrZgAAAADNnc0jzIsWLbKpXFlZmdzc3Ko9FxcXpzNnzkiSQkNDbW0aAAAAaDI2B+Z+/frZ3Vi3bt30/vvvS5L8/f3trg8AAABoaA7ZGttWbdq0UZs2bRqzSQAAAMAujRqYHa2kpERRUVHauXOnsrKy5O3trUGDBum+++6r10OFGRkZ+vrrr/XTTz/p4sWL8vT0VIcOHTR8+HBNnz69AT4BAAAAmjuHPvTXmEpKSrR48WJ99dVXKioqUnh4uNq1a6etW7fq+eefV3p6ep3qO3jwoJ599llt2rRJPj4+GjZsmEJCQpSZmakffvihgT4FAAAAmju7RpgzMzN1/PhxnTt3TllZWSoqKlJJSYk8PDzk7e2tLl26qFevXurTp48MBoOj+ixJWrt2rRISEhQaGqqFCxfKw8NDkrRu3TqtWLFCy5Yt00svvWRTXefPn9dbb70lT09PLVy4UH369DGfq6io0OnTpx3adwAAALQc9QrMu3fv1tdff21eU7k2gYGBmjRpkqZNmyYXF/tngZSVlSk6OlqSNHfuXHNYlqRp06Zp27ZtiouLU1JSknr06FFrfStWrFBpaameffZZi7AsSU5OTrVuxgIAAIDWq05TMoqLi7V06VK9/fbbNodlScrKytKXX36p3/3ud+Zl5ewRHx+vwsJCBQcHKyQkpMr54cOHS5L27dtnU98OHTqk4OBg3XTTTXb3DQAAAK1LnYZ733rrLR0+fFjS1VHjG2+8UX5+fiosLNTx48d15swZubi46OGHH1aPHj2UlpampKQk7dmzR1lZWUpNTdXixYu1aNEide3atd6dTklJkaRqw7Ik86iyLeE8Li5ORqNRoaGhKi8v1+7du3XixAlVVFTohhtu0MiRI+Xt7V3vvgIAAKBlszkw79y50xyWZ8yYoXvvvVdOTpYD1LGxsfrwww/1xRdf6PXXX9eYMWM0ZswY/fznP9fmzZu1fPlyFRQU6L333tPSpUurXG+rrKwsSVK7du2qPW9aISMzM7PWus6dOydJ8vDw0IsvvqiEhASL86tWrdIzzzyjAQMG1KuvAAAAaNlsTqzbtm2TJI0YMUIzZ86sNuyOHj1a9957r4qLi7Vy5cr/NeLkpFtvvVUvvPCCDAaDzpw5o5iYmHp3uqioSJLk7u5e7XnTnGZTOWsKCgokSZs3b1Zqaqp+85vf6NNPP9W7776rMWPGqKCgQG+99ZZycnKs1lNaWqrCwkLz/65cuWI+ZzAYHPI/oDly1M93Q/4PaG6a+p7g3kFL1VQ/4zaPMCcnJ0uSIiIirJaLiIjQ6tWrdeDAAZWVlVk85DdgwACNHj1aMTEx2rVrV611NQaj0ShJKi8v12OPPaaRI0dKkry9vfXkk08qNTVVp06d0vfff6/777+/xnqioqIUGRlpfh8SEqKlS5eqffv2DutrqsNqAhynY8eOTd0FoMVpOfcNv3nQvDTVvWNzYM7Pz5dU+5bWpvPl5eXKyclRUFCQxflhw4YpJibGPA+5PkwjyMXFxdWeN40sV149o7a6PDw8NGLEiCrnx48fr1OnTikuLs5qPXfffbemTZtmfm/61pKZmamysrJa+wG0VGlpaU3dBaDF4b4B6sfR946Li4tNg5s2B2YvLy/l5+fr4sWL6t69e43lLl68aNGJawUGBkqS8vLybG26xjqys7OrPW+aPmHLP4CpTGBgYLVD86bztfXX1dVVrq6u1Z4zjWIDrRE/30Ddcd8A9dNU947Nc5i7desm6X9zmWuyfft2SVdHbP38/KqcN4222jL6W1tfatpQxLTknS0rcZjC/+XLl6s9b5rjbE9/AQAA0HLZHJiHDRsmSdq1a5ciIyOrTfg7d+40z+MdOHBgtQ8Gnj17VlLtUzusCQsLk5eXl9LT081zqyvbvXu3JCk8PLzWuvr06SMfHx/l5uYqNbXqXC3TVAxro+oAAABovWwOzBMmTDBPtF6zZo2efPJJffTRR/r3v/+tzz77TM8//7z++te/qry8XM7Ozrr33nurrefAgQOSpF69etW70y4uLpoyZYok6ZNPPrFYDWPdunVKSUlRv379LHb5i46O1tNPP22xeockOTs764477pDRaNQnn3yiwsJC87nDhw9r69atMhgMmjRpUr37CwAAgJbL5jnMrq6uev7557VkyRLl5OQoMzNTmzZtqlLOyclJjz32WLUjsjk5OYqLi5O7u7sGDhxoV8fvueceHTlyRCdOnNBTTz2lsLAwZWVlKSEhQb6+vpo3b55F+by8PKWmplrMsTaZPn26jh07piNHjuipp55S7969lZ+fr4SEBFVUVOi+++6zK+ADAACg5arTTn+dOnXS0qVLtWbNGsXGxlqMxjo7O2vAgAGaOXOmevfuXe31AQEB+uyzz+zr8f/Pzc1NixYtUlRUlGJjY7V37155e3tr3Lhxmj17do2bmlTHxcVFf/jDH/Tdd99p+/bt+umnn+Ti4qJ+/frpjjvu0NChQx3SZwAAALQ8BmM9HzesqKjQhQsXlJ+fLw8PD3Xo0KHGjUSuZ5mZmSotLXVIXWm/e9Qh9QCO1PHNj5u6C7V6ZPmupu4CYOHzh6suY9ocff8ty9+hebltumPXYXZ1dXXssnLXcnJyUqdOnep7OQAAANAi2PzQHwAAAHA9IjADAAAAVhCYAQAAACsIzAAAAIAVBGYAAADACgIzAAAAYAWBGQAAALCCwAwAAABYQWAGAAAArCAwAwAAAFYQmAEAAAArXBxVUUVFhXJyciRJgYGBdpcDAAAAmgOHBebz58/rueeek8Fg0KpVq+wuBwAAADQHDp+SYTQaHVoOAAAAaErMYQYAAACsIDADAAAAVhCYAQAAACsIzAAAAIAVBGYAAADAinotKxcZGVnl2KVLl6yenzFjRn2aAgAAAJpUvQLzmjVr6nyewAwAAICWqF6BuW/fvjIYDBbHioqKlJSUJEnq16+f/T0DAAAAmoF6BeaXXnqpyrGzZ8/queeekyQtWrTIrk4BAAAAzQUP/QEAAABWEJgBAAAAKwjMAAAAgBUEZgAAAMAKAjMAAABgBYEZAAAAsKJey8pVJygoyKbl5GwtBwAAADQHDgvM7u7uNm1YYms5AAAAoDlgSgYAAABgBYEZAAAAsILADAAAAFhhU2DevXt3g3YiJydHJ0+ebNA2AAAAgPqwKTC//fbb+t3vfqcff/zRoY1nZWXp448/1pNPPqnDhw87tG4AAADAEWxaJSM4OFhnzpzRO++8o8DAQI0ePVqjR4/WDTfcUOcGi4qKtGfPHu3YsUOHDx9WRUWFnJycFBwcXOe6AAAAgIZmU2B+++23tX79en3zzTfKysrS119/ra+//lodO3ZU79691bNnT4WEhKht27Zq06aN2rRpo5KSEhUUFOjy5ctKS0tTYmKiTp06pcTERJWUlJjrHjZsmO6//3516tSpwT4kAAAAUF82BWYXFxdNnz5dkydP1vfff6/vv/9e2dnZSktLU1pamrZv3163Rl1cNHz4cN1+++3q1atXvToOAAAANIY6bVzi4eGhn/3sZ7rzzjt1+PBh7dq1S8eOHVNmZmat17q6uqp3794KDw9XRESEvL29691pAAAAoLHUa6c/JycnDR48WIMHD5Z0dZWLEydOKDs7W3l5eSooKJCbm5t8fX3l6+urrl27qkePHnJxcdjGggAAAECjcEiCDQgI0IgRIxxRFQAAANCssHEJAAAAYAWBGQAAALCiRU8qLikpUVRUlHbu3KmsrCx5e3tr0KBBuu+++xQQEGBzPfPnz7f64OI777yjzp07O6LLAAAAaGFabGAuKSnR4sWLlZCQIH9/f4WHhyszM1Nbt27VgQMH9Morr9R5M5SIiIhqj3t5eTmiywAAAGiBWmxgXrt2rRISEhQaGqqFCxfKw8NDkrRu3TqtWLFCy5Yt00svvVSnOufPn98APQUAAEBL1iLnMJeVlSk6OlqSNHfuXHNYlqRp06apW7duiouLU1JSUlN1EQAAAK1EiwzM8fHxKiwsVHBwsEJCQqqcHz58uCRp3759jd01AAAAtDItckpGSkqKJFUbliWpR48ekqQzZ87Uqd5vv/1WFy5ckKurq2644QYNGzZMvr6+9nUWAAAALVqLDMxZWVmSpHbt2lV73rRChi1bdlf2xRdfWLxfvny5fvGLX2jChAm1XltaWqrS0lLze4PBIE9PT/NroLXi5xuoO+4boH6a6t5pkYG5qKhIkuTu7l7tedOcZlO52gwdOlQDBgxQjx495Ovrq/T0dG3ZskXr16/X3//+d/n4+Ojmm2+2WkdUVJQiIyPN70NCQrR06VK1b9/epj7YItVhNQGO07Fjx6buAtDitJz7ht88aF6a6t5pkYHZ0X75y19avL/hhhv00EMPqVOnTvroo4/0r3/9q9bAfPfdd2vatGnm96ZvQJmZmSorK3N8p4FmIi0tram7ALQ43DdA/Tj63nFxcbFpcNNhgbmiokLHjh3TyZMnlZubq+LiYt1///3y9/c3lykrK1N5ebmcnJzk6upa77ZMI8jFxcXVnjeNLFdePaM+JkyYoH//+99KTU1VRkaGgoKCaizr6upa42cyGo129QNozvj5BuqO+waon6a6dxwSmPfv36/PPvusypzh6dOnWwTmTZs26dNPP5WHh4f+/ve/1zvQBgYGSpKys7OrPZ+TkyNJdk+HcHJyUnBwsC5duqTc3FyrgRkAAACtk93Lyv33v//VG2+8YQ7LPj4+NZadOHGivLy8VFRUpD179tS7zW7dukmSTp8+Xe150/rLXbt2rXcbJpcvX5ZU83xpAAAAtG52Bea0tDR98sknkqQBAwbo7bff1scff1xjeRcXF/MayYcPH653u2FhYfLy8lJ6erqSk5OrnN+9e7ckKTw8vN5tSNLZs2eVmpoqd3d3de7c2a66AAAA0DLZFZi/++47VVRU6IYbbtAf/vAHm0Jl3759JdU8OmwLFxcXTZkyRZL0ySefWKyGsW7dOqWkpKhfv37m9ZglKTo6Wk8//bRWrlxpUdeBAwd09OjRKm2kpKTo7bffltFo1IQJE+TiwvORAAAA1yO7UqApaN5+++02B8oOHTpIqnn+sa3uueceHTlyRCdOnNBTTz2lsLAwZWVlKSEhQb6+vpo3b55F+by8PKWmpurixYsWxxMTExUZGan27durW7ducnNzU0ZGhk6fPq3y8nL1799fDz74oF19BQAAQMtlV2A2hd7u3bvbfI1pLnBNK1zYys3NTYsWLVJUVJRiY2O1d+9eeXt7a9y4cZo9e3aNm5pca/DgwcrOztapU6d04sQJFRYWytPTU2FhYRo9erTGjx8vJ6cWuYM4AAAAHMCuwGxaa7gu4begoECS5OXlZU/Tkq6G5tmzZ2v27Nm1lp01a5ZmzZpV5XhoaKhCQ0Pt7gsAAABaJ7uGTk1bUKenp9t8TXx8vCQpODjYnqYBAACARmFXYO7Xr58kadu2bTaVLyws1A8//CDp6qoaAAAAQHNnV2CeNGmSJCkuLk5bt261WjY/P19vvvmmcnNz5ezsbL4WAAAAaM7smsMcEhKi22+/XevXr9eyZct08OBB8zrLknTixAklJycrPj5eO3bsUGFhoSTp3nvvtXsXPgAAAKAx2L248EMPPaTS0lL98MMP+vHHH/Xjjz+az3300UdVyt9+++2699577W0WAAAAaBR2B2aDwaBHH31Uw4YN09dff624uDgZjcYq5UJDQ3XPPfdoyJAh9jYJAAAANBqHbV83cOBADRw4UFeuXNHp06eVl5eniooKeXt7q3v37vL19XVUUwAAAECjcfh+z56enubVMwAAAICWji3sAAAAACsIzAAAAIAVdk3J+PDDD+t9rcFg0Lx58+xpHgAAAGhwdgVmW3f4qwmBGQAAAM2dXYE5MDCw1jLFxcXKz883v/f19ZWbm5s9zQIAAACNxq7A/MEHH9hUrqCgQDt27NDq1avl5eWlF154QZ06dbKnaQAAAKBRNMpDf97e3rrtttu0ZMkS5eXl6dVXX1VBQUFjNA0AAADYpVFXyejUqZOmTp2qzMxMrVu3rjGbBgAAAOql0ZeVu/HGGyVJu3fvbuymAQAAgDpr9MDs4eEhScrKymrspgEAAIA6a/TAfPr0aUmSi4vDd+UGAAAAHK5RA3NGRobWrFkjSerevXtjNg0AAADUS4NvXGI0GlVQUKCkpCTt3btXJSUlkqRJkybZ0zQAAADQKJpka+ypU6dq5MiR9jQNAAAANIpGm0js5eWlvn376rbbbtOgQYMaq1kAAADALnYF5vfff7/WMgaDQZ6enmrTpo09TQEAAABNwq7A3L59e0f1AwAAAGiWGn1ZOQAAAKAlITADAAAAVhCYAQAAACtsmsP8xBNPOLxhg8Gg9957z+H1AgAAAI5kU2DOzMxs6H4AAAAAzZJNgTkiIqKh+wEAAAA0SzYF5l//+tcN3Q8AAACgWeKhPwAAAMAKAjMAAABgBYEZAAAAsMKurbGrk5GRofz8fJWUlMhoNFot269fP0c3DwAAADiUQwJzamqq1q5dq/3796uwsNCmawwGg1atWuWI5gEAAIAGY3dg3rNnj9577z2VlJTU6braRp8BAACA5sCuwJyVlWUOywEBAbrzzjvl7u6ujz76SJL0pz/9SQUFBUpKStL27dt18eJFhYWFaebMmXJyYvo0AAAAmj+7AvOGDRtUUlIiDw8PvfLKKwoICNDZs2fN5wcMGCBJuuWWWzRjxgwtW7ZMO3fu1ObNm/Wb3/zGvp4DAAAAjcCuYd4jR45Ikm677TYFBARYLevm5qYnn3xSISEh2rFjh3788Ud7mgYAAAAahV2BOTMzU5IUGhpqPmYwGMyvy8vLLRtzctLUqVMlSVu2bLGnaQAAAKBR2BWYi4qKJEmBgYHmY25ububX1a2YccMNN0iSUlJS7GkaAAAAaBR2BWYvLy9Jslghw8fHx/w6PT29yjWmEJ2fn29P0wAAAECjsOuhv06dOunkyZPKyMgwT8vw9PRUYGCgsrKy9NNPP6lXr14W1xw+fFjS/8K2PUpKShQVFaWdO3cqKytL3t7eGjRokO67775a51Rbk5aWpueee06lpaW68cYb9ac//cnuvgIAAKBlsmuE2RSST548aXF86NChkqRvv/1WR48eNR/fuXOn1q9fL0nq06ePPU2rpKREixcv1ldffaWioiKFh4erXbt22rp1q55//vlqR7dt9dFHH6msrMyu/gEAAKB1sGuEeciQIVq3bp327NmjRx55xLy28vTp07VlyxYVFRVpyZIl8vb2VmlpqYqLiyVdffhv+vTpdnV87dq1SkhIUGhoqBYuXCgPDw9J0rp167RixQotW7ZML730Up3r3bx5s44dO6Zbb71V//3vf+3qIwAAAFo+u0aY+/fvrxkzZmjcuHHKyckxHw8MDNQzzzxjnnZRUFBgDssuLi56/PHHLVbWqKuysjJFR0dLkubOnWsOy5I0bdo0devWTXFxcUpKSqpTvbm5ufrnP/+pgQMHatSoUfXuHwAAAFoPm0eY169fr9GjR8vX19d8zGAwaObMmdWWHzJkiP72t7/pxx9/1NmzZ1VRUaEOHTpo5MiRds0vlqT4+HgVFhYqODhYISEhVc4PHz5cKSkp2rdvn3r06GFzvZ9//rlKSkr06KOPKjs7264+AgAAoHWwOTAvX75cX3zxhQYOHKiIiAjdfPPNcnGxfrmPj48mTZpkdyevZVqSrrqwLMkcks+cOWNznQcOHNDOnTs1a9YsdejQgcAMAAAASXWcw1xeXq6DBw/q4MGD8vLy0ogRIzR27FiFhYU1VP+qlZWVJUlq165dtedNI9imjVVqU1RUpE8++USdOnXSXXfdVa8+lZaWqrS01PzeYDDI09PT/Bporfj5BuqO+waon6a6d2wOzI899phiYmIUHx8v6ep6yps2bdKmTZsUFBSkiIgIjR07VkFBQQ3WWRPThinu7u7VnjfNaTaVq82qVauUmZmpRYsW1TpqXpOoqChFRkaa34eEhGjp0qVq3759veqrTqrDagIcp2PHjk3dBaDFaTn3Db950Lw01b1jczq89dZbdeuttyojI0Pbt29XbGys0tLSJEkZGRlas2aN1qxZoz59+mjs2LEaOXKkQ9ZabminTp3Shg0bNHbsWPXv37/e9dx9992aNm2a+b3pG1BmZiZL1KFVM/13AIDtuG+A+nH0vePi4mLT4Gadh1ODgoI0Y8YMzZgxQ4mJidq+fbt27txp3rnvxIkTOnHihD7//HPddNNNioiI0JAhQ8xLzjmCaQTZtPLGtUwjy5VXz6hOeXm5/v73v6tNmzZ66KGH7OqTq6urXF1dqz1nNBrtqhtozvj5BuqO+waon6a6d+xah7lXr17q1auXHn74YR08eFDbt2/X/v37VVZWptLSUu3evVu7d++Wr6+vRo0apbFjx9Zp1YqaBAYGSlKND+aZlrir7RtDdna2kpOT5efnp7ffftvi3OXLlyVJSUlJ5vWc67OuMwAAAFo2uwKzibOzs8LDwxUeHq7CwkLt2rVL27dvN893zsvL04YNG7RhwwZ16dJFERERGj16dL2Xl+vWrZsk6fTp09WeN62/3LVrV5vqy83NVW5ubrXnLl++rLi4uLp3EgAAAK2CQwJzZV5eXpo4caImTpyozMxMbd++XTExMeY5J+fOndO//vUvffnll/ryyy/r1UZYWJi8vLyUnp6u5ORkde/e3eL87t27JUnh4eFW6wkKCtLq1aurPXfs2DEtXrxYN954o/70pz/Vq58AAABo+Rw3sbga7du317333qt3331XS5YsUZcuXcznKioq6l2vi4uLpkyZIkn65JNPLFbDWLdunVJSUtSvXz+L6R/R0dF6+umntXLlynq3CwAAgOuPw0eYr5WUlKSYmBjt2LFDly5dcli999xzj44cOaITJ07oqaeeUlhYmLKyspSQkCBfX1/NmzfPonxeXp5SU1N18eJFh/UBAAAArV+DBOasrCzFxsZq+/btOn/+fJXz/fv3V0REhF1tuLm5adGiRYqKilJsbKz27t0rb29vjRs3TrNnz65xUxMAAACgLhwWmK9cuaJdu3YpJiZGx48fr7LsR6dOnTR27FiNGTPGvMqFvdzc3DR79mzNnj271rKzZs3SrFmzbK67f//+Nc5vBgAAwPXDrsBcUVFhXk7uwIEDKikpsTjv7e1tXk6uV69ednUUAAAAaAr1CsymDUt27dqlvLw8ywpdXDRkyBBFRETopptukrOzs0M6CgAAADQFmwNzRkaGYmJiLJaIq6xXr14aO3asRo0aJW9vb4d2EgAAAGgqNgfmJ598ssqxwMBAjRkzRmPHjlWnTp0c2jEAAACgOajzlAwPDw8NHz5cERER6t+/f0P0CQAAAGg2bA7MAwcOVEREhIYNGyY3N7eG7BMAAADQbNgcmBcsWNCQ/QAAAACapQbdGhsAAABo6QjMAAAAgBUEZgAAAMAKAjMAAABgBYEZAAAAsILADAAAAFhBYAYAAACsIDADAAAAVtgVmHNzcx3UDQAAAKB5snmnv+rMmzdPQ4YM0fjx4zV06FA5OTFgDQAAgNbFrsBcUVGh/fv3a//+/fL19dXYsWM1fvx4denSxVH9AwAAAJqUXYH5jjvuUExMjPLy8pSXl6d169Zp3bp16tWrl8aPH69Ro0bJ09PTUX0FAAAAGp1dgfmhhx7SnDlzdODAAW3evFkHDx5URUWFEhMTlZiYqOXLl2v48OEaP368+vfv76g+AwAAAI3GrsAsSU5OTgoPD1d4eLguXbqk7du3a+vWrTp37pxKSkoUExOjmJgYBQUFafz48YqIiFC7du0c0XcAAACgwdkdmCtr27at7rzzTt15551KTEzUli1btHPnThUWFiojI0P//ve/tWbNGg0YMEATJkzQzTffLBcXh3YBAAAAcKgGS6u9evVSr1699Mgjj2j37t3aunWrjh49qoqKCh0+fFiHDx+Wt7e3xowZo1tvvZUHBQEAANAsNfg6cK6urgoNDVVoaKh8fHwszhUUFGjDhg169tln9dZbbykjI6OhuwMAAADUSYONMJeUlOjHH3/Uli1bdPz4cRmNRvO5Ll26aMyYMTpz5oz27t2rkpIS7d27V3FxcXr55ZcZbQYAAECz4fDAfOLECW3ZskW7du1SUVGR+bi7u7tGjBihiRMnKjQ01Hy8sLBQ69evV1RUlC5fvqxVq1bpueeec3S3AAAAgHpxSGDOyckxr46RlpZmca5nz56aMGGCRo8eLQ8PjyrXenl5acaMGfL29tZnn32mkydPOqJLAAAAgEPYFZh37typbdu26fDhw6qoqDAfb9OmjcaMGaOJEyeqa9euNtU1YMAASdKlS5fs6RIAAADgUHYF5r/+9a8W7/v3768JEyZo+PDhcnV1rVtHWF4OAAAAzZDdKdXPz0/jxo3ThAkTFBwcXO96goKC9P7779vbHQAAAMCh7ArMv/vd73TTTTfJycn+1emcnJzUvn17u+sBAAAAHMmuwBweHu6ofgAAAADNUoNvXAIAAAC0ZHYF5jNnzuiJJ57Qb37zG+Xk5NRaPicnR08++aSefPJJpaam2tM0AAAA0CjsCszbt29XZmamOnTooICAgFrLBwQEqFOnTsrIyFBMTIw9TQMAAACNwq7AfPz4cUl1m8tsKnv06FF7mgYAAAAahV2B2TStwtbNSSTphhtusLgWAAAAaM7sCsxFRUWSVO2W1zUxlS0sLLSnaQAAAKBR2BWYvb29JUm5ubk2X2Mq6+npaU/TAAAAQKOwKzB36NBBknTo0CGbrzGVtWdXQAAAAKCx2BWYBw0aJEnatGmTzp07V2v5s2fPatOmTZKkwYMH29M0AAAA0CjsCsyTJ0+Wu7u7SkpK9PLLL2v//v01lt23b5+WLFmikpISubm56bbbbrOnaQAAAKBR2LU1tq+vrx577DG9//77unTpkt544w0FBwcrLCxMfn5+kq7OWT5+/LgyMjLM1z322GPm8wAAAEBzZldglqQxY8bIaDTqH//4h0pKSpSenq709PRqy7q7u+vRRx/V2LFj7W0WAAAAaBR2B2ZJGjt2rAYOHKj169fr4MGDOnPmjPmcwWDQDTfcoKFDh2rKlCkOHVkuKSlRVFSUdu7cqaysLHl7e2vQoEG67777bNp5UJLKy8v11Vdf6dSpUzp//rzy8vJUXl6udu3aaeDAgfrZz36m9u3bO6zPAAAAaFkcEpglyc/PTw888IAeeOABlZeXq6CgQNLVpeecnZ0d1YxZSUmJFi9erISEBPn7+ys8PFyZmZnaunWrDhw4oFdeecWmlThKS0sVGRkpDw8PdevWTT169FBZWZmSk5O1ceNGxcTE6MUXX1TPnj0d/hkAAADQ/DksMFfm7Oystm3bNkTVZmvXrlVCQoJCQ0O1cOFC84Yo69at04oVK7Rs2TK99NJLtdbj6uqql19+Wb1797YI9hUVFVq1apW+/vpr/eMf/9Drr7/eUB8FAAAAzZhdq2Q0lbKyMkVHR0uS5s6da7HT4LRp09StWzfFxcUpKSmp1rqcnZ0VFhZWZRTcyclJs2fPlqurq5KSktiZEAAA4DrVIgNzfHy8CgsLFRwcrJCQkCrnhw8fLunqUnb2MBgMcnJyksFgkItLgwzGAwAAoJmzKQUuXrxY0tUA+eKLL1Y5Xh/X1lUXKSkpklRtWJakHj16SJLFw4d1ZTQa9fXXX6u4uFgDBgyQm5tbvesCAABAy2VTYI6Li6vT8YaWlZUlSWrXrl21500rZGRmZtap3i+++EKXLl3SlStXlJKSovT0dHXu3Fm/+tWv7OswAAAAWiybAnPfvn1lMBhsPt7QioqKJF1d17k6pjnNpnK22r17t8Ua0t26ddOTTz6poKCgWq8tLS1VaWmp+b3BYJCnp6f5NdBa8fMN1B33DVA/TXXv2BSYa1ptwpZVKFqS9957T5KUl5enpKQkrVq1Si+88IIef/xxjRs3zuq1UVFRioyMNL8PCQnR0qVLHbqGc6rDagIcp2PHjk3dBaDFaTn3Db950Lw01b3TIp9kM40gFxcXV3veNLJcefWMuvD19dXgwYMVGhqqZ599Vh9//LEGDBigwMDAGq+5++67NW3aNPN70zegzMxMlZWV1asfQEuQlpbW1F0AWhzuG6B+HH3vuLi42DS42SIDsym4ZmdnV3s+JydHkuwe3fXy8tLQoUO1ceNGHT58WBMmTKixrKurq1xdXas9ZzQa7eoH0Jzx8w3UHfcNUD9Nde+0yGXlunXrJkk6ffp0tedN6y937drV7rZ8fX0lXZ2mAQAAgOtPiwzMYWFh8vLyUnp6upKTk6uc3717tyQpPDzc7rZMK4F06NDB7roAAADQ8tg0JWP27NkOb9hgMGjVqlX1utbFxUVTpkzR2rVr9cknn2jBggUWW2OnpKSoX79+5vWYJSk6OlrR0dEaNmyYHnjgAfPxAwcOqE2bNurTp49FG8XFxVq7dq3i4uLk5+enwYMH16uvAAAAaNmabA6zvXNQ7rnnHh05ckQnTpzQU089pbCwMGVlZSkhIUG+vr6aN2+eRfm8vDylpqbq4sWLFscTExMVGRmpgIAAde/eXV5eXsrNzVVycrIKCgrk5eWl3/72t/V+gBAAAAAtm02BecaMGQ3djzpzc3PTokWLFBUVpdjYWO3du1fe3t4aN26cZs+eXeOmJtcaPny4ioqKdPz4cZ06dUoFBQVyc3NThw4ddOutt2rq1Kny9/dv4E8DAACA5spg5FHdBpWZmWmxoYk90n73qEPqARyp45sfN3UXavXI8l1N3QXAwucPj2jqLtjk+29Z/g7Ny23THbsOs6urq02rqrXIh/4AAACAxkJgBgAAAKxw+EN/ubm5Onv2rAoKCiRJ3t7euuGGG+Tn5+fopgAAAIAG55DAbDQa9d///lfR0dE6d+5ctWW6dOmiqVOnauLEieZtowEAAIDmzu7AXFBQoDfeeEMnTpywWu7cuXP6xz/+oe3bt+uFF15QmzZt7G0aAAAAaHB2BWaj0ag333zTHJa9vb01YsQI9e7d2zwFIzc3V4mJidq1a5fy8/N14sQJvfHGG1q8eLHdnQcAAAAaml2BOTY2VvHx8ZKk0aNH69FHH5Wnp2eVchEREXrwwQf18ccfKyYmRvHx8YqNjdXo0aPtaR4AAABocHatkhEbGytJ6tevn5588slqw7KJh4eHnnjiCfXr10+SFBMTY0/TAAAAQKOwKzCfPn1akjRlyhSbrzGVTU5OtqdpAAAAoFHYFZhNS8cFBQXZfI2prOlaAAAAoDmzKzB7eXlJki5evGjzNaay1qZvAAAAAM2FXYH5hhtukCRt2bLF5mu2bt1qcS0AAADQnNkVmG+55RZJ0p49e7R69WoZjUar5SMjI7V7925J0ogRI+xpGgAAAGgUdi0rN3HiREVHRys1NVVfffWV9uzZo3HjxqlXr15q27atDAaDeR3mbdu26cyZM5Kkzp07a+LEiQ75AAAAAEBDsiswu7i46A9/+IOWLFmijIwMnT17Vv/85z+tXhMUFKQ//vGPcnZ2tqdpAAAAoFHYNSVDuhqA33zzTU2bNs38EGB1vLy8dOedd+rNN99UYGCgvc0CAAAAjcKuEWYTDw8P/fznP9f999+vpKQknTlzxrxsnLe3t7p27aoePXrIxcUhzQEAAACNxqEJ1sXFRaGhoQoNDXVktQAAAECTqXNgvnjxotatW6dDhw4pKytLFRUV8vf3V//+/XXHHXeoS5cuDdFPAAAAoEnUKTDHx8frjTfe0OXLly2Op6enKz09Xdu2bdOvfvUrjR071qGdBAAAAJqKzQ/9Xb58We+8845FWPb29pafn5/5fXl5uf7+97+bl48DAAAAWjqbR5g3b96s3NxcSVc3LHnwwQcVFBQkScrLy9M333yjdevWqaysTP/5z380f/78BukwAAAA0JhsDsyHDh2SJPXt21e//e1vLc75+vrq5z//ua5cuaJNmzaZywIAAAAtnc1TMkzTLCZPnlxjmalTp0q6OuKcl5dnZ9cAAACAplenOcyS1KlTpxrLdOzYsUp5AAAAoCWzOTCXl5dLktXNRyqfM5UHAAAAWjK7t8YGAAAAWjMCMwAAAGBFnXf6+/DDD+Xu7m53OYPBoBdffLGuzQMAAACNqs6B+dSpUw4tBwAAADRnTMkAAAAArLB5hPnf//53Q/YDAAAAaJYYYQYAAACsIDADAAAAVhCYAQAAACsIzAAAAIAVBGYAAADACgIzAAAAYAWBGQAAALCCwAwAAABYQWAGAAAArCAwAwAAAFYQmAEAAAArCMwAAACAFQRmAAAAwAoXWwo98cQTDm/YYDDovffes6uOkpISRUVFaefOncrKypK3t7cGDRqk++67TwEBATbVcfnyZR08eFD79u1TQkKCcnJy5Orqqi5dumj06NGaPHmyXFxs+mcCAABAK2RTEszMzGzoftRZSUmJFi9erISEBPn7+ys8PFyZmZnaunWrDhw4oFdeeUXBwcG11vOf//xHa9eulcFgUPfu3dW7d2/l5eXpxIkTSkxM1I8//qgFCxbI3d29ET4VAAAAmhubAnNERITV88nJyUpJSZEkeXl5KSQkRG3btpUkXbp0ScnJybp8+bIkqXv37urWrZs9fZYkrV27VgkJCQoNDdXChQvl4eEhSVq3bp1WrFihZcuW6aWXXqq1Hnd3d02fPl1TpkxRYGCg+XhaWpqWLFmi+Ph4ffXVV3rggQfs7jMAAABaHpsC869//esaz23evFmxsbEKCAjQQw89pGHDhsnZ2dmiTEVFhXbv3q0vvvhC586d02233aYJEybUu9NlZWWKjo6WJM2dO9ccliVp2rRp2rZtm+Li4pSUlKQePXpYrevuu++u9njHjh31wAMP6G9/+5t27NhBYAYAALhO2fXQ36lTp/SPf/xDPj4+euWVVzRixIgqYVmSnJycNGLECP35z3+Wt7e3Pv74Y506dare7cbHx6uwsFDBwcEKCQmpcn748OGSpH379tW7DenqaLgkXbx40a56AAAA0HLZFZi/++47VVRU6O6777bpITt/f3/dfffdKi8v17p16+rdrmn6R3VhWZJ5VPnMmTP1bkOS0tPTJUl+fn521QMAAICWy67lH44fPy5J6t27t83XmMrGx8fXu92srCxJUrt27ao9bwrv9j6suH79eklSeHh4rWVLS0tVWlpqfm8wGOTp6Wl+DbRW/HwDdcd9A9RPU907dgXmvLw8SbIIirUxlTVdWx9FRUWSVOPKFaY5zaZy9bFx40YdOXJEbdq00V133VVr+aioKEVGRprfh4SEaOnSpWrfvn29+3CtVIfVBDhOx44dm7oLQIvTcu4bfvOgeWmqe8euwOzr66ucnBwdOnRIYWFhNl1z8OBB87XN1fHjx/X555/LYDBo3rx5Nk03ufvuuzVt2jTze9M3oMzMTJWVlTVYX4GmlpaW1tRdAFoc7hugfhx977i4uNg0uGlXYB4wYIC2b9+udevWafDgwbWG5hMnTui7774zX1tfphHk4uLias+bRpYrr55hqzNnzuiNN95QWVmZfvGLX2jYsGE2Xefq6ipXV9dqzxmNxjr3A2gp+PkG6o77Bqifprp37ArMd911l3bu3KnS0lItWbJEkyZN0rhx49StWzfzCKvRaFRKSoq2bt2qH374QWVlZXJxcbFpmkNNTOslZ2dnV3s+JydHkuo8HSIjI0OvvPKKLl++rJkzZ2rq1Kn17iMAAABaB7sCc+fOnTV//ny99957Kisr04YNG7Rhwwa5uLjI29tbBoNB+fn5FlMSnJyc9Otf/1qdO3eud7umjU9Onz5d7fmkpCRJUteuXW2u8+LFi1qyZIkuXryo22+/XTNnzqx3/wAAANB62BWYJWnkyJEKCgrSJ598Yg6qZWVlys3NrVI2JCREjz76qHr16mVXm2FhYfLy8lJ6erqSk5PN6yWb7N69W5Jtq1tIUkFBgV555RWlp6dr3Lhxevjhh+3qHwAAAFoPuwOzJPXq1UuvvfaaTp06pSNHjujMmTMqKCiQJLVp00Zdu3bVjTfeaHdQNnFxcdGUKVO0du1affLJJ1qwYIHF1tgpKSnq16+fxS5/0dHRio6O1rBhwyx27SsuLtbrr7+uM2fOaMSIEfrVr37Fcj8AAAAwc0hgNunZs6d69uzpyCprdM899+jIkSM6ceKEnnrqKYWFhSkrK0sJCQny9fXVvHnzLMrn5eUpNTW1yq59X375pU6ePCknJyc5Oztr2bJl1bY3f/78BvssAAAAaL4cGpgbk5ubmxYtWqSoqCjFxsZq79698vb21rhx4zR79uwaNzW51uXLlyVJFRUVio2NrbEcgRkAAOD6ZDA6eH2O7Oxs5ebmqri4WL169ZKbm5sjq29xMjMz67SxizVpv3vUIfUAjtTxzY+bugu1emT5rqbuAmDh84dHNHUXbPL9t6wXjebltumO3bjE1dW14ddhNrly5Yq++eYbbdu2zbykmyT95S9/UZcuXczvd+zYod27d8vLy0u/+tWvHNE0AAAA0KDsDsxpaWl67bXXlJ6eXmvZ3r1767333pPRaNS4ceNs3h0QAAAAaCpO9lxcUlKi119/Xenp6XJ3d9f06dP1wgsv1Fg+KChI/fv3lyTt27fPnqYBAACARmHXCPPGjRt14cIFubu76+WXX66yHnJ1hgwZoqNHj+rkyZP2NA0AAAA0CrtGmPfs2SNJuv32220Ky9L/dulLS+NBAgAAADR/dgXm8+fPS5IGDhxo8zU+Pj6SpMLCQnuaBgAAABqFXYG5qKhIksy77NnCtMSas7OzPU0DAAAAjcKuwOzt7S3p6lrDtjp79qwkyc/Pz56mAQAAgEZhV2AOCQmRJB0/ftzma7Zv3y5JCg0NtadpAAAAoFHYFZhvueUWSdJ///tfZWVl1Vr+u+++M4frUaNG2dM0AAAA0CjsCsxjx45V165dVVpaqpdeekkHDx7UtTttG41GJSYm6m9/+5tWrFghSerbt6+GDBliT9MAAABAo7BrHWYnJye98MIL+tOf/qTMzEy9/vrrcnd3N59fvHixrly5Yn7QT5KCg4P129/+1p5mAQAAgEZj1wizJAUGBurNN9/UqFGj5OTkpOLiYvO5vLw8i7A8YsQIvfrqq2rbtq29zQIAAACNwq4RZhNvb2/95je/0f33368DBw7o1KlTysvLU0VFhby9vRUSEqKhQ4eqU6dOjmgOAAAAaDQOCcwm7du312233ebIKgEAAIAmZVdgNq2MERAQICcn22Z3VFRUKCcnR9LV6RwAAABAc2ZXYJ4/f74MBoPeeustdenSxaZrMjIy9NRTT8lgMGjVqlX2NA8AAAA0OLsf+rt2GbmGvg4AAABoTHYH5no3bOMUDgAAAKApNXpqvXjxoiTJw8OjsZsGAAAA6syhq2RYU1ZWpvT0dK1du1aSWGIOAAAALUKdAvPs2bOrPf7ss8/WueFbbrmlztcAAAAAja1JJhKPGDFCd9xxR1M0DQAAANRJnUaYZ8yYYfE+MjJSkjRp0iSr210bDAa5urrK399foaGh6tChQz26CgAAADS+OgXmmTNnWrw3BeYpU6bYvA4zAAAA0JLY9dDfvHnzJF3d6Q8AAABojewKzOPGjXNQNwAAAIDmid1DAAAAACscug5zQUGBUlJSlJ+fr5KSklq3v46IiHBk8wAAAIDDOSQwHzt2TKtXr1Z8fLzN1xgMBgIzAAAAmj27A/PGjRv16aef1jqafK26lgcAAACagl2B+dy5c/rss89kNBrVtWtXzZ49W87Oznr99dclSe+9954KCgp06tQpbdq0SadPn1ZYWJgee+wxubu7O+QDAAAAAA3Jrof+oqOjVVFRIV9fX7388ssKDw9XYGCg+XxQUJB69OihSZMm6bXXXtOdd96p+Ph4ffbZZ2rfvr3dnQcAAAAaml2BOS4uTpI0depUeXp6Wi1rMBg0Z84cDRgwQEePHtXmzZvtaRoAAABoFHYF5uzsbElSSEiI+ZjBYDC/Lisrq3LNxIkTJUkxMTH2NA0AAAA0CrsCc2lpqSTJ39/ffKzy3OTLly9XuaZDhw6Srs5/BgAAAJo7uwKzt7e3JKm4uNh8zNfX1/w6LS2tyjX5+fmSpMLCQnuaBgAAABqFXYG5c+fOkiyDsbu7uzp27ChJ2rdvX5Vr9uzZI8kyWAMAAADNlV2BuU+fPpJUZcOSYcOGSZI2bNigLVu2qKioSJcuXdI333xjftivf//+9jQNAAAANAq7AvPQoUMlSXv37lVJSYn5+J133ilvb2+VlZXp//7v//Twww/r//2//6eVK1eqoqJCbm5uuuuuu+zqOAAAANAY7ArMvXv31rx58/Tggw9aPODn4+OjBQsWKCgoqMo1vr6+eu6559SlSxd7mgYAAAAahd1bY48bN67a4z169NA777yjo0eP6ty5cyovL1fHjh01aNAgdvkDAABAi2F3YLZauYuLBg8erMGDBzdkMwAAAECDsWtKBgAAANDaNegIc0MrKSlRVFSUdu7cqaysLHl7e2vQoEG67777FBAQYHM9cXFxOnbsmBITE5WYmKj8/Hy1b99eH3zwQQP2HgAAAC2BwwJzfn6+Tp48qfT0dBUVFamioqLWa2bMmFHv9kpKSrR48WIlJCTI399f4eHhyszM1NatW3XgwAG98sorCg4Otqmuzz77TCkpKfXuCwAAAFovuwNzbm6uli9frt27d6u8vLxO19oTmNeuXauEhASFhoZq4cKF8vDwkCStW7dOK1as0LJly/TSSy/ZVNegQYM0YsQI9ezZU+3atdMzzzxT734BAACgdbErMOfl5WnhwoXKzMx0VH9sUlZWpujoaEnS3LlzzWFZkqZNm6Zt27YpLi5OSUlJ6tGjR631zZkzx/w6NzfX4f0FAABAy2VXYF69erU5LN9yyy2aPHmyunXrpjZt2shgMDikg9WJj49XYWGhgoODFRISUuX88OHDlZKSon379tkUmAEAAICa2BWY9+/fL0kaO3as5s+f75AO2cI037i6sCzJHJLPnDnTaH0CAABA62T3lAxJGj9+vEM6Y6usrCxJUrt27ao9b1ohozGnipSWlqq0tNT83mAwyNPT0/waaK34+QbqjvsGqJ+munfsCswBAQHKyMiwmEPcGIqKiiSpxh0DTf0xlWsMUVFRioyMNL8PCQnR0qVL1b59e4e1keqwmgDH6dixY1N3AWhxWs59w28eNC9Nde/YFZj79u2rjIwMnTlz5rqfK3z33Xdr2rRp5vemb0CZmZkqKytrqm4BDS4tLa2puwC0ONw3QP04+t5xcXGxaXDTrsB85513aseOHfrPf/6jkSNHys3NzZ7qbGYaQS4uLq72vGlkuTFHvl1dXeXq6lrtOaPR2Gj9ABobP99A3XHfAPXTVPeOXVtj33DDDZo3b55SU1P1yiuvKDW1cf50ExgYKEnKzs6u9nxOTo4kOXQ6BAAAAK5Pdm9cMnr0aHXs2FGvv/66nnnmGXXr1k0dO3asdbTZYDBo3rx59WqzW7dukqTTp09Xez4pKUmS1LVr13rVDwAAAJjYHZhTU1O1YsUK84oZycnJSk5Otuna+gbmsLAweXl5KT09XcnJyerevbvF+d27d0uSwsPD61U/AAAAYGLXlIysrCwtWrRI8fHx5mMeHh5q166dAgMDa/1ffbm4uGjKlCmSpE8++cRiNYx169YpJSVF/fr1s3gQMTo6Wk8//bRWrlxZ73YBAABw/bFrhDkyMlJ5eXkyGAyaNm2aJk+erKCgIEf1zap77rlHR44c0YkTJ/TUU08pLCxMWVlZSkhIkK+vb5XR67y8PKWmpurixYtV6tq0aZM2b94sSeYVLS5evKgFCxaYy8ydO/e6XwkEAADgemRXYD5y5Igk6fbbb9ecOXMc0iFbubm5adGiRYqKilJsbKz27t0rb29vjRs3TrNnz65xU5PqZGdnKyEhweJYWVmZxbErV644rO8AAABoOewKzJcuXZIkDR8+3CGdqSs3NzfNnj1bs2fPrrXsrFmzNGvWrDqfAwAAwPXNrjnM/v7+kq7OKQYAAABaI7sC84033ihJOnXqlEM6AwAAADQ3dgXm6dOny8PDQ998840KCgoc1ScAAACg2bArMHfo0EHPPfecrly5oj/96U86fPiwo/oFAAAANAt2TT5evHixJMnHx8e8PXabNm1s3unvxRdftKd5AAAAoMHZFZjj4uKqHLt8+bISExPtqRYAAABoNuwKzH379pXBYHBUXwAAAIBmx67A/NJLLzmoGwAAAEDzZNdDfwAAAEBrR2AGAAAArCAwAwAAAFbYNIc5KyvL/DowMLDa4/VRuS4AAACgObIpMM+fP1/S1bWTV61aVeV4fVxbFwAAANAc1WlKhtFodFjDjqwLAAAAaCg2jTDPmzevTscBAACA1sKmwDxu3Lg6HQcAAABaC1bJAAAAAKywa6e/bdu2SZJuvvlmeXl52XRNUVGRdu/eLUmKiIiwp3kAAACgwdkVmD/88ENJ0l/+8hebA3Nubq4+/PBDGQwGAjMAAACavSabksEqGQAAAGgJGj0wV1RUSJKcnZ0bu2kAAACgzho9MKempkqSvL29G7tpAAAAoM7qNIc5Li6u2uOJiYnKy8uzem1ZWZkuXLig//znP5Kk7t2716VpAAAAoEnUKTAvXry42uPLli2rc8O33nprna8BAAAAGptdq2TUR0BAgO6++24NGzassZsGAAAA6qxOgXnRokXm10ajUS+//LKkq1tkBwUFWb3Wzc1Nfn5+CgwMrEc3AQAAgKZRp8Dcr1+/ao/36tVLXbp0cUiHAAAAgObErikZ77//vqSr0ywAAACA1siuwNy+fXtH9QMAAABolhr8ob99+/Zp165dys/PV1BQkCZMmKAePXo0dLMAAACAQ9gVmI8ePap3331Xrq6ueuutt9SmTRuL86tWrVJUVJTFsU2bNmnevHkaO3asPU0DAAAAjcKunf4OHjyo/Px89ezZs0pYTklJsQjLpvMVFRX66KOPlJGRYU/TAAAAQKOwKzDHx8dLkgYOHFjl3MaNGyVdDcqvv/66Pv30U7366qvy9vZWaWmpfvjhB3uaBgAAABqFXYE5NzdXkqpdUu7AgQOSpClTpigkJESS1LNnT912222SpCNHjtjTNAAAANAo7ArMeXl5klRlOsaFCxeUk5MjSVV29Ovbt68kKT093Z6mAQAAgEZhV2A2Go2SpMLCQovjpqkaXl5e6t69u8U5Hx8fSVJxcbE9TQMAAACNwq7A7OfnJ0k6f/68xfFDhw5Jkvr06VPlmqKiIklVR6UBAACA5siuwNy7d29JVx/wM40Yp6ena9++fZKqfxgwLS1N0v/CNgAAANCc2bUO88SJE7Vz506lpKTo2WefVUhIiI4fP67S0lK5ublp9OjRVa45fvy4JKljx472NA0AAAA0CrtGmAcMGKCpU6dKkjIzM7Vnzx7l5+dLkn7+85/L19fXonxJSYn27t0rSerXr589TQMAAACNwu6tsR955BHdeOON2rVrly5duiQ/Pz9FRERowIABVcru27dPXl5e8vLy0tChQ+1tGgAAAGhwdgdmSRo6dKhNAXjkyJEaOXKkI5oEAAAAGoVdUzIAAACA1o7ADAAAAFhh85SM5cuXS5LuuusutW3btsr5iooK8+5+gYGBNdaTnp6ut99+W5K0dOnSOnX2WiUlJYqKitLOnTuVlZUlb29vDRo0SPfdd58CAgLqVFdBQYHWrFmjvXv3Kjc3V35+fho2bJhmzpzJmtEAAADXMZtHmNevX6/169ebV8G41vnz5zV//nw98cQTVuspKSlRcnKykpOT69TR6upZvHixvvrqKxUVFSk8PFzt2rXT1q1b9fzzz9dp6+28vDz98Y9/1IYNG+Ts7Kybb75Znp6eWr9+vf74xz+qoKDArr4CAACg5XLIQ3+VmbbLbmhr165VQkKCQkNDtXDhQnl4eEiS1q1bpxUrVmjZsmV66aWXbKrr888/14ULFzRs2DD99re/lbOzsyTp008/VXR0tJYvX6758+c31EcBAABAM9Yi5zCXlZUpOjpakjR37lxzWJakadOmqVu3boqLi1NSUlKtdV28eFE7duyQi4uLHn30UXNYlv63lnRMTIwuXbrk+A8CAACAZq9FBub4+HgVFhYqODhYISEhVc4PHz5cksxbdFtz6NAhGY1G9e3bt8p23a6urho6dKgqKip08OBBh/QdAAAALUuLDMwpKSmSVG1YlqQePXpIks6cOWN3XabjpnIAAAC4vrTIwJyVlSVJateuXbXnTStkZGZm2lxXTatqmNqwpS4AAAC0Pg5/6K8xFBUVSZLc3d2rPW+a02wqZ09dpuO11VVaWqrS0lLze4PBIE9PT7m4OO6f2LN7T4fVBTiKq6trU3ehVr2C/Zq6C4CFlnDfSFK7QM+m7gJgwdH3jq05rUUG5uYoKipKkZGR5vejRo3SU089JX9/f4e10f6V9xxWF3A9+fCXE5u6C0CLNH1G+6buAtAstMgpGaYR5OLi4mrPm0aDK6+eUd+6TMdrq+vuu+/W559/bv7fY489ZjHijObjypUreuGFF3TlypWm7grQonDvAHXHfdM61HmE+fvvv692p7/Ky65VHmm1Vq6+TDsJZmdnV3vetONg+/a1fzM21WW65lqmNmqry9XVtcX8ie16ZzQadfr06UZbMxxoLbh3gLrjvmkd6hyYN27cWGuZNWvW1KszturWrZsk6fTp09WeN62/3LVrV7vrMh03lQMAAMD1pUVOyQgLC5OXl5fS09Or3WJ79+7dkqTw8PBa6xo8eLAMBoOOHz9eZfS7tLRU+/fvl5OTk4YMGeKQvgMAAKBlsXmEedGiRQ3ZjzpxcXHRlClTtHbtWn3yySdasGCBxdbYKSkp6tevn3k9ZkmKjo5WdHS0hg0bpgceeMB83N/fX6NGjVJsbKw+/vhjPf300+bd/r744gvl5eUpIiKi2mkoaJlcXV01Y8YMptAAdcS9A9Qd903rYDC20Ek1JSUlWrx4sRISEuTv76+wsDBlZWUpISFBvr6+euWVVxQcHGwuv3r1akVGRioiIkLz58+3qCsvL08LFixQenq6goOD1bNnT509e1Znz55Vx44d9corr8jb27uxPyIAAACagRY5JUOS3NzctGjRIt17771yc3PT3r17lZmZqXHjxmnp0qUWYbk2vr6+eu211zRlyhSVlZVpz549Kiws1NSpU/Xqq68SlgEAAK5jLXaEGQAAAGgMLXaEGQAAAGgM7PSHZm3r1q368MMPJV198LR///7mc6Z56ddycnJSmzZt1LlzZw0ZMkSTJk2ymFZz9OhRvfzyy5Kk+fPnKyIiosb2//CHP+jUqVN1KtuxY0f99a9/rdsHBRpQ5fuoMldXV3l5ecnHx0ddu3ZVr169dMstt5jXp69ORkaGnnjiCfN7Pz8//d///Z+cnKyPv6xcuVJff/21+f2MGTM0a9asun8YoAk05D1Umbu7u/z9/dWrVy+NHz9eN954o8M+A+zDCDNanYqKCuXn5ys+Pl5ffvmlnnnmGSUmJprPh4aGmldCOX78eI31FBUVWazPbWvZvn372vsRgEZRWlqqS5cu6dy5c9q5c6dWrFihJ554Qm+99VaNmzldKzc3V4cPH7Zaxmg0KiYmxhFdBpoVR9xDlRUXF+vChQuKjY3VkiVLtGzZMjY8aSYYYUar8Pbbb5u/0ZeXlysjI0PR0dHasmWLcnNztXTpUv31r3+Vl5eX3Nzc1LNnT508eVLx8fE11nny5ElVVFTI1dVVpaWlNpWVpH79+jn2wwEO9Ic//MH8pa6iokKXL19Wdna24uPjtW3bNp0/f1579uxRXFycfv/73ys0NLTGujw9PXXlyhVt375dgwcPrrHcsWPHlJ2dbS4PtGSOvIfuuusu3XPPPZKufrHMzs5WXFycVq1apYKCAm3ZskVdunTRnXfe2SifDTVjhBmtgpubmzw8POTh4aE2bdooJCRE8+bN07hx4yRd3ZL9v//9r7l8WFiYJCk1NbXG7dpNI8ojRoyQm5ubTWUlRpjRvFW+V7y8vNS+fXuFhYXprrvu0ttvv62HHnpITk5OKigo0Jtvvqns7Owa67rlllskSXv37lVRUVGN5bZv3y7p6r0EtHSOvIdcXFzMdXl6eqpLly6aPHmyFi5cKIPBIOnq/hJoegRmtGozZswwvz5y5Ij5deVR4JqmWphGlAcMGKCePXvaVDYwMFDt27e3r9NAEzEYDJo2bZp5c6dLly5pzZo1NZYPCwtTUFCQiouLzTusXqvyubFjxzq+00AzUtd7qCY9evTQwIEDJUkXL17UhQsXHNpP1B2BGa1aUFCQeRfIyt/yw8LCzN/eq5tqUVZWpoSEBHPZPn362FwWaOnuvPNOdezYUdLV0eG8vLxqyxkMBo0ZM8Zcrjp79+7VlStX1L59e/76guuGrfeQNZ06dTK/rs/1cCwCM1o9UzCu/OCEl5eXunbtKqn6UeOkpCSVlJTI399fHTp0MAdha2Ul5i+jdTAYDBo/frykq18I4+LiaixrCsxHjx6t9iGnbdu2mcuZ7kWgtavLPVSTyr+z2rRp47C+oX4IzGjVcnJyzA8ZBQQEWJwzjXalpKSosLDQ4pwpGJtGlvv06SODwWC1bOU6gZbO9LMvXX2otSadOnVS7969q10JIzc31zwViukYuN7Yeg/V5Pz585KuLjXHVL+mR2BGq7Z27Vrz68prOEuyeMr52v+YmUKwaWS5TZs26tKli9Wybdu2VefOnR37AYAmYvpzsnQ1+FpT07SMmJgYVVRUqGfPnhZ/XgauB3W5h66VlJSko0ePSpLGjx8vNzc3R3YN9UBgRqtTUVGhCxcuaPny5frhhx8kXQ28t956q0W5yqPBlUeJjUajTpw4UaVMddMyKpdl/jJak8p/Ai4oKLBadtSoUXJ2dtbZs2eVnJxsPm4K0NY2/AFaK1vuobKyMhUVFamoqEhXrlzRuXPntHHjRv35z3+W0WhUWFiY+QFCNC3WYUarUNOuSdLV+crPPPOMfH19LY77+fmpY8eOSktLswjBZ8+e1eXLl+Xp6alu3bqZj4eFhemHH36otqzEdAy0LpXnT9Y299jHx0dDhgzRvn37tH37dnXv3l1nzpxRSkqKnJ2dNXLkyIbuLtDs2HIPff311xY7YFZ2//33a/r06eaNttC0GGFGq+Ti4qKuXbtq+vTpevvtt2vcXtQUck+dOqXS0lJJ/xtBDg0Ntdju1zQfrbqylesCWoPKc/VteeDINEc5NjZWFRUV5of9Bg8eXOXLKnA9qOs9dK2vv/7avAITmh4jzGgVKu/05+TkZPN8r759+2rz5s0qLS1VYmKi+vbtW2X+sklQUJACAgKUk5NTpayXl5fFaDTQ0qWlpZlf+/n51Vp+6NChatOmjXJzc3Xo0CHFxsZKYjoGrl+23EMzZszQrFmzJF3dZjszM1ObN2/Wf/7zH125ckV/+ctf9NZbb6lt27aN0WVYwQgzWoXKOy/V5eGI6uYxm9Zarm5O8rXzmE1l+/TpYzEaDbR0prn5kuXT/jVxdXU17+T32Wef6eLFi2rTpo2GDh3aYH0EmrP63EOdOnXSnDlz9PDDD0u6uvHJv/71rwbrI2zHb3hc14KCgtSuXTtJV0NwRkaGcnJy5OzsrN69e1cpb/qPXuWyEtMx0LoYjUZt2bJF0tXpTbb+fJumZaSnp0u6unW2q6trw3QSaMbqew+ZTJ061fz7Zvv27UpNTXV4H1E3BGZc90z/ITt58qSOHTsm6eq2pNWNVJtGmCuXrVwH0Bp899135j8nR0RE2DwHOSwsTMHBweb3TMfA9aq+95CJwWDQ/fffL+nqyk+Vl0hF02AOM657YWFhio2N1ZUrVxQdHW0+Vp1u3brJ09PToqybm5t69uzZaP0FGorRaNSGDRvMfwL28/PTzJkz61TH4sWLzZsFsfYyrjeOuIdM+vXrp/79++vYsWPasWOHZs2apaCgIEd2F3VAYMZ1r/J21qdPn5ZU84ixk5OTevfurcOHD5vL9u7dWy4u3EpoGUpKSlRUVCTp6shVYWGhsrKydPLkSW3ZssW8u5iPj49+97vfVdkhszZ1LQ+0NA19D1U2a9YsLVq0SOXl5YqKitLjjz/ukM+AuuO3PK57nTt3lo+Pj/Lz8yVd/VOYtQc0wsLCdPjwYfN7pmOgJXnttdesnndyctLNN9+sX/7yl/L392+kXgEtR2PeQ3379tWAAQN09OhRbdu2TTNmzDA/d4PGRWDGdc9gMCgsLEx79+6VdPXPyD4+PjWWv3a6RuURaqAlcXV1laenp3x8fNStWzf16tVLt9xyi3mJRgDWNcY9NGvWLB09elRlZWX65ptv9Mtf/tJhdcN2BmPlrWgAAAAAWGCVDAAAAMAKAjMAAABgBYEZAAAAsILADAAAAFhBYAYAAACsIDADAAAAVhCYAQAAACsIzAAAAIAVBGYAAADACgIzAAAAYAWBGQAAALDCpak7AABouV566SXFxcVJkn79619r3LhxTdshAGgABGYA9VJRUaFjx45p//79OnHihHJzc5WXlyc3Nzf5+fkpODhYQ4YMUXh4uNq1a2dzvRkZGXriiSdqPG8wGOTh4SFvb291795dAwYM0NixY9WmTZt611mb9u3b64MPPqj39ZJ07NgxLV68uN7X9+vXTy+99JJdfQAA1A+BGUCdHTx4UP/85z917ty5KudKS0t1+fJlnT9/XgcOHNDnn3+uyZMna+bMmfL29ra7baPRqCtXrujKlSvKzMzU3r179eWXX+rnP/+5Jk2aZHf916vVq1crMjJSkhQREaH58+c3cY9ahspfhBzxxQpA80RgBmAzo9Gozz//XBs2bLA47uzsrKCgIPn7+6uoqEhZWVnKy8uTJJWXl2vDhg368ccftWDBAnXt2rVObfbs2bNK0C4oKNCZM2dUWloqSSoqKtI//vEPXbp0STNmzKhXnda0bdu2Tn22Rd++feXm5mZz+W7dujm8DwAA2xCYAdjEaDTqvffeU2xsrPmYj4+PZs6cqZEjR8rX19eibEJCgr799lvt2bNHknTx4kUtWrRICxYsUK9evWxud86cOerfv3+V48XFxYqOjtaqVatUXl4uSVqzZo0GDx5ca/011dmY5s+fr6CgoCbtgyMwTQTA9YBVMgDYZP369RZhuVevXnrnnXc0ZcoUi7AsXZ1nHBoaqueee05PPPGEnJyu/qfm8uXLeuedd1RYWGh3f9zd3fWzn/1Mv/71r83HjEajoqKi7K4bAIDKCMwAanX+/HmtXLnS/L5z585asGBBlaBcnbFjx+rRRx81v8/MzNTnn3/usL6NGTNGPXr0ML8/cuSIysrKHFY/AABMyQBQq2+//dY8X9hgMOjxxx+3uirFtW699Vbt3r1bP/30kyRp+/btmjlzptq3b++Q/g0ePFhJSUmSrs5nzszMVMeOHR1Sd0tx+PBhxcbG6tSpU8rOzlZRUZFcXV3l7e2t4OBghYSEqH///ho0aJBcXV3N11VeFs5k27Zt2rZtW7XtvP/++xZTSWxZVq6mB+POnj2rH374QUeOHFF2draMRqM6deqk0aNHa8qUKRb9lKS8vDxFR0dr//79SktLU3l5uQIDAzVkyBDddddd8vPzs+nfKikpSYcPH1Z8fLzOnz+v3NxclZWVydvbW4GBgerbt6/Gjx+vG264ocY6Kj8kaZKZmalZs2ZVW97aknslJSXatm2b9u3bp7Nnz+rSpUtydXWVn5+f+vbtq5EjR+rGG2+s9XNt3bpVH374oSTLVVUSExO1detWHT9+XDk5Obp8+XK1q65cuXJFMTEx2r9/v86cOaP8/HyVl5fL3d1d/v7+6tixo3r37q2hQ4fW+VkEoKUjMAOwKi8vz2IqxuDBgxUWFlbneu6//35zYK6oqNCGDRv00EMPOaSPgYGBFu/z8/Ovm8B8+fJlvfvuu+Z/28qKi4tVXFys7OxsxcXF6bvvvtPkyZMtRvybynfffacvvvjCPP/c5PTp0zp9+rR+/PFHLVy4UJ6enpKkQ4cO6W9/+5sKCgosyqelpSktLU3btm2rdX58QUGBFixYoLS0tGrPX7p0SZcuXdKpU6f03XffadKkSXrkkUfk4tJwvyp/+ukn/f3vf1dWVpbF8dLSUhUWFio1NVWbNm3S4MGDNX/+/Do9gFpWVqYvvvhC69evr7Xs0aNH9d577+nixYtVzhUWFqqwsFDnz5/Xvn379OWXX+rVV1+t07MIQEtHYAZg1aFDh8yjy5I0fvz4etXTo0cPde3aVWfOnJEk7du3z2GB+dopGA0ZcJqTiooKLV26VPHx8eZjrq6u6ty5s3x8fFRWVqb8/HzzaKzpmsp69eolV1dXXbhwQenp6ZIkf3//GkcQ67KyR002btyo5cuXS5K8vLzUpUsXOTs7KyUlxTy/PSEhQW+//bYWLFigY8eO6Y033lBZWZmcnZ3VtWtXeXl56cKFC8rOzpZ09YvD0qVL9e6779b414/i4mKLsOzm5qaOHTuqTZs2MhgMysnJ0YULF2Q0GmU0GrVx40bl5eXpmWeeqVJXhw4dNGjQIBUUFOjUqVOSrv7b9+vXr9q2AwICqhzbvXu33n33XYsvDW3btlWnTp1UUlJisRLMoUOH9OKLL2rRokXV1lWdzz//XBs3bpQkeXh4qEuXLnJ1dVVGRoZFuZSUFL322msW97mvr686duwoNzc3FRUVKTs7Wzk5Oebz1/4cAa3d9fFbBUC9VQ5jBoNBAwcOrHddAwcONAfmCxcu6NKlSw5Zsi01NdXifUMsA9cc7dmzx/z/j4uLi+bMmaOJEyfK3d3dolxZWZni4uIUGxtbJfDOmTNHkuUUg4EDBzbYOsx5eXlavny53N3d9cgjjygiIsL8Bae0tFRffvml1q1bJ+nq6OvevXv18ccfq6ysTHfccYfuvfdeiyUBY2Ji9OGHH6q8vFyXLl3Sf/7zH9133301tu/n56cJEyZo2LBh6t69u/mBVJOcnBytW7dO3333nYxGo3788UfFxsZq9OjRFuXGjh2rsWPHWkw38fPz04IFC2z6d8jIyNAHH3xgDst+fn569NFHFR4ebu5TYWGhvv76a33zzTcyGo1KS0vTBx98oIULF8pgMFit//Tp04qLi5OHh4fmzJmj8ePHW0xxMX05kqQvv/zSHJY7d+6sxx9/vNq/Il26dEn79+/Xpk2bbPqMQGtCYAZg1enTp82vO3ToIC8vr3rXVfnhPEk6deqUbrrppnrXJ10Ng6al66SrI3l12VmwJTt48KD59fTp03X77bdXW87FxUUDBw7UwIEDm3xksLi4WE5OTlqwYEGV0VhXV1c99NBDSk5O1tGjRyVJ77zzjsrKynTffffpnnvuqVLfmDFjlJaWZg7727dvrzEwt23bVh9++KHVv0AEBATooYceUmBgoPnh1O+++65KYLbXihUrVFRUJOnqKPuiRYvUuXNnizJeXl564IEH5OPjo3/+85+Srj7UumvXLo0cOdJq/VeuXJGTk5N+//vfVzvqHRwcLOnqOulHjhyRdPUL8fPPP1/jdKa2bdtqwoQJmjBhQpP/HAGNjcAMwKpLly6ZX9v7kN61c41Nm5vYY8WKFRZ/Kh42bFit19R1i2prD2zVV1236l60aFGVtaMrf25b55VfO6LaFCZOnFjj1AXTeVNgLisr0w033KC77rqrxvK33nqrvvrqKxmNRmVlZSk7O7vaL011maozdepUrVu3TllZWTp16pQuXrwof39/m6+3JisrS/v27TO/nzVrVpWwXNm0adP0448/KiEhQZIUHR1da2CWpEmTJln9d5au3oOm0WXTNAxbNIefI6Ax8RMPwKrKD1nZM7pc3fXXPsBlC6PRqIKCAh06dEh//vOfFR0dbT7n6elpNVi1NpX/xJ6SktKEPambCRMmWD1/7cNkERERVgNaQECAxbze8+fP29dBXR1trdyPxMREu+s0OXDggHmE1t3dvdZ/D4PBYPHXg/j4eJu+bN566621lqn8M5SXl6fc3NxarwGuR4wwA7Cq8oNA9j5Md+0yYSUlJbVeY+tosJubm5599lmbHoiq69bYtj5kVRd13Rq7uv726NHDPFIZGRkpf39/jRo1Ss7Ozg7rp6O5uLioe/fuVstcuzxcaGhorfX6+flZPABYm+LiYv300086ffq0MjMzVVhYqLKyMhmNRnMZ03x7yXI0316mkWLp6vJvHh4etV4zZMgQGQwGc/8SExOtTmfy8vKyaek3b29vBQUFKSMjQ0ajUW+++aYef/xxlo0DrkFgBmBVmzZtzKNZV65csauua6+vS2i15sYbb9QvfvELdenSxabyrWVr7AkTJujbb79VUVGRiouL9f7772v58uUaMmSI+vXrp7CwMHXq1MlBPXYMHx+fWgP9tQ8t2rJBTuVriouLayxXUlKiyMhIff/993X6eXbE7pQmFy5cML+2ttZzZV5eXgoMDFRmZmaVOqoTFBRU64OBJrfffrt5vnZCQoKee+45devWTYMGDVLfvn3Vp08fh92rQEtFYAZgVeXAXJ8pFJVde70tv4SvHQ12cnKSu7u7fHx81L17dw0YMOC6WXP5WgEBAXr22WctthvPz8/X9u3btX37dklSu3btFB4erokTJ9Y6stsY6vNXCkctE3jlyhX9+c9/thjhtVXlv7TYq3L49vHxsfk6X19fc2CubRTdtH61LaZOnarU1FTzEnTS1Sk+KSkp+vbbb2UwGNSzZ0+NHDlSEyZMsHtqFtASEZgBWBUcHGxeu/bs2bMyGo02j1xdq/KfuCXZNMLaHEaDm7NBgwbp3Xff1bp16xQTE1Nl44ns7Gx9//332rhxoyIiIjR37twqI7jXi3/+858WYXnw4MEaOXKkQkJCFBAQIA8PD4tpQx988EGNOx7ao77TnCqXrS3A1+UeNRgMevTRRzVy5EitX79eBw8etKjfaDQqMTFRiYmJ+uqrrzR37lyHrxoCNHcEZgBW9enTR4cOHZIk825ftk59uFblB6fc3d0VEhLiiC5e9/z8/DRnzhzNmTNH586d07FjxxQXF6ejR48qPz9f0tXQs3XrVhUUFOj5559v4h43vvz8fG3evNn8fs6cOZo+fbrVa+ydglSTyiO0dWmjctm6bE1vq379+qlfv34qKSnRiRMnFBcXp7i4OJ08edK8XvTly5f1t7/9Ta6urho+fLjD+wA0VwRmAFZduyzVjh07NHv27DrXU1RUpP3795vfh4aGNuuH01qqLl26qEuXLrrttttUUVGhn376SStXrjSvorFv3z4dP35cffv2beKeNq6jR4+aV6YICgrSnXfeWes11W0T7QiV52Rfu+teTYxGo0VZW+Z115ebm5tuvPFG3XjjjZKuTqXatGmTIiMjzfPDv/jiCwIzrissKwfAqrCwMIs5wlu2bLFpdYtrbd261eJhrIkTJzqkf6iZk5OThgwZoj/96U8Wc2V/+umnasuaVF4porXIysoyv+7Ro0etUxZKSkqUnJxca72V67H1363yX1ZM22rX5uzZs+aNTq6to6F5e3vrZz/7mebOnWs+lp6eXuuDh0BrQmAGYJXBYNAdd9xhfp+Tk6M1a9bUqY5Lly7p3//+t/l9YGAgo1ONyNfXV3369DG/r7wZjUnlec31+ULU3JmmFNhqx44dNj3oV3lJOFv/3SqP7p89e7bK3P7qxMTEmF+3adOmSZZ9u/nmmy3eV/dzBLRWBGYAtZowYYLFttbffvutYmNjbbq2qKhIb775psVT/b/85S+ZjuEAdRkJrjw6Wd3qJJXXPk5PT7erX81R5c+XkJBgdWvny5cvW3zBs7Xe/Px8m5afGzhwoMVOhCtXrrRaPisry2KDnto2cqmLuvwMNdSykEBLQGAGUCsXFxc99dRT5tE0o9Go999/X2vWrFFZWVmN1yUnJ2vRokU6efKk+djkyZMVHh7e4H2+Hrz88svauHFjrSHtwIEDOnbsmPl9dfOXK/+JPzk52bw1dWtReS5+dna2vvrqq2rL5eXl6bXXXrN5o5KAgAC1bdtW0tX7Yv369bVe4+TkpJ/97Gfm9wcOHNC//vWvakN8bm6uli5dap7O5OHhYfEXH3sdP35cr7/+uo4dO2Y1PFdUVFgEez8/v+t2OUdcn3joD4BNOnbsqAULFuj111/X5cuXVVFRoTVr1mjTpk0aMWKEevfuLT8/PxUVFSkzM1P79+/XkSNHLP4UHhERoV/84hdN+Cmu+uKLL+o8Ovboo48qODjYYX344IMP6rTTnyQ9++yzFlMAMjIy9PHHH2v58uUaNGiQQkND1blzZ3l7e6uiokKZmZk6cOCAdu/ebQ5DPXv21ODBg6vU3aVLF3Xv3l3JyckyGo16+eWX1a1bN7Vr185iNPPxxx83B8SWJCgoSOHh4eadEdesWaPExESNGTNG7dq1U2FhoeLj47Vp0yYVFBTI399f3bt318GDB2ute9SoUeagvHr1am3evFmdO3e2WKLu9ttv14ABA8zvb7vtNu3Zs8f8xeSbb77R0aNHNX78eHXq1EmlpaU6ceKEfvjhB/NKJ5L00EMPqX379g75N5GuhvwDBw7owIEDateunYYMGaIePXooMDBQHh4eKioq0pkzZ7Rt2zadPXvWfN1dd93lsFFuoCUgMAOwWZ8+fbRkyRJ98MEH5oeVcnJy9N1331m9zs3NTXfffbfuueeeeq/h7Ei2PmhVmaOXGDt+/Hidr6lpHm5paan27dtnDoM16dixo5599tkag87jjz+uP//5z+bpM6bNKyp75JFH6tzv5uKxxx7T6dOnzVtoHzx4sNpA7OXlpd/+9rfatGmTTfXOmjVLR44cMQfKrKwsi4cMJWnYsGEW7w0Gg373u9/pjTfeMI/+nzp1yurP5pw5c3Trrbfa1Kf6yM7O1n//+99ay02aNElTp05tsH4AzRGBGUCddOnSRa+++qpiY2O1ceNGnTx5ssY/5bZt21Y333yz7rnnHgUGBjZyT1u/+++/X7t27dKxY8esTsvw8fHRxIkTdc8991iMUF+rZ8+e+stf/qKNGzfqyJEjSktL05UrV+r8wFxz5e/vr1dffVUfffSRxRKHJgaDQQMHDtRjjz2moKAgmwOzl5eXXnvtNW3evFn79u3T2bNnVVBQUOtDg56enlq4cKG+//57ffPNNzUuY9enTx89+OCDCgsLs6k/ddG1a1fNnDlTBw4c0OnTp63O7Q4JCdG9995bJfwD1wODsTWuHwSg0eTn5+vkyZPKzc1Vfn6+XF1d1bZtW3Xo0EE9evTgz7aNoKKiQufOnVNqaqpycnJ05coVubi4yMfHR127dlX37t0dtr10a5Genq7jx4/r4sWLcnNzU0BAgPr06aOAgIAm6Y/RaNSpU6d05swZ5eXlycXFRX5+furbt6/FA4INqaioSMnJybpw4YLy8vJUWloqDw8PBQQEqGfPnjbtzAm0VgRmAAAAwAqGfgAAAAArCMwAAACAFQRmAAAAwAoCMwAAAGAFgRkAAACwgsAMAAAAWEFgBgAAAKwgMAMAAABWEJgBAAAAKwjMAAAAgBUEZgAAAMAKAjMAAABgBYEZAAAAsILADAAAAFjx/wEPNtE5o0FCDAAAAABJRU5ErkJggg==\n"
},
"metadata": {}
}
],
"source": [
"# estimate the policy value of Uniform Random\n",
"estimated_policy_value_c, estimated_interval_c = ope.summarize_off_policy_estimates(\n",
" action_dist=action_dist_random,\n",
" estimated_rewards_by_reg_model=estimated_rewards_by_reg_model\n",
")\n",
"print(estimated_interval_c, '\\n')\n",
"\n",
"# visualize the estimated policy values of Uniform Random\n",
"ope.visualize_off_policy_estimates(\n",
" action_dist=action_dist_random,\n",
" estimated_rewards_by_reg_model=estimated_rewards_by_reg_model,\n",
" n_bootstrap_samples=1000, # number of resampling performed in bootstrap sampling\n",
" random_state=12345,\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "-gL7S2sToGgS"
},
"source": [
"OPE procedure estimates that IPWLearners outperform the Uniform Random policy by a large margin.\n",
"\n",
"Moreover, IPWLearner with Logistic Regression seems to be the best one."
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"id": "YkxCISNOoGgS"
},
"outputs": [],
"source": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "PfXLtO7ZoGgS"
},
"source": [
"## (4) Evaluation of OPE estimators\n",
"Our final step is **the evaluation of OPE**, which evaluates and compares the estimation accuracy of OPE estimators."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Cj7pqf2CoGgS"
},
"source": [
"### (4-1) Approximate the Ground-truth Policy Value\n",
"With synthetic data, we can calculate the ground-truth policy value of the evaluation policies as follows.\n",
"\n",
"$$V(\\pi_e) \\approx \\frac{1}{|\\mathcal{D}_{te}|} \\sum_{i=1}^{|\\mathcal{D}_{te}|} \\mathbb{E}_{a \\sim \\pi_e(a|x_i)} [q(x_i, a)], \\; \\, where \\; \\, q(x,a) := \\mathbb{E}_{r \\sim p(r|x,a)} [r]$$"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {
"tags": [],
"id": "wGiMxRNHoGgS",
"outputId": "64e6dac2-6034-4650-845e-a78ef02bacdd",
"colab": {
"base_uri": "https://localhost:8080/"
}
},
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"policy value of IPW Learner with Logistic Regression: 0.9568150699980109\n",
"policy value of IPWLearner with Random Forest: 0.889654207952693\n",
"policy value of Unifrom Random: 0.6838835043816496\n"
]
}
],
"source": [
"# we first calculate the true policy values of the three evaluation policies based on the expected rewards of the test data\n",
"policy_names = [\"IPW Learner with Logistic Regression\", \"IPWLearner with Random Forest\", \"Unifrom Random\"]\n",
"for name, action_dist in zip(policy_names, [action_dist_ipw_lr, action_dist_ipw_rf, action_dist_random]):\n",
" true_policy_value = dataset.calc_ground_truth_policy_value(\n",
" expected_reward=bandit_feedback_test[\"expected_reward\"],\n",
" action_dist=action_dist,\n",
" )\n",
" print(f'policy value of {name}: {true_policy_value}')"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "hD8NQ9z-oGgT"
},
"source": [
"In fact, IPWLearner with Logistic Regression reveals the best performance among the three evaluation policies.\n",
"\n",
"Using the true policy values, we evaluate the estimation accuracy of the OPE estimators."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ncZ3IM9ToGgT"
},
"source": [
"### (4-2) Evaluation of OPE\n",
"\n",
"We can then evaluate the estimation performance of OPE estimators by comparing the estimated policy values of the evaluation with its ground-truth as follows.\n",
"\n",
"- $\\textit{relative-ee} (\\hat{V}; \\mathcal{D}_b) := \\left| \\frac{V(\\pi_e) - \\hat{V} (\\pi_e; \\mathcal{D}_b)}{V(\\pi_e)} \\right|$ (relative estimation error; relative-ee)\n",
"- $\\textit{SE} (\\hat{V}; \\mathcal{D}_b) := \\left( V(\\pi_e) - \\hat{V} (\\pi_e; \\mathcal{D}_b) \\right)^2$ (squared error; se)"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {
"id": "THH8TDfKoGgT",
"outputId": "2790e412-cdd9-483b-be7b-ad15de58dbd7",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 143
}
},
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
" relative-ee\n",
"ipw 0.002699\n",
"dm 0.163266\n",
"dr 0.001249"
],
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>relative-ee</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>ipw</th>\n",
" <td>0.002699</td>\n",
" </tr>\n",
" <tr>\n",
" <th>dm</th>\n",
" <td>0.163266</td>\n",
" </tr>\n",
" <tr>\n",
" <th>dr</th>\n",
" <td>0.001249</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
]
},
"metadata": {},
"execution_count": 17
}
],
"source": [
"# evaluate the estimation performances of OPE estimators for IPWLearner with Logistic Regression\n",
"# `summarize_estimators_comparison` returns a pandas dataframe containing estimation performances of given estimators\n",
"relative_ee_for_ipw_lr = ope.summarize_estimators_comparison(\n",
" ground_truth_policy_value=dataset.calc_ground_truth_policy_value(\n",
" expected_reward=bandit_feedback_test[\"expected_reward\"],\n",
" action_dist=action_dist_ipw_lr,\n",
" ),\n",
" action_dist=action_dist_ipw_lr,\n",
" estimated_rewards_by_reg_model=estimated_rewards_by_reg_model,\n",
" metric=\"relative-ee\", # \"relative-ee\" (relative estimation error) or \"se\" (squared error)\n",
")\n",
"\n",
"# estimation performance of the estimators (lower means accurate)\n",
"relative_ee_for_ipw_lr"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {
"id": "FZIUaF-YoGgT",
"outputId": "343e5682-1e17-4869-e121-2b36c45f0f40",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 143
}
},
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
" relative-ee\n",
"ipw 0.010925\n",
"dm 0.108399\n",
"dr 0.000508"
],
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>relative-ee</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>ipw</th>\n",
" <td>0.010925</td>\n",
" </tr>\n",
" <tr>\n",
" <th>dm</th>\n",
" <td>0.108399</td>\n",
" </tr>\n",
" <tr>\n",
" <th>dr</th>\n",
" <td>0.000508</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
]
},
"metadata": {},
"execution_count": 18
}
],
"source": [
"# evaluate the estimation performance of OPE estimators for IPW with Random Forest\n",
"relative_ee_for_ipw_rf = ope.summarize_estimators_comparison(\n",
" ground_truth_policy_value=dataset.calc_ground_truth_policy_value(\n",
" expected_reward=bandit_feedback_test[\"expected_reward\"],\n",
" action_dist=action_dist_ipw_rf,\n",
" ),\n",
" action_dist=action_dist_ipw_rf,\n",
" estimated_rewards_by_reg_model=estimated_rewards_by_reg_model,\n",
" metric=\"relative-ee\", # \"relative-ee\" (relative estimation error) or \"se\" (squared error)\n",
")\n",
"\n",
"# estimation performance of the estimators (lower means accurate)\n",
"relative_ee_for_ipw_rf"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {
"id": "4e6bg2l0oGgT",
"outputId": "a9f3f254-1901-4a1e-996a-b8ec7b5d535a",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 143
}
},
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
" relative-ee\n",
"ipw 0.001289\n",
"dm 0.048438\n",
"dr 0.000959"
],
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>relative-ee</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>ipw</th>\n",
" <td>0.001289</td>\n",
" </tr>\n",
" <tr>\n",
" <th>dm</th>\n",
" <td>0.048438</td>\n",
" </tr>\n",
" <tr>\n",
" <th>dr</th>\n",
" <td>0.000959</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
]
},
"metadata": {},
"execution_count": 19
}
],
"source": [
"# evaluate the estimation performance of OPE estimators for Uniform Random\n",
"relative_ee_for_random = ope.summarize_estimators_comparison(\n",
" ground_truth_policy_value=dataset.calc_ground_truth_policy_value(\n",
" expected_reward=bandit_feedback_test[\"expected_reward\"],\n",
" action_dist=action_dist_random,\n",
" ),\n",
" action_dist=action_dist_random,\n",
" estimated_rewards_by_reg_model=estimated_rewards_by_reg_model,\n",
" metric=\"relative-ee\", # \"relative-ee\" (relative estimation error) or \"se\" (squared error)\n",
")\n",
"\n",
"# estimation performance of the estimators (lower means accurate)\n",
"relative_ee_for_random"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "fd4Ltv_roGgT"
},
"source": [
"We can iterate the above process several times to get more reliable results.\n",
"\n",
"Please see [../examples/synthetic](../synthetic) for a more sophisticated example of the evaluation of OPE with synthetic bandit data."
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {
"id": "TV7gexWYoGgT"
},
"outputs": [],
"source": []
}
],
"metadata": {
"interpreter": {
"hash": "2ff39f3b22306140fd87fd114528320b56c4f8c8e196b421a3ea939a2b6b4692"
},
"kernelspec": {
"display_name": "py310",
"name": "py310"
},
"orig_nbformat": 2,
"colab": {
"provenance": [],
"include_colab_link": true
}
},
"nbformat": 4,
"nbformat_minor": 0
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment