Skip to content

Instantly share code, notes, and snippets.

@mattbullen

mattbullen/unit07-ex3-multi-layer-perceptron.ipynb

Last active October 16, 2025 12:00
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save mattbullen/f34eb03709863466ace421a666d30361 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save mattbullen/f34eb03709863466ace421a666d30361 to your computer and use it in GitHub Desktop.
Download ZIP
unit07-ex3-multi-layer-perceptron.ipynb
Display the source blob
Display the rendered blob
Raw
unit07-ex3-multi-layer-perceptron.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/gist/mattbullen/f34eb03709863466ace421a666d30361/unit07-ex3-multi-layer-perceptron.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "8tBFC1qoqAcq"
},
"source": [
"### Author: Dr Mike Lakoju"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "QrgDwyKOqAcr"
},
"source": [
" * If X is high, the value is approximately 1\n",
" * if X is small, the value is approximately 0"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "DEC81gtrqAcr"
},
"source": [
"## Import Libraries"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "zagQTPSsqAcr"
},
"outputs": [],
"source": [
"import numpy as np"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "fD5lg7qCqAcr"
},
"source": [
"## Define the Sigmoid Function"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "RO8pnlS4qAcr"
},
"outputs": [],
"source": [
"def sigmoid(sum_func):\n",
" return 1 / (1 + np.exp(-sum_func))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "rZfW7OGEqAcr",
"outputId": "c0365218-2ab8-4021-9733-d03bca62494e"
},
"outputs": [
{
"data": {
"text/plain": [
"0.5"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"sigmoid(0)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "B1bQ461mqAcs",
"outputId": "09d3cee6-2556-44d1-cffd-d1e955e33492"
},
"outputs": [
{
"data": {
"text/plain": [
"7.38905609893065"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"np.exp(2)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "IHDOx92BqAcs",
"outputId": "1d5de01a-b565-49db-e756-4aa92e09f3d6"
},
"outputs": [
{
"data": {
"text/plain": [
"2.718281828459045"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"np.exp(1)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "RJah_yP9qAcs",
"outputId": "561fd71a-0c9e-4218-c63f-54d9fabf4a9e"
},
"outputs": [
{
"data": {
"text/plain": [
"1.0"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"sigmoid(40)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "J-sORjD1qAcs",
"outputId": "0c90d064-72fd-4dd2-a1e8-910f90e529fd"
},
"outputs": [
{
"data": {
"text/plain": [
"1.2501528648238605e-09"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"sigmoid(-20.5)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "euvFzbaaqAcs"
},
"source": [
"# Input Layer to Hidden Layer"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "73WWcJDdqAcs"
},
"source": [
"### Define \"Inputs, outputs and weights\" as Numpy arrays"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "7lF9qPClqAcs"
},
"source": [
"#### Inputs"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "5AeI3yiuqAcs"
},
"outputs": [],
"source": [
"inputs = np.array([[0,0],\n",
" [0,1],\n",
" [1,0],\n",
" [1,1]])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "vMNF3jWSqAcs",
"outputId": "235cf247-7b01-43a5-a12d-f57b66c6c56d"
},
"outputs": [
{
"data": {
"text/plain": [
"array([[0, 0],\n",
" [0, 1],\n",
" [1, 0],\n",
" [1, 1]])"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"inputs"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "dL6d9-2vqAcs",
"outputId": "b4e494f4-2c48-4d10-dffc-14b7fe1efaa2"
},
"outputs": [
{
"data": {
"text/plain": [
"(4, 2)"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"inputs.shape"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "mUoffeinqAct"
},
"source": [
"#### Outputs"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "0RaM1nQDqAct"
},
"outputs": [],
"source": [
"outputs = np.array([[0],\n",
" [1],\n",
" [1],\n",
" [0]])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "gTTlycuXqAct",
"outputId": "1f0faaef-f61f-4db1-db68-d43fde20a189"
},
"outputs": [
{
"data": {
"text/plain": [
"(4, 1)"
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"outputs.shape"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "RLM6-mjzqAct"
},
"source": [
"### Weights"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "8gsRh6NHqAct"
},
"source": [
"#### These weights are for the connection between the inputs and the hidden layer"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "UdbiOFdjqAct",
"outputId": "6a689392-ee20-4fcd-f567-3ba86c00f21b"
},
"outputs": [
{
"data": {
"text/plain": [
"(2, 3)"
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# First row holds the weights for x1, 2nd row contains the weights for x2\n",
"\n",
"weights_0 = np.array([[-0.424, -0.740, -0.961],\n",
" [0.358, -0.577, -0.469]])\n",
"weights_0.shape"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "bbZOIonRqAct"
},
"source": [
"#### These weights are for the connection between the hidden layer and the output"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "4EgiLMZXqAct",
"outputId": "163ddf2e-d561-4616-f8e9-6174a050f7d8"
},
"outputs": [
{
"data": {
"text/plain": [
"(3, 1)"
]
},
"execution_count": 15,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"weights_1 = np.array([[-0.017],\n",
" [-0.893],\n",
" [0.148]])\n",
"weights_1.shape"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "6Zo0uDO8qAct"
},
"source": [
"#### Epochs & Learning Rate"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "8o-8BaUMqAct"
},
"outputs": [],
"source": [
"epochs = 100\n",
"learning_rate = 0.3"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "4mny4DvuqAct"
},
"outputs": [],
"source": [
"#for epoch in epochs:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Q6wE-S7UqAct",
"outputId": "45233c67-b567-4e75-8769-40a57a89fdac"
},
"outputs": [
{
"data": {
"text/plain": [
"array([[0, 0],\n",
" [0, 1],\n",
" [1, 0],\n",
" [1, 1]])"
]
},
"execution_count": 18,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"input_layer = inputs\n",
"input_layer"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "YwCm1boPqAct",
"outputId": "bdea99df-1b46-4c2e-d367-7495da528640"
},
"outputs": [
{
"data": {
"text/plain": [
"array([[ 0. , 0. , 0. ],\n",
" [ 0.358, -0.577, -0.469],\n",
" [-0.424, -0.74 , -0.961],\n",
" [-0.066, -1.317, -1.43 ]])"
]
},
"execution_count": 19,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# \"sum_synapse_0\" This holds the sum function total of weights for the hidden layer\n",
"# For the Output: Each row holds the sum_func for each input data [0,0,0 -> data 0,0],[0.358, -0.577, -0.469 --> 0,1]\n",
"# The dot product does the matrix multiplication and also the sum\n",
"\n",
"sum_synapse_0 = np.dot(input_layer, weights_0)\n",
"sum_synapse_0"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "-qZK6fn5qAct",
"outputId": "b3b8108b-0137-4da6-da43-9198c63ea765"
},
"outputs": [
{
"data": {
"text/plain": [
"array([[0.5 , 0.5 , 0.5 ],\n",
" [0.5885562 , 0.35962319, 0.38485296],\n",
" [0.39555998, 0.32300414, 0.27667802],\n",
" [0.48350599, 0.21131785, 0.19309868]])"
]
},
"execution_count": 20,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Computing the Sigmoid function for the Hidden layer\n",
"\n",
"hidden_layer = sigmoid(sum_synapse_0)\n",
"hidden_layer"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "b7v1NakZqAct"
},
"source": [
"Dealing with the 2nd side"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "arlNsISrqAcu",
"outputId": "981018ef-6857-4043-d659-24a3ee82712c"
},
"outputs": [
{
"data": {
"text/plain": [
"array([[-0.017],\n",
" [-0.893],\n",
" [ 0.148]])"
]
},
"execution_count": 21,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"weights_1"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "VyDCFNMiqAcu",
"outputId": "07f7a770-e58a-4188-adde-a0d7a39d795f"
},
"outputs": [
{
"data": {
"text/plain": [
"array([[-0.381 ],\n",
" [-0.27419072],\n",
" [-0.25421887],\n",
" [-0.16834784]])"
]
},
"execution_count": 22,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# \"sum_synapse_1\" This holds the sum function total of weights for the output layer\n",
"# For the Output: Each row holds the sum_func for each input data\n",
"\n",
"sum_synapse_1 = np.dot(hidden_layer, weights_1)\n",
"sum_synapse_1"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "HZIqsARQqAcu",
"outputId": "ba33778e-bb38-47f3-df70-0cd15a7256cf"
},
"outputs": [
{
"data": {
"text/plain": [
"array([[0.40588573],\n",
" [0.43187857],\n",
" [0.43678536],\n",
" [0.45801216]])"
]
},
"execution_count": 23,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"output_layer = sigmoid(sum_synapse_1)\n",
"output_layer"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "JcDaEYaJqAcv",
"outputId": "4c064d85-9e32-4773-ee51-4d4e904c8994"
},
"outputs": [
{
"data": {
"text/plain": [
"array([[0],\n",
" [1],\n",
" [1],\n",
" [0]])"
]
},
"execution_count": 24,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"outputs"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Ap31u1PrqAcv",
"outputId": "d7d2f216-91e7-4a0a-b406-167644e0446d"
},
"outputs": [
{
"data": {
"text/plain": [
"array([[0.40588573],\n",
" [0.43187857],\n",
" [0.43678536],\n",
" [0.45801216]])"
]
},
"execution_count": 25,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"output_layer"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Lswsw2hgqAcv",
"outputId": "6192d18e-d869-4e0b-c5da-c7964bc733c9"
},
"outputs": [
{
"data": {
"text/plain": [
"array([[-0.40588573],\n",
" [ 0.56812143],\n",
" [ 0.56321464],\n",
" [-0.45801216]])"
]
},
"execution_count": 26,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"error_output_layer = outputs - output_layer\n",
"error_output_layer"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "EGMiI5tnqAcv",
"outputId": "cb3d36c5-a39c-4334-d723-29a0f454d66f"
},
"outputs": [
{
"data": {
"text/plain": [
"0.49880848923713045"
]
},
"execution_count": 27,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"average_error = np.mean(abs(error_output_layer))\n",
"average_error"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "OXjlMKivqAcv"
},
"source": [
"Sigmoid Derivative"
]
},
{
"cell_type": "markdown",
"source": [],
"metadata": {
"id": "DdDS0W2F48_e"
}
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "L_VH4g2AqAcv"
},
"outputs": [],
"source": [
"def sigmoid_derivative(sigmoid):\n",
" return sigmoid * (1 - sigmoid)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "EAsSgsHpqAcv"
},
"source": [
"# Delta output Calculation"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Y--xrV5HqAcv",
"outputId": "8c4ee9f0-c415-4c7c-e087-dbdc59c1e45b"
},
"outputs": [
{
"data": {
"text/plain": [
"array([[0.40588573],\n",
" [0.43187857],\n",
" [0.43678536],\n",
" [0.45801216]])"
]
},
"execution_count": 29,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# output_layer holds the results of our application of the sigmoid, computed above\n",
"\n",
"output_layer"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "oV4jRnGXqAcv",
"outputId": "9f3c99fe-e52e-4102-a51c-57c5e9fe0ac5"
},
"outputs": [
{
"data": {
"text/plain": [
"array([[0.2411425 ],\n",
" [0.24535947],\n",
" [0.24600391],\n",
" [0.24823702]])"
]
},
"execution_count": 30,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# derivative_output is our Derivative of the activation function (sigmoid) which we have on the slide\n",
"# each row is for each instance of our input dataset\n",
"\n",
"derivative_output = sigmoid_derivative(output_layer)\n",
"derivative_output"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "S4Y48lY3qAcv",
"outputId": "4dd8fae5-b0d6-4fa3-f2d1-aa27e3282247"
},
"outputs": [
{
"data": {
"text/plain": [
"array([[-0.40588573],\n",
" [ 0.56812143],\n",
" [ 0.56321464],\n",
" [-0.45801216]])"
]
},
"execution_count": 31,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"error_output_layer"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "b0vIxroyqAcv",
"outputId": "bed08a4b-4ed7-4235-ae6e-92c4c9e6f7b5"
},
"outputs": [
{
"data": {
"text/plain": [
"array([[-0.0978763 ],\n",
" [ 0.13939397],\n",
" [ 0.138553 ],\n",
" [-0.11369557]])"
]
},
"execution_count": 32,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Delta output\n",
"# each row is for each instance of our input dataset\n",
"\n",
"delta_output = error_output_layer * derivative_output\n",
"delta_output"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "iT8P-280qAcw"
},
"source": [
"# Delta calculations for the Hidden Layer"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "h-OHZANIqAcw",
"outputId": "e55ee1be-92a4-4a25-e4ea-3c1415407537"
},
"outputs": [
{
"data": {
"text/plain": [
"array([[-0.0978763 ],\n",
" [ 0.13939397],\n",
" [ 0.138553 ],\n",
" [-0.11369557]])"
]
},
"execution_count": 33,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"delta_output"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "gE0GCA7MqAcw",
"outputId": "c262b665-f71b-4658-dd64-7afe6cbbced0"
},
"outputs": [
{
"data": {
"text/plain": [
"array([[-0.017],\n",
" [-0.893],\n",
" [ 0.148]])"
]
},
"execution_count": 34,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"weights_1"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "EEAIW4LMqAcw"
},
"source": [
"#### NOTE THAT:\n",
" * Lets deal with this part first (Weight * delta_output)\n",
" * Notice that we will get an error below becuase of the shape of the weights_1 (Transpose)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "ZO8ECwLzqAcw",
"outputId": "8b37f58b-5b18-412e-9739-4e9a2031f611"
},
"outputs": [
{
"ename": "ValueError",
"evalue": "shapes (4,1) and (3,1) not aligned: 1 (dim 1) != 3 (dim 0)",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mValueError\u001b[0m Traceback (most recent call last)",
"\u001b[0;32m<ipython-input-35-50b740e5a31c>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mdelta_output_x_weight\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mdelta_output\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdot\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mweights_1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
"\u001b[0;31mValueError\u001b[0m: shapes (4,1) and (3,1) not aligned: 1 (dim 1) != 3 (dim 0)"
]
}
],
"source": [
"delta_output_x_weight = delta_output.dot(weights_1)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "eOGSmwgzqAcw",
"outputId": "78106652-5fe2-4582-fb4a-ed9b1da412f5"
},
"outputs": [
{
"data": {
"text/plain": [
"(3, 1)"
]
},
"execution_count": 36,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"weights_1.shape"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "2VmrhC6mqAcw",
"outputId": "caa44f62-3154-4b5e-98fb-cc44747e5d77"
},
"outputs": [
{
"data": {
"text/plain": [
"array([[-0.017, -0.893, 0.148]])"
]
},
"execution_count": 37,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"weights_1T = weights_1.T\n",
"weights_1T"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Ec6fpu1wqAcw",
"outputId": "0f899f0b-ec31-4f50-eb2c-c7e0c1931492"
},
"outputs": [
{
"data": {
"text/plain": [
"(1, 3)"
]
},
"execution_count": 38,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"weights_1T.shape"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "fV_XdKC_qAcw"
},
"source": [
"#### Each one of the weights will have to be multiplied by each delta_output for each data instance \n",
" \n",
" array([[-0.017],\n",
" [-0.893],\n",
" [ 0.148]])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "i6FOg58jqAcy",
"outputId": "80ccfb87-e046-4a9d-e4e7-e8917fbf45d4"
},
"outputs": [
{
"data": {
"text/plain": [
"array([[ 0.0016639 , 0.08740354, -0.01448569],\n",
" [-0.0023697 , -0.12447882, 0.02063031],\n",
" [-0.0023554 , -0.12372783, 0.02050584],\n",
" [ 0.00193282, 0.10153015, -0.01682694]])"
]
},
"execution_count": 39,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"delta_output_x_weight = delta_output.dot(weights_1T)\n",
"delta_output_x_weight"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ZHMDzHlSqAcy"
},
"source": [
"#### NOTE THAT:\n",
" * Now we need to deal with the last part of the equation \n",
" * sigmoid_derivative * delta_output_x_weight"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "0G_6CfrOqAcy",
"outputId": "53c938e0-42af-49f1-808e-74e739078124"
},
"outputs": [
{
"data": {
"text/plain": [
"array([[0.5 , 0.5 , 0.5 ],\n",
" [0.5885562 , 0.35962319, 0.38485296],\n",
" [0.39555998, 0.32300414, 0.27667802],\n",
" [0.48350599, 0.21131785, 0.19309868]])"
]
},
"execution_count": 40,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"hidden_layer"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "SgLBM-fBqAcy",
"outputId": "b2fcccf6-4b58-4cfd-e745-d301c30c580e"
},
"outputs": [
{
"data": {
"text/plain": [
"array([[ 0.00041597, 0.02185088, -0.00362142],\n",
" [-0.00057384, -0.02866677, 0.00488404],\n",
" [-0.00056316, -0.02705587, 0.00410378],\n",
" [ 0.00048268, 0.01692128, -0.00262183]])"
]
},
"execution_count": 41,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Each row in the output of delta_hidden_layer is for the data input values\n",
"\n",
"delta_hidden_layer = delta_output_x_weight * sigmoid_derivative(hidden_layer)\n",
"delta_hidden_layer"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "urPoyUlUqAcy"
},
"source": [
"#### We will deal with the (input * delta) first\n",
"* The first column in \"hidden_layer\" holds the activation value for the first neuron"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "fPGF1TOKqAcy",
"outputId": "1a020639-8ae5-4f0b-b6aa-4f178c622bf8"
},
"outputs": [
{
"data": {
"text/plain": [
"array([[0.5 , 0.5 , 0.5 ],\n",
" [0.5885562 , 0.35962319, 0.38485296],\n",
" [0.39555998, 0.32300414, 0.27667802],\n",
" [0.48350599, 0.21131785, 0.19309868]])"
]
},
"execution_count": 42,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"hidden_layer"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "t5rM4mxUqAcz",
"outputId": "7e3ca281-7748-4fb0-cbe5-655202108cdf"
},
"outputs": [
{
"data": {
"text/plain": [
"array([[-0.0978763 ],\n",
" [ 0.13939397],\n",
" [ 0.138553 ],\n",
" [-0.11369557]])"
]
},
"execution_count": 43,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"delta_output"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "BAnIbfyuqAcz"
},
"source": [
"* We need to multiply the \"inputs\" by \"delta\" however, for the matrix multiplication we need to transpose the values in the hidden_layer, so we have all of them on one row for each neuron"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "5SQ839HoqAcz",
"outputId": "999d4638-87a1-483d-a59c-a56cac78c0b3"
},
"outputs": [
{
"data": {
"text/plain": [
"array([[0.5 , 0.5885562 , 0.39555998, 0.48350599],\n",
" [0.5 , 0.35962319, 0.32300414, 0.21131785],\n",
" [0.5 , 0.38485296, 0.27667802, 0.19309868]])"
]
},
"execution_count": 44,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"hidden_layerT = hidden_layer.T\n",
"hidden_layerT"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "JNvJ6-yrqAcz",
"outputId": "76306471-a9f3-41dc-f88d-dad11eadd791"
},
"outputs": [
{
"data": {
"text/plain": [
"array([[0.03293657],\n",
" [0.02191844],\n",
" [0.02108814]])"
]
},
"execution_count": 45,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"input_x_delta1 = hidden_layerT.dot(delta_output)\n",
"input_x_delta1"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "nMXRkXH5qAcz"
},
"source": [
"#### Let us now update the \"weights_1\""
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "PEDTED82qAcz",
"outputId": "51e040e3-6f74-4595-80e4-b8f8ac0b23ea"
},
"outputs": [
{
"data": {
"text/plain": [
"array([[-0.00711903],\n",
" [-0.88642447],\n",
" [ 0.15432644]])"
]
},
"execution_count": 48,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"weights_1 = weights_1 + (input_x_delta1 * learning_rate)\n",
"weights_1"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Y3a5a-2zqAcz"
},
"source": [
"## Dealing with the Hidden Layer to Input Layer"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "pDLTEKTOqAcz",
"outputId": "d21620e4-fa9d-4544-c4f0-d9d9c4ce5403"
},
"outputs": [
{
"data": {
"text/plain": [
"array([[0, 0],\n",
" [0, 1],\n",
" [1, 0],\n",
" [1, 1]])"
]
},
"execution_count": 54,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# First column is X1, and 2nd column is X2 (our input values )\n",
"\n",
"input_layer"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "gVMMsw2jqAcz",
"outputId": "cf6b8e2a-6c9a-4110-b522-1605607f3b6e"
},
"outputs": [
{
"data": {
"text/plain": [
"array([[ 0.00041597, 0.02185088, -0.00362142],\n",
" [-0.00057384, -0.02866677, 0.00488404],\n",
" [-0.00056316, -0.02705587, 0.00410378],\n",
" [ 0.00048268, 0.01692128, -0.00262183]])"
]
},
"execution_count": 50,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"delta_hidden_layer"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "VpI0j_WCqAcz",
"outputId": "517631b4-f294-4e52-8578-3a75350b2b0c"
},
"outputs": [
{
"data": {
"text/plain": [
"array([[0, 0, 1, 1],\n",
" [0, 1, 0, 1]])"
]
},
"execution_count": 51,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# we need to transpose the values just as we did before\n",
"\n",
"input_layerT = input_layer.T\n",
"input_layerT"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "U1L1WkwtqAcz",
"outputId": "7b8b0eaa-c3fe-4aaa-c5de-75267ec140e2"
},
"outputs": [
{
"data": {
"text/plain": [
"array([[-8.04778516e-05, -1.01345901e-02, 1.48194623e-03],\n",
" [-9.11603819e-05, -1.17454886e-02, 2.26221011e-03]])"
]
},
"execution_count": 52,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"input_x_delta0 = input_layerT.dot(delta_hidden_layer)\n",
"input_x_delta0"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Y8JYuTJHqAcz",
"outputId": "2789fd6f-c8bd-4aa9-b03c-3c75defed723"
},
"outputs": [
{
"data": {
"text/plain": [
"array([[-0.42402414, -0.74304038, -0.96055542],\n",
" [ 0.35797265, -0.58052365, -0.46832134]])"
]
},
"execution_count": 53,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"weights_0 = weights_0 + (input_x_delta0 * learning_rate)\n",
"weights_0"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "NueQni1tqAcz"
},
"source": [
"### So all the lines of code above, has allowed us to complete our first epoch. we will need to put all the code together so we can run multiple epochs"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "pZpeA5k5qAc0"
},
"source": [
"# Complete Artificial Neural Network"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "7-HT1z8sqAc0"
},
"outputs": [],
"source": [
"#Importing Numpy\n",
"import numpy as np\n",
"\n",
"# This is the sigmoid Function\n",
"def sigmoid(sum):\n",
" return 1 / (1 + np.exp(-sum))\n",
"\n",
"#This is the sigmoid derivative as used before\n",
"def sigmoid_derivative(sigmoid):\n",
" return sigmoid * (1 - sigmoid)\n",
"\n",
"# Our input values\n",
"inputs = np.array([[0,0],\n",
" [0,1],\n",
" [1,0],\n",
" [1,1]])\n",
"#Our output values\n",
"outputs = np.array([[0],\n",
" [1],\n",
" [1],\n",
" [0]])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "w2URHhU6qAc0"
},
"outputs": [],
"source": [
"# weights_0 = np.array([[-0.424, -0.740, -0.961],\n",
"# [0.358, -0.577, -0.469]])\n",
"\n",
"# weights_1 = np.array([[-0.017],\n",
"# [-0.893],\n",
"# [0.148]])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "TyVUiXytqAc0"
},
"source": [
"### Initializing our weights with random values\n",
"* <b>Note:</b> Multiplying the random number by 2 and subtracting by 1, allows us to have a mix of both positive and negative random numbers for the weights"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "CRvokXFcqAc0"
},
"outputs": [],
"source": [
"weights_0 = 2 * np.random.random((2, 3)) - 1\n",
"weights_1 = 2 * np.random.random((3, 1)) - 1"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "kNHnmgpeqAc0",
"outputId": "92aa16df-e06c-4401-dd90-d724d14e5d1a"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch: 1 Error: 0.4999269690201742\n",
"Epoch: 100001 Error: 0.015563199459768319\n",
"Epoch: 200001 Error: 0.010326071808337757\n",
"Epoch: 300001 Error: 0.008192022809586367\n"
]
}
],
"source": [
"epochs = 400000\n",
"learning_rate = 0.6\n",
"error = []\n",
"\n",
"for epoch in range(epochs):\n",
" input_layer = inputs\n",
" sum_synapse0 = np.dot(input_layer, weights_0)\n",
" hidden_layer = sigmoid(sum_synapse0)\n",
"\n",
" sum_synapse1 = np.dot(hidden_layer, weights_1)\n",
" output_layer = sigmoid(sum_synapse1)\n",
"\n",
" error_output_layer = outputs - output_layer\n",
" average = np.mean(abs(error_output_layer))\n",
" #print after every specified range of the value\n",
" if epoch % 100000 == 0:\n",
" print('Epoch: ' + str(epoch + 1) + ' Error: ' + str(average))\n",
" error.append(average)\n",
"\n",
" derivative_output = sigmoid_derivative(output_layer)\n",
" delta_output = error_output_layer * derivative_output\n",
"\n",
" weights1T = weights1.T\n",
" delta_output_weight = delta_output.dot(weights1T)\n",
" delta_hidden_layer = delta_output_weight * sigmoid_derivative(hidden_layer)\n",
"\n",
" hidden_layerT = hidden_layer.T\n",
" input_x_delta1 = hidden_layerT.dot(delta_output)\n",
" weights_1 = weights_1 + (input_x_delta1 * learning_rate)\n",
"\n",
" input_layerT = input_layer.T\n",
" input_x_delta0 = input_layerT.dot(delta_hidden_layer)\n",
" weights_0 = weights_0 + (input_x_delta0 * learning_rate)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "UrNetloPqAc0"
},
"source": [
"#### At this point after runing for 1million epochs you can see the value is very low."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "3G0qCYE6qAc0",
"outputId": "787fe7bc-c9fc-47cb-be5c-c6f4b31eefa6"
},
"outputs": [
{
"data": {
"text/plain": [
"0.9903290320690693"
]
},
"execution_count": 67,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"#1 million epochs with a learning rate of 0.3\n",
"1 - 0.009670967930930745"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "UviN0IGAqAc0",
"outputId": "9a52dc57-affa-457f-d9ad-c9a9bb1f7543"
},
"outputs": [
{
"data": {
"text/plain": [
"0.9918079771904136"
]
},
"execution_count": 92,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"#after 400,000 epochs, with a learning rate of 0.6\n",
"1- 0.008192022809586367"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "csQl8orzqAc0"
},
"source": [
"# Let's visualize this result"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "OgiBiWkKqAc0"
},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "wgYwKrx3qAc0",
"outputId": "8a8f8f34-2435-4bd7-f9f1-d69eda514fd0"
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de5xcdX3/8dd7Zi9JdmcTIAuzBCThYiZoK9pIrRekP2mL1YK/KhWKtvanpbZF+Vmtl9YHRbReqvXSX3nU4qV4v5RajUpFrULVViUCohACIQJJSMJCSMh1b/P5/XHO7p5ddpNN2LMzs+f9fDzmsec2Zz5zZnff8z3fme9RRGBmZsVVanQBZmbWWA4CM7OCcxCYmRWcg8DMrOAcBGZmBecgMDMrOAdBg0i6QdKrcn6M5ZJCUluOj3GxpG/mtf+5MJuvhaR3SHpI0rbZ2F8rm4vfv7kyF3+vjeQgyJGkeyXtl7RH0nZJ10jqPsx9NPUfU0R8JiJ+s9F1zBZJr5D0/SO87xOA1wOnR0R1diub0eNfkf6u/F5mWVu6bPlc13Mo6d/Hg5K6MsteJemGGd7/GknvyK3AAnEQ5O93IqIbeBqwGnhrg+tpSc0ahJM8AXg4Ih6cauUcPYcdwNsklfN+oFl6jDJw2SzsJxdKzPv/k/P+CTaLiNgC/Afw5MnrJJUkvVXSfek7pE9KWpyu/q/05860ZfFrU9z/TElrJT2atjzeP2mTiyXdn56y+OvM/TolfVDSA+ntg5I603U3SnpxOv2s9F3lC9L550m6NZ2e8A463e7Vku6WtFPSVZKUritL+vu0jl9IuvRgrZ30HeObJN0G7E3f3T5D0n+n+/6ppLMz279C0kZJu9P9X5wuv0LSpzPbTdnKkrQK+DDwa+mx3pku/21Jd6T73SLpDVPUeg7wLeD49L7XZB7nlZLuB75zsNc6s/0fSdok6ZH0WD5d0m3pc/7HqY5VxjeAQeBl0xzTTknvS38ftkv6sKSFU72W6bKQdGo6fY2kf5J0naS9wK9LeoGkW9LfvU2SrjhEfZO9F3iDpCXT1FuT9C1JOyStV9rakXQJcDHwxvR4fzU9bl/N3PduSf+amd8k6Yx0+pmSbpK0K/35zMx2N0j6W0k/APYBJ0+qqS99Pf7yMJ9r84oI33K6AfcC56TTJwK3A29P528AXpVO/x9gA8kvXDfwJeBT6brlQABtB3mc/wFenk53A8+YdN+PAAuBpwADwKp0/ZXAD4FjgV7gvzP1XQn8v3T6r4B7gPdk1n0onX4F8P1MLQF8DVhC8g65Hzg3Xfdq4A7gBOAo4NsHe27p8bs1PXYLgWXAw8Bvk7yJ+Y10vhfoAh4FVqb37QOelE5fAXw6s98Jx3TSazHh+aTLtgLPSaePAp42Tb1nA5uneJxPpvUtnOFr/WFgAfCbwAHgy+lrtAx4EHjuNI9/BfBp4DxgI9AOtKX7XJ5u8wFgDXA0UAG+CrzrIM89gFPT6WuAXcCz0uO/IH3Ov5TO/zKwHXjRTH5309f3nPQYvCNd9irghnS6C9gE/FH6PJ4KPERy6m20nndk9ncysDOt5XjgvtHXI133SLru6HT65el+L0rnj8n8PtwPPCld354uexWwArgLuKTR/19m8+YWQf6+nL6z/D5wI/DOKba5GHh/RGyMiD3AW4ALp3unPIUh4FRJSyNiT0T8cNL6t0XE/oj4KfBTkkAYfdwrI+LBiOgH3kbyx0Fa63PT6bOAd2Xmn5uun867I2JnRNwPfBc4I13+eyQBsjkiHgHePYPn9g8RsSki9pO8y70uIq6LiHpEfAtYSxIMAHXgyZIWRsTWiLh9BvufiSHgdEk9EfFIRNx8mPe/IiL2ps9hJq/12yPiQER8E9gLfC59jbYA3yP5hzitiFhDEsATOjfTltklwOsiYkdE7Cb5fbzwMJ7LVyLiB+nxPxARN0TEz9L524DPMf57MlOXA6+R1Dtp+QuBeyPiXyJiOCJuAf4NuGCqnUTERmA3ye/bWcD1wAOSamlN34uIOvAC4O6I+FS6388BdwK/k9ndNRFxe7p+KF12Osnv899ExNWH+RybmoMgfy+KiCURcVJE/Fn6z2Cy0Xcvo+4jeSdy3Awf45XAE4E702buCyetz36CZR/JO9HpHvf4dPp/gCdKOo7kD+uTwImSlgJnMn7KaioHe7xNmXXZ6elktzkJuCA9RbIzDdhnA30RsRd4KUmrY6ukr6f/AGbDi0nC5j4lp8wec3ruELLPYSav9fbM9P4p5mfygYO3An9N8q59VC+wCPhJ5vh9I10+UxNeM0m/Kum7kvol7SI5/ksPY39ExM9JWpFvnrTqJOBXJ73eFwMH64i/kaSVclY6fQNJCGTfvEx+DUjnl2Xmp/rdvBjYAlx78GfUehwEzeEBkl/6UU8Ahkn+ARxyeNiIuDsiLiI5ffAe4FplPolxmI/7QLrPfcBPSDryfh4RgySnjv4CuCciHprB/ifbSnJaaNSJM7hP9vlvIjmNsiRz64qId6c1Xx8Rv0FyWuhOklNikLyrXpTZz8H+kTzmeEfETRFxPsnx/TLwxRnUPd0+D/Zaz5q0tbQB+LPM4odIguRJmeO3OJIPM8Ck4yRpquM0+fh8luRU04kRsZjktJaOoOS/Af6Yx/4zvnHS690dEX86TS0wHgTPSadHW7bZIJj8GkDyOmzJzE+17ytIjuFnNQed8XPJQdAcPge8TtIKJR8vfSfwhYgYJmni15nUYZUl6WWSetNm7850cX2Gj/tWSb3pO/3LSc4xj7oRuJTxP6AbJs0fri8Cl0lalnYOvukw7/9p4Hck/ZaSjucFks6WdIKk4ySdnwbgALCH8WNwK3CWpCekHbNvOchjbAdOkNQBIKlDyXclFqenCB5lZsd2Ogd7rWfbXwNvHJ1Jfz8+AnxA0rEA6WvxW+kmPwWeJOkMSQtI/vEdSgXYEREHJJ0J/P6RFBoRG4AvAK/NLP4aSav05ZLa09vTlXTqQ/JaTf67uBH4dWBhRGwmOZV2LnAMcEu6zXXpfn9fyQcQXkpy2udrhyhziOS0VBfwSc2jTxPNmyfS4j4OfIrkdMsvSDoIXwNj78z/FvhB2jx+xhT3Pxe4XdIe4EPAhdOcgprsHSTn2G8DfgbcnC4bdSPJH/p/TTN/uD4CfDN9vFtI/iCHgZGZ3DkiNgHnk3Re95O8Y/xLkt/jEklr5QGSj1A+F/jT9H7fIvkncxtJK+dgf/DfIenU3yZptNXzcuBeSY+SnPq4eEbPdmrTvtazLSJ+APx40uI3kbQUfpg+n28DK9Pt7yL5IMC3gbtJ+rUO5c+AKyXtJnkjcbitpawrSf7Jjta/m6TD/EKS13UbSYu3M93kYyR9NzslfTnzHPaQBAAR8ShJx/kPImIkXfYwSf/D60k+bPBG4IUzaeWmLePfJTmV9/H5EgaK8IVprDEkPR/4cERMbqab2RyaF2lmrUHSQiWfyW+TtIzkvPC/N7ous6Jzi8DmjKRFJKeXaiSdll8HLkub72bWIA4CM7OC86khM7OCa4WBvCZYunRpLF++vNFlmJm1lJ/85CcPRcSUXx5suSBYvnw5a9eubXQZZmYtRdLkb1OP8akhM7OCcxCYmRWcg8DMrOAcBGZmBecgMDMruFyDQNK56eXlNkiaPNb46KXx+iXdmt5eNdV+zMwsP7l9fDQdr/sqkssJbgZukrQmIu6YtOkXIuLSvOowM7ODy7NFcCawIb0k3yDweZIhhBti7b07eM837sRDapiZTZRnECxj4uXeNjPx6kOjXizpNknXSpryilWSLpG0VtLa/v7+Iyrm51t28U833EP/7oEjur+Z2XzV6M7irwLLI+KXgW8Bn5hqo4i4OiJWR8Tq3t7DubzquFpfDwDrtu0+wlLNzOanPINgCxOvSXsCE68JSkQ8HBGjb9E/CvxKXsXUqhUA7tzqEY/NzLLyDIKbgNPSa7N2kFxubk12A0l9mdnzgHV5FbNkUQfVngXc6RaBmdkEuX1qKCKGJV0KXA+UgY9HxO2SrgTWRsQa4LWSziO5bu0O4BV51QNQ66s4CMzMJsl19NGIuI7kAuXZZZdnpt8CvCXPGrJq1R5+sGEjQyN12suN7h4xM2sOhfpvuKqvwtBIsLF/b6NLMTNrGoUKglo1+eTQndvcYWxmNqpQQXBybxftZbFuq/sJzMxGFSoI2sslTuntdovAzCyjUEEAsKqvh/X+5JCZ2ZjCBUGtWmHrrgPs3DfY6FLMzJpC8YKgb7TD2K0CMzMoYBCs8lATZmYTFC4IeiudHN3V4RaBmVmqcEEgiZXHVTwKqZlZqnBBAMmYQ3dt20297ovUmJkVMghWVXvYPzTC/Tv2NboUM7OGK2QQ1PrSDmN/sczMrJhBcNqxFUrCQ02YmVHQIFjYUWb50i63CMzMKGgQQPINY3+E1Mys0EHQw/079rF3YLjRpZiZNVSBg6BCBNy13a0CMyu2wgbBKo85ZGYGFDgIli1ZSHdnm8ccMrPCK2wQlEpiZdVDTZiZFTYIAFZWK6zftpsIDzVhZsVV6CBYVa2wa/8Q2x490OhSzMwaptBBMHaRGn/D2MwKrNBBsDK9SM06f8PYzAqs0EHQs6CdZUsWukVgZoVW6CAAWNVX8ZhDZlZohQ+CldUKG/v3MjA80uhSzMwaovBBUKv2MFwP7nlwb6NLMTNriMIHwSpfpMbMCq7wQbD8mC462koec8jMCivXIJB0rqT1kjZIevNBtnuxpJC0Os96ptJWLvHE47pZ5zGHzKygcgsCSWXgKuD5wOnARZJOn2K7CnAZ8KO8ajmUWrXHLQIzK6w8WwRnAhsiYmNEDAKfB86fYru3A+8BGjbOQ61aoX/3AA/vGWhUCWZmDZNnECwDNmXmN6fLxkh6GnBiRHz9YDuSdImktZLW9vf3z3qhtWoy1MR6twrMrIAa1lksqQS8H3j9obaNiKsjYnVErO7t7Z31Wmp9o0NNOAjMrHjyDIItwImZ+RPSZaMqwJOBGyTdCzwDWNOIDuOl3Z0s7e70RWrMrJDyDIKbgNMkrZDUAVwIrBldGRG7ImJpRCyPiOXAD4HzImJtjjVNKxlqwi0CMyue3IIgIoaBS4HrgXXAFyPidklXSjovr8c9UrVqhbu272ak7ovUmFmxtOW584i4Drhu0rLLp9n27DxrOZSV1R4Ghuvc+/BeTuntbmQpZmZzqvDfLB5VS69N4CGpzaxoHASpU4/tplySxxwys8JxEKQWtJc5eWkX69wiMLOCcRBk1Pp63CIws8JxEGTUqhU2P7Kf3QeGGl2KmdmccRBkjHYY37Xdp4fMrDgcBBm1vmTMIfcTmFmROAgyjl+8gMqCNvcTmFmhOAgyJLGq2uPvEphZoTgIJqn1VVi/bTcRHmrCzIrBQTBJrdrD7oFhtuzc3+hSzMzmhINgkpUeasLMCsZBMMlYELjD2MwKwkEwSXdnG084epGvVmZmheEgmEKtWvHVysysMBwEU6j19fCLh/ZyYGik0aWYmeXOQTCFVdUK9YAND+5pdClmZrlzEExhtMN4nU8PmVkBOAimcNIxXSxoL/li9mZWCA6CKZRLYuVxFX+E1MwKwUEwjVq1h3VbPdSEmc1/DoJp1Poq7Ng7SP+egUaXYmaWKwfBNGrV5NoE691PYGbznINgGjWPOWRmBeEgmMZRXR0c19PJOncYm9k85yA4iJovUmNmBeAgOIhaX4UND+5haKTe6FLMzHLjIDiIVdUeBkfq/OKhvY0uxcwsNw6Cgxi/NoFPD5nZ/OUgOIhTertpK8lDUpvZvOYgOIiOthKnHtvtFoGZzWu5BoGkcyWtl7RB0punWP9qST+TdKuk70s6Pc96joQvUmNm811uQSCpDFwFPB84Hbhoin/0n42IX4qIM4C/A96fVz1HqtbXwwO7DrBr31CjSzEzy0WeLYIzgQ0RsTEiBoHPA+dnN4iI7FvtLqDpRngb/Ybx+u0+PWRm81OeQbAM2JSZ35wum0DSn0u6h6RF8Noc6zkio2MOeUhqM5uvGt5ZHBFXRcQpwJuAt061jaRLJK2VtLa/v39O6zuup5Mli9pZ528Ym9k8lWcQbAFOzMyfkC6bzueBF021IiKujojVEbG6t7d3Fks8NElJh7FbBGY2T+UZBDcBp0laIakDuBBYk91A0mmZ2RcAd+dYzxGrVXtYv2039XrTdWGYmT1ubXntOCKGJV0KXA+UgY9HxO2SrgTWRsQa4FJJ5wBDwCPAH+ZVz+Oxqq/CvsERNj2yj5OO6Wp0OWZmsyq3IACIiOuA6yYtuzwzfVmejz9bxjuMdzsIzGzeaXhncSt44nEVJF+kxszmJwfBDCzsKLP8mC53GJvZvOQgmKHkk0NuEZjZ/HPIIJBUkvTMuSimmdWqPdz78F72DQ43uhQzs1l1yCCIiDrJmEGFVuurEAF3bd/T6FLMzGbVTE8N/aekF0tSrtU0sVXpJ4fWu5/AzOaZmQbBnwD/CgxKelTSbkmF+o94wlELWdRR9lATZjbvzOh7BBFRybuQZlcqiZUeasLM5qEZf6FM0nnAWensDRHxtXxKal61ag//8fOtRAQFPktmZvPMjE4NSXo3cBlwR3q7TNK78iysGa3qq7Bz3xDbHx1odClmZrNmpi2C3wbOSD9BhKRPALcAb8mrsGY0OtTEum2PUl28oMHVmJnNjsP5QtmSzPTi2S6kFawcvVqZv1hmZvPITFsE7wRukfRdQCR9BY+5GP18t3hhO8cvXuCL2ZvZvHLIIJBUAurAM4Cnp4vfFBHb8iysWdX6ejzUhJnNKzP9ZvEbI2JrRKxJb4UMAUjGHNrw4B4Gh+uNLsXMbFbMtI/g25LeIOlESUeP3nKtrEnV+noYrgf39HuoCTObH2baR/DS9OefZ5YFcPLsltP8VqUdxndue5RVfT0NrsbM7PGbaR/BmyPiC3NQT9NbsbSLjnLJ/QRmNm/MtI/gL+eglpbQVi5x6rHdvlqZmc0b7iM4ArU+jzlkZvOH+wiOwKpqD1+6eQs79g5ydFdHo8sxM3tcZjr66Iq8C2kltb7xDuNnnrK0wdWYmT0+Bz01JOmNmekLJq17Z15FNbva2EVq3E9gZq3vUH0EF2amJw8wd+4s19IyeiudLO3ucIexmc0LhwoCTTM91Xyh+CI1ZjZfHCoIYprpqeYLpVbtYf323YzUC30YzGweOFRn8VPSaxMLWJi5TrGAQg/IX6tWODBU576H93Jyb3ejyzEzO2IHDYKIKM9VIa1mdHiJO7ftdhCYWUs7nAvTWMapx3ZTEh5qwsxanoPgCC1oL3Nyb7cvUmNmLc9B8Dgknxxyi8DMWpuD4HFYVa1w/4597BkYbnQpZmZHLNcgkHSupPWSNkh6zDWOJf2FpDsk3SbpPyWdlGc9s83fMDaz+SC3IJBUBq4Cng+cDlwk6fRJm90CrI6IXwauBf4ur3rykB1zyMysVeXZIjgT2BARGyNiEPg8cH52g4j4bkTsS2d/CJyQYz2zbtmShVQ629wiMLOWlmcQLAM2ZeY3p8um80rgP6ZaIekSSWslre3v75/FEh8fScm1CTzmkJm1sKboLJb0MmA18N6p1kfE1RGxOiJW9/b2zm1xh7CyWmHdtkeJ8FATZtaa8gyCLcCJmfkT0mUTSDoH+GvgvIgYyLGeXNSqPew+MMwDuw40uhQzsyOSZxDcBJwmaYWkDpIhrddkN5D0VOCfSULgwRxryc2q0Q5jf7HMzFpUbkEQEcPApcD1wDrgixFxu6QrJZ2XbvZeoBv4V0m3Slozze6a1hOPG/3kkPsJzKw1zfSaxUckIq4Drpu07PLM9Dl5Pv5cqCxo58SjFzoIzKxlNUVncaurVXt8asjMWpaDYBbUqhU2PrSXA0MjjS7FzOywOQhmQa3aw0g92PDgnkaXYmZ22BwEs2B8qAn3E5hZ63EQzILlx3TR2VZyP4GZtSQHwSwol8TKaoX1290iMLPW4yCYJbVqhXUec8jMWpCDYJasrPbw0J4B+ne33CgZZlZwDoJZsqqadBh7SGozazUOglmysuqL1JhZa3IQzJJjujs5ttLpj5CaWctxEMyiWl+PWwRm1nIcBLNoVbXCXdv3MDxSb3QpZmYz5iCYRSurFQaH69z78N5Gl2JmNmMOgllUq/YA+PsEZtZSHASz6JRju2gryf0EZtZSHASzqLOtzCm93f4ugZm1FAfBLKv1eagJM2stDoJZVqv2sGXnfh49MNToUszMZsRBMMtqHmrCzFqMg2CWjV2kxtcmMLMW4SCYZdWeBSxe2M46twjMrEU4CGaZJGrVik8NmVnLcBDkYFVfD+u37aZej0aXYmZ2SA6CHKysVtgzMMyWnfsbXYqZ2SE5CHIw+smhde4wNrMW4CDIwROPqyDhaxOYWUtwEOSgq7ONk45e5DGHzKwlOAhyUqv2uEVgZi3BQZCTWl+Fex/ay/7BkUaXYmZ2UA6CnNSqFeoBdz/oVoGZNbdcg0DSuZLWS9og6c1TrD9L0s2ShiW9JM9a5troRWru9EikZtbkcgsCSWXgKuD5wOnARZJOn7TZ/cArgM/mVUejPOHoRSxsL7POHcZm1uTactz3mcCGiNgIIOnzwPnAHaMbRMS96bp5d7X3Ukms9FATZtYC8jw1tAzYlJnfnC47bJIukbRW0tr+/v5ZKW4urOqrsG7ro0R4qAkza14t0VkcEVdHxOqIWN3b29vocmasVu3hkX1D9O8eaHQpZmbTyjMItgAnZuZPSJcVxsrRoSZ8esjMmlieQXATcJqkFZI6gAuBNTk+XtMZHXPIF6kxs2aWWxBExDBwKXA9sA74YkTcLulKSecBSHq6pM3ABcA/S7o9r3oaYcmiDvoWL/A3jM2sqeX5qSEi4jrguknLLs9M30RyymjeqlUrDgIza2ot0Vncymp9PWx4cDdDI/PuE7JmNk84CHJWq1YYGgk29u9tdClmZlNyEORsbKgJf8PYzJqUgyBnJ/d20V4W6zzmkJk1KQdBztrLJU49tuIWgZk1LQfBHFjlMYfMrIk5COZAra/C1l0H2LlvsNGlmJk9hoNgDox3GLtVYGbNx0EwBzzUhJk1MwfBHOitdHJ0V4dbBGbWlBwEc0AStWrFo5CaWVNyEMyRWrWHu7btpl73RWrMrLk4COZIra/C/qER7t+xr9GlmJlN4CCYI6s81ISZNSkHwRw57bhuSsJDTZhZ03EQzJEF7WWWL+1yi8DMmo6DYA6tqvb4I6Rm1nQcBHOoVq1w/4597B0YbnQpZmZjHARzqNbXQwTctd2tAjNrHg6COTQ21IRPD5lZE3EQzKETjlpId2ebxxwys6bS1ugCikQSK6sVbrirn499/xcsW7KAZUsWcfySBRzd1YGkRpdoZgXkIJhjz39ylfd9cz1v/9odE5YvaC9x/JKFLEtvx0/6WV28gI42N+DMbPY5CObYq55zMq989goe2TfEAzv3s2XnfrY8sp8Hdu7ngV3J9Lqtu3loz8CE+0lwbKVzQlgsO2ohxy9Ow+KohfQsaHOrwswOm4OgASRxdFcHR3d18ORli6fc5sDQCFt3HZgyLH6+ZRffvH07gyP1Cffp7mxLWxELxsIh26o4ttJJW9mtCjObyEHQpBa0l1mxtIsVS7umXF+vBw/vHWTLzjQgdu5ncxoWW3bu59ZNO3lk39CE+5RLotqzYCwsjunupKujTFdnG12dbXSnP7s6y3R1ZJcl86WSWxtm85GDoEWVSqK30klvpZMzTlwy5Tb7BofTYEhbFpmgWHvfI+zcN8TewWFihiNjL2wvp+EwHh6jQTIeIsn6RR0Tg6W7sy2zzMFi1kwcBPPYoo42Tj22wqnHVqbdpl4P9g+NsHdwmL0DI+wdGGbPwPDYz32Dk5cl8/sGk2UP7Rnkvof3jW97GMGyKA2RBe0lOsolOtvKdLSV6Ggr0Tl2K4/NT/yZblsu0Tl6//by2Hzn2PLy2PrJ928vy30qZjgICq9U0tg7eabPixkbC5ax8BgNmcz8wHBm2QgDQyMMjNQZGKozOFJnYGiEPQPD7NhbZ2C4zuBwnYHhkfRnMj88Sxf4yQZMe7lEW1m0l5KfbaUS7WXRVi7RVtLY+uzy9pKSZWPTE/fRnt63rZzeZ2x5Zj+ZbdvLJcolKJeS+5WU7L9cEmUlP9vKmelSiVIJ2kqlZJuSKAkHnB0WB4HNqmywHJvj44zU4zEBMTBFYEwXJGPzaQANDNcZHkkCZmikzvBIMFyvM5T5uW9wOF0f0287Umeonvxs5MXo2kqiVBJtaTiUs9MS5TSASpoYIqPblTLBIzF2v0MtL6VBVE5DbDycRLkEJR16+fhjJIGW7DfZZmw+DbvsY0mM7WN0upTZTyndfnR5KfO4B11eGp8WQmPrmfC4JQmRfMKv1YLYQWAtqVwSCzvKLOwoN7qUadXrwVA9DYqR8emhTIg8NkiCkQjq9WC4HozU64zUYbheZ6QeY7fhelCPdPt6cp+R+vj9R+rJY4zvZ9J9Jy0fe6wguW9a0+BIErr1GN92dDqCscetj9UAEfGY5fU6yf0iZnzqsJWNh8N4YE0OiwkBkw0W0vlMII3u77XPO43znnL8rNfrIDDLSakkOktlOv1XNkHEeHhlA6Jen2J5Gjj1iMwtna8zxbLM9KRtIsZDrR7jgTU6XU+DLNk2WT66/XgNybbZ+ez2o9tm57P7n+k2Mc19lixsz+U1yfVXVNK5wIeAMvDRiHj3pPWdwCeBXwEeBl4aEffmWZOZNZbSfg/nY/PI7dtFksrAVcDzgdOBiySdPmmzVwKPRMSpwAeA9+RVj5mZTS3Pr5meCWyIiI0RMQh8Hjh/0jbnA59Ip68FnqdW62UxM2txeQbBMmBTZn5zumzKbSJiGNgFHDN5R5IukbRW0tr+/v6cyjUzK6aWGHgmIq6OiNURsbq3t7fR5ZiZzSt5BsEW4MTM/Anpsim3kdQGLCbpNDYzszmSZxDcBJwmaYWkDuBCYM2kbdYAf5hOvwT4TkQRPmVsZtY8cvsEV0QMS7oUuJ7k46Mfj4jbJV0JrI2INcDHgE9J2gDsIAkLMzObQ7l+lDcirgOum7Ts8sz0AeCCPGswM7ODU6udiZHUD9x3hCa+gnsAAAYwSURBVHdfCjw0i+W0Oh+PiXw8xvlYTDQfjsdJETHlp21aLggeD0lrI2J1o+toFj4eE/l4jPOxmGi+H4+W+PiomZnlx0FgZlZwRQuCqxtdQJPx8ZjIx2Ocj8VE8/p4FKqPwMzMHqtoLQIzM5vEQWBmVnCFCQJJ50paL2mDpDc3up5GkXSipO9KukPS7ZIua3RNzUBSWdItkr7W6FoaTdISSddKulPSOkm/1uiaGkXS69K/k59L+pykBY2uKQ+FCIIZXiSnKIaB10fE6cAzgD8v8LHIugxY1+gimsSHgG9ERA14CgU9LpKWAa8FVkfEk0mGypmXw+AUIgiY2UVyCiEitkbEzen0bpI/8snXiSgUSScALwA+2uhaGk3SYuAsknHAiIjBiNjZ2Koaqg1YmI6OvAh4oMH15KIoQTCTi+QUjqTlwFOBHzW2kob7IPBGoN7oQprACqAf+Jf0VNlHJXU1uqhGiIgtwPuA+4GtwK6I+GZjq8pHUYLAJpHUDfwb8H8j4tFG19Mokl4IPBgRP2l0LU2iDXga8E8R8VRgL1DIPjVJR5GcOVgBHA90SXpZY6vKR1GCYCYXySkMSe0kIfCZiPhSo+tpsGcB50m6l+SU4f+S9OnGltRQm4HNETHaSryWJBiK6BzgFxHRHxFDwJeAZza4plwUJQhmcpGcQpAkkvO/6yLi/Y2up9Ei4i0RcUJELCf5vfhORMzLd30zERHbgE2SVqaLngfc0cCSGul+4BmSFqV/N89jnnac53o9gmYx3UVyGlxWozwLeDnwM0m3psv+Kr12hBnAa4DPpG+aNgJ/1OB6GiIifiTpWuBmkk/b3cI8HWrCQ0yYmRVcUU4NmZnZNBwEZmYF5yAwMys4B4GZWcE5CMzMCs5BYE1LUkj6+8z8GyRdMUv7vkbSS2ZjX4d4nAvSETy/O2n5ckn7Jd2auf3BLD7u2R5J1WaqEN8jsJY1APyupHdFxEONLmaUpLaIGJ7h5q8E/jgivj/Funsi4oxZLM3siLhFYM1smOQLPK+bvGLyO3pJe9KfZ0u6UdJXJG2U9G5JF0v6saSfSTols5tzJK2VdFc65tDodQneK+kmSbdJ+pPMfr8naQ1TfNNW0kXp/n8u6T3pssuBZwMfk/TemT5pSXskfSAdB/8/JfWmy8+Q9MO0rn9Px8JB0qmSvi3pp5JuzjzH7sx1BT6TfjuW9Jjcke7nfTOty+axiPDNt6a8AXuAHuBeYDHwBuCKdN01wEuy26Y/zwZ2An1AJ8mYUm9L110GfDBz/2+QvBk6jWSMnQXAJcBb0206gbUkg46dTTIA24op6jyeZDiCXpJW9neAF6XrbiAZz37yfZYD+4FbM7fnpOsCuDidvhz4x3T6NuC56fSVmefyI+B/p9MLSIZLPhvYRTKuVgn4H5JQOgZYz/iXSZc0+nX2rfE3twisqUUyMuonSS4QMlM3RXLdhQHgHmB06OCfkfwDHvXFiKhHxN0kQynUgN8E/iAdfuNHJP84T0u3/3FE/GKKx3s6cEMkg5MNA58hGdP/UO6JiDMyt++ly+vAF9LpTwPPTq8TsCQibkyXfwI4S1IFWBYR/w4QEQciYl+m3s0RUScJmuUk4XCApJXyu8DotlZgDgJrBR8kOdeeHRd/mPT3V1IJ6MisG8hM1zPzdSb2i00eXyUAAa/J/HNeEeNj0O99XM/iyB3pODDZ4zACjPZtnEkyqugLSVpFVnAOAmt6EbED+CJJGIy6F/iVdPo8oP0Idn2BpFJ6Tv1kklMm1wN/mg7VjaQnzuDCLD8GnitpaXpZ1IuAGw9xn4MpAaP9H78PfD8idgGPSHpOuvzlwI2RXGVus6QXpfV2Slo03Y7T61AsjmSQwdeRXIrSCs6fGrJW8ffApZn5jwBfkfRTkne1R/Ju/X6Sf+I9wKsj4oCkj5KcQrk57VztB150sJ1ExFZJbwa+S9Ki+HpEfGUGj39KZgRYSEbF/QeS53KmpLcCDwIvTdf/IfDh9B99dlTQlwP/LOlKYAi44CCPWSE5bgvSWv9iBnXaPOfRR82ajKQ9EdHd6DqsOHxqyMys4NwiMDMrOLcIzMwKzkFgZlZwDgIzs4JzEJiZFZyDwMys4P4/H3UmvY/SQdkAAAAASUVORK5CYII=\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"plt.xlabel('Number of Epochs')\n",
"plt.ylabel('Error')\n",
"plt.title('Plot showing results from Neural Network')\n",
"plt.plot(error)\n",
"plt.show()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "8pLBUsrUqAc0",
"outputId": "3d3130f3-e540-48bc-9faf-a9fc2523c9ff"
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deZwcdZ3/8dd7jtwhgWS4cpA7CMg5hEO5k4Ci4IoIiqy4sIjIsUB03d/6cJF1dz24RFFEZREPBFmPiGgSjoT7SLgD5CTk4MiEkPua4/P7o2qkM5mjk0xPTU+/n49HP6aOb1d/qqun3/2t6q5SRGBmZqWrLOsCzMwsWw4CM7MS5yAwMytxDgIzsxLnIDAzK3EOAjOzEucgyIik6ZIuKPBjDJMUkioK+BjnSJpaqOV3hPbcFpK+JWmFpLfbY3nFrCNefx2lI/5fs+QgKCBJiyRtlLRO0juSbpfUZzuX0an/mSLi1xExMes62ouk8yQ9uoP3HQpcBewXEXu2b2V5Pf7V6Wvl0znTKtJpwzq6nrak/x/LJfXOmXaBpOl53v92Sd8qWIElxEFQeB+PiD7AoUA18PWM6ylKnTUImxgKvBsRy5ub2UHrsBL4pqTyQj9QOz1GOXB5OyynIJTo8u+TXX4FO4uIWAb8FTig6TxJZZK+LumN9BPSHZL6pbMfTv+uSnsWRzVz/3GSZkpak/Y8rm/S5BxJi9NdFv+ec7/ukm6U9GZ6u1FS93TeDElnpMMfSj9VnpqOnyTp+XR4q0/QabuLJM2TtErSzZKUziuXdF1ax+uSLmmtt5N+YvxXSS8C69NPt0dKejxd9guSjs9pf56khZLWpss/J51+taRf5bRrtpcl6QPALcBR6XO9Kp3+UUmvpMtdJmlSM7WOB6YBe6f3vT3ncc6XtBh4sLVtndP+C5KWSHovfS4Pl/Rius4/bO65yvE3YAvwuRae0+6Srk1fD+9IukVSz+a2ZTotJI1Kh2+X9GNJ90laD5wg6VRJz6WvvSWSrm6jvqa+B0yS1L+FeveVNE3SSklzlPZ2JF0InAN8NX2+/5w+b3/Oue88Sb/LGV8i6eB0+GhJz0hanf49OqfddEn/JekxYAMwoklNe6Xb4yvbua6dV0T4VqAbsAgYnw4PAWYD/5mOTwcuSIf/CZhP8oLrA/we+GU6bxgQQEUrj/MEcG463Ac4ssl9fwr0BA4CNgMfSOdfAzwJ7A5UAY/n1HcN8IN0+P8BC4Dv5Mz7fjp8HvBoTi0B3Av0J/mEXAOcks67CHgFGAzsCtzf2rqlz9/z6XPXExgEvAt8lORDzIR0vAroDawBxqb33QvYPx2+GvhVznK3ek6bbIut1ied9hZwTDq8K3BoC/UeDyxt5nHuSOvrmee2vgXoAUwENgF/TLfRIGA5cFwLj3818CvgNGAhUAlUpMsclra5AZgM7Ab0Bf4M/E8r6x7AqHT4dmA18KH0+e+RrvMH0/EDgXeAT+Tz2k237/j0OfhWOu0CYHo63BtYAnwhXY9DgBUku94a6/lWzvJGAKvSWvYG3mjcHum899J5u6XD56bL/Uw6PiDn9bAY2D+dX5lOuwAYDswFLsz6/aU9b+4RFN4f00+WjwIzgP9ups05wPURsTAi1gH/Bpzd0iflZtQCoyQNjIh1EfFkk/nfjIiNEfEC8AJJIDQ+7jURsTwiaoBvkvxzkNZ6XDp8LPA/OePHpfNb8u2IWBURi4GHgIPT6Z8mCZClEfEe8O081u2miFgSERtJPuXeFxH3RURDREwDZpIEA0ADcICknhHxVkTMzmP5+agF9pO0S0S8FxHPbuf9r46I9ek65LOt/zMiNkXEVGA9cGe6jZYBj5C8IbYoIiaTBPBWBzfTntmFwBURsTIi1pK8Hs/ejnX5U0Q8lj7/myJiekS8lI6/CNzJ+6+TfH0DuFRSVZPpHwMWRcT/RkRdRDwH/B9wZnMLiYiFwFqS19uxwBTgTUn7pjU9EhENwKnAvIj4ZbrcO4HXgI/nLO72iJidzq9Np+1H8nr+j4i4dTvXsVNzEBTeJyKif0TsExEXp28GTTV+emn0BsknkT3yfIzzgTHAa2k392NN5ud+g2UDySfRlh5373T4CWCMpD1I/rHuAIZIGgiM4/1dVs1p7fGW5MzLHW5Jbpt9gDPTXSSr0oD9MLBXRKwHziLpdbwl6S/pG0B7OIMkbN5Qsstsm91zbchdh3y29Ts5wxubGc/nCwdfB/6d5FN7oyqgFzAr5/n7Wzo9X1ttM0lHSHpIUo2k1STP/8DtWB4R8TJJL/JrTWbtAxzRZHufA7R2IH4GSS/l2HR4OkkI5H54aboNSMcH5Yw399o8B1gG3NP6GhUfB0Hn8CbJi77RUKCO5A2gzdPDRsS8iPgMye6D7wD3KOebGNv5uG+my9wAzCI5kPdyRGwh2XV0JbAgIlbksfym3iLZLdRoSB73yV3/JSS7Ufrn3HpHxLfTmqdExASS3UKvkewSg+RTda+c5bT2RrLN8x0Rz0TE6STP7x+Bu/Oou6Vltrat203aW5oPXJwzeQVJkOyf8/z1i+TLDNDkeZLU3PPU9Pn5DcmupiER0Y9kt5Z2oOT/AP6Zbd+MZzTZ3n0i4kst1ALvB8Ex6XBjzzY3CJpuA0i2w7Kc8eaWfTXJc/gbdcDB+I7kIOgc7gSukDRcyddL/xu4KyLqSLr4DTQ5YJVL0uckVaXd3lXp5IY8H/frkqrST/rfINnH3GgGcAnv/wNNbzK+ve4GLpc0KD04+K/bef9fAR+XdLKSA889JB0vabCkPSSdngbgZmAd7z8HzwPHShqaHpj9t1Ye4x1gsKRuAJK6KfmtRL90F8Ea8ntuW9Latm5v/w58tXEkfX38FLhB0u4A6bY4OW3yArC/pIMl9SB542tLX2BlRGySNA747I4UGhHzgbuAy3Im30vSKz1XUmV6O1zJQX1ItlXT/4sZwAlAz4hYSrIr7RRgAPBc2ua+dLmfVfIFhLNIdvvc20aZtSS7pXoDd6gLfZuoy6xIkbsN+CXJ7pbXSQ4QXgp//2T+X8Bjaff4yGbufwowW9I64PvA2S3sgmrqWyT72F8EXgKeTac1mkHyj/5wC+Pb66fA1PTxniP5h6wD6vO5c0QsAU4nOXhdQ/KJ8Sskr+Mykt7KmyRfoTwO+FJ6v2kkbzIvkvRyWvuHf5DkoP7bkhp7PecCiyStIdn1cU5ea9u8Frd1e4uIx4Cnm0z+V5KewpPp+twPjE3bzyX5IsD9wDyS41ptuRi4RtJakg8S29tbynUNyZtsY/1rSQ6Yn02yXd8m6fF2T5v8nOTYzSpJf8xZh3UkAUBErCE5cP5YRNSn094lOf5wFcmXDb4KfCyfXm7aM/4kya6827pKGCjCF6axbEj6CHBLRDTtpptZB+oSaWbFQVJPJd/Jr5A0iGS/8B+yrsus1LlHYB1GUi+S3Uv7khy0/Atwedp9N7OMOAjMzEqcdw2ZmZW4YjiR11YGDhwYw4YNy7oMM7OiMmvWrBUR0eyPB4suCIYNG8bMmTOzLsPMrKhIavpr6r/zriEzsxLnIDAzK3EOAjOzEucgMDMrcQ4CM7MSV9AgkHRKenm5+ZKanmu88dJ4NZKeT28XNLccMzMrnIJ9fTQ9X/fNJJcTXAo8I2lyRLzSpOldEXFJoeowM7PWFbJHMA6Yn16SbwvwW5JTCGfi5WWr+fZfX8On1DAz21ohg2AQW1/ubSlbX32o0RmSXpR0j6Rmr1gl6UJJMyXNrKmp2aFiZr3xHrfMWMAj83bkwlpmZl1X1geL/wwMi4gDgWnAL5prFBG3RkR1RFRXVW3P5VXfd/a4IQzq35Nrp85xr8DMLEchg2AZW1+TdjBbXxOUiHg3Ijanoz8DDitUMd0ryrl8/GheXLqaKbPb9fKwZmZFrZBB8AwwOr02azeSy81Nzm0gaa+c0dOAVwtYD588ZBAjqnpz/bQ51De4V2BmBgUMgvRi3JcAU0je4O+OiNmSrpF0WtrsMkmzJb1ActHq8wpVD0BFeRlXjB/D3HfWMfmFZW3fwcysBBTdhWmqq6tjZ84+2tAQnPqDR1m/uY4HrjqOyvKsD5OYmRWepFkRUd3cvJJ7FywrE5MmjmHxyg38bubSrMsxM8tcyQUBwIn77s6hQ/tz0wPz2FRbn3U5ZmaZKskgkMSkk8fy9ppN/OrJFq/VYGZWEkoyCACOHjmQD40awI+mL2Dd5rqsyzEzy0zJBgHApIljWbl+C//76OtZl2JmlpmSDoJDhu7K+A/swa2PLGTVhi1Zl2NmlomSDgKAqyaOYd3mOn7y8MKsSzEzy0TJB8EH9tqFjx+4N7c/tojlazdlXY6ZWYcr+SAAuGLCGLbUN/CjhxZkXYqZWYdzEADDB/bmzMMG85unFrNs1casyzEz61AOgtSlJ40G4Kb752VciZlZx3IQpAb178lnjxjKPc8uZWHNuqzLMTPrMA6CHF8+YRTdysu4wb0CMyshDoIcVX2784UPDePPL7zJK2+uybocM7MO4SBo4ovHjqRvjwqunzYn61LMzDqEg6CJfr0q+eKxI7j/1eU8u/i9rMsxMys4B0EzvvCh4Qzo3Y1rp7hXYGZdn4OgGb27V3DxCaN4fMG7PD5/RdblmJkVlIOgBeccMZS9+vXge1PnUGyX8zQz2x4Oghb0qCznspNG89ziVTzw6vKsyzEzKxgHQSs+ddhg9hnQi2unzqGhwb0CM+uaHAStqCwv44rxY3jt7bXc+9JbWZdjZlYQDoI2fPygvRm7R19unDaXuvqGrMsxM2t3DoI2lJeJKyeOYeGK9fz+2WVZl2Nm1u4cBHmYuN8eHDS4H99/YB6b6+qzLsfMrF05CPIgiUknj2XZqo3c+dTirMsxM2tXDoI8fXjUQI4Yvhs/fGg+G7bUZV2OmVm7cRDkSRJfOXksK9Zt4fbHF2VdjplZu3EQbIfqYbtxwtgqfjJjIas31mZdjplZu3AQbKerJo5l9cZafvbIwqxLMTNrFw6C7XTAoH6c+sG9uO3R13l33easyzEz22kFDQJJp0iaI2m+pK+10u4MSSGpupD1tJcrJoxhY209P56+IOtSzMx2WsGCQFI5cDPwEWA/4DOS9mumXV/gcuCpQtXS3kbt3odPHjqYO558g7dWb8y6HDOznVLIHsE4YH5ELIyILcBvgdObafefwHeATQWspd1dftJoIoIfPDg/61LMzHZKIYNgELAkZ3xpOu3vJB0KDImIv7S2IEkXSpopaWZNTU37V7oDhuzWi7MPH8rdzyzhjXfXZ12OmdkOy+xgsaQy4HrgqrbaRsStEVEdEdVVVVWFLy5Pl544iopyceP987IuxcxshxUyCJYBQ3LGB6fTGvUFDgCmS1oEHAlMLpYDxgC779KDzx81jD8+v4y576zNuhwzsx1SyCB4BhgtabikbsDZwOTGmRGxOiIGRsSwiBgGPAmcFhEzC1hTu7vouJH07lbBdVN9oXszK04FC4KIqAMuAaYArwJ3R8RsSddIOq1Qj9vRdu3djQuOGc6U2e/wwpJVWZdjZrbdVGwXZq+uro6ZMztXp2HtplqO/e5DHDCoH788/4isyzEz24akWRHR7K53/7K4HfTtUcmXjh/JI/NW8NTCd7Mux8xsuzgI2sk/HjWM3ft259qpcyi2XpaZlTYHQTvpUVnOpSeO4plF7zF9buf4rYOZWT4cBO3orMOHMnjXnlw7ZQ4NDe4VmFlxcBC0o24VZfzL+DHMfnMNf5v9dtblmJnlxUHQzv7hkEGM2r0P10+bS717BWZWBBwE7ay8TFw5YQzzl6/jj88ta/sOZmYZcxAUwCn778n+e+/CDffPZUtdQ9blmJm1ykFQAGVlYtLJY1n63kbumrmk7TuYmWXIQVAgx4+ponqfXfnBA/PYVFufdTlmZi1yEBSIJL5y8liWr93MHU8syrocM7MWOQgK6IgRAzhm9EB+PH0BazfVZl2OmVmzHAQFNmniWN7bUMvPH30961LMzJrlICiwg4b05+T99+Bnj7zOe+u3ZF2Omdk2HAQd4KqJY1m/pY5bHl6QdSlmZttwEHSAMXv05RMHD+IXjy9i+ZpNWZdjZrYVB0EH+Zfxo6mrD3740PysSzEz24qDoIPsM6A3Z1YP4c6nF7Nk5YasyzEz+zsHQQe67KRRSOL7D8zLuhQzs79zEHSgvfr15Nwj9+H3zy5l/vJ1WZdjZgY4CDrcxcePpGdlOTdMm5t1KWZmgIOgww3o051/+vBw/vLSW7y8bHXW5ZiZOQiycMExI9ilRwXXTZ2TdSlmZg6CLPTrWclFx4/koTk1zHpjZdblmFmJcxBk5LyjhzGwT3e++7c5RPiSlmaWHQdBRnp1q+DLJ4zkqddX8uj8FVmXY2YlzEGQoc8eMZS9+/Xg2inuFZhZdhwEGepeUc7l40fzwtLVTHvlnazLMbMS5SDI2BmHDmb4wN5cN3Uu9Q3uFZhZx3MQZKyivIwrJoxhzjtruffFN7Mux8xKkIOgE/jYB/di3z37cv20udTWN2RdjpmVmIIGgaRTJM2RNF/S15qZf5GklyQ9L+lRSfsVsp7OqqxMTJo4ljfe3cA9s5ZmXY6ZlZiCBYGkcuBm4CPAfsBnmnmj/01EfDAiDga+C1xfqHo6u5M+sDsHD+nPTQ/MY1NtfdblmFkJKWSPYBwwPyIWRsQW4LfA6bkNImJNzmhvoGSPlkriqyeP5a3Vm/j1U4uzLsfMSkghg2AQsCRnfGk6bSuSvixpAUmP4LIC1tPpHT1qIEePHMCPHprP+s11WZdjZiUi84PFEXFzRIwE/hX4enNtJF0oaaakmTU1NR1bYAebdPJY3l2/hf997PWsSzGzElHIIFgGDMkZH5xOa8lvgU80NyMibo2I6oiorqqqascSO59Dh+7K+A/szk8eXsjqDbVZl2NmJaCQQfAMMFrScEndgLOBybkNJI3OGT0V8DUcgSsnjGXtpjpufWRB1qWYWQkoWBBERB1wCTAFeBW4OyJmS7pG0mlps0skzZb0PHAl8PlC1VNM9tt7Fz5+0N7c9ugiatZuzrocM+viVGwnO6uuro6ZM2dmXUbBLaxZx4QbHuYfj9qH//j4/lmXY2ZFTtKsiKhubl7mB4uteSOq+nDGoYP49ZOLWbZqY9blmFkX5iDoxC47KTmE8oMHfOjEzArHQdCJDd61F589Yii/m7WU11esz7ocM+ui2gwCSWWSju6IYmxbF58wkspyccO0uVmXYmZdVJtBEBENJOcMsgzs3rcHX/jQcP784pu89vaatu9gZrad8t019ICkMySpoNVYs7547Aj6dKvguqnuFZhZ+8s3CL4I/A7YImmNpLWS/PG0g/Tv1Y0Ljx3BtFfe4bnF72Vdjpl1MXkFQUT0jYiyiKiMiF3S8V0KXZy97wsfHs5uvbu5V2Bm7S7vbw1JOk3StentY4UsyrbVp3sFFx8/kkfnr+DxBSuyLsfMupC8gkDSt4HLgVfS2+WS/qeQhdm2PnfkPuy5Sw+unTKHYvtFuJl1Xvn2CD4KTIiI2yLiNuAUkpPEWQfqUVnOpSeN4tnFq3hozvKsyzGzLmJ7flDWP2e4X3sXYvn5dPUQhu7Wi+9NmUtDg3sFZrbz8g2C/waek3S7pF8As4D/KlxZ1pLK8jKumDCaV99aw30vv5V1OWbWBeT1y2KgATgS+D3wf8BREXFXgWuzFpx20CBG796H66fOpa6+IetyzKzI5fvL4q9GxFsRMTm9vd0BtVkLysvEVRPHsnDFen7/XGsXfTMza1u+u4bulzRJ0hBJuzXeClqZterk/ffgwMH9+P7989hcV591OWZWxPINgrOALwMPkxwfmAV0/avDdGKSmDRxLMtWbeS3Ty/JuhwzK2L5HiP4WkQMb3Ib0QH1WSuOGT2QccN344cPzWfjFvcKzGzH5HuM4CsdUIttJ0l85eSx1KzdzC+eWJR1OWZWpHyMoMgdPmw3jh9bxY+nL2DNptqsyzGzIuRjBF3ApIljWb2xlp898nrWpZhZEcr37KNNjw/4GEEncsCgfnzkgD35+SMLeXfd5qzLMbMi02oQSPpqzvCZTeb9d6GKsu135YQxbKyt55YZC7IuxcyKTFs9grNzhv+tybxT2rkW2wmj9+jLJw4ZxB1PvMHbqzdlXY6ZFZG2gkAtDDc3bhm7YvwYGiL4wYPzsi7FzIpIW0EQLQw3N24ZG7JbL846fAh3PbOExe9uyLocMysSbQXBQY3XKAYOTIcbxz/YAfXZdrr0xNGUl4kb7/clLc0sP60GQUSU51yjuCIdbhyv7KgiLX977NKDzx89jD88v4x576zNuhwzKwLbc2EaKxIXHTeS3t0quH6aewVm1jYHQRe0W+9unP/h4fz15bd5aenqrMsxs07OQdBFXXDMcPr3quTaqXOyLsXMOjkHQRfVt0clFx03khlza3j69ZVZl2NmnVhBg0DSKZLmSJov6WvNzL9S0iuSXpT0gKR9CllPqfn8UcOo6tuda6fMIcLf9jWz5hUsCCSVAzcDHwH2Az4jab8mzZ4DqiPiQOAe4LuFqqcU9exWzqUnjuLpRSt5eN6KrMsxs06qkD2CccD8iFgYEVuA3wKn5zaIiIciovGXT08CgwtYT0k6+/ChDN61p3sFZtaiQgbBICD3GopL02ktOR/4a3MzJF0oaaakmTU1Ne1YYtfXraKMy08azUvLVjNl9ttZl2NmnVCnOFgs6XNANfC95uZHxK0RUR0R1VVVVR1bXBfwD4cMYkRVb66bOpf6BvcKzGxrhQyCZcCQnPHB6bStSBoP/DtwWkT4ZPoFUFFexlUTxjJv+Tr+9Pw2m8DMSlwhg+AZYLSk4ZK6kZzSenJuA0mHAD8hCYHlBayl5H3kgD3Zb69duPH+eWypa8i6HDPrRAoWBBFRB1wCTAFeBe6OiNmSrpF0Wtrse0Af4HeSnpc0uYXF2U4qKxOTTh7D4pUbuHvmkrbvYGYlo6KQC4+I+4D7mkz7Rs7w+EI+vm3thLG7c9g+u/KDB+fxqcMG06OyPOuSzKwT6BQHi61jSGLSxLG8s2Yzv3ryjazLMbNOwkFQYo4aOYBjRg/kR9MXsG5zXdblmFkn4CAoQVdNHMvK9Vu47dHXsy7FzDoBB0EJOnhIfybstwc/fXghqzZsybocM8uYg6BEXTVxDOu21HHLjIVZl2JmGXMQlKh999yF0w7am9sff53lazdlXY6ZZchBUMKuGD+G2vrg5gfnZ12KmWXIQVDChg3szaerB/Obpxez9L0Nbd/BzLokB0GJu/TE0Qhx0wPzsi7FzDLiIChxe/fvyeeO3Id7Zi1lQc26rMsxsww4CIyLTxhJj8pybpg2N+tSzCwDDgJjYJ/u/NOHhnPvi28x+83VWZdjZh3MQWAA/POxI9ilRwXXT3WvwKzUOAgMgH49K/nicSN54LXlzHrjvazLMbMO5CCwvzvv6GEM7NONa6fMyboUM+tADgL7u97dK7j4+FE8sfBdHpu/IutyzKyDOAhsK589Yih79+vBd6fMIcIXujcrBQ4C20qPynIuO2k0LyxZxf2v+jLSZqXAQWDbOOOwwQwb0Ivrps6hocG9ArOuzkFg26gsL+OKCWN47e21/PnFN7Mux8wKzEFgzfr4gXuz7559uWHaXGrrG7Iux8wKyEFgzSorE1dOGMOidzfwf7OWZl2OmRWQg8BaNGG/PThoSH9uemAem+vqsy7HzArEQWAtksRXJo7lzdWb+M1Ti7Mux8wKxEFgrfrQqAEcNWIANz80nw1b6rIux8wKwEFgrZLEpJPHsmLdFv73sUVZl2NmBeAgsDYdts+unLjv7vxkxgJWb6zNuhwza2cOAsvLVRPHsGZTHT99eGHWpZhZO3MQWF7237sfpx64F7c99jor1m3Ouhwza0cOAsvblRPGsKm2nh89tCDrUsysHTkILG8jq/pwxqGD+dVTb/DW6o1Zl2Nm7aSgQSDpFElzJM2X9LVm5h8r6VlJdZI+VcharH1cdtJoIoKbHpifdSlm1k4KFgSSyoGbgY8A+wGfkbRfk2aLgfOA3xSqDmtfQ3brxWfHDeXumUtYtGJ91uWYWTsoZI9gHDA/IhZGxBbgt8DpuQ0iYlFEvAj4rGZF5MsnjqKyXNx4vy90b9YVFDIIBgFLcsaXptO2m6QLJc2UNLOmpqZdirMdt3vfHpx39HD+9MKbzHl7bdblmNlOKoqDxRFxa0RUR0R1VVVV1uUYcNFxI+jTrYLrpvpC92bFrpBBsAwYkjM+OJ1mXUD/Xt244JgRTH3lHV5YsirrcsxsJxQyCJ4BRksaLqkbcDYwuYCPZx3s/GOGs1vvblzrXoFZUStYEEREHXAJMAV4Fbg7ImZLukbSaQCSDpe0FDgT+Imk2YWqx9pfn+4VfOm4kTwybwVPLHg363LMbAcporguTl5dXR0zZ87MugxLbaqt57jvPcTgXXtxz0VHISnrksysGZJmRUR1c/OK4mCxdV49Ksu59MTRzHrjPabP8Te6zIqRg8B22qerhzBkt55cO3UODQ3F1cM0MweBtYNuFWVcMX4Ms99cw19ffjvrcsxsOzkIrF2cfvAgRu/eh+unzaHevQKzouIgsHZRXiaunDCGBTXr+cNz/rmIWTFxEFi7OeWAPfngoH7ceP9cttT59FFmxcJBYO1GEldNHMPS9zZy1zOLsy7HzPLkILB2ddyYKsYN242bHpzPxi31WZdjZnlwEFi7ksSkk8dSs3YzdzyxKOtyzCwPDgJrd+OG78axY6r48YwFrN1Um3U5ZtYGB4EVxFcmjmXVhlp+9sjrWZdiZm1wEFhBfHBwP07Zf09+/ujrrFy/JetyzKwVDgIrmKsmjmH9ljpumbEg61LMrBUOAiuY0Xv05R8OHsQvHl/EO2s2ZV2OmbXAQWAF9S/jx1DfEPzwwflZl2JmLXAQWEENHdCLsw4fwp1PL2bJyg1Zl2NmzXAQWMFdeuJoysvEjffPy7oUM2uGg8AKbs9+PTj3yH34w3NLmb98bdblmFkTDgLrEF86fiQ9K8u5ftrcrEsxsyYcBNYhBvTpzvkfHs59L73Ny8tWZ12OmeVwEFiHueDYEfTrWcm1U+dkXYqZ5XAQWAGVS74AAArdSURBVIfZpUclFx03kulzapi5aGXW5ZhZykFgHerzR+/DwD7d+e6UOUT4kpZmnYGDwDpUr24VXHriKJ5+fSWPzFuRdTlmhoPAMnD2uCEM6t+Ta6e6V2DWGTgIrMN1ryjn8vGjeXHpaqbMfifrcsxKnoPAMvHJQwYxoqo310+bQ32DewVmWarIugArTRXlZVwxfgyX3vkcp970CH26V1BRLirLy6gsL6OiTFRWlFFZJirSaZXloqKsjMoKUVlWltM+nZ6OV5TnDJflLLNcOdNz26vJYybTKsqEpKyfKrOCcxBYZk794F68/OZqXn1rLXX1DdTWN7Cpto66hgbq6oMt9cnfuvoGttRHk+kNdERHYtuQaRJWjUHUSvhUlJXRrSL5W1Euum01PWlfUV5Gt3Q5FWVKp2/bfnvrcJBZPhwElpmyMvFvH/nADt+/viGorW+grqExLBqDIw2LNDhq6xuoTQOltiGorUvm1abz6uqD2oaGdHrkTE/a19U3abvV9HTZDQ3U1gXr6upyHrOxtvfD6/3aokN2iTWGRHO9p617WtuGV5mgTKK8TCgdbpymdLi87P3hxvlK79O0bZlEWdm2bZPHaKatSNtv3bbVx5UoK9u6bbJc0nnbLquscbismWWl99267q2Hld6vXM09TxRFGDsIrGiVl4nysvKsy9hhDQ1JADUXXltyQqeuISdYcoOtISeItgmftBeVjm+p23bZdQ0NbKnbuqe1YUvd38MwImiIoCGgIYJI/zZE0NCQMxykbZNw3qZtOj+Zl/Wz3vG2Co0mQVHWQmA2H75w+fgxnHbQ3u1eY0GDQNIpwPeBcuBnEfHtJvO7A3cAhwHvAmdFxKJC1mTWWZSVie5l5XQvsY9jkRMubYVK07aREza5bRvSoGktgLZnWU1ra1xufW7bhqbtc9qm8+qj+UDNJzC3np8ss3/PyoJsk4K9BCWVAzcDE4ClwDOSJkfEKznNzgfei4hRks4GvgOcVaiazCx7yW4XKKfz7zIpFYX8+ug4YH5ELIyILcBvgdObtDkd+EU6fA9wkophh5qZWRdSyCAYBCzJGV+aTmu2TUTUAauBAU0XJOlCSTMlzaypqSlQuWZmpakoflAWEbdGRHVEVFdVVWVdjplZl1LIIFgGDMkZH5xOa7aNpAqgH8lBYzMz6yCFDIJngNGShkvqBpwNTG7SZjLw+XT4U8CD4bOQmZl1qIJ9aygi6iRdAkwh+frobRExW9I1wMyImAz8HPilpPnASpKwMDOzDlTQbzBHxH3AfU2mfSNneBNwZiFrMDOz1hXFwWIzMyscFdsueUk1wBs7ePeBQFe5LJbXpfPpKusBXpfOamfWZZ+IaPZrl0UXBDtD0syIqM66jvbgdel8usp6gNelsyrUunjXkJlZiXMQmJmVuFILgluzLqAdeV06n66yHuB16awKsi4ldYzAzMy2VWo9AjMza8JBYGZW4rpkEEg6RdIcSfMlfa2Z+d0l3ZXOf0rSsI6vMj95rMt5kmokPZ/eLsiizrZIuk3SckkvtzBfkm5K1/NFSYd2dI35ymNdjpe0OmebfKO5dlmTNETSQ5JekTRb0uXNtCmK7ZLnuhTLdukh6WlJL6Tr8s1m2rTve1ikl1LrKjeS8xotAEYA3YAXgP2atLkYuCUdPhu4K+u6d2JdzgN+mHWteazLscChwMstzP8o8FdAwJHAU1nXvBPrcjxwb9Z15rEeewGHpsN9gbnNvL6KYrvkuS7Fsl0E9EmHK4GngCObtGnX97Cu2CPoSldGy2ddikJEPExyYsGWnA7cEYkngf6S9uqY6rZPHutSFCLirYh4Nh1eC7zKthePKortkue6FIX0uV6Xjlamt6bf6mnX97CuGATtdmW0TiCfdQE4I+223yNpSDPzi0G+61osjkq79n+VtH/WxbQl3bVwCMmnz1xFt11aWRcoku0iqVzS88ByYFpEtLhd2uM9rCsGQan5MzAsIg4EpvH+pwTLzrMk53U5CPgB8MeM62mVpD7A/wH/EhFrsq5nZ7SxLkWzXSKiPiIOJrmg1zhJBxTy8bpiEHSlK6O1uS4R8W5EbE5HfwYc1kG1tbd8tltRiIg1jV37SE7FXilpYMZlNUtSJckb568j4vfNNCma7dLWuhTTdmkUEauAh4BTmsxq1/ewrhgEXenKaG2uS5P9taeR7BstRpOBf0y/pXIksDoi3sq6qB0hac/G/bWSxpH8n3W6DxppjT8HXo2I61toVhTbJZ91KaLtUiWpfzrcE5gAvNakWbu+hxX0wjRZiC50ZbQ81+UySacBdSTrcl5mBbdC0p0k39oYKGkp8B8kB8GIiFtILmD0UWA+sAH4QjaVti2PdfkU8CVJdcBG4OxO+kHjQ8C5wEvp/miA/wcMhaLbLvmsS7Fsl72AX0gqJwmruyPi3kK+h/kUE2ZmJa4r7hoyM7Pt4CAwMytxDgIzsxLnIDAzK3EOAjOzEucgsE5LUki6Lmd8kqSr22nZt0v6VHssq43HOVPSq5IeajJ9mKSNOWfCfF7SP7bj4x4v6d72Wp51bV3udwTWpWwGPinpfyJiRdbFNJJUkZ7fJR/nA/8cEY82M29BehoBs0y5R2CdWR3JNVqvaDqj6Sd6SevSv8dLmiHpT5IWSvq2pHPS87u/JGlkzmLGS5opaa6kj6X3L5f0PUnPpCfy+2LOch+RNBl4pZl6PpMu/2VJ30mnfQP4MPBzSd/Ld6UlrZN0g5Jz0T8gqSqdfrCkJ9O6/iBp13T6KEn3pydTezZnHfsoORHha5J+nfOr2m8rOW//i5Kuzbcu68KyON+2b77lcwPWAbsAi0jOpTIJuDqddzvwqdy26d/jgVUkv87sTnJOlm+m8y4Hbsy5/99IPgyNJjmrZg/gQuDraZvuwExgeLrc9cDwZurcG1gMVJH0sh8EPpHOmw5UN3OfYSS/bn0+53ZMOi+Ac9Lhb5BebwJ4ETguHb4mZ12eAv4hHe4B9ErrXU1ybqAy4AmSUBoAzOH9H5P2z3o7+5b9zT0C69QiOYPkHcBl23G3ZyI5P/1mkgv7TE2nv0TyBtzo7ohoiIh5wEJgX2Aiybl1nid5gx1AEhQAT0fE68083uHA9IioiWSX0a9JLl7TlgURcXDO7ZF0egNwVzr8K+DDkvqRvGnPSKf/AjhWUl9gUET8ASAiNkXEhpx6l0ZEA0nQDCMJh00kvZRPkpw2wkqcg8CKwY0k+9p750yrI339SiojuYJbo805ww054w1sfVys6flVguTqUJfmvDkPj4jGIFm/U2ux43b0PDC5z0M90HhsYxzJxUw+RtIrshLnILBOLyJWAneThEGjRbx/yu3TSE/6tp3OlFSW7lMfQbLLZArJickqASSNkdS7tYUATwPHSRqYnijsM8CMNu7TmjKSE6QBfBZ4NCJWA+9JOiadfi4wI5KrcS2V9Im03u6SerW0YCXn6+8XyWmYrwAO2ok6rYvwt4asWFwHXJIz/lPgT5JeIPlUuyOf1heTvInvAlwUEZsk/YxkF8qz6cHVGuATrS0kIt6S9DWS88YL+EtE/CmPxx+Zc6ZMSM4uexPJuoyT9HWSK1Sdlc7/PHBL+ka/kPfPBHou8JP07JS1wJmtPGZfkuetR1rrlXnUaV2czz5q1slIWhcRfbKuw0qHdw2ZmZU49wjMzEqcewRmZiXOQWBmVuIcBGZmJc5BYGZW4hwEZmYl7v8DoxGxuy8RZjwAAAAASUVORK5CYII=\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"plt.xlabel('Number of Epochs')\n",
"plt.ylabel('Error')\n",
"plt.title('Plot showing results from Neural Network')\n",
"plt.plot(error)\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "_CPGHGIUqAc1"
},
"source": [
"#### Compearing the outputs and the predictions"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "P8k6xUoLqAc1",
"outputId": "69e2c004-e912-490d-985e-14f86ba5b80a"
},
"outputs": [
{
"data": {
"text/plain": [
"array([[0],\n",
" [1],\n",
" [1],\n",
" [0]])"
]
},
"execution_count": 74,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"outputs"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "WHl5K1IlqAc1",
"outputId": "42d32027-a0a3-4794-ff35-0971d77856fa"
},
"outputs": [
{
"data": {
"text/plain": [
"array([[0.0105802 ],\n",
" [0.9826343 ],\n",
" [0.98124727],\n",
" [0.02222762]])"
]
},
"execution_count": 75,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"output_layer"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "CEC0iCDfqAc1"
},
"source": [
"#### * We see that our neural network was able to get values close to the actual values from the results.\n",
"#### * This shows that our neural network can handle the complexity of the XOR operator dataset."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "QmVC2QavqAc1"
},
"source": [
"* Let us see the updated weights. These are the weights we will require if we want to make future predictions"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "Sjt5y3XgqAc1",
"outputId": "a29d52fe-c626-408e-beeb-d4014529b09f"
},
"outputs": [
{
"data": {
"text/plain": [
"array([[-0.04578885, -5.45125419, -1.01868082],\n",
" [ 1.04772434, -5.25578328, -0.5958262 ]])"
]
},
"execution_count": 77,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"weights_0"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "VN-RO2wsqAc1",
"outputId": "b27e8087-aa30-4952-cc42-3e7d9b26b425"
},
"outputs": [
{
"data": {
"text/plain": [
"array([[-14.93268248],\n",
" [-37.16040415],\n",
" [ 43.01681387]])"
]
},
"execution_count": 78,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"weights_1"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "g3QxkIBYqAc1"
},
"outputs": [],
"source": [
"# This function accepts an instance of a dataset\n",
"\n",
"def calculate_output(instance):\n",
" #input to hidden layer\n",
" hidden_layer = sigmoid(np.dot(instance, weights_0))\n",
" #hidden to output layer\n",
" output_layer = sigmoid(np.dot(hidden_layer, weights_1))\n",
" return output_layer[0]"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "0vTs9KUbqAc1",
"outputId": "9e9bd79d-5806-4ca8-a568-8801e437fc9c"
},
"outputs": [
{
"data": {
"text/plain": [
"0"
]
},
"execution_count": 95,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"round(calculate_output(np.array([0, 0])))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "pYxALdNOqAc1",
"outputId": "b78d1f10-bd23-40c6-93be-89d9c86b931f"
},
"outputs": [
{
"data": {
"text/plain": [
"1"
]
},
"execution_count": 96,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"round(calculate_output(np.array([0, 1])))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "yj9-9ifZqAc1",
"outputId": "44c89328-2891-4968-cf7e-cd2b30e367a3"
},
"outputs": [
{
"data": {
"text/plain": [
"1"
]
},
"execution_count": 97,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"round(calculate_output(np.array([1, 0])))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "A6n4cav9qAc1",
"outputId": "0fbef7fe-c267-4fce-8d67-7ab56d4a3141"
},
"outputs": [
{
"data": {
"text/plain": [
"0"
]
},
"execution_count": 98,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"round(calculate_output(np.array([1, 1])))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "y11IEZdKqAc1"
},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.6"
},
"colab": {
"provenance": [],
"include_colab_link": true
}
},
"nbformat": 4,
"nbformat_minor": 0
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment