diff --git "a/\133Master\135Fine_Tune_BERT_for_Text_Classification_with_TensorFlow.ipynb" "b/\133Master\135Fine_Tune_BERT_for_Text_Classification_with_TensorFlow.ipynb" new file mode 100644 index 0000000..36e3f4c --- /dev/null +++ "b/\133Master\135Fine_Tune_BERT_for_Text_Classification_with_TensorFlow.ipynb" @@ -0,0 +1,1375 @@ +{ + "nbformat": 4, + "nbformat_minor": 0, + "metadata": { + "colab": { + "name": "[Master]Fine-Tune-BERT-for-Text-Classification-with-TensorFlow.ipynb", + "provenance": [], + "collapsed_sections": [], + "machine_shape": "hm" + }, + "kernelspec": { + "name": "python3", + "display_name": "Python 3" + }, + "accelerator": "GPU" + }, + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "zGCJYkQj_Uu2" + }, + "source": [ + "

Fine-Tune BERT for Text Classification with TensorFlow

" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "4y2m1S6e12il" + }, + "source": [ + "
\n", + " \n", + "

Figure 1: BERT Classification Model

\n", + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "eYYYWqWr_WCC" + }, + "source": [ + "In this project, you will learn how to fine-tune a BERT model for text classification using TensorFlow and TF-Hub." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "5yQG5PCO_WFx" + }, + "source": [ + "The pretrained BERT model used in this project is [available](https://tfhub.dev/tensorflow/bert_en_uncased_L-12_H-768_A-12/2) on [TensorFlow Hub](https://tfhub.dev/)." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "7pKNS21u_WJo" + }, + "source": [ + "### Learning Objectives" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "_3NHSMXv_WMv" + }, + "source": [ + "By the time you complete this project, you will be able to:\n", + "\n", + "- Build TensorFlow Input Pipelines for Text Data with the [`tf.data`](https://www.tensorflow.org/api_docs/python/tf/data) API\n", + "- Tokenize and Preprocess Text for BERT\n", + "- Fine-tune BERT for text classification with TensorFlow 2 and [TF Hub](https://tfhub.dev)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "o6BEe-3-AVRQ" + }, + "source": [ + "### Prerequisites" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Sc9f-8rLAVUS" + }, + "source": [ + "In order to be successful with this project, it is assumed you are:\n", + "\n", + "- Competent in the Python programming language\n", + "- Familiar with deep learning for Natural Language Processing (NLP)\n", + "- Familiar with TensorFlow, and its Keras API" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "MYXXV5n3Ab-4" + }, + "source": [ + "### Contents" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "XhK-SYGyAjxe" + }, + "source": [ + "This project/notebook consists of several Tasks.\n", + "\n", + "- **[Task 1]()**: Introduction to the Project.\n", + "- **[Task 2]()**: Setup your TensorFlow and Colab Runtime\n", + "- **[Task 3]()**: Download and Import the Quora Insincere Questions Dataset\n", + "- **[Task 4]()**: Create tf.data.Datasets for Training and Evaluation\n", + "- **[Task 5]()**: Download a Pre-trained BERT Model from TensorFlow Hub\n", + "- **[Task 6]()**: Tokenize and Preprocess Text for BERT\n", + "- **[Task 7]()**: Wrap a Python Function into a TensorFlow op for Eager Execution\n", + "- **[Task 8]()**: Create a TensorFlow Input Pipeline with `tf.data`\n", + "- **[Task 9]()**: Add a Classification Head to the BERT `hub.KerasLayer`\n", + "- **[Task 10]()**: Fine-Tune BERT for Text Classification\n", + "- **[Task 11]()**: Evaluate the BERT Text Classification Model" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "IaArqXjRAcBa" + }, + "source": [ + "## Task 2: Setup your TensorFlow and Colab Runtime." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "GDDhjzZ5A4Q_" + }, + "source": [ + "You will only be able to use the Colab Notebook after you save it to your Google Drive folder. Click on the File menu and select “Save a copy in Drive…\n", + "\n", + "![Copy to Drive](https://drive.google.com/uc?id=1CH3eDmuJL8WR0AP1r3UE6sOPuqq8_Wl7)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "mpe6GhLuBJWB" + }, + "source": [ + "### Check GPU Availability\n", + "\n", + "Check if your Colab notebook is configured to use Graphical Processing Units (GPUs). If zero GPUs are available, check if the Colab notebook is configured to use GPUs (Menu > Runtime > Change Runtime Type).\n", + "\n", + "![Hardware Accelerator Settings](https://drive.google.com/uc?id=1qrihuuMtvzXJHiRV8M7RngbxFYipXKQx)\n" + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "8V9c8vzSL3aj" + }, + "source": [ + "!nvidia-smi" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Obch3rAuBVf0" + }, + "source": [ + "### Install TensorFlow and TensorFlow Model Garden" + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "bUQEY3dFB0jX", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 35 + }, + "outputId": "a4e2f5bd-d19b-4565-cf97-900f98767939" + }, + "source": [ + "import tensorflow as tf\n", + "print(tf.version.VERSION)" + ], + "execution_count": null, + "outputs": [ + { + "output_type": "stream", + "text": [ + "2.3.0\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "aU3YLZ1TYKUt" + }, + "source": [ + "!pip install -q tensorflow==2.3.0" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "AFRTC-zwUy6D" + }, + "source": [ + "!git clone --depth 1 -b v2.3.0 https://github.com/tensorflow/models.git" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "3H2G0571zLLs" + }, + "source": [ + "# install requirements to use tensorflow/models repository\n", + "!pip install -Uqr models/official/requirements.txt\n", + "# you may have to restart the runtime afterwards" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "GVjksk4yCXur" + }, + "source": [ + "## Restart the Runtime\n", + "\n", + "**Note** \n", + "After installing the required Python packages, you'll need to restart the Colab Runtime Engine (Menu > Runtime > Restart runtime...)\n", + "\n", + "![Restart of the Colab Runtime Engine](https://drive.google.com/uc?id=1xnjAy2sxIymKhydkqb0RKzgVK9rh3teH)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "IMsEoT3Fg4Wg" + }, + "source": [ + "## Task 3: Download and Import the Quora Insincere Questions Dataset" + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "GmqEylyFYTdP" + }, + "source": [ + "import numpy as np\n", + "import tensorflow as tf\n", + "import tensorflow_hub as hub\n", + "import sys\n", + "sys.path.append('models')\n", + "from official.nlp.data import classifier_data_lib\n", + "from official.nlp.bert import tokenization\n", + "from official.nlp import optimization" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "ZuX1lB8pPJ-W", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 90 + }, + "outputId": "65894a32-cb22-4201-b767-b2e5a49a5624" + }, + "source": [ + "print(\"TF Version: \", tf.__version__)\n", + "print(\"Eager mode: \", tf.executing_eagerly())\n", + "print(\"Hub version: \", hub.__version__)\n", + "print(\"GPU is\", \"available\" if tf.config.experimental.list_physical_devices(\"GPU\") else \"NOT AVAILABLE\")" + ], + "execution_count": null, + "outputs": [ + { + "output_type": "stream", + "text": [ + "TF Version: 2.3.0\n", + "Eager mode: True\n", + "Hub version: 0.9.0\n", + "GPU is available\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "QtbwpWgyEZg7" + }, + "source": [ + "A downloadable copy of the [Quora Insincere Questions Classification data](https://www.kaggle.com/c/quora-insincere-questions-classification/data) can be found [https://archive.org/download/fine-tune-bert-tensorflow-train.csv/train.csv.zip](https://archive.org/download/fine-tune-bert-tensorflow-train.csv/train.csv.zip). Decompress and read the data into a pandas DataFrame." + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "0nI-9itVwCCQ", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 35 + }, + "outputId": "137234df-54aa-4d93-963c-70cd78cdd1cd" + }, + "source": [ + "import numpy as np\n", + "import pandas as pd\n", + "from sklearn.model_selection import train_test_split\n", + "\n", + "df = pd.read_csv('https://archive.org/download/fine-tune-bert-tensorflow-train.csv/train.csv.zip',\n", + " compression='zip', low_memory=False)\n", + "df.shape" + ], + "execution_count": null, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "(1306122, 3)" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 3 + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "yeHE98KiMvDd", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 677 + }, + "outputId": "4b6cb84c-01d4-479e-9293-604ce2a6282d" + }, + "source": [ + "df.tail(20)" + ], + "execution_count": null, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
qidquestion_texttarget
1306102ffff3778790af9baae76What steps can I take to live a normal life if...0
1306103ffff3f0a2449ffe4b9ffIsn't Trump right after all? Why should the US...1
1306104ffff41393389d4206066Is 33 too late for a career in creative advert...0
1306105ffff42493fc203cd9532What is difference between the filteration wor...0
1306106ffff48dd47bee89fff79If the universe \"popped\" into existence from n...0
1306107ffff5fd051a032f32a39How does a shared service technology team meas...0
1306108ffff6d528040d3888b93How is DSATM civil engineering?0
1306109ffff8776cd30cdc8d7f8Do you know any problem that depends solely on...0
1306110ffff94d427ade3716cd1What are some comic ideas for you Tube videos ...0
1306111ffffa382c58368071dc9If you had $10 million of Bitcoin, could you s...0
1306112ffffa5b0fa76431c063fAre you ashamed of being an Indian?1
1306113ffffae5dbda3dc9e9771What are the methods to determine fossil ages ...0
1306114ffffba7c4888798571c1What is your story today?0
1306115ffffc0c7158658a06fd9How do I consume 150 gms protein daily both ve...0
1306116ffffc404da586ac5a08fWhat are the good career options for a msc che...0
1306117ffffcc4e2331aaf1e41eWhat other technical skills do you need as a c...0
1306118ffffd431801e5a2f4861Does MS in ECE have good job prospects in USA ...0
1306119ffffd48fb36b63db010cIs foam insulation toxic?0
1306120ffffec519fa37cf60c78How can one start a research project based on ...0
1306121ffffed09fedb5088744aWho wins in a battle between a Wolverine and a...0
\n", + "
" + ], + "text/plain": [ + " qid ... target\n", + "1306102 ffff3778790af9baae76 ... 0\n", + "1306103 ffff3f0a2449ffe4b9ff ... 1\n", + "1306104 ffff41393389d4206066 ... 0\n", + "1306105 ffff42493fc203cd9532 ... 0\n", + "1306106 ffff48dd47bee89fff79 ... 0\n", + "1306107 ffff5fd051a032f32a39 ... 0\n", + "1306108 ffff6d528040d3888b93 ... 0\n", + "1306109 ffff8776cd30cdc8d7f8 ... 0\n", + "1306110 ffff94d427ade3716cd1 ... 0\n", + "1306111 ffffa382c58368071dc9 ... 0\n", + "1306112 ffffa5b0fa76431c063f ... 1\n", + "1306113 ffffae5dbda3dc9e9771 ... 0\n", + "1306114 ffffba7c4888798571c1 ... 0\n", + "1306115 ffffc0c7158658a06fd9 ... 0\n", + "1306116 ffffc404da586ac5a08f ... 0\n", + "1306117 ffffcc4e2331aaf1e41e ... 0\n", + "1306118 ffffd431801e5a2f4861 ... 0\n", + "1306119 ffffd48fb36b63db010c ... 0\n", + "1306120 ffffec519fa37cf60c78 ... 0\n", + "1306121 ffffed09fedb5088744a ... 0\n", + "\n", + "[20 rows x 3 columns]" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 4 + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "leRFRWJMocVa", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 281 + }, + "outputId": "8de6f91c-57c6-44cb-c4c7-1199d1e820e0" + }, + "source": [ + "df.target.plot(kind='hist', title='Target distribution');" + ], + "execution_count": null, + "outputs": [ + { + "output_type": "display_data", + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEICAYAAABS0fM3AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAWdklEQVR4nO3dfbRddX3n8fdHEBF5qiaOmIDxIShR6UivoOPqiJW2gMvQ1pYhBa0WiWPFNRWxUGuB0RkfaqsztliMVlGsPE7LiiWIg6UyVYKEikiiaIoRAlQiIKigGPjOH2fHOXO5N/eEe/c5nLvfr7XOYj/8zt7fX27I5+7fb599UlVIkrrrMaMuQJI0WgaBJHWcQSBJHWcQSFLHGQSS1HEGgSR1nEEgDSjJa5P8c9/6j5I8Y46O/fYkH2uWlySpJDvP0bH3a2rdaS6Op/nHINDQNf8obXs9lOT+vvVjh1TDoUk2z+YYVbV7Vd00F+epqndX1etnU0/fOTclOazv2Dc3tT44F8fX/DMnv3FIO6Kqdt+2nGQT8PqqunxHjpFk56raOte1jcJ86ovGk1cEetRIcnCSq5L8IMntSf4qyS59+yvJm5J8G/h2s+2Pmra3JXl90+ZZzb7HJfnzJDcn+V6Ss5I8PskTgEuBp/ZdiTx1inqelGR1knuTfAV45qT9/ec6MsmGJD9McmuSk6c7T5IzklyU5NNJ7gVe22z79KQSfr/p1+1JTu4779lJ/lvf+s+vOpKcA+wHfLY53x9NHmpqalid5K4kG5Oc0HesM5JckORTTV/WJ5nY8Z+mxolBoEeTB4G3AAuAFwMvB/5gUpvfAA4BliU5HDgJOAx4FnDopLbvBfYH/n2zfxFwWlX9GDgCuK0ZMtm9qm6bop4zgZ8A+wC/37ym8zfAG6pqD+B5wD/OcJ6jgIuAvYG/neaYLwOWAr8GnNI/3DOdqno1cDPwyuZ8fzZFs/OAzcBTgd8G3p3kV/r2L2/a7A2sBv5qpvNqvI1lECT5eJI7ktwwYPujm9/W1if5TNv16ZGpqmuram1Vba2qTcBHgJdOavaeqrqrqu4HjgY+UVXrq+o+4IxtjZIEWAm8pWn/Q+DdwDGD1NJMrL6KJjiq6gbgk9t5y8/ohdOeVXV3Vf3LDKe4qqourqqHmr5M5b825/468AlgxSC1b0+SfYGXAKdU1U+q6jrgY8Br+pr9c1WtaeYUzgF+cbbn1aPbWAYBcDZw+CANkywF/hh4SVU9F/jD9srSbCTZP8k/JPm3Zsjk3fSuDvrd0rf81Enr/csLgd2Aa5uhph8An2u2D2IhvTm0/mN+dzvtXwUcCXw3yReTvHiG498yw/7Jbb5Lr7+z9VRgWzD2H3tR3/q/9S3fB+w6V3cw6dFpLIOgqq4E7urfluSZST6X5Nok/yfJc5pdJwBnVtXdzXvvGHK5GtxfA98EllbVnsDbgUxq0/+43NuBxX3r+/Ytfx+4H3huVe3dvPbqm6ie6bG7W4Ctk46533SNq+qaqjoKeDJwMXDBDOcZ5LG/k8+9bVjpx/RCbpun7MCxbwOemGSPSce+dYB6NE+NZRBMYxXw5qr6JeBk4MPN9v2B/ZN8KcnaZlxZj057APcCP2qC/I0ztL8AeF2SA5LsBvzpth1V9RDwUeCDSZ4MkGRRkl9vmnwPeFKSvaY6cDMs8nfAGUl2S7IM+L2p2ibZJcmxSfaqqp81fXhokPPM4E+bcz8XeB1wfrP9OuDIJE9M8hQefpX7PWDKzzdU1S3Al4H3JNk1yYHA8cDkiWp1yLwIgiS7A/8BuDDJdfTGlvdpdu9Mb8LtUHpjrB9Nsvfwq9QATgZ+F/ghvX/Ez99e46q6FPgQcAWwEVjb7Ppp899Ttm1vhpouB57dvPebwLnATc3Q0VTDLicCu9MbKjmb3jj9dF4NbGrO85+BY3fgPNP5YlP/F4A/r6rPN9vPAb4GbAI+z8P/nN4DvKM538k83ApgCb2rg78HTt/R23c1v2Rcv5gmyRLgH6rqeUn2BG6sqn2maHcWcHVVfaJZ/wJwalVdM9SC1bokBwA3AI/zvnxpcPPiiqCq7gW+k+R3oHfHSJJtdzpcTHNbYZIF9IaKtvtpUI2PJL+Z3ucFfgF4H/BZQ0DaMWMZBEnOBa4Cnp1kc5Lj6V2KH5/ka8B6evdpA1wG3JlkA70hhLdV1Z2jqFuteANwB/Cv9D6HMNO8gqRJxnZoSJI0N8byikCSNHfG7kMiCxYsqCVLloy6DEkaK9dee+33q2rKD1SOXRAsWbKEdevWjboMSRorSab9ZLxDQ5LUcQaBJHWcQSBJHWcQSFLHGQSS1HEGgSR1nEEgSR1nEEhSxxkEktRxY/fJ4tlYcuolIzv3pve+YmTnlqTtae2KIMnHk9yR5IZp9h+b5PokX0/y5b7vD5AkDVGbQ0NnA9v7fuDvAC+tqucD76L3ncOSpCFrbWioqq5svk5yuv1f7ltdCyxuqxZJ0vQeLZPFxwOXTrczycok65Ks27JlyxDLkqT5b+RBkORl9ILglOnaVNWqqpqoqomFC6d8nLYk6REa6V1DSQ4EPgYc4fcIS9JojOyKIMl+wN8Br66qb42qDknqutauCJKcCxwKLEiyGTgdeCxAVZ0FnAY8CfhwEoCtVTXRVj2SpKm1edfQihn2vx54fVvnlyQNZuSTxZKk0TIIJKnjDAJJ6jiDQJI6ziCQpI4zCCSp4wwCSeo4g0CSOs4gkKSOMwgkqeMMAknqOINAkjrOIJCkjjMIJKnjDAJJ6jiDQJI6ziCQpI4zCCSp4wwCSeo4g0CSOs4gkKSOMwgkqeMMAknqOINAkjrOIJCkjmstCJJ8PMkdSW6YZn+SfCjJxiTXJzmorVokSdNr84rgbODw7ew/AljavFYCf91iLZKkabQWBFV1JXDXdpocBXyqetYCeyfZp616JElTG+UcwSLglr71zc22h0myMsm6JOu2bNkylOIkqSvGYrK4qlZV1URVTSxcuHDU5UjSvDLKILgV2LdvfXGzTZI0RKMMgtXAa5q7h14E3FNVt4+wHknqpJ3bOnCSc4FDgQVJNgOnA48FqKqzgDXAkcBG4D7gdW3VIkmaXmtBUFUrZthfwJvaOr8kaTBjMVksSWqPQSBJHWcQSFLHGQSS1HEGgSR1nEEgSR1nEEhSxxkEktRxBoEkdZxBIEkdZxBIUscZBJLUcQaBJHWcQSBJHWcQSFLHGQSS1HEGgSR1nEEgSR1nEEhSxxkEktRxBoEkdZxBIEkdZxBIUscZBJLUca0GQZLDk9yYZGOSU6fYv1+SK5J8Ncn1SY5ssx5J0sO1FgRJdgLOBI4AlgErkiyb1OwdwAVV9QLgGODDbdUjSZpam1cEBwMbq+qmqnoAOA84alKbAvZslvcCbmuxHknSFHZu8diLgFv61jcDh0xqcwbw+SRvBp4AHNZiPZKkKYx6sngFcHZVLQaOBM5J8rCakqxMsi7Jui1btgy9SEmaz9oMgluBffvWFzfb+h0PXABQVVcBuwILJh+oqlZV1URVTSxcuLClciWpm9oMgmuApUmenmQXepPBqye1uRl4OUCSA+gFgb/yS9IQDRQESZ6/oweuqq3AicBlwDfo3R20Psk7kyxvmr0VOCHJ14BzgddWVe3ouSRJj9ygk8UfTvI44Gzgb6vqnkHeVFVrgDWTtp3Wt7wBeMmANUiSWjDQFUFV/TJwLL0x/2uTfCbJr7ZamSRpKAaeI6iqb9P7ANgpwEuBDyX5ZpLfaqs4SVL7Bp0jODDJB+mN9f8K8MqqOqBZ/mCL9UmSWjboHMFfAh8D3l5V92/bWFW3JXlHK5VJkoZi0CB4BXB/VT0I0Hzoa9equq+qzmmtOklS6wadI7gceHzf+m7NNknSmBs0CHatqh9tW2mWd2unJEnSMA0aBD9OctC2lSS/BNy/nfaSpDEx6BzBHwIXJrkNCPAU4D+1VZQkaXgGCoKquibJc4BnN5turKqftVeWJGlYduT7CF4ILGnec1ASqupTrVQlSRqagYIgyTnAM4HrgAebzQUYBJI05ga9IpgAlvlkUEmafwa9a+gGehPEkqR5ZtArggXAhiRfAX66bWNVLZ/+LZKkcTBoEJzRZhGSpNEZ9PbRLyZ5GrC0qi5PshuwU7ulSZKGYdDHUJ8AXAR8pNm0CLi4pZokSUM06GTxm+h9peS98PMvqXlyW0VJkoZn0CD4aVU9sG0lyc70PkcgSRpzgwbBF5O8HXh8813FFwKfba8sSdKwDBoEpwJbgK8DbwDW0Pv+YknSmBv0rqGHgI82L0nSPDLos4a+wxRzAlX1jDmvSJI0VDvyrKFtdgV+B3ji3JcjSRq2geYIqurOvtetVfU/6H2h/XYlOTzJjUk2Jjl1mjZHJ9mQZH2Sz+xY+ZKk2Rp0aOigvtXH0LtC2O57k+wEnAn8KrAZuCbJ6qra0NdmKfDHwEuq6u4kfjZBkoZs0KGhv+hb3gpsAo6e4T0HAxur6iaAJOcBRwEb+tqcAJxZVXcDVNUdA9YjSZojg9419LJHcOxFwC1965uBQya12R8gyZfoPbvojKr63OQDJVkJrATYb7/9HkEpkqTpDDo0dNL29lfVB2Zx/qXAocBi4Mokz6+qH0w6/ipgFcDExISfaJakObQjdw29EFjdrL8S+Arw7e2851Zg3771xc22fpuBq6vqZ8B3knyLXjBcM2BdkqRZGjQIFgMHVdUPAZKcAVxSVcdt5z3XAEuTPJ1eABwD/O6kNhcDK4BPJFlAb6jopoGrlyTN2qCPmPh3wAN96w8026ZVVVuBE4HLgG8AF1TV+iTvTLLtm80uA+5MsgG4AnhbVd25Ix2QJM3OoFcEnwK+kuTvm/XfAD4505uqag295xL1bzutb7mAk5qXJGkEBr1r6L8nuRT45WbT66rqq+2VJUkalkGHhgB2A+6tqv8JbG7G/iVJY27Qr6o8HTiF3qeAAR4LfLqtoiRJwzPoFcFvAsuBHwNU1W3AHm0VJUkankGD4IFmYrcAkjyhvZIkScM0aBBckOQjwN5JTgAuxy+pkaR5Yca7hpIEOB94DnAv8GzgtKr63y3XJkkaghmDoKoqyZqqej7gP/6SNM8MOjT0L0le2GolkqSRGPSTxYcAxyXZRO/OodC7WDiwrcIkScMx07eM7VdVNwO/PqR6JElDNtMVwcX0njr63ST/q6peNYSaJElDNNMcQfqWn9FmIZKk0ZgpCGqaZUnSPDHT0NAvJrmX3pXB45tl+H+TxXu2Wp0kqXXbDYKq2mlYhUiSRmNHHkMtSZqHDAJJ6jiDQJI6ziCQpI4zCCSp4wwCSeo4g0CSOs4gkKSOMwgkqeNaDYIkhye5McnGJKdup92rklSSiTbrkSQ9XGtBkGQn4EzgCGAZsCLJsina7QH8F+DqtmqRJE2vzSuCg4GNVXVTVT0AnAccNUW7dwHvA37SYi2SpGm0GQSLgFv61jc3234uyUHAvlV1yfYOlGRlknVJ1m3ZsmXuK5WkDhvZZHGSxwAfAN46U9uqWlVVE1U1sXDhwvaLk6QOaTMIbgX27Vtf3GzbZg/gecA/JdkEvAhY7YSxJA1Xm0FwDbA0ydOT7AIcA6zetrOq7qmqBVW1pKqWAGuB5VW1rsWaJEmTtBYEVbUVOBG4DPgGcEFVrU/yziTL2zqvJGnHzPRVlbNSVWuANZO2nTZN20PbrEWSNDU/WSxJHWcQSFLHGQSS1HEGgSR1nEEgSR1nEEhSxxkEktRxBoEkdZxBIEkdZxBIUscZBJLUcQaBJHWcQSBJHWcQSFLHGQSS1HEGgSR1nEEgSR1nEEhSxxkEktRxBoEkdZxBIEkdZxBIUscZBJLUcQaBJHWcQSBJHddqECQ5PMmNSTYmOXWK/Scl2ZDk+iRfSPK0NuuRJD1ca0GQZCfgTOAIYBmwIsmySc2+CkxU1YHARcCftVWPJGlqbV4RHAxsrKqbquoB4DzgqP4GVXVFVd3XrK4FFrdYjyRpCm0GwSLglr71zc226RwPXDrVjiQrk6xLsm7Lli1zWKIk6VExWZzkOGACeP9U+6tqVVVNVNXEwoULh1ucJM1zO7d47FuBffvWFzfb/j9JDgP+BHhpVf20xXokSVNo84rgGmBpkqcn2QU4Bljd3yDJC4CPAMur6o4Wa5EkTaO1IKiqrcCJwGXAN4ALqmp9kncmWd40ez+wO3BhkuuSrJ7mcJKklrQ5NERVrQHWTNp2Wt/yYW2eX5I0s0fFZLEkaXQMAknqOINAkjrOIJCkjjMIJKnjDAJJ6jiDQJI6ziCQpI4zCCSp4wwCSeo4g0CSOs4gkKSOMwgkqeMMAknquFYfQy1J882SUy8Z2bk3vfcVrRzXKwJJ6jiDQJI6ziCQpI4zCCSp4wwCSeo4g0CSOs4gkKSOMwgkqeMMAknqOINAkjqu1SBIcniSG5NsTHLqFPsfl+T8Zv/VSZa0WY8k6eFaC4IkOwFnAkcAy4AVSZZNanY8cHdVPQv4IPC+tuqRJE2tzSuCg4GNVXVTVT0AnAccNanNUcAnm+WLgJcnSYs1SZImafPpo4uAW/rWNwOHTNemqrYmuQd4EvD9/kZJVgIrm9UfJbnxEda0YPKxhyWju9YZWZ9HyD53Q+f6nPfNqs9Pm27HWDyGuqpWAatme5wk66pqYg5KGhv2uRvscze01ec2h4ZuBfbtW1/cbJuyTZKdgb2AO1usSZI0SZtBcA2wNMnTk+wCHAOsntRmNfB7zfJvA/9YVdViTZKkSVobGmrG/E8ELgN2Aj5eVeuTvBNYV1Wrgb8BzkmyEbiLXli0adbDS2PIPneDfe6GVvocfwGXpG7zk8WS1HEGgSR13LwMgi4+2mKAPp+UZEOS65N8Icm09xSPi5n63NfuVUkqydjfajhIn5Mc3fys1yf5zLBrnGsD/N3eL8kVSb7a/P0+chR1zpUkH09yR5IbptmfJB9q/jyuT3LQrE9aVfPqRW9i+l+BZwC7AF8Dlk1q8wfAWc3yMcD5o657CH1+GbBbs/zGLvS5abcHcCWwFpgYdd1D+DkvBb4K/EKz/uRR1z2EPq8C3tgsLwM2jbruWfb5PwIHATdMs/9I4FIgwIuAq2d7zvl4RdDFR1vM2OequqKq7mtW19L7XMc4G+TnDPAues+w+skwi2vJIH0+ATizqu4GqKo7hlzjXBukzwXs2SzvBdw2xPrmXFVdSe8uyukcBXyqetYCeyfZZzbnnI9BMNWjLRZN16aqtgLbHm0xrgbpc7/j6f1GMc5m7HNzybxvVV0yzMJaNMjPeX9g/yRfSrI2yeFDq64dg/T5DOC4JJuBNcCbh1PayOzo/+8zGotHTGjuJDkOmABeOupa2pTkMcAHgNeOuJRh25ne8NCh9K76rkzy/Kr6wSiLatkK4Oyq+oskL6b32aTnVdVDoy5sXMzHK4IuPtpikD6T5DDgT4DlVfXTIdXWlpn6vAfwPOCfkmyiN5a6eswnjAf5OW8GVlfVz6rqO8C36AXDuBqkz8cDFwBU1VXArvQeSDdfDfT/+46Yj0HQxUdbzNjnJC8APkIvBMZ93Bhm6HNV3VNVC6pqSVUtoTcvsryq1o2m3DkxyN/ti+ldDZBkAb2hopuGWONcG6TPNwMvB0hyAL0g2DLUKodrNfCa5u6hFwH3VNXtszngvBsaqkfnoy1aNWCf3w/sDlzYzIvfXFXLR1b0LA3Y53llwD5fBvxakg3Ag8Dbqmpsr3YH7PNbgY8meQu9iePXjvMvdknOpRfmC5p5j9OBxwJU1Vn05kGOBDYC9wGvm/U5x/jPS5I0B+bj0JAkaQcYBJLUcQaBJHWcQSBJHWcQSFLHGQSS1HEGgSR13P8FdJXGehzQddcAAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "tags": [], + "needs_background": "light" + } + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "ELjswHcFHfp3" + }, + "source": [ + "## Task 4: Create tf.data.Datasets for Training and Evaluation" + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "fScULIGPwuWk", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 35 + }, + "outputId": "45769c51-82d2-4347-ece7-738af314777e" + }, + "source": [ + "train_df, remaining = train_test_split(df, random_state=42, train_size=0.0075, stratify=df.target.values)\n", + "valid_df, _ = train_test_split(remaining, random_state=42, train_size=0.00075, stratify=remaining.target.values)\n", + "train_df.shape, valid_df.shape" + ], + "execution_count": null, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "((9795, 3), (972, 3))" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 6 + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "qQYMGT5_qLPX", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 53 + }, + "outputId": "65e133ba-89fe-41b0-f666-db96b50c2f68" + }, + "source": [ + "with tf.device('/cpu:0'):\n", + " train_data = tf.data.Dataset.from_tensor_slices((train_df.question_text.values, train_df.target.values))\n", + " valid_data = tf.data.Dataset.from_tensor_slices((valid_df.question_text.values, valid_df.target.values))\n", + "\n", + " for text, label in train_data.take(1):\n", + " print(text)\n", + " print(label)" + ], + "execution_count": null, + "outputs": [ + { + "output_type": "stream", + "text": [ + "tf.Tensor(b'Why are unhealthy relationships so desirable?', shape=(), dtype=string)\n", + "tf.Tensor(0, shape=(), dtype=int64)\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "e2-ReN88Hvy_" + }, + "source": [ + "## Task 5: Download a Pre-trained BERT Model from TensorFlow Hub" + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "EMb5M86b4-BU" + }, + "source": [ + "\"\"\"\n", + "Each line of the dataset is composed of the review text and its label\n", + "- Data preprocessing consists of transforming text to BERT input features:\n", + "input_word_ids, input_mask, segment_ids\n", + "- In the process, tokenizing the text is done with the provided BERT model tokenizer\n", + "\"\"\"\n", + "\n", + "label_list = [0, 1] # Label categories\n", + "max_seq_length = 128 # maximum length of (token) input sequences\n", + "train_batch_size = 32\n", + "\n", + "# Get BERT layer and tokenizer:\n", + "# More details here: https://tfhub.dev/tensorflow/bert_en_uncased_L-12_H-768_A-12/2\n", + "bert_layer = hub.KerasLayer(\"https://tfhub.dev/tensorflow/bert_en_uncased_L-12_H-768_A-12/2\",\n", + " trainable=True)\n", + "vocab_file = bert_layer.resolved_object.vocab_file.asset_path.numpy()\n", + "do_lower_case = bert_layer.resolved_object.do_lower_case.numpy()\n", + "tokenizer = tokenization.FullTokenizer(vocab_file, do_lower_case)" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "wEUezMK-zkkI", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 35 + }, + "outputId": "2deee318-ff99-4a1c-f285-7cf3258f8a93" + }, + "source": [ + "tokenizer.wordpiece_tokenizer.tokenize('hi, how are you doing?')" + ], + "execution_count": null, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "['hi', '##,', 'how', 'are', 'you', 'doing', '##?']" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 9 + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "5AFsmTO5JSmc", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 35 + }, + "outputId": "15409374-b640-4dc9-a36a-f38aac5e2b29" + }, + "source": [ + "tokenizer.convert_tokens_to_ids(tokenizer.wordpiece_tokenizer.tokenize('hi, how are you doing?'))" + ], + "execution_count": null, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "[7632, 29623, 2129, 2024, 2017, 2725, 29632]" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 10 + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "9QinzNq6OsP1" + }, + "source": [ + "## Task 6: Tokenize and Preprocess Text for BERT" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "3FTqJ698zZ1e" + }, + "source": [ + "
\n", + " \n", + "

Figure 2: BERT Tokenizer

\n", + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "cWYkggYe6HZc" + }, + "source": [ + "We'll need to transform our data into a format BERT understands. This involves two steps. First, we create InputExamples using `classifier_data_lib`'s constructor `InputExample` provided in the BERT library." + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "m-21A5aNJM0W" + }, + "source": [ + "# This provides a function to convert row to input features and label\n", + "\n", + "def to_feature(text, label, label_list=label_list, max_seq_length=max_seq_length, tokenizer=tokenizer):\n", + " example = classifier_data_lib.InputExample(guid = None,\n", + " text_a = text.numpy(), \n", + " text_b = None, \n", + " label = label.numpy())\n", + " feature = classifier_data_lib.convert_single_example(0, example, label_list,\n", + " max_seq_length, tokenizer)\n", + " \n", + " return (feature.input_ids, feature.input_mask, feature.segment_ids, feature.label_id)" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "A_HQSsHwWCsK" + }, + "source": [ + "You want to use [`Dataset.map`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset#map) to apply this function to each element of the dataset. [`Dataset.map`](https://www.tensorflow.org/api_docs/python/tf/data/Dataset#map) runs in graph mode.\n", + "\n", + "- Graph tensors do not have a value.\n", + "- In graph mode you can only use TensorFlow Ops and functions.\n", + "\n", + "So you can't `.map` this function directly: You need to wrap it in a [`tf.py_function`](https://www.tensorflow.org/api_docs/python/tf/py_function). The [`tf.py_function`](https://www.tensorflow.org/api_docs/python/tf/py_function) will pass regular tensors (with a value and a `.numpy()` method to access it), to the wrapped python function." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "zaNlkKVfWX0Q" + }, + "source": [ + "## Task 7: Wrap a Python Function into a TensorFlow op for Eager Execution" + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "AGACBcfCWC2O" + }, + "source": [ + "def to_feature_map(text, label):\n", + " input_ids, input_mask, segment_ids, label_id = tf.py_function(to_feature, inp=[text, label], \n", + " Tout=[tf.int32, tf.int32, tf.int32, tf.int32])\n", + "\n", + " # py_func doesn't set the shape of the returned tensors.\n", + " input_ids.set_shape([max_seq_length])\n", + " input_mask.set_shape([max_seq_length])\n", + " segment_ids.set_shape([max_seq_length])\n", + " label_id.set_shape([])\n", + "\n", + " x = {\n", + " 'input_word_ids': input_ids,\n", + " 'input_mask': input_mask,\n", + " 'input_type_ids': segment_ids\n", + " }\n", + " return (x, label_id)" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "dhdO6MjTbtn1" + }, + "source": [ + "## Task 8: Create a TensorFlow Input Pipeline with `tf.data`" + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "LHRdiO3dnPNr" + }, + "source": [ + "with tf.device('/cpu:0'):\n", + " # train\n", + " train_data = (train_data.map(to_feature_map,\n", + " num_parallel_calls=tf.data.experimental.AUTOTUNE)\n", + " #.cache()\n", + " .shuffle(1000)\n", + " .batch(32, drop_remainder=True)\n", + " .prefetch(tf.data.experimental.AUTOTUNE))\n", + "\n", + " # valid\n", + " valid_data = (valid_data.map(to_feature_map,\n", + " num_parallel_calls=tf.data.experimental.AUTOTUNE)\n", + " .batch(32, drop_remainder=True)\n", + " .prefetch(tf.data.experimental.AUTOTUNE)) " + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "KLUWnfx-YDi2" + }, + "source": [ + "The resulting `tf.data.Datasets` return `(features, labels)` pairs, as expected by [`keras.Model.fit`](https://www.tensorflow.org/api_docs/python/tf/keras/Model#fit):" + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "B0Z2cy9GHQ8x", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 90 + }, + "outputId": "0cb5b236-fb1f-48ac-a23e-81c9c2b228f6" + }, + "source": [ + "# data spec\n", + "train_data.element_spec" + ], + "execution_count": null, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "({'input_mask': TensorSpec(shape=(32, 128), dtype=tf.int32, name=None),\n", + " 'input_type_ids': TensorSpec(shape=(32, 128), dtype=tf.int32, name=None),\n", + " 'input_word_ids': TensorSpec(shape=(32, 128), dtype=tf.int32, name=None)},\n", + " TensorSpec(shape=(32,), dtype=tf.int32, name=None))" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 14 + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "DGAH-ycYOmao", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 90 + }, + "outputId": "4030e117-9959-4ef2-be4b-125ac7d2882a" + }, + "source": [ + "# data spec\n", + "valid_data.element_spec" + ], + "execution_count": null, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "({'input_mask': TensorSpec(shape=(32, 128), dtype=tf.int32, name=None),\n", + " 'input_type_ids': TensorSpec(shape=(32, 128), dtype=tf.int32, name=None),\n", + " 'input_word_ids': TensorSpec(shape=(32, 128), dtype=tf.int32, name=None)},\n", + " TensorSpec(shape=(32,), dtype=tf.int32, name=None))" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 15 + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "GZxe-7yhPyQe" + }, + "source": [ + "## Task 9: Add a Classification Head to the BERT Layer" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "9THH5V0Dw2HO" + }, + "source": [ + "
\n", + " \n", + "

Figure 3: BERT Layer

\n", + "
" + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "G9il4gtlADcp" + }, + "source": [ + "# Building the model\n", + "def create_model():\n", + " input_word_ids = tf.keras.layers.Input(shape=(max_seq_length,), dtype=tf.int32,\n", + " name=\"input_word_ids\")\n", + " input_mask = tf.keras.layers.Input(shape=(max_seq_length,), dtype=tf.int32,\n", + " name=\"input_mask\")\n", + " input_type_ids = tf.keras.layers.Input(shape=(max_seq_length,), dtype=tf.int32,\n", + " name=\"input_type_ids\")\n", + "\n", + " pooled_output, sequence_output = bert_layer([input_word_ids, input_mask, input_type_ids])\n", + "\n", + " drop = tf.keras.layers.Dropout(0.4)(pooled_output)\n", + " output = tf.keras.layers.Dense(1, activation=\"sigmoid\", name=\"output\")(drop)\n", + "\n", + " model = tf.keras.Model(\n", + " inputs={\n", + " 'input_word_ids': input_word_ids,\n", + " 'input_mask': input_mask,\n", + " 'input_type_ids': input_type_ids\n", + " },\n", + " outputs=output)\n", + " return model" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "S6maM-vr7YaJ" + }, + "source": [ + "## Task 10: Fine-Tune BERT for Text Classification" + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "ptCtiiONsBgo", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 417 + }, + "outputId": "e16ceba2-bea2-416f-fc8f-165cf9393cc7" + }, + "source": [ + "model = create_model()\n", + "model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=2e-5),\n", + " loss=tf.keras.losses.BinaryCrossentropy(),\n", + " metrics=[tf.keras.metrics.BinaryAccuracy()])\n", + "model.summary()" + ], + "execution_count": null, + "outputs": [ + { + "output_type": "stream", + "text": [ + "Model: \"functional_1\"\n", + "__________________________________________________________________________________________________\n", + "Layer (type) Output Shape Param # Connected to \n", + "==================================================================================================\n", + "input_word_ids (InputLayer) [(None, 128)] 0 \n", + "__________________________________________________________________________________________________\n", + "input_mask (InputLayer) [(None, 128)] 0 \n", + "__________________________________________________________________________________________________\n", + "input_type_ids (InputLayer) [(None, 128)] 0 \n", + "__________________________________________________________________________________________________\n", + "keras_layer (KerasLayer) [(None, 768), (None, 109482241 input_word_ids[0][0] \n", + " input_mask[0][0] \n", + " input_type_ids[0][0] \n", + "__________________________________________________________________________________________________\n", + "dropout (Dropout) (None, 768) 0 keras_layer[0][0] \n", + "__________________________________________________________________________________________________\n", + "output (Dense) (None, 1) 769 dropout[0][0] \n", + "==================================================================================================\n", + "Total params: 109,483,010\n", + "Trainable params: 109,483,009\n", + "Non-trainable params: 1\n", + "__________________________________________________________________________________________________\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "6GJaFnkbMtPL", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 338 + }, + "outputId": "3ae08dbc-ccca-403f-bda8-8cf83d19825c" + }, + "source": [ + "tf.keras.utils.plot_model(model=model, show_shapes=True, dpi=76, )" + ], + "execution_count": null, + "outputs": [ + { + "output_type": "execute_result", + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAA+AAAAFBCAYAAAD3+MUfAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nOzdeVhT57o//G+YCRGRUUwUqlY9aHXjiFZFAae2WJzqKaWKFRzQo6Xaglr27rFbrFRgo2J7RCgqWltBN+iWWnAo1aK1agcQrMNW0R8IDsxhSHjeP3xZm8iUhJAVyP25Lq6WNTzrfp64btadNQkYYwyEEEIIIYQQQgjpTGkGfEdACCGEEEIIIYToAyrACSGEEEIIIYQQLaACnBBCCCGEEEII0QKjpr+cP38eH374IV+xEKJ19fX1qK+vh1Ao5DsUrWloaEBVVRV69OjBdyhdgkQiwZEjRzrczp9//onFixdrICJCOq6mpgYCgQCmpqZ8h6I1+pjvO2LWrFn461//qpG2fHx88OjRI420RYgukcvlkEqlEIlEfIeiVeXl5ejRowcEAgHfoXQJ2dnZCr8rFOBPnz5FdXU1IiIitBoUIXw5c+YMsrKy8Mknn/AditYUFhZi7dq1+Pbbb/kORefdunUL0dHRGmmrsrIS9+/fR0JCgkbaI6Qjdu3aBTs7OyxcuJDvULRGH/O9uo4fP46bN29qrL0rV64gLCwMTk5OGmuTEF2Ql5eH6OhobN++ne9QtGrmzJk4evQofaHZjurqasydO7fZdKMXJ/Tq1QszZszQSlCE8K2kpAR5eXl69W/+9u3bMDEx0as+q8vOzk5jBTgAmJub07gTnZCWlgaJRKJX/x71Md+r69atW7h48aJG23z11VcxdOhQjbZJCN8sLS0hEon0Mq94enrC0tKS7zB0Wnl5eYvT6R5wQgghhBBCCCFEC6gAJ4QQQgghhBBCtIAKcEIIIYQQQgghRAuoACeEEEIIIYQQQrSACnBC1LRkyRIEBQXxHUankkgkEAgEEAgEzZ7wGRcXh/DwcAwaNAjm5uawtbXF7NmzcffuXZW28ejRI6xYsQKrV6/mpjU0NGDkyJGwsrKCUCjE8OHDkZSUpLBednY2xo8fD5FIBLFYjODgYMjlcgBAamoqDh8+rLD89u3bub44OzurFCMh5D/0OffFxcXh6NGjyM/P71DuaynvAZT7CNEn+pxLgc49jgQ6lk87O5dSAU6ImhhjYIxpbXt79+5FZmam1rbXKCYmBgUFBQgODuamvf/++6ivr4eHhwd8fX3x9OlT5Obmor6+HosWLVK67ePHjyMyMhLXrl2DTCbjpjc0NMDJyQm3b9/G48ePsWbNGixatAiXL1/m5vv4+GDChAkoKirCiRMnsH//fuzZswcA8Oabb+Lp06dYt24d12ZwcDAKCgoQExPT0SEhRK9pM/fxlfeA5rmvMe/NnTsXpaWlaue+1vIeQLmPEH1Cx5GddxwJdCyfdnYupQKcEDUlJibiiy++0Nr2Dhw4oLVtNTVo0CBIJBIYGhoCAL766itkZ2cjKCgIbm5u+OSTT2Bubg4HBwe8++67yM7O5s7GtMfb2xsRERHo27evwnQjIyMcO3YMNjY2EAqFCAgIQI8ePXDhwgUAwOPHj1FcXIyFCxdCJBLB1dUVY8aMQU5ODtdGUFAQfvzxRyQmJgIADA0NIZFIMGjQIA2MCiH6S5u5j6+8ByjmvqZ5D0CHcl9reQ+g3EeIPqHjyM47jgQ6nk87M5dSAU6IGg4dOgQLCwu88cYbAICwsDAYGBggICAAU6dOhVAohJubG549ewYACAkJgUAgwPTp0+Hg4ABra2sEBASguroaAODn5weBQIATJ04AAGbPng0DAwNup/fz80NWVhamTZuGIUOGIDAwEF5eXtrvOIDIyEgEBga2OK+yshJmZmZcktWUhoYGVFVVwd7eHsDz93OPGzcOqampkEqlyMvLw9WrVzFz5kyF9QICAhAZGanRWAjRZ6rkPk3nPQC85b628h5AuY8Qohq+jyMB3cynnZVLAfXyaWflUirACVGDr6+vwqU0n376KVxcXODq6oqMjAzk5eXh5s2bSE1NBQBs27YNYrEY3t7euH//Pk6fPo309HREREQAAJKSkmBjY8O1l5aWpvANW1JSEiwsLJCRkYH8/HzI5XI0NDRoqbf/kZ+fj9zcXIwaNUphem1tLS5evIidO3fC19dX49vNysqCpaUlvL29AQACgQAHDx7E7t27IRQK4eLiAh8fH25+I1dXV+Tk5ODGjRsaj4kQfaRK7tN03gPAS+5rLe8BlPsIIerh+zgS0K182tm5FFAvn3ZWLqUCnBANMjU1hZGREZycnPDyyy+jtLRUYb6joyNMTU3h6uoKf39/HD9+XK3tJCQk4MyZM5oIWSW3b98G8Pxbw6bCw8Mxa9YseHh4aPw+Q5lMhk2bNiE+Ph49evQA8PwbUk9PT4SHh6OiogI5OTnIysrCl19+qbBuY5yNcRNCOkdbuU9TeQ/gJ/e1lvcAyn2EEM3S1nEkoFv5tDNzKaB+Pu2sXEoFOCE8sbe3R3FxMd9hqKTxUidTU1OF6Y6OjkhOTkZMTAzMzMw0tr2GhgYEBAQgICAAc+bM4aafPHkSpaWlWLlyJUQiEYYOHYrAwEDs2rVLYX2hUKgQNyGEX90p7wGU+wgh/OlO+bSzcinQsXzaWbnUSKOtEUKUdu/ePYjFYr7DUImFhQUAoK6uTmH6ihUrNL4tmUyG5cuXw8fHBz4+Pgrz6urqIBAImq1TW1vbbDngPwmUEMKv7pT3AMp9hBD+dKd82hm5FOh4Pu2sXEpnwAnRoqqqKshkMly+fBlJSUl4++23uXmWlpY4f/486uvrUVRUhJqaGoV1jY2NkZubC6lUivfeew+enp7aDh/9+/cHAJSUlChM9/HxwcKFCzW2HalUisWLF8Pf379ZwgSeP4G4qqoKcXFxqK6uxo0bNxAfHw93d3eF5YqKigAAAwYM0FhshBDVaCrvNZ7F0Hbuay3vAZT7CCHa1V3zqaZzKaCZfNppuZQ1kZqaytzd3Rkh+uLAgQPM29tb5fXCwsKYubk5MzQ0ZHPnzmUff/wxEwgEzNrammVnZ7OQkBBmYGDAevXqxX799VfGGGNisZj16tWLGRkZsb59+7KNGzey+vp6rs3Y2FgmEomYo6MjW7lyJevfvz+ztbVlf/zxB2OMsaCgIGZiYsJcXFzYwoULmYeHh1p9vnXrFrO1tVVqWbFYzNLT0xWmDRs2jCUkJChMe/PNN9lbb72lMK2mpoaNGzeOhYSEtNr+5s2b2cCBA5mBgQEzNDRkgwcPZrt27WKHDx9mAJr9zJs3j1v3yJEj7JVXXmHm5uasd+/ezN/fnz158kSh/T179rBhw4YpTEtPT2dOTk5K9f/KlStswIABSi2rzbYI6aigoCAWHh6u8nqq5j5N5r2Kigq2dOlStXOfKvn+xdzXUt5jrHnu60jeY4zpTO7btWsX8/PzU2pZZUgkEpaTk6Ox9gjRFT/99BNzcXFReT2+jyM7mk8BsLKysnaX4+s4kjHN5NOO5tKysjL2QrnNGGOpVIATvaZuAa4OsVjMUlJStLKttqhagMfExLAHDx4wuVzOGGMsMTGRTZ06Van1L168yCZOnKh2rB01efJktm/fPsYYY3K5nD148IDFxMRQAU70nroFuKp0Je8xpnoB3jT3daW8x1jHcx8V4IQoR90CXB26lE9VKcD1+TiytQKcLkEnRIv4eHVYR61duxYSiQRRUVEAgMWLF2PEiBGIjY1tc73y8nKEhYVhxowZ2gizmR07dmDMmDFYtGgRACAqKgoSiQRr167lJR5C9FVXzHuAYu7rKnkPoNxHSHfWFfMpHUc2p3YBvmTJEgQFBXU4gK5u1qxZMDQ0xGeffab0Om2NnTrtvSg5ORn29vYQCASws7PD4cOH1W5LXbGxsbCysoJAIIC9vT33nkJ9tXTpUjx8+BCBgYFITEzkOxylPXjwAIwxMMawfv16bnp0dDSMjY2RkpLS6ropKSkYMWIENmzYoI1QFRw7dgy2trbYvn07N239+vVcX+7evav1mFShr/k1ODgYRkZG+Pjjj9Van+/cR3lPUVfNe0DLuU/X8x5AuU9fdPfjSMqlzXXVfErHkS1T+ynojQFow969e+Hs7AwvLy+tbE8V6enpmDJlikrrtDV26rT3ovnz58PZ2RljxoxBeno6Ro8e3aH2lNX0c1q1ahXEYjHmzJmDq1evQiKRaCUGXRUfH4/4+Hi+w9CoZcuWtTl/yZIlWoqkuaavmeiK9DW/RkdH4+bNm2qvz0fuo7zXOsp72ke5TzWq5j9dyZfd/TiScmlzlE+1q7NzqdpnwBMTE/HFF19oMpZWHThwQCvbUZeBgWrD2N7YqdqertD1z4mQroLya9dB40eI5mgz9wGq77+6sr/TcSQhXZtae+ihQ4dgYWGBN954AwAQFhYGAwMDBAQEYOrUqRAKhXBzc8OzZ88AACEhIRAIBJg+fTocHBxgbW2NgIAAVFdXw8/PDwKBACdOnAAAzJ49GwYGBtzlFX5+fsjKysK0adMwZMiQdmNzcXGBQCCARCLBtWvXms1fsGABDA0NuW9Vrly5gkmTJsHS0hJOTk4IDQ3l3vm2bt06GBgYYP/+/di4cSPGjh0LALh8+TImTpwIkUiEnj174uLFi2qPXXvtyeVyLFu2DD179kSPHj3g6+sLAAgMDFTpG9iOfEYANP45NQoNDYW9vT1MTEwwYsQI/Pnnn5gzZw4EAgFsbW1x+vRplJSUYPDgwTA1NUVycjIA4IcffsDIkSMhEokwYcIE5ObmAmj9MyOkq9BkfgXa3ndV3W9DQkJgYGCAfv36wc7ODsbGxrC3t8fYsWPRu3dvmJiYwMXFBbdu3eLWaWkfB1rPbU1lZGTAzMwMgwcP5t7LqUu5T92819q4qJv7KO+R7kCbua9xvir774vLt7e/thdfa8cxqo4TQMeRdBxJuhq1CnBfX18EBwdzv3/66adwcXGBq6srMjIykJeXh5s3byI1NRUAsG3bNojFYnh7e+P+/fs4ffo00tPTERERgaSkJNjY2HBtpaWlYdCgQdzvSUlJsLCwQEZGBvLz89uN7cqVK7CyssI//vEPuLq6AgAmTpyIM2fOAAAOHz6M1157DV999RUqKyvx2muvYfr06Xjw4AFSUlJw4MABREdHAwAiIyPh4uKCGzduYNmyZXB3d0dlZSVmzZqFUaNG4eHDhygqKuK2o87Ytddeeno6Ll68iIKCAty+fRu9evUC8DyhqvIgho58RgA0/jk1qqqqwqVLl/D48WMYGhoiPj4eycnJ6Nu3L4KDg+Hp6Qk7OzvEx8dj48aNmD9/PsrKyjB37ly8++67KCwsxPjx4+Hn5weg5c+MkK5Ek/kVaHvfVXW/3bZtG4YPH4733nsPd+/exZ07d2BoaIjp06cjLy8PRUVFaGhowNdff82t09I+DrSe25oyMTHBunXrcP36dZiamgLQrdynbt5rbVzUzX2U90h3oM3c1zhflf33xeXb21/biq+t4xhVx4mOI+k4knQ9at8D3hJTU1MYGRnByckJL7/8MkpLSxXmOzo6wtTUFK6urvD398fx48fxySefaDIEmJub45133sGePXswf/585OXl4dKlS4iLi4OHhwdOnTqFWbNmAXj+7VdZWRlCQ0NhbGyM0aNH47333kNKSgpCQkK4Nl966SU4Ozvj888/x7Fjx1BRUYGtW7dCKBQCeP5ie3VlZGS02Z5IJMKtW7fwzTffwNfXl3tiYEJCglrb04XPqKmdO3dy/z98+HCUlZXB0NAQy5cvR0JCAjZu3AiBQIBDhw4hLCwMAHDhwgVUVlZi7dq1MDAwwOLFixEVFYWnT5/C2toagOJn1p67d+92ah91zbNnzyCVSvWqz+oqLCzkrojhmy7tu71794aFhQUsLCwwePBg9OjRgzuoGzZsGMrKyrhlW9rHgdZzW6O0tDRcvXoVW7ZsUZhOua/l3AeolvfkcjlOnz7NXVmgD37//XfcunWLcp8SLl++jB49evAdBgDd23cbtbe/thXf2LFj2z2OURYdR/J/HFlWVqaXeWXr1q3cl+OkZa39jdVoAa4Ke3t7FBcXd0rbgYGBcHV1xb///W/s2bMHn332GTZt2oQnT57g22+/5c5wFxUVwcrKSiFROTo6orCwsNW2Hz58CAcHBy7JdVR77U2ZMgXbt29HREQEPvjgA6xfvx5/+9vfNLLt9nTmZwQAUqkUq1evxsmTJ/Hs2TPU1tZi+fLlAJ5/hps3b8a5c+fg5uaG+vp6ODo6AgAePXqEuro6GBoaKrT37Nkzlf9wAYBMJkN5eXnHO9RFVFRUAIBe9VldVVVVfIegls7ed1/U2sOA2trH28ptJ0+exKlTp1BbW4uQkBCYm5trrS9A18x96qipqdGrPCCVSiGXy/Wqz+qqqanRmQJcFdrOfW3tr23Fp8njGDqO5P84kjGml3mloqJCr77EVUdrJ3F4K8Dv3bsHsVjcKW2PGDECo0ePxq5du1BQUIDIyEjs2LEDO3bsgIGBAXemxtHREaWlpaivr+eK8MLCQvTp06fVtm1sbPD06VPI5fJmO646lGkvKCgIQUFBSE9Px+zZs+Ht7Y2RI0d2eNvt6azP6MiRI3BwcMCdO3dw8uRJnD17FgMGDEBgYCC3jL29PebPn4/4+HgUFhbinXfe4ebZ2tpCJBKhvLwcAoGgw/EMHDiQezehPrh9+zaOHz+uV31W19WrV1V6xoOu6Mz8qopvvvmm1X0caDm3AcBf/vIX7N69GxMmTMCaNWsQFxen1bg7Y/wa897kyZPbHBdt5T5DQ0O8/vrrvL0uiw9JSUn49ttvKfcpITY2lnKfEtraX1vSGJ8m92V9O47UtVwKAFZWVnqXV6KjoxEeHg5LS0u+Q9Fp5eXlLb7vXKuPSayqqoJMJsPly5eRlJSEt99+GwBgaWmJ8+fPo76+HkVFRaipqVFYz9jYGLm5uZBKpUrfr7Js2TL84x//wLx587iHRmzZsoXbJgC4u7ujZ8+e2Lp1K8rLy/HLL78gPj4eCxYsaLXdKVOmQC6XY/PmzaioqMAvv/yi8MAhVbXX3r59+3DkyBHU1NTglVdegZWVFQAgICAAnp6eam+3Na19RkDHPyfGGCorK5Geng4LCwtUV1dDKBSid+/eKC4uxr179xSWX716NVJSUnD27FmFV2pMnDgRAoEAW7ZswZMnT1BXV4eCggLNDgQhXYy6+646+VVZbe3jreU2AOjTpw/MzMyQnJyM5ORkJCUlcfN0LfepmveAtscFoNxHiCo687jlRS0t39r+2lZ8mtyX9eU4knIp6VZYE6mpqczd3Z21JywsjJmbmzNDQ0M2d+5c9vHHHzOBQMCsra1ZdnY2CwkJYQYGBqxXr17s119/ZYwxJhaLWa9evZiRkRHr27cv27hxI6uvr2eMMRYbG8tEIhFzdHRkK1euZP3792e2trbsjz/+YIwxFhQUxExMTJiLiwurqKhoNz7GGKuoqGAvvfQSq6mpYYwx9v/+3/9j/fv3Z3K5XGG5y5cvswkTJjALCwvm5OTENmzYwOrq6hhjjK1bt44JBAJmaWnJvvjiC26do0ePskGDBjELCws2depUNmrUKNazZ0+WnZ2t8ti1197333/P+vXrx4yMjJiDgwMLCwtjjDG2dOlS5uHh0eI2kpOTmb29PQPA7O3t2TfffNPhz0jVzykiIoJZWVkxAC3+XL9+nRUVFTFXV1dmZmbG3N3d2fz585m5uTn79ttvuW26urqyXbt2NetjZmYmc3V1ZUKhkDk5ObHIyMg2P7PWHDhwgHl7e7e7XHdy69YtZmtry3cYXcKVK1fYgAEDtNqWpvMrY23vu6rk148++ogJBALWs2dPduzYMbZixQpmYGDARCIR279/P9uwYQMzMjJiQqGQHT16tM19vKXctmPHDmZkZMQsLS3ZH3/8wS5cuMBMTU2ZkZERS0hIYIzpXu5TNe8xxjSe+1TNe4w9z9fh4eFKLdtd6GO+V9euXbuYn5+fxtqTSCQsJyenzWW0nfsYU/34srXlW9tf24qvteOY9ujDcSQAZmRkpPVcypjqx5E//fQTc3FxaXe57gYAKysr4zsMnVdWVsZeKLcZYyxVrQJcHWKxmKWkpHRK20QzdPEzCgoKYuXl5Z3Wvj4ekFEBrjw+CnB16OK+25Xo4vh1du6jApy0hY8CXB26su+2tr/qSnzaoov97excSgU4aUtrBbhWL0Hv6OWNd+/ehUAgaPXH399fM4F20/iUoelLUNWRmpqKqqoq7N+/H7179+6SD4IhRNu6e37tbJT7COmaNLHvqpP/lN1fVYmvO+RhyqWEtE8rBfjSpUvx8OFDBAYGIjExUe12nJ2dwRhr9acjbWuCrsfXFk19Rppw9OhR2Nra4tSpUwqvgyPaJ5FIuD/827dvV5gXFxeH8PBwDBo0CObm5rC1tcXs2bNx9+5dlbbx6NEjrFixAqtXr+amNTQ0YOTIkbCysoJQKMTw4cMV7gMGgOzsbIwfPx4ikQhisRjBwcGQy+UAnv/xPXz4sMLy27dv5/ri7OysUoy6TF/ya2eh3Eda0lrui4uLw9GjR5Gfn9+h3NdS3gMo96lCk/uuOvmvvf1Vnfi6ch6mXEpawtdxJNCxfNrpuVThfHgnXoJOiC7S1iWJcXFxLCMjQyfaU+USdLFYzGJiYlhBQQGTyWTc9LVr17LY2FiWnZ3N/va3v7Hq6mpWVFTEZs6cySZNmqR0LGlpaezDDz9kY8eOZcuXL+em19fXMx8fH/b48WNWVVXF4uLimEAgYD///DNjjDG5XM7s7e3ZBx98wCoqKtjVq1eZtbU12717N9dGbGws++CDD7jfZTIZKygoYDExMczJyUmp+LrKJeiEqEpbl6DrUu5TJd+3lPsa8x5jrEO5r7W8x5ju5L6ucgk6IXzT5iXomsynHW0LSl6CztdxJGMdz6eayKU6cQk6IfrqwIEDOt1eWwYNGgSJRMK93uSrr75CdnY2goKC4Obmhk8++QTm5uZwcHDAu+++i+zsbO5sTHu8vb0RERGBvn37Kkw3MjLCsWPHYGNjA6FQiICAAPTo0QMXLlwAADx+/BjFxcVYuHAhRCIRXF1dMWbMGOTk5HBtBAUF4ccff+S+iTc0NIREIsGgQYM0MCqEEGV0l9zXNO8B6FDuay3vAZT7CCGt02T+6+7HkUDH82ln5lIqwAlR0pUrVzBp0iRYWlrCyckJoaGhqKurAwD4+flBIBDgxIkTAIDZs2fDwMAAiYmJ8PPzQ1ZWFqZNm4YhQ4YgJCQEAoEA06dPh4ODA6ytrREQEIDq6up222qc37Q9AAgMDISXl5dWxiEyMrLZ+5wbVVZWwszMrNV3kaqroaEBVVVVsLe3BwDY2dlh3LhxSE1NhVQqRV5eHq5evYqZM2cqrBcQEIDIyEiNxkKIPlE37zXO7y65r628B1DuI4S0T1v5VJdzKcDPcSSgXj7trFxKBTghSqisrMRrr72G6dOn48GDB0hJScGBAwcQHR0NAEhKSoKNjQ23fFpaGvcNWVJSEiwsLJCRkYH8/Hxs27YNYrEY3t7euH//Pk6fPo309HRERES021ZL7QGAXC7XyoNP8vPzkZubi1GjRilMr62txcWLF7Fz5074+vpqfLtZWVmwtLSEt7c3AEAgEODgwYPYvXs3hEIhXFxc4OPjw81v5OrqipycHNy4cUPjMRHS3XUk7zXO7w65r7W8B1DuI4QoR5v5VFdzKcDfcSSgXj7trFxKBTghSvjhhx9QVlaG0NBQWFpaYvTo0XjvvfeQkpKidpuOjo4wNTWFq6sr/P39cfz4cbXbSkhIwJkzZ9ReX1m3b98G8Pxbw6bCw8Mxa9YseHh4ICYmRqPblMlk2LRpE+Lj47knmVZWVsLT0xPh4eGoqKhATk4OsrKy8OWXXyqs2xhnY9yEEOV1Rt4Dul7uay3vAZT7CCHK0fV82p2PIwH182ln5VIqwAlRQlFREaysrGBsbMxNc3R0RGFhoUbat7e3R3FxsUba6kyNl4qampoqTHd0dERycjJiYmJgZmamse01NDQgICAAAQEBmDNnDjf95MmTKC0txcqVKyESiTB06FAEBgZi165dCusLhUKFuAkhyuvsvAd0jdzXWt4DKPcRQpRD+fQ5bR9HAh3Lp52VS4002hoh3ZSjoyNKS0tRX1/PJc/CwkL06dNHI+3fu3cPYrFYI211JgsLCwDg7llqtGLFCo1vSyaTYfny5fDx8YGPj4/CvLq6OggEgmbr1NbWNlsO+E8CJYQor7PzHtA1cl9reQ+g3EcIUQ7l0+e0eRwJdDyfdlYupTPghCjB3d0dPXv2xNatW1FeXo5ffvkF8fHxWLBgAbeMpaUlzp8/j/r6ehQVFaGmpoabZ2xsjNzcXEilUu4em6qqKshkMly+fBlJSUl4++23lWqrpfYCAgLg6enZyaMA9O/fHwBQUlKiMN3HxwcLFy7U2HakUikWL14Mf3//ZgkTeP4E4qqqKsTFxaG6uho3btxAfHw83N3dFZYrKioCAAwYMEBjsRGiLzqa94Dukftay3sA5T5CiHK0nU91MZcC2juOBDSTTzstlyq8lIzeA070jCrvhb18+TKbMGECs7CwYE5OTmzDhg2srq6Omx8bG8tEIhFzdHRkK1euZP3792e2trbsjz/+YEFBQczExIS5uLiwiooKJhaLWa9evZiRkRHr27cv27hxI6uvr1eqLcZYs/aWLl3KPDw8lOqHqu8BT09PV5g2bNgwlpCQoDDtzTffZG+99ZbCtJqaGjZu3DgWEhLSavubN29mAwcOZAYGBszQ0JANHjyY7dq1ix0+fJgBaPYzb948bt0jR46wV155hZmbm7PevXszf39/9uTJE4X29+zZw4YNG6YwLT09nd4DTvSesu8B70jea9yOruQ+Vd8D3jT3tZT3GGue+zqS9xhjOpP76D3ghChHlfeAazOfdmYuZUy194DzcRzJmGbyaUdzaWvvAacCnOg1VQ7INEksFrOUlBStb5cx1QvwmJTW1bEAACAASURBVJgY9uDBAyaXyxljjCUmJrKpU6cqtf7FixfZxIkT1Y61oyZPnsz27dvHGGNMLpezBw8esJiYGCrAid5TtgDXND5zn6oFeNPc15XyHmMdz31UgBOiHFUKcE3jM5+qUoDr83FkawU4XYJOCE+08boHTVi7di0kEgmioqIAAIsXL8aIESMQGxvb5nrl5eUICwvDjBkztBFmMzt27MCYMWOwaNEiAEBUVBQkEgnWrl3LSzyEkOe6Yu7rKnkPoNxHiD7pCvmUjiObowKcEC1bunQpHj58iMDAQCQmJvIdTpsePHgAxhgYY1i/fj03PTo6GsbGxm2+PiMlJQUjRozAhg0btBGqgmPHjsHW1hbbt2/npq1fv57ry927d7UeEyH6rqvnPl3PewDlPkL0RVfJp3Qc2TJ6CjohWhYfH4/4+Hi+w+iwZcuWtTl/yZIlWoqkuaavmSCE6IbukPt0Oe8BlPsI0ReUTztXZ+dSOgNOCCGEEEIIIYRoARXghBBCCCGEEEKIFlABTgghhBBCCCGEaEGze8DLy8uRnZ3NRyyEaN2tW7fw7Nkzvfo3//DhQ8hkMoU+19TUAADMzMz4Cksn/fnnnxptr6amRq/+rRHdVVRUhIaGBr3696iP+V5d//73vzXe5m+//Yby8nKNt9sVSaVSGBkZwdjYmO9QSAfl5ORAKpXqZV75+eefYWFhwXcYOq26urrF6QLGGGv85dy5c1i1apXWgiKEb/X19ZDJZDA3N+c7FK1paGhAdXU1RCIRN628vBwPHz6EUCiEhYUFLCws9GpM2tKvXz+kp6d3uJ28vDzMnz9fAxER8h+1tbWorq5Gr169VF5PIBDAxMSkkyLTPfqY7zti9uzZ2Lp1q0ba8vLyQmFhoUba6ooYY5BKpaiqqkJVVRWkUin69etHxUs3IJfLUVNTo3efZUVFBUQiEQQCAd+hdAm5ublNf01TKMAJIfqrpKQE586dQ2ZmJjIyMlBSUgI3Nzd4eXnBy8sLI0eOpERLiI5JS0tDdHQ0zp49y3cohJAm7ty5g8zMTO5vqrW1Nff31MPDAzY2NnyHSAjhBxXghJCWNT14OHPmDIyMjDB58mR4eXlh5syZ6NevH98hEqL3qAAnRDc8evQIWVlZyMzMRHp6OqRSKaZOnQovLy9MmzYNL730Et8hEkJ0AxXghJD2NTQ0IC8vDxcuXEBmZia+++472NnZcd/me3l5qXwJLCGk46gAJ4QfVVVVyM7O5r6ozsvLw4QJE7i/ia6urjAwoGcdE0KaoQKcEKI6mUyG3377jTvwOH/+PIYOHcodeEycOJEe6EaIFlABToh20N89QoiGUAFOCOm4yspKXLx4kc4EEKJlVIAT0nma3or1/fffw8bGhvu75unpCWtra75DJIR0PWnNXkNGCCGqEolE3EEJoHgv3Ny5c1FVVYXx48dj4sSJ8PLywqhRo3iOmBBCCFFUVFSEH3/8EZmZmTh58iRqa2sxZcoUeHl5ISIiAs7OznyHSAjpBqgAJ4RonIODAxYsWIAFCxYAUDyLsG3bNpiZmXHF+Ouvvw6xWMxzxIQQQvRNW1dv/fOf/6SrtwghnYIuQSeEaJVcLsevv/6qcB9dnz59uDPo06dPR8+ePfkOk5AugS5BJ0R57d3HPWnSJJiamvIdJiGke6N7wAkh/JJKpdzT1TMzM/H7779j+PDh3AHR5MmTYWJiwneYhOgkKsAJaVvTK7BOnToFW1tbeoMHIYRPdA84IYRf5ubmCvePP378GGfPnsX58+cRGhqK/Px8jB8/nltm5MiREAgEPEdNCCFEFxUWFuL8+fPIzMzEv/71L9TX18Pd3R1eXl74/PPP4eTkxHeIhBA9RwU4IUSn2NraKtw/3vRg6s0331Q4mJo+fTo9FIcQQvTYi/dxN/3SNjU1lb60JYToHLoEnRDSpdBrYQj5D7oEnegbuo+bENLF0T3ghJCuq60DsVdffRVTpkxBjx49+A6TkE5DBTjRB02/eP3uu+9gZ2fHFdzTpk2DlZUV3yESQoiyqAAnhHQfVVVVyM7ObvWBbu7u7jA2NuY7TEI0hgpw0h3duXMH58+fx4ULF3DixAnI5XJMnjwZXl5emDlzJvr168d3iIQQoi56CBshpPuwsLBQeKBbcXExfvjhB2RmZiIgIABPnjyBm5sbPdCNEEJ0SOPDNxu/PH306BF3H3daWhrlakJIt0IFOCGk27K3t1d4oFvTyxgjIiJgYmKCSZMmwcvLC7NmzULfvn15jpgQQrq/tl4/+X//93/0+klCSLdGl6ATQvRSQ0MDrl27pnD/eJ8+fei+QtKl0CXopCtoL99Onz4dPXv25DtMQgjRBroHnBBCgOdnZK5cucKdlaEn65KugApwoquaXnF0+vRphSuOXnvtNUgkEr5DJIQQPlABTgghLXnx3bJ5eXmYMGECV5C7urrCwMCA7zCJnqMCnOiKkpISnDt3DpmZmcjIyEBJSQk9c4MQQpqjh7ARQkhLRCKRwgPdioqK8OOPPyIzMxNz5syBVCrF1KlTuWX69+/Pc8SEEKI91dXV+Omnn1q8j3vPnj301glCCGkFnQEnhBA1NL28MiMjA9bW1nj11VcxceJEvPHGG+jTpw/fIRI9QGfAibbI5XL8+uuvXN778ccfIRaLuS8hZ8yYAUtLS77DJIQQXUeXoBNCSEfRgSnhCxXgpDM1/aIxMzMTZmZmmDhxIry8vPD6669DLBbzHSIhhHQ1VIATQoimtXVpppeXF71ih2gMFeBEk5rex33q1Ck8efJE4T7uUaNG8R0iIYR0dXQPOCGEaJpQKFS4f7zpQe2yZctQXFyM8ePH08OJCCG8auvLwr1799J93IQQ0gnoDDghhGhZ42Wd58+fx+nTpyGTyeDu7s5dru7k5MR3iKSLoDPgRBXt3S4zc+ZM9OjRg+8wCSGkO6NL0AkhhG9N77M8deoUbG1tuQNiT09PWFtb8x0i0VFUgJP2vPjASHNzc+4+bnpgJCGEaB0V4IQQoktkMhl+++037oD5/PnzGDp0KFeQT5w4EWZmZnyHSXQEFeDkRcXFxfjhhx+QmZmJ7777DlVVVRg/fjxXdNN93IQQwisqwAkhRJdVVVUhOzubK8jz8vIwYcIEeHl54dVXX4WbmxuMjOhxHvqitLQUDg4OqKura3WZv//979i0aZMWoyJ8aitHeHl5wdXVFQYGBnyHSQgh5DkqwAkhpCt59OgRsrKyuLNbz549w7hx4+gpxXpkxowZyMjIQEt/vk1MTJCbm4uBAwfyEBnRhhfv46arZAghpEuhApwQQrqytt7T+9prr0EikfAdItGwb7/9FkuXLkVlZWWzea+88gp+//13HqIinanpfv7999/DxsaGK7g9PDxgY2PDd4iEEEKUQwU4IYR0Fy2dGevTpw93oD59+nT07NmT7zBJB9XU1MDa2hpSqVRhulAoxNatW7FmzRqeIiOa0vRKl/T0dEilUkydOhVeXl6YNm0aXnrpJb5DJIQQoh4qwAkhpLuSSqW4cOFCi+/49fLywqRJk2Bqaqp0e/v27cN//dd/YezYsZ0YNVHGW2+9hZSUFDQ0NHDTjI2Ncf/+ffTu3ZvHyMg///lPGBsb4/XXX1d6HbqPmxBC9AYV4IQQoi8qKipw6dIl7iA/Pz8f48eP5w7yR44cCYFA0Or6w4cPx/Xr17F+/Xps3rwZJiYmWoyeNJWeno4FCxagqqqKmzZhwgRcuHCBx6j0W2lpKZYtW4bk5GQsXLgQX3/9davL0tsOCCFEb1EBTggh+qqwsBDnz59HZmYm/vWvf6Gurg5TpkzhLld3dnbmlq2oqIC1tTVkMhnMzc3Ru3dvJCcnY+TIkfx1QI/JZDLY2NigvLwcACASibBz5074+/vzG5ie+v777+Hn54fKykpIpVJYW1vjyZMnCsu0dR+3p6cnrK2teYqeEEKIFlEBTggh5Lm2CoS6ujqsXLkSFRUVAAADAwMYGxvjf/7nf7BlyxY6G86DwMBAJCYmQiaTwdjYGMXFxbCysuI7LL1SXV2NdevWYd++fQr35JuamiIzMxOFhYVKfcFFCCFEb1ABTgghpLn6+npcvHgRp0+fRmZmJu7cuYPi4mLI5XKF5YRCIcRiMY4cOYIRI0bwFK1+On/+PKZPn46amhrMmDED6enpfIekV3766Se89dZbePLkCWpqahTmCYVCGBoaYtKkSdwZ7ldeeaXNWzwIIYToBSrACSGEtM/Z2Rn37t1rcV7j2fC//vWvCAkJgaGhoZaj00+MMdjb26O0tBSHDh3CggUL+A5JL9TU1GDTpk3YvXs3amtrW3wfOwB4eXkhIyNDy9ERQgjRcVSAE0J0h1wuR35+Pt9hkBeUlJTAy8ur2dnvF5mYmKB///6IiopCv379tBSdftu+fTu+/vpr/PTTTyo90Z6o5/fff8f69evx5MkT1NbWtrmsqakpfv75Z/pCSgfZ2dnB3t6e7zAIIfqJCnBCiO4oKSmBvb09JBIJ36GQJqqrq/Hs2TPuTF/jZbQCgYCbxhiDQCCAQCCAkZERevXqBWNjY95i7qjG+3nNzc15jqRt9fX13APyNNGWVCqFpaWlBiLrfmpqalBWVga5XM69/q21faGRvb09PR9Bx5SVlWHVqlXYunUr36EQQvRTmhHfERBCSFMmJiYoKCjgOwzSRExMDE6dOoU+ffqgT58+sLGxga2tLWxsbGBnZwc7OzvY2NjAwsKC71A1ZsOGDQDQJQ7S8/PzMWTIkA63k5aWhqioKJw7d67jQemByspKPHnyBCUlJXj8+DGePHnC/bewsBCPHj3Cu+++S7cG6JjGfZsQQvhCBTghhJA2rV27FmvXruU7DNIKTRTfRHUikQgikQhOTk58h0IIIaQLMeA7AEIIIYQQQgghRB9QAU4IIYQQQgghhGgBFeCEEEIIIYQQQogWUAFOCCGEaNCSJUsQFBTEdxidSiKRcE+93759Ozc9Li4OR48eRX5+PgYNGgRzc3PY2tpi9uzZuHv3rtLtP3r0CCtWrMDq1asVpjc0NGDkyJGwsrKCUCjE8OHDkZSUpLBMdnY2xo8fD5FIBLFYjODgYO4VeqmpqTh8+LDa/e7O/euKfdu+fTv379DZ2VmtfhNCiLZRAU4IIYRoEGMM2nrD5969e5GZmamVbb0oJiYGBQUFCA4OBgC8//77qK+vx9y5c1FaWgpfX188ffoUubm5qK+vx6JFi5Rq9/jx44iMjMS1a9cgk8kU5jU0NMDJyQm3b9/G48ePsWbNGixatAiXL1/m5vv4+GDChAkoKirCiRMnsH//fuzZswcA8Oabb+Lp06dYt26dyv3tzv3rqn0LDg5GQUEBYmJiVOovIYTwihFCiI4oLi5mJiYmfIdBCAsNDWWhoaF8h9GuyZMns4yMDI20lZqaytzd3ZVaViwWs/T0dO73hIQENnbs2FaXP3jwIDMyMmIymUzpeObNm8eWL1/e7nKWlpYsOjqaMcbYo0ePGAB26dIlbv6MGTNYUFCQwjpjxoxhX331ldKxdOf+dYe+paenMycnJ6Vi6yr7NiGk20qlM+CEEEKIhhw6dAgWFhZ44403AABhYWEwMDBAQEAApk6dCqFQCDc3Nzx79gwAEBISAoFAgOnTp8PBwQHW1tYICAhAdXU1/Pz8IBAIcOLECQDA7NmzYWBggMTERACAn58fsrKyMG3aNO5VZIGBgfDy8tJ6vyMjIxEYGNjq/MrKSpiZmcHQ0FCj221oaEBVVRXs7e0BAHZ2dhg3bhxSU1MhlUqRl5eHq1evYubMmQrrBQQEIDIyUuntdOf+dee+EUKILqICnBBCCNEQX19f7pJsAPj000/h4uICV1dXZGRkIC8vDzdv3kRqaioAYNu2bRCLxfD29sb9+/dx+vRppKenIyIiAklJSbCxseHaSktLw6BBg7jfk5KSYGFhgYyMDOTn5wMA5HI5GhoatNTb5/Lz85Gbm4tRo0Y1m1dbW4uLFy9i586d8PX11fi2s7KyYGlpCW9vbwCAQCDAwYMHsXv3bgiFQri4uMDHx4eb38jV1RU5OTm4ceNGu9vozv3rzn0jhBBdRQU4IYQQ0slMTU1hZGQEJycnvPzyyygtLVWY7+joCFNTU7i6usLf3x/Hjx9XazsJCQk4c+aMJkJW2u3btwE8P4P5ovDwcMyaNQseHh4av09XJpNh06ZNiI+PR48ePQA8P1vr6emJ8PBwVFRUICcnB1lZWfjyyy8V1m2MtTH2tnTn/nXnvhFCiK6iApwQQgjRIfb29iguLuY7DKVVV1cDeP4lw4scHR2RnJyMmJgYmJmZaWybDQ0NCAgIQEBAAObMmcNNP3nyJEpLS7Fy5UqIRCIMHToUgYGB2LVrl8L6QqFQIfa2dOf+dee+EUKIrjLiOwBCCCGE/Me9e/cgFov5DkNpFhYWAIC6urpm81asWKHx7clkMixfvhw+Pj7w8fFRmFdXVweBQNBsndra2mbLAf8p5trSnfvXnftGCCG6is6AE0IIITyrqqqCTCbD5cuXkZSUhLfffhsAYGlpifPnz6O+vh5FRUWoqalRWM/Y2Bi5ubmQSqXcmUVPT0+txt6/f38AQElJSbN5Pj4+WLhwoca2JZVKsXjxYvj7+zcr4ADAzc0NVVVViIuLQ3V1NW7cuIH4+Hi4u7srLFdUVAQAGDBgQLvb7M796859I4QQXUUFOCGEEKIhf/3rXxEVFYXvvvsO8+bNQ1hYGK5fv46QkBBcvHgRoaGhuHz5MjZv3ozffvuNWy84OBjm5uaYN28eAgMDsWrVKgDA+vXrERsbCycnJ2zevBmGhob48MMPkZOTA+D5Q98++ugjjB49mrdLcocMGYJhw4bh2rVrSi1fW1sLNzc3hIaGtjj/008/xcsvv4xjx45h7969GDJkCGJjYwE8fxDdoUOHMHnyZAgEAu5n/vz5AICBAwfi0KFD2LlzJ2xtbTFlyhSMGzcOERERCtu4du0ahg0bhsGDB7cbjyr9a68tbffP2dm52/Zt8ODB7cZMCCE6ie8XoRFCSCN6DzjRFdp8V7BYLGYpKSla2VZbVH0PeExMDHvw4AGTy+UsMTGRTZ06VeltXbx4kU2cOFHNSDtu8uTJbN++fUrHo0r/+O4bY4r96859k8vl7MGDBywmJobeA04I6SroPeCEEEII37T96jBNWLt2LSQSCaKiorB48WKMGDGCO9vZlvLycoSFhWHGjBlaiLK5HTt2YMyYMVi0aJHS8SjbP777Bij2rzv3DQCioqIgkUiwdu1a3mIihBBVUQFOCCGE8GTp0qV4+PAhAgMDkZiYyHc4Snvw4AEYY2CMYf369QCA6OhoGBsbIyUlpc11U1JSMGLECGzYsEEboSo4duwYbG1tsX37dpXjUaZ/fPYNaN6/7tw34PktGo3/Du/evctLXIQQoioBY4zxHQQhhADPHwQkkUiaPfX2RcHBwdi5cydCQ0Px97//XUvRKSc0NBQxMTGYMWMG/vnPf/IdDgAgNjYWmzZtQllZGfr06YMDBw7Aw8MD9+/fx6uvvoqHDx/Cy8sL33//vdbisLOzw/r16/HRRx916jbV1VhkbN26ledItCctLQ1RUVE4d+4c36EQ0mn0cd8mhOiUNHoNGSGky4mOjsbNmzf5DqNFn332GQAgPz+f50j+Y9WqVRCLxZgzZw4uXboEiUQCALCzs8PEiRPh7++vlctKm8Zx9epVLg5CCCGEEH1Bl6ATQogeKi0txX//939jzZo1vN7TSQghhBCiT6gAJ4R0eRkZGTAzM+NeKQQAP/zwA0aOHAmRSIQJEyYgNzcX69atg4GBAfbv34+NGzdi7NixAJ5fNm5vbw8TExOMGDECf/75J+RyOZYtW4aePXuiR48e8PX1VTu+ltqfM2cOBAIBbG1tcfr0aZSUlGDw4MEwNTVFcnKyyn0IDAyEl5eXUvEUFhbi7bffRnh4OMaPH99sPt9j11KbADQ+ZoQQQggh2kYFOCGkyzMxMcG6detw/fp1mJqaoqysDHPnzsW7776LwsJCjB8/Hn5+foiMjISLiwtu3LiBZcuWwd3dHQBQVVWFS5cu4fHjxzA0NER8fDzS09Nx8eJFFBQU4Pbt2+jVq5fa8bXUfnJyMvr27Yvg4GB4enrCzs4O8fHx2LhxI+bPn69yH+RyuVJP0r558yYmTJiA6upqDBo0qNl8XRi7ltoEoPExI4QQQgjRNroHnBDSpaWlpeHq1avYsmULN+3ChQuorKzE2rVrYWBggMWLFyMqKgpPnz4FALz00ktwdnbG559/DgDYuXMnt+7w4cNRVlYGkUiEW7du4ZtvvoGvr69Sr1dqTUvtGxoaYvny5UhISMDGjRshEAhw6NAhhIWFqdWHhIQEpWLZsmULli1bho0bN3Lbb0oXxq6lNgFofMzaExsbi6SkJJVi78qkUinkcjn69u3LdyiEdJqysjKsWrWK7zAIIXqMCnBCSJd18uRJnDp1CrW1tQgJCYG5uTkA4NGjR6irq4OhoaHC8s+ePWvWhlQqxerVq3Hy5Ek8e/YMtbW1WL58OaZMmYLt27cjIiICH3zwAdavX4+//e1vKsfYWvvA88vGN2/ejHPnzsHNzQ319fVwdHRUuQ+qSExMhEQiQWlpKSIiIvDyyy8rvFaI77Fra7wA7Y7ZwoUL8f7776u1bld09uxZ7N+/H1999RXfoRDSaf7xj3/wHQIhRM9RAU4I6bL+8pe/YPfu3ZgwYQLWrFmDuLg4AICtrS1EIhHKy8shEAjabOObb77ByZMncfbsWQwYMACBgYHcvKCgIAQFBSE9PR2zZ8+Gt7c3Ro4cqVKMbbVvb2+P+fPnIz4+HoWFhXjnnXe4ear0QR1bt27Fb7/9hk2bNmHgwIFYsGCBytvV5NgdOXIEDg4OuHPnTqttAtodM1tbWwwdOrTD7XQVt2/fhlAo1Ks+E/1ja2vLdwiEED1H94ATQrqsPn36wMzMDMnJyUhOTuYuF544cSIEAgG2bNmCJ0+eoK6uDgUFBS22UV1dDaFQiN69e6O4uBj37t0DAOzbtw9HjhxBTU0NXnnlFVhZWakVY2vtN1q9ejVSUlJw9uxZTJkyhZuuSh8AICAgAJ6enkrHZWBggK+//hr9+/fH4sWLcenSJZW3q4mxY4yhsrIS6enpsLCwaHe8AM2NGSGEEEKI1jFCCNERxcXFzMTEpN3lduzYwYyMjJilpSX7448/2IULF5ipqSkzMjJiCQkJjDHGMjMzmaurKxMKhczJyYlFRkaydevWMYFAwCwtLdkXX3zBGGOsqKiIubq6MjMzM+bu7s7mz5/PzM3NWWBgIOvXrx8zMjJiDg4OLCwsTKk+bNiwgZmZmTFDQ0M2b968Vtv/9ttvuXVcXV3Zrl27mrWlbB8YY2zp0qXMw8OjxZh2797NrKysGADm6OjI9u3bx83LyclhIpGImZiYsI8//rhTx65pHC39XL9+Xanx0tSYtSU0NJSFhoYqtWx3kZqaytzd3fkOg5BOpY/7NiFEp6QKGGOMx/qfEEI4JSUlkEgk3KvE9MWqVavw2WefoUePHnyH0mV09pg13he/devWTmlfF6WlpSEqKgrnzp3jOxRCOo0+7tuEEJ2SRpegE0KIku7evQuBQNDqj7+/v9JtpaamoqqqCvv370fv3r2p+FYCjRkhhBBCujoqwAkhREnOzs5gjLX6k5iYqHRbR48eha2tLU6dOoWQkJDOC7oboTHruiQSCfdF1fbt2xXmxcXFITw8HIMGDYK5uTlsbW0xe/Zs3L17V6m2ZTJZi1+ImZmZQSqVAgB27NiB/v37w8TEBGKxGF9//TW3fnZ2NsaPHw+RSASxWIzg4GDI5XIAz7/0OXz4sNr9jouLw9GjR5Gfn692/4DnT/hfsWIFVq9erTC9oaEBI0eOhJWVFYRCIYYPH97s1Xlt9a89yowt0Pr4qjq227dv57bh7Oys9PgQQkhXQgU4IYTwYN++fZBKpTh48CBMTEz4DqdL6I5jtnfvXmRmZupse5oUExODgoICBAcHc9Pef/991NfXw8PDA76+vnj69Clyc3NRX1+PRYsWKd32ihUrFL4Mu3r1KhYvXgxzc3PExMTg888/x8GDB1FRUYE9e/bg4cOHAJ4XsD4+PpgwYQKKiopw4sQJ7N+/H3v27AEAvPnmm3j69CnWrVuncn8b+zZ37lyUlpaq3b/jx48jMjIS165dg0wmU5jX0NAAJycn3L59G48fP8aaNWuwaNEiXL58Wan+KaOtsQXQ6viqM7bBwcEoKChATEyM0vERQkiXw8ON54QQ0iJlH8JGSGfT1oOaJk+ezDIyMnSivc58CJtYLGbp6ekK0xISEtjYsWNbXP7gwYPMyMiIyWQytbb31ltvsRs3brDa2lpma2vLEhMTW1zu0aNHDAC7dOkSN23GjBksKChIYbkxY8awr776Suntt9U3xtTr37x589jy5cvbXc7S0pJFR0czxpTvnyoax5Yx1ub4dmRs09PTmZOTk9oxtoUewkYI4VkqnQEnhBBC1HTlyhVMmjQJlpaWcHJyQmhoKOrq6gAAfn5+EAgEOHHiBABg9uzZMDAw4G5V8PPzQ1ZWFqZNm4YhQ4YgJCQEAoEA06dPh4ODA6ytrREQEIDq6mq12gOAwMBAeHl5aXFElBcZGdnsPe+NKisrYWZmBkNDQ5XbvXHjBhoaGjBo0CBcu3YNjx8/hoeHR4vL2tnZYdy4cUhNTYVUKkVeXh6uXr2KmTNnKiwXEBCAyMhIpWNoq29Ax/rXloaGBlRVVcHe3h6A8v1TVtOxBdDm+HbW2BJCSFdHBTghhBCihsrKSrz22muYPn06Hjx4gJSUFBw4cADR0dEAgKSkJNjY2HDLp6WlcYVL43wLCwtkZGQgPz8f27Ztg1gshre3N+7fv4/Tp08jPT0dERERarUHAHK5Lx40eQAAIABJREFUHA0NDZ06DurIz89Hbm4uRo0apTC9trYWFy9exM6dO+Hr66tW25999hl3WXPje+D9/PxgZ2cHoVCIiRMn4vr16wAAgUCAgwcPYvfu3RAKhXBxcYGPjw+8vb0V2nR1dUVOTg5u3Lihdt801b+2ZGVlwdLSkotf2f4pq+nYAm2Pb2eMLSGEdAdUgBNCCCFq+OGHH1BWVobQ0FBYWlpi9OjReO+995CSktKhdh0dHWFqagpXV1f4+/vj+PHjareVkJCAM2fOdCieznD79m0Az8+SNhUeHo5Zs2bBw8NDrfuA7927h4KCAri5uQEA9+XD//7v/+L27dt4+PAhLC0tubPTlZWV8PT0RHh4OCoqKpCTk4OsrCx8+eWXCu02xtkYtzp900T/2iKTybBp0ybEx8dzbwhQtn/KeHFsgbbHtzPGlhBCugMqwAkhhBA1FBUVwcrKCsbGxtw0R0dHFBYWamwb9vb2KC4u1lh7uqLxsnpTU1OF6Y6OjkhOTkZMTAzMzMxUbjciIgLvv/8+97uDgwMAYNiwYbC0tESvXr2wePFiXL58GXK5HCdPnkRpaSlWrlwJkUiEoUOHIjAwELt27VJoVygUKsStTt800b/WNDQ0ICAgAAEBAZgzZw43Xdn+KePFsQXaHt/jx49rfGwJIaQ7oAKcEEIIUYOjoyNKS0tRX1/PTSssLESfPn00to179+5BLBZrrD1dYWFhAQDc/fKNVqxYAU9PT7XafPToES5duoTXX3+dmzZy5EiYmpri559/5qbV19fD3NwchoaGqKurg0AgaNZWbW2twu+NcTYWi21prW9Ax/rXGplMhsDAQPj4+GDJkiUK85TtX3taGlug7fGVy+UaH1tCCOkOqAAnhBBC1ODu7o6ePXti69atKC8vxy+//IL4+HgsWLCAW8bS0hLnz59HfX09ioqKUFNTo9CGsbExcnNzIZVKuct5q6qqIJPJcPnyZSQlJeHtt99Wu72AgACNF3ya0L9/fwBASUmJwnQfHx8sXLhQrTajoqKwevVqhaKvR48eWLJkCT7++GPcuXMHJSUliI+PxxtvvAEAcHNzQ1VVFeLi4lBdXY0bN24gPj4e7u7uCm0XFRUBAAYMGKB23zrav5ZIpVIsXrwY/v7+8PHxaTZf2f61p6WxBdoe384YW0II6Q6oACeEEELUYGFhgX/96184deoU+vTpg/nz58Pf3x9r167lllm/fj1iY2Ph5OSEzZs3w9DQEB9++CFycnIAAL6+vvjoo48wevRo7hLc4OBgmJubY968eQgMDMSqVas61J4uGjJkCIYNG4Zr1661u2xtbS3c3NwQGhra6jKlpaVIS0vDO++802xeVFQUxo4di1GjRmHIkCFwcHDAjh07AAADBw7EoUOHsHPnTtja2mLKlCkYN24c9+C7RteuXcOwYcMwePDgduPRdN8+/fRTvPzyyzh27Bj27t2LIUOGIDY2FsDzB/EdOnQIkydPhkAg4H7mz5+vVP86OrZA6+OrztgSQog+EDDGGN9BEEII8PyMkUQiUfnySEI0bcOGDQCArVu3anW7EokEO3bswNy5c7W6XeB5MRcVFYVz585pvG2JRIKPPvoI8+bNg6OjIwwMDLBv3z7s27dPqYfEXbp0CevXr8ePP/6o8diU4e7ujqVLl2LRokVKxdOV+sb39puObUNDAwoLC5GSkoKoqCjcvXtX49vja98mhJD/XxqdASeEEEJ0iC6+NkwT1q5dC4lEgqioKADA4sWLMWLECO5sbmvKy8sRFhaGGTNmaCPMZnbs2IExY8Zwxbcy8XSVvvG9/RfHNioqChKJROEqEkII6W6oACeEEEJ0wNKlS/Hw4UMEBgYiMTGR73A06sGDB2CMgTGG9evXc9Ojo6NhbGzc5qvbUlJSMGLEiP+vvXuPqrLO9zj+2Vzc3MQbykKcoSyJhbYcvEJjR/NuBnmpPBIqyt6OaWflBZNm5Rwzj6UpLjCqpeAlsZpV5AItp2V28jKhoVLnAOI0NioykjqKiiJsYJ8/Wu4TeQOEZwu8X2vxx35+v/39fX87XfbhefbzOM5cGmnbtm3y8/PTqlWr6t3P/b43Z69/q882Pj7e8eekKc5+A8D9gEvQAdw3uAQd94vWeJlqU16CDtwvWuPfbQD3FS5BBwAAAADACARwAAAAAAAMQAAHAAAAAMAAbs5uAAB+qaqqShaLxdltoJX77rvvJP18X4LW4vTp0zpx4gR//9CiHTp0SGPGjHF2GwBaMW7CBuC+UV5ernfffdfZbQDNRmFhof76178qLi7O2a0AzcaAAQM0aNAgZ7cBoHXKIoADANBMZWVlac2aNfrv//5vZ7cCAADujrugAwAAAABgBAI4AAAAAAAGIIADAAAAAGAAAjgAAAAAAAYggAMAAAAAYAACOAAAAAAABiCAAwAAAABgAAI4AAAAAAAGIIADAAAAAGAAAjgAAAAAAAYggAMAAAAAYAACOAAAAAAABiCAAwAAAABgAAI4AAAAAAAGIIADAAAAAGAAAjgAAAAAAAYggAMAAAAAYAACOAAAAAAABiCAAwAAAABgAAI4AAAAAAAGIIADAAAAAGAAAjgAAAAAAAYggAMAAAAAYAACOAAAAAAABiCAAwAAAABgADdnNwAAAOrm0qVL+vOf/+x4/f333+uf//yn1q1b5zgWGhqqQYMGOaM9AABwFya73W53dhMAAODuampq5O/vrytXrsjN7ebfoV+/fl3p6en693//dyd0BwAA7iKLS9ABAGgmXFxc9Pzzz8tut+vq1as3/bi5uSkyMtLZbQIAgNsggAMA0IxMmTJFLi43//NtMpn01FNPydvb2wldAQCAuiCAAwDQjPTt21edO3e+6bi3t7dmzJjhhI4AAEBdEcABAGhmYmNj5eHhUeuY3W7X8OHDndQRAACoCwI4AADNzNSpU1VTU+N47erqqueee05t2rRxYlcAAOBuCOAAADQzDz/8sB588EHHa7PZrNjYWOc1BAAA6oQADgBAM2SxWOTl5SVJ8vT05NnfAAA0AwRwAACaoejoaNlsNrm7u2vq1Km3vDM6AAC4v/CvNQAAzVDXrl3Vu3dv2Ww2xcTEOLsdAABQB27ObgAAcO/27NmjzMxMZ7cBg7Vr107t2rVTenq60tPTnd2OIWpqamQymWQymZzdilN16NBBixcvdnYbAIB6IoADQAtw+PBhff3114qKinJ2KzDQ999/r+DgYPn6+jq7FcNkZWWpc+fOioiIcHYrTnPu3DmtW7eOAA4AzRABHABaiD59+mjJkiXObgMG+vjjj7Vq1Sr927/9m7NbMczp06cVGhqq+fPnO7sVp8nPz1dWVpaz2wAANADfAQcAoBlzd3d3dgsAAKCOCOAAAAAAABiAAA4AAAAAgAEI4AAAAAAAGIAADgBAKzN9+nTNnj3b2W0YYv369fr0009VWFio4OBgeXp6ys/PT1FRUTpx4sRd319VVeV47Nkvfzw8PFReXu6Yl5ycrO7du6tNmzYKDAzUhx9+6BjLzs5WRESEfHx8FBgYqHnz5qm6ulqSlJmZqY8++qjR9w0AuD8RwAEAaGXsdrvsdrth66WmpurLL780bL0b5s6dK5vNpgkTJqi0tFTR0dG6cOGC8vPzZbPZNHXq1DrVmTVrluMzs9vtOnLkiKZNmyZPT09JUlJSkt566y1t3bpVV65c0bp161RcXCzp5+eWjxs3To899phKSkq0Y8cOvf/++1q3bp0k6emnn9aFCxe0YMGCpvkQAAD3FQI4AACtzKZNm/Tuu+8att6WLVsMW+uGjRs3Kjs723GmPzw8XEuWLJGnp6f8/f01ZcoUZWdnO85E346bm9tNn9Wbb77pCMyVlZVatmyZli1bpoiICJnNZo0dO1bx8fGSpPPnz+vs2bOaNGmSfHx8FBYWpv79+ysvL89Rb/bs2dq3b582bdrUiJ8AAOB+RAAHAKAV+eCDD+Tt7a2nnnpKkrR48WK5uLjIYrHoiSeekJeXl8LDw3Xx4kVJ0qJFi2QymTRy5Ej5+/urY8eOslgsunbtmiQpJiZGJpNJO3bskCRFRUXJxcXFESZjYmK0d+9ejRgxQiEhIZIkq9Wq4cOHN+k+V69eLavVetvxsrIyeXh4yNXVtV51jx07ppqaGgUHB0uScnNzdf78eQ0dOvSW8zt37qyBAwcqMzNT5eXlOnr0qI4cOaLRo0fXmmexWLR69ep69QIAaH4I4AAAtCLR0dGaN2+e4/Xrr7+u0NBQhYWFadeuXTp69Kh++OEHZWZmSpJWrFihwMBARUZG6tSpU9q9e7d27typlStXSpLS09PVqVMnR72srCxHOL0x7u3trV27dqmwsFCSVF1drZqamibbY2FhofLz89W3b9+bxioqKnTgwAGtXbtW0dHR9a79y7PfklRUVCTp5180dO7cWV5eXho0aJAKCgokSSaTSVu3btU777wjLy8vhYaGaty4cYqMjKxVNywsTHl5eTp27Fi9ewIANB8EcAAAILPZLDc3NwUFBalHjx4qLS2tNR4QECCz2aywsDDFxsZq+/btDV5rw4YN+uqrr+615ds6fvy4pJ/PPv/a8uXLNWbMGA0dOlRJSUn1qnvy5EkVFRUpPDzccezGLxJee+01HT9+XMXFxfL19XWcfS8rK9OwYcO0fPlyXblyRXl5edq7d6/ee++9WrVv9HqjdwBAy0QABwAA9dKlSxedPXvW2W3c1o3L481m801jAQEB+uSTT5SUlCQPD4961V25cqXmzp1b65i/v78kqVevXvL19VWHDh00bdo05eTkqLq6Wp9//rlKS0v1wgsvyMfHRz179pTVatXbb79dq46Xl1et3gEALRMBHAAA1MvJkycVGBjo7DZuy9vbW9LPN0j7tVmzZmnYsGH1rvnTTz/p4MGDGjt2bK3jffr0kdls1rfffus4ZrPZ5OnpKVdXV1VWVspkMt1Ur6KiotbrG73eCOIAgJaJAA4AAO7q6tWrqqqqUk5OjtLT0zV58mTHmK+vr/bv3y+bzaaSkhJdv3691nvd3d2Vn5+v8vJy1dTUyGKxNCgE11X37t0lSefOnbtpbNy4cZo0aVK9ayYmJurFF1+8KUy3bdtW06dP16uvvqoff/xR586dU1pamuMmd+Hh4bp69arWr1+va9eu6dixY0pLS9PgwYNr1SkpKZEkPfTQQ/XuDQDQfBDAAQBoRf70pz8pMTFRf/nLXzRx4kQtXrxYBQUFWrRokQ4cOKCEhATl5ORo6dKl+v777x3vmzdvnjw9PTVx4kRZrVbNmTPHMRYfH6+UlBQFBQVp6dKlcnV11cKFCx2P2oqOjtbLL7+sfv36GXKJdUhIiHr16qXc3Ny7zq2oqFB4eLgSEhJuO6e0tFRZWVl6/vnnbzmemJioAQMGqG/fvgoJCZG/v7+Sk5MlSQ8//LA++OADrV27Vn5+fhoyZIgGDhzouIndDbm5uerVq5ceeeSReuwUANDcmOx2u93ZTQAA7k1iYqIKCgqUmprq7FZgoJ49eyo1NVURERFNuk63bt2UnJysCRMmNOk6dWGxWBQaGqr58+ffcd7mzZu1efPmOt3s7eDBg4qPj9e+ffsaq816Gzx4sOLi4jR16tS7zs3Pz9fo0aMdd2AHADQbWZwBBwAAd9WUjw1rCtOmTVPv3r2VkpJyx3mXL1/W4sWLNWrUKIM6u1lycrL69+9fp/ANAGjeCOAA0IqNGTNGrq6uevPNN53dSqNISUlR+/btZTKZHD9eXl4KCQnRnDlzdOLECWe32OzExcWpuLhYVqtVmzZtcnY79bJmzRq5u7srIyPjtnMyMjLUu3dvvfLKKwZ29v+2bdsmPz8/rVq1yinrAwCM5ebsBgAAzrNz504NGTLE2W3cUWpqqh544AENHz78rnPnzJmjwMBAjR8/XkVFRfL399eJEye0e/duLVu2TFu3btXXX3+t3/3udwZ0Xn/12atR0tLSlJaW5uw2GmzmzJl3HJ8+fbpBndza+PHjnbo+AMBYnAEHgFbOxeX+/qdgy5YtDX6vu7u7evTooVmzZunw4cPy8PCodfOw+8297BUAANz/7u//6wIANLqcnBwNGjRIPj4+ateunQ4cOCBJWrBggVxcXPT+++/rj3/8owYMGCBJOnz4sB5//HH5+voqKChICQkJjmcWL1q0SCaTSSNHjpS/v786duwoi8VS607Xd3p/TEyMTCaTduzYIUmKioqSi4uL41LnmJgY7d27VyNGjFBISIgkyWq1NugMsb+/v/7whz/om2++0dmzZ+u9X2fsFQAAtCwEcABoRcrKyjRmzBj17dtXxcXFKikpUVhYmCRp9erVCg0N1bFjxzRz5kwNHjxYZWVlevLJJzVy5EidPn1aGRkZ2rJli9asWSNJWrFihQIDAxUZGalTp05p9+7d2rlzp+MRS3d7f3p6ujp16uToLysrS8HBwY7X6enp8vb21q5du1RYWChJqq6ubvANwR599FFJUlFRUb3364y9AgCAloXvgANAK7Jr1y5duXJFb7zxhry8vCT9fJn2Lz344IN64IEH9NZbb+mzzz7TpUuXlJCQIHd3d/Xr108zZsxQRkaGFi1a5HhPQECAzGazwsLCFBsbq+3bt2vJkiXas2dPnd5fHxs2bGjw/svLyyXVvuy+vvs1cq93U11drYMHD+ry5ctNUv9+9M9//lOurq764osvnN2K05w8eVLV1dXObgMA0AAEcABoRYqLi+Xv7+8I33dTUlKi9u3b1wrpAQEBOnPmzG3f06VLF509e7bB729K//jHP2QymfTggw/ecry+/Tp7rxUVFUpNTVXbtm2bbI37zalTp1RQUKD/+Z//cXYrTlNeXi6bzebsNgAADUAAB4BWpFOnTrpw4YKqq6vl6up61/kBAQEqLS2VzWZzBMszZ86oa9eut33PyZMnFRgY2OD3N5Wqqipt3LhRQ4YMUfv27W85p779OnuvXl5eWr9+vSIiIppsjfuNxWJRaGio5s+f7+xWnCY/P1+jR492dhsAgAbgO+AA0IoMGTJE1dXVWrp0qa5cuaJDhw7p73//+23nDx48WO3atdMbb7yhy5cv69ChQ0pLS9Ozzz5ba97Vq1dVVVWlnJwcpaena/LkyXV+v6+vr/bv3y+bzaaSkhJdv369Vm13d3fl5+ervLxcNTU1slgsGjZs2F33ev36ddntdtlsNn333XcaOXKkLl68qKSkpHvar5F7BQAALYwdANDsrV692h4XF1enuZ9++qk9ODjY7u3tbX/iiSfsffv2tbdr187++9//3m4ymey+vr72d9991zE/JyfH/thjj9m9vb3tQUFB9ldeecVeWVnpGA8MDLR36NDB7ubmZv/Nb35j/+Mf/2i32Wx1fn9KSordx8fHHhAQYH/hhRfs3bt3t/v5+dn/93//12632+2zZ8+2t2nTxh4aGmq/cuWKPS4uzj506NBb7m39+vX2wMBAu9lstru6utol2d3d3e2//e1v7XFxcfbjx4875i5YsKDe+zV6r3cTGhpq/+abb+46ryWJi4uzr1692tltOFVeXp69W7duzm4DAFB/mSa73W539i8BAAD3JjExUQUFBUpNTTV87W7duik5OVkTJkwwfG2j3W977dmzp1JTU7kEvZW5cQl6UVGRs1sBANRPFpegAwDuWWu6XLo17RUAADQuAjgAoMHi4uJUXFwsq9WqTZs2ObudJtWa9tqSrF+/Xp9++qkKCwsVHBwsT09P+fn5KSoqSidOnLjr+6uqqmQymW768fDwcDzWTpKSk5PVvXt3tWnTRoGBgfrwww8dY9nZ2YqIiJCPj48CAwM1b948x2PEMjMz9dFHHzX6vgEA9ycCOACgwdLS0mS323Xx4kXFxsY6u50m1Zr2+mupqan68ssv79t6tzN37lzZbDZNmDBBpaWlio6O1oULF5Sfny+bzaapU6fWqc6sWbNkt9sdP0eOHNG0adPk6ekpSUpKStJbb72lrVu36sqVK1q3bp2Ki4sl/XzFxLhx4/TYY4+ppKREO3bs0Pvvv69169ZJkp5++mlduHBBCxYsaJoPAQBwXyGAAwCAO9qyZct9Xe9WNm7cqOzsbM2ePVuSFB4eriVLlsjT01P+/v6aMmWKsrOzHWeib8fNzU3vvvturWNvvvmmIzBXVlZq2bJlWrZsmSIiImQ2mzV27FjFx8dLks6fP6+zZ89q0qRJ8vHxUVhYmPr376+8vDxHvdmzZ2vfvn1cWQEArQABHACAFu7w4cN6/PHH5evrq6CgICUkJKiyslKSFBMTI5PJpB07dkiSoqKi5OLi4giDMTEx2rt3r0aMGKGQkBAtWrRIJpNJI0eOlL+/vzp27CiLxaJr1641qJ4kWa1WDR8+vFH3vHr1almt1tuOl5WVycPDQ66urvWqe+zYMdXU1Cg4OFiSlJubq/Pnz2vo0KG3nN+5c2cNHDhQmZmZKi8v19GjR3XkyJGbnuNtsVi0evXqevUCAGh+COAAALRgZWVlevLJJzVy5EidPn1aGRkZ2rJli9asWSNJSk9PV6dOnRzzs7KyHOHyxri3t7d27dqlwsJCrVixQoGBgYqMjNSpU6e0e/du7dy5UytXrmxQPUmqrq5u1JvbFRYWKj8/X3379r1prKKiQgcOHNDatWsVHR1d79q/PPstyXEn8piYGHXu3FleXl4aNGiQCgoKJEkmk0lbt27VO++8Iy8vL4WGhmrcuHGKjIysVTcsLEx5eXk6duxYvXsCADQfBHAAAFqwPXv26NKlS0pISJCvr6/69eunGTNmKCMj457qBgQEyGw2KywsTLGxsdq+fXuDa23YsEFfffXVPfXzS8ePH5f089nnX1u+fLnGjBmjoUOHKikpqV51T548qaKiIoWHhzuO3fjFwWuvvabjx4+ruLhYvr6+jrPvZWVlGjZsmJYvX64rV64oLy9Pe/fu1XvvvVer9o1eb/QOAGiZCOAAALRgJSUlat++vdzd3R3HAgICdObMmUZbo0uXLjp79myj1btXNy6HN5vNN40FBATok08+UVJSkjw8POpVd+XKlZo7d26tY/7+/pKkXr16ydfXVx06dNC0adOUk5Oj6upqff755yotLdULL7wgHx8f9ezZU1arVW+//XatOl5eXrV6BwC0TARwAABasICAAJWWlspmszmOnTlzRl27dm20NU6ePKnAwMBGq3evvL29JcnxPfdfmjVrloYNG1bvmj/99JMOHjyosWPH1jrep08fmc1mffvtt45jNptNnp6ecnV1VWVlpUwm0031Kioqar2+0euNIA4AaJkI4AAAtGCDBw9Wu3bt9MYbb+jy5cs6dOiQ0tLS9Oyzzzrm+Pr6av/+/bLZbCopKdH169dr1XB3d1d+fr7Ky8sdl1xfvXpVVVVVysnJUXp6uiZPntzgehaLpUGh+Ha6d+8uSTp37txNY+PGjdOkSZPqXTMxMVEvvvjiTWG6bdu2mj59ul599VX9+OOPOnfunNLS0vTUU09J+vnu61evXtX69et17do1HTt2TGlpaRo8eHCtOiUlJZKkhx56qN69AQCaDwI4AAAtmLe3tz777DN98cUX6tq1q5555hnFxsbqpZdecsyJj49XSkqKgoKCtHTpUrm6umrhwoWOR2VFR0fr5ZdfVr9+/RyXSM+bN0+enp6aOHGirFar5syZc0/1GlNISIh69eql3Nzcu86tqKhQeHi4EhISbjuntLRUWVlZev755285npiYqAEDBqhv374KCQmRv7+/kpOTJUkPP/ywPvjgA61du1Z+fn4aMmSIBg4c6Lhp3Q25ubnq1auXHnnkkXrsFADQ3Jjsdrvd2U0AAO5NYmKiCgoKlJqa6uxWYKCePXsqNTVVERERhq7brVs3JScna8KECYauK/38uK7Q0FDNnz//jvM2b96szZs31+nmbgcPHlR8fLz27dvXWG3W2+DBgxUXF6epU6fedW5+fr5Gjx7tuAM7AKDZyOIMOAAAqLfGfGxYU5g2bZp69+6tlJSUO867fPmyFi9erFGjRhnU2c2Sk5PVv3//OoVvAEDzRgAHAAB1FhcXp+LiYlmtVm3atMnZ7dzRmjVr5O7ufsdHrmVkZKh379565ZVXDOzs/23btk1+fn5atWqVU9YHABjLzdkNAACA5iMtLU1paWnObqPOZs6cecfx6dOnG9TJrY0fP96p6wMAjMUZcAAAAAAADEAABwAAAADAAARwAAAAAAAMQAAHAAAAAMAAPAccAFqAxMRELViwwNltADBIt27deA44ADQ/WQRwAGgBKisrdf36dWe3AYN9/vnnSklJ0WeffebsVmAwFxcX+fj4OLsNAED9ZPEYMgBoAdq0aaM2bdo4uw0YzMvLS25ubvL19XV2KwAAoA74DjgAAAAAAAYggAMAAAAAYAACOAAAAAAABiCAAwAAAABgAAI4AAAAAAAGIIADAAAAAGAAAjgAAAAAAAYggAMAAAAAYAACOAAAAAAABiCAAwAAAABgAAI4AAAAAAAGIIADAAAAAGAAAjgAAAAAAAYggAMAAAAAYAACOAAAAAAABiCAAwAAAABgAAI4AAAAAAAGIIADAAAAAGAAAjgAAAAAAAYggAMAAAAAYAACOAAAAAAABiCAAwAAAABgAAI4AAAAAAAGIIADAAAAAGAAAjgAAAAAAAZwc3YDAACgbi5evKghQ4aovLxcklRZWalr164pODhYkuTi4qL/+I//0Jw5c5zZJgAAuA0COAAAzUSHDh3k4uKiH374odbxc+fOSZLc3Nw0YMAAZ7QGAADqgEvQAQBoRqxWq7y9vW851rlzZ/Xr18/gjgAAQF0RwAEAaEaee+45VVRU3HTcbDYrLi5OJpPJCV0BAIC6IIADANCM+Pn56fe///0txyZPnmxwNwAAoD4I4AAANDNxcXHy8fGpdSwoKEihoaFO6ggAANQFARz/ySOYAAAT70lEQVQAgGZm3LhxqqysdLz29PSUxWJxYkcAAKAuCOAAADQzbdu21ahRo+Ti8vM/41VVVXruueec3BUAALgbAjgAAM3QjBkz5OXlJUl69NFHFRQU5OSOAADA3RDAAQBohsaMGaOamhq1adOGy88BAGgmCOAAADRDZrNZEyZMUHV1tZ599llntwMAAOrAzdkNAACMN3DgQB09etTZbeBXampqZLfb5erqWqf51dXVMplM6t69exN31nTsdruqqqrk7u7u7Facbtu2bRo2bJiz2wAANCECOAC0QmVlZUpPT1efPn2c3Qp+Yf369frb3/6mt956q07zq6ur9c033+jxxx9v4s6azuHDh7Vw4UJ99dVXzm7FqZ5++mlVVVU5uw0AQBMjgANAK9W5c2d169bN2W3gF9q1aydvb+96/Xdp7jdfKyoqkru7e6v/s9imTRtntwAAMADfAQcAAAAAwAAEcAAAAAAADEAABwAAAADAAARwAAAAAAAMQAAHAKAFmD59umbPnu3sNgyxfv16ffrppyosLFRwcLA8PT3l5+enqKgonThx4q7vr6qqkslkuunHw8ND5eXljnnJycnq3r272rRpo8DAQH344YeOsezsbEVERMjHx0eBgYGaN2+eqqurJUmZmZn66KOPGn3fAIDmjwAOAEALYLfbZbfbDVkrNTVVX375pSFr/drcuXNls9k0YcIElZaWKjo6WhcuXFB+fr5sNpumTp1apzqzZs1yfGZ2u11HjhzRtGnT5OnpKUlKSkrSW2+9pa1bt+rKlStat26diouLJf38vPZx48bpscceU0lJiXbs2KH3339f69atk/TzI8UuXLigBQsWNM2HAABotgjgAAC0AJs2bdK7775ryFpbtmwxZJ1f27hxo7Kzsx1n+sPDw7VkyRJ5enrK399fU6ZMUXZ2tuNM9O24ubnd9Fm9+eabjsBcWVmpZcuWadmyZYqIiJDZbNbYsWMVHx8vSTp//rzOnj2rSZMmycfHR2FhYerfv7/y8vIc9WbPnq19+/Zp06ZNjfgJAACaOwI4AADN3AcffCBvb2899dRTkqTFixfLxcVFFotFTzzxhLy8vBQeHq6LFy9KkhYtWiSTyaSRI0fK399fHTt2lMVi0bVr1yRJMTExMplM2rFjhyQpKipKLi4u2rRpk2JiYrR3716NGDFCISEhkiSr1arhw4c3+T5Xr14tq9V62/GysjJ5eHjI1dW1XnWPHTummpoaBQcHS5Jyc3N1/vx5DR069JbzO3furIEDByozM1Pl5eU6evSojhw5otGjR9eaZ7FYtHr16nr1AgBo2QjgAAA0c9HR0Zo3b57j9euvv67Q0FCFhYVp165dOnr0qH744QdlZmZKklasWKHAwEBFRkbq1KlT2r17t3bu3KmVK1dKktLT09WpUydHvaysLEc4TU9Pl7e3t3bt2qXCwkJJUnV1tWpqapp0j4WFhcrPz1ffvn1vGquoqNCBAwe0du1aRUdH17v2L89+S1JRUZGkn38R0blzZ3l5eWnQoEEqKCiQJJlMJm3dulXvvPOOvLy8FBoaqnHjxikyMrJW3bCwMOXl5enYsWP17gkA0DIRwAEAaKHMZrPc3NwUFBSkHj16qLS0tNZ4QECAzGazwsLCFBsbq+3btzdonQ0bNuirr75qjJZv6/jx45J+Pvv8a8uXL9eYMWM0dOhQJSUl1avuyZMnVVRUpPDwcMexG79MeO2113T8+HEVFxfL19fXcfa9rKxMw4YN0/Lly3XlyhXl5eVp7969eu+992rVvtHrjd4BACCAAwAAdenSRWfPnnV2G7d14/J4s9l801hAQIA++eQTJSUlycPDo151V65cqblz59Y65u/vL0nq1auXfH191aFDB02bNk05OTmqrq7W559/rtLSUr3wwgvy8fFRz549ZbVa9fbbb9eq4+XlVat3AAAI4AAAQCdPnlRgYKCz27gtb29vST/fIO3XZs2apWHDhtW75k8//aSDBw9q7NixtY736dNHZrNZ3377reOYzWaTp6enXF1dVVlZKZPJdFO9ioqKWq9v9HojiAMAQAAHAKCVunr1qqqqqpSTk6P09HRNnjzZMebr66v9+/fLZrOppKRE169fd4y5u7srPz9f5eXlqqmpkcViaVAAro/u3btLks6dO3fT2Lhx4zRp0qR610xMTNSLL754U5hu27atpk+frldffVU//vijzp07p7S0NMdN7sLDw3X16lWtX79e165d07Fjx5SWlqbBgwfXqlNSUiJJeuihh+rdGwCgZSKAAwDQzP3pT39SYmKi/vKXv2jixIlavHixCgoKtGjRIh04cEAJCQnKycnR0qVL9f333zveN2/ePHl6emrixImyWq2aM2eOYyw+Pl4pKSkKCgrS0qVL5erqqoULFyovL0/R0dF6+eWX1a9fP8Murw4JCVGvXr2Um5t717kVFRUKDw9XQkLCbeeUlpYqKytLzz///C3HExMTNWDAAPXt21chISHy9/dXcnKyJOnhhx/WBx98oLVr18rPz09DhgzRwIEDHTexuyE3N1e9evXSI488Uo+dAgBaMpPdbrc7uwkAgLF69uyp1NRURUREOLsV/EJiYqIKCgqUmpra5Gt169ZNycnJmjBhQpOvdSfZ2dmyWCzKz8+/69zNmzdr8+bNdbrh28GDBxUfH699+/Y1RpsNMnjwYMXFxWnq1Kl3nRsREaElS5Zo1KhRBnQGAHCSLM6AAwDQSjX1o8Ma27Rp09S7d2+lpKTccd7ly5e1ePFip4bZ5ORk9e/fv07hGwDQehDAAQAtUkpKitq3by+TyeT48fLyUkhIiObMmaMTJ044u0WniYuLU3FxsaxWqzZt2uTsduplzZo1cnd3V0ZGxm3nZGRkqHfv3nrllVcM7Oz/bdu2TX5+flq1apVT1gcA3L/cnN0AAKDlSk1N1QMPPKDhw4cbXm/OnDkKDAzU+PHjVVRUJH9/f504cUK7d+/WsmXLtHXrVn399df63e9+1yi9NSdpaWlKS0tzdhsNNnPmzDuOT58+3aBObm38+PFOXR8AcP/iDDgAoMls2bLlvqnn7u6uHj16aNasWTp8+LA8PDxq3XQMAACgqRHAAQC3dfjwYT3++OPy9fVVUFCQEhISHM82jomJkclk0o4dOyRJUVFRcnFxcVzSHBMTo71792rEiBEKCQnRokWLZDKZNHLkSPn7+6tjx46yWCyOu2jXt54kWa3WBp1d9/f31x/+8Ad98803Onv2rCRpz5496tOnj3x8fPTYY48pPz9fixcvlouLiywWi5544gl5eXkpPDxcFy9elCRVV1dr5syZateundq2bavo6GjHGreqBwAAWjcCOADglsrKyvTkk09q5MiROn36tDIyMrRlyxatWbNGkpSenq5OnTo55mdlZSk4ONjxOj09Xd7e3tq1a5cKCwu1YsUKBQYGKjIyUqdOndLu3bu1c+dOx6Ob6ltP+jkAN/RGYo8++qgkqaioSJcuXdKECRM0ZcoUnTlzRhEREYqJidHrr7+u0NBQhYWFadeuXTp69Kh++OEHZWZmSpJ27typAwcOqKioSMePH1eHDh0k6bb1AABA68Z3wAEAt7Rnzx5dunRJCQkJcnd3V79+/TRjxgxlZGRo0aJFDa4bEBAgs9mssLAwxcbGavv27VqyZEmDam3YsKHBfZSXl0uSXFxc9Ne//lVlZWV66aWX5OLiomnTpikxMVEXLlyQJJnNZrm5uSkoKEg9evRQaWmpJMnHx0d///vf9ec//1nR0dGOu3PfqV7Hjh3v2Ne+ffv07LPPNnhfzc2//vUvlZSUtKo938rp06ed3QIAwAAEcADALZWUlKh9+/Zyd3d3HAsICNCZM2cabY0uXbo4LgE32j/+8Q+ZTCY9+OCD+u6771RZWSlXV9dac25can47Q4YM0apVq7Ry5UrNnz9f8fHx+s///E/99NNPt613twDetWtXjR49umGbaoaOHz+uv/3tb61qz7eSl5fn7BYAAAYggAMAbikgIEClpaWy2WyOEH7mzBl17dq10dY4efKkAgMDG61eXVVVVWnjxo0aMmSI2rdvLz8/P/n4+Ojy5csymUz1qjV79mzNnj1bO3fuVFRUlCIjI++p3kMPPaS4uLh6vac5y87OVmZmZqva862kpqY6uwUAgAH4DjgA4JYGDx6sdu3a6Y033tDly5d16NAhpaWl1bpU2NfXV/v375fNZlNJSYmuX79eq4a7u7vy8/NVXl7u+K721atXVVVVpZycHKWnp2vy5MkNrmexWDRs2LC77uX69euy2+2y2Wz67rvvNHLkSF28eFFJSUmSpEGDBslkMum//uu/9K9//UuVlZUqKiq6a93Nmzfr448/1vXr1/Xoo4+qffv291QPAAC0bARwAMAteXt767PPPtMXX3yhrl276plnnlFsbKxeeuklx5z4+HilpKQoKChIS5culaurqxYuXOi4nDY6Olovv/yy+vXr57jb+bx58+Tp6amJEyfKarXWehRYQ+rdTmpqql588UWZzWaFhITIxcVF3t7eevrpp9W9e3cdOXLEcSO2Dh06aNu2bfr000/129/+VsHBwfr444+1ePFiFRQUaNGiRTpw4IASEhKUk5OjpUuX6vvvv1fXrl0VHx+vtm3bql+/fnrhhRfUp0+f29YDAACtm8lut9ud3QQAwFg9e/ZUamqqIiIiDF23W7duSk5O1oQJEwxdt7lITExUQUFBq7ocOTs7WxaLpdU/pi0iIkJLlizRqFGjnN0KAKDpZHEGHABgqIY+NgwAAKC5I4ADAAwRFxen4uJiWa1Wbdq0ydntoAVbv369li9fruDgYHl6esrPz09RUVE6ceJEver89NNPmjVrll588cVaxzMzM/XRRx81YscAgNaCAA4AMERaWprsdrsuXryo2NhYZ7fTqqWmpurLL7+8b+vdi7lz58pms2no0KGKjo7WhQsXlJ+fL5vNpqlTp9a5zvbt27V69Wrl5uaqqqqq1tjTTz+tCxcuaMGCBY3dPgCghSOAAwDQymzZsuW+rtdQGzduVHZ2tmbPnq3w8HAtWbJEnp6e8vf315QpU5Sdna3q6uo61YqMjNTKlSv1m9/85pbjs2fP1r59+7iaAwBQLwRwAACaocOHD+vxxx+Xr6+vgoKClJCQoMrKSsXExMhkMmnHjh2SpKioKLm4uDiCYkxMjPbu3asRI0YoJCREixYtkslk0siRI+Xv76+OHTvKYrE47jJf33qSZLVaNXz4cGM/EEmrV6+W1Wq95VhZWZk8PDzk6uraaOtZLBatXr260eoBAFo+AjgAAM1MWVmZnnzySY0cOVKnT59WRkaGtmzZojVr1ig9PV2dOnVyzM3KylJwcLDjdXp6ury9vbVr1y4VFhZqxYoVCgwMVGRkpE6dOqXdu3dr586dWrlypWN+fepJUnV1teE32yssLFR+fr769u1b63hFRYUOHDigtWvXKjo6ulHXDAsLU15eno4dO9aodQEALRcBHACAZmbPnj26dOmSEhIS5Ovrq379+mnGjBnKyMhocM2AgACZzWaFhYUpNjZW27dvb3CtDRs26Kuvvmrw+xvi+PHjkqTOnTvXOr58+XKNGTNGQ4cOVVJSUqOueWOtG2sDAHA3bs5uAAAA1E9JSYnat28vd3d3x7GAgACdOXOmUep36dJFZ8+ebZRaRrlxybzZbK51PCAgQJ988omGDRvW6Gt6eXnVWhsAgLshgAMA0MwEBASotLRUNpvNEcLPnDmjrl27Nkr9kydPKjAwsFFqGcXb21uSVFlZWev4rFmzmmzNG2vdCOIAANwNl6ADANDMDB48WO3atdMbb7yhy5cv69ChQ0pLS9Ozzz4rSfL19dX+/ftls9lUUlKi69ev13q/u7u78vPzVV5e7viu9tWrV1VVVaWcnBylp6dr8uTJjvn1rWexWJrkjPOddO/eXZJ07ty5WsfHjRunSZMmNcmaJSUlkqSHHnqoSeoDAFoeAjgAAM2Mt7e3PvvsM33xxRfq2rWrnnnmGcXGxuqll16SJMXHxyslJUVBQUFaunSpXF1dtXDhQuXl5UmSoqOj9fLLL6tfv36Oy6fnzZsnT09PTZw4UVarVXPmzHGs15B6RgsJCVGvXr2Um5t717kVFRUKDw9XQkLCbee8/vrr6tGjh7Zt26bU1FSFhIQoJSWl1pzc3Fz16tVLjzzyyD33DwBoHUx2u93u7CYAAMbq2bOnUlNTFRER4exW8AuJiYkqKChQamqqoet269ZNycnJmjBhgqHrSlJ2drYsFovy8/PvudbmzZu1efPmOt0A7uDBg4qPj9e+ffsavN7gwYMVFxenqVOnNrjGDREREVqyZIlGjRp1z7UAAPetLM6AAwAAwx8b1hSmTZum3r1733Sm+tcuX76sxYsX31PYTU5OVv/+/RslfAMAWg8COAAArVhcXJyKi4tltVq1adMmZ7dzz9asWSN3d/c7PpItIyNDvXv31iuvvNKgNbZt2yY/Pz+tWrWqoW0CAFop7oIOAEArlpaWprS0NGe30ahmzpx5x/Hp06ffU/3x48ff0/sBAK0XZ8ABAAAAADAAARwAAAAAAAMQwAEAAAAAMADfAQeAVmrHjh0qKChwdhv4hYMHD+rMmTMt7jvZd3L8+HGVlpa2qj3fytmzZ53dAgDAADwHHABaofnz56uoqMjZbeBXbDabampqZDabnd2KYaqrq1VRUSEvLy9nt+J0r776qnr37u3sNgAATSeLAA4AAAAAQNPL4jvgAAAAAAAYgAAOAAAAAIAB3CT96OwmAAAAAABo4X76PynHupvTgKC5AAAAAElFTkSuQmCC\n", + "text/plain": [ + "" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 18 + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "OcREcgPUHr9O", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 146 + }, + "outputId": "0a193357-3361-4c8f-9268-a21d106abf62" + }, + "source": [ + "# Train model\n", + "epochs = 4\n", + "history = model.fit(train_data,\n", + " validation_data=valid_data,\n", + " epochs=epochs,\n", + " verbose=1)" + ], + "execution_count": null, + "outputs": [ + { + "output_type": "stream", + "text": [ + "Epoch 1/2\n", + "306/306 [==============================] - ETA: 0s - loss: 0.1679 - binary_accuracy: 0.9391WARNING:tensorflow:Callbacks method `on_test_batch_end` is slow compared to the batch time (batch time: 0.0122s vs `on_test_batch_end` time: 0.1396s). Check your callbacks.\n" + ], + "name": "stdout" + }, + { + "output_type": "stream", + "text": [ + "WARNING:tensorflow:Callbacks method `on_test_batch_end` is slow compared to the batch time (batch time: 0.0122s vs `on_test_batch_end` time: 0.1396s). Check your callbacks.\n" + ], + "name": "stderr" + }, + { + "output_type": "stream", + "text": [ + "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r306/306 [==============================] - 147s 480ms/step - loss: 0.1679 - binary_accuracy: 0.9391 - val_loss: 0.1348 - val_binary_accuracy: 0.9531\n", + "Epoch 2/2\n", + "306/306 [==============================] - 146s 478ms/step - loss: 0.1040 - binary_accuracy: 0.9608 - val_loss: 0.1600 - val_binary_accuracy: 0.9563\n" + ], + "name": "stdout" + } + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "kNZl1lx_cA5Y" + }, + "source": [ + "## Task 11: Evaluate the BERT Text Classification Model" + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "dCjgrUYH_IsE" + }, + "source": [ + "import matplotlib.pyplot as plt\n", + "\n", + "def plot_graphs(history, metric):\n", + " plt.plot(history.history[metric])\n", + " plt.plot(history.history['val_'+metric], '')\n", + " plt.xlabel(\"Epochs\")\n", + " plt.ylabel(metric)\n", + " plt.legend([metric, 'val_'+metric])\n", + " plt.show()" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "v6lrFRra_KmA", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 279 + }, + "outputId": "dd4cfcf2-f6f1-4e92-c17c-e50291e0bb09" + }, + "source": [ + "plot_graphs(history, 'binary_accuracy')" + ], + "execution_count": null, + "outputs": [ + { + "output_type": "display_data", + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY8AAAEGCAYAAACdJRn3AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAA3rUlEQVR4nO3deViV1fbA8e8SRVAUFZxR0RxxysQhtTJvZmlpal4rG2ywwaFu3UqzulnqTxtug2WDlpZl10ycMytN07ISnABxRlNwwglFQKb1++McjQyUoxwOw/o8jw/nHc/aMqyz9/u+a4uqYowxxriilKcDMMYYU/RY8jDGGOMySx7GGGNcZsnDGGOMyyx5GGOMcVlpTwdQUAIDAzU4ONjTYRhjTJGybt26I6pa9fz1JSZ5BAcHExER4ekwjDGmSBGRP3Jab8NWxhhjXGbJwxhjjMsseRhjjHFZibnmkZP09HTi4uJITU31dCimEPLx8SEoKIgyZcp4OhRjCp0SnTzi4uKoUKECwcHBiIinwzGFiKpy9OhR4uLiqF+/vqfDMabQKdHDVqmpqQQEBFjiMH8jIgQEBFiv1JhclOjkAVjiMLmynw1jclfik4cxxhRXOw8nMeHbLbhj6o0Sfc3DGGOKo/TMLKasiuWdZTvw9fZiUPt61A0ol6/vYT0PD9uzZw8tWrT42/qHHnqImJgYD0RkjCnKouMT6fPeL7z+3Ta6h1Rn2VPX5XviAOt5FFoff/xxvpwnIyOD0qUL57c5MzMTLy8vT4dhTLGQmp7JO8t3MGVVLFXKe/Ph3W25qUUNt71f4fyr4gEvL9pMzP6T+XrOkFoVeenW5hfdLyMjg0GDBrF+/XqaN2/OjBkz6NmzJ2+88QahoaH4+fnxxBNPsHjxYnx9fVmwYAHVq1dn0aJFjBs3jrS0NAICApg5cybVq1dnzJgx7Nq1i9jYWOrWrUt8fDyTJk3iyiuvBKBLly5MnjyZ1q1b/y2WtWvX8sQTT5Camoqvry/Tp0+nSZMmZGZmMnLkSJYuXUqpUqUYMmQII0aMIDw8nCeeeILTp09TtmxZli9fTlhYGBEREbz33nsA3HLLLTz99NN07doVPz8/HnnkEZYtW8bkyZP58ccfWbRoESkpKXTq1ImPPvoIEWHnzp08+uijJCQk4OXlxddff83LL79Mv379uO222wAYNGgQ//znP+nTp0++fc+MKYrC9xxj5JxIYo+c5p+hQTzfMwT/cu59Psntw1YicpOIbBORnSIyKoft9URkuYhEishKEQnKtq2uiHwvIltEJEZEgp3r64vI785zfiUi3u5uhztt27aNoUOHsmXLFipWrMj777//l+2nT5+mY8eObNq0iWuvvZapU6cCjiTw22+/sWHDBu644w5ee+21c8fExMSwbNky/ve///Hggw/y6aefArB9+3ZSU1NzTBwATZs2ZfXq1WzYsIFXXnmF0aNHAzBlyhT27NnDxo0biYyMZNCgQaSlpTFw4EDeeecdNm3axLJly/D19b1gW0+fPk2HDh3YtGkTXbp0Yfjw4YSHhxMdHU1KSgqLFy8GHIlh2LBhbNq0iTVr1lCzZs2/tCMxMZE1a9bQq1cvl/+/jSkuks5k8J8F0Qz48FfSMrP44sEOvHZ7a7cnDnBzz0NEvIDJQHcgDggXkYWqmn0w/w1ghqp+JiLdgAnAPc5tM4DxqvqDiPgBWc71rwJvqeosEfkQeBD44HJizUsPwV3q1KlD586dAbj77ruZNGnSX7Z7e3tzyy23ANC2bVt++OEHwPGQ48CBAzlw4ABpaWl/eZitd+/e5/6QDxgwgLFjx/L6668zbdo0Bg8enGssiYmJ3HfffezYsQMRIT09HYBly5bx6KOPnhsCq1KlClFRUdSsWZN27doBULFixYu21cvLi/79+59bXrFiBa+99hrJyckcO3aM5s2b07VrV+Lj4+nbty/geNIb4LrrrmPo0KEkJCQQFhZG//79C+2QnDHutnLbYZ6fF83+xBTu7xzM0zc2oXzZgvt9cHfPoz2wU1VjVTUNmAWcP8YQAvzofL3i7HYRCQFKq+oPAKqapKrJ4rj5vhswx3nMZ8Btbm2Fm53/PMH5y2XKlDm3zsvLi4yMDABGjBjB8OHDiYqK4qOPPvrLA23ly5c/97pcuXJ0796dBQsWMHv2bAYNGpRrLC+++CLXX3890dHRLFq06JIekitdujRZWVnnlrOfw8fH59x1jtTUVIYOHcqcOXOIiopiyJAhF32/e++9ly+++ILp06fzwAMPuBybMUXd8dNpPDV7I4Onh+Pr7cWcRzvx0q3NCzRxgPuTR21gX7blOOe67DYB/Zyv+wIVRCQAaAycEJG5IrJBRF539mQCgBOqmnGBcwIgIg+LSISIRCQkJORTk/Lf3r17+fXXXwH48ssv6dKlS56OS0xMpHZtR9M/++yzC+770EMP8fjjj9OuXTsqV66cp3OeHSIC6N69Ox999NG5xHXs2DGaNGnCgQMHCA8PB+DUqVNkZGQQHBzMxo0bycrKYt++faxduzbH9zqbKAIDA0lKSmLOHMfngQoVKhAUFMT8+fMBOHPmDMnJyQAMHjyYt99+G4CQkJALttmY4kRV+SbyAN3f+omFG/fzeLeGfPN4F9rWy/332Z0Kw626TwPXicgG4DogHsjEMaR2jXN7O6ABMNiVE6vqFFUNVdXQqlX/NhFWodGkSRMmT55Ms2bNOH78OI899liejhszZgwDBgygbdu2BAYGXnDftm3bUrFiRe6///4L7vfss8/y3HPP0aZNm3OJAhzJp27durRq1YrWrVvz5Zdf4u3tzVdffcWIESNo3bo13bt3JzU1lc6dO1O/fn1CQkJ4/PHHueqqq3J8r0qVKjFkyBBatGhBjx49zg1/AXz++edMmjSJVq1a0alTJw4ePAhA9erVadas2UXbYUxxcvhkKo98vo5hX66npr8vC4d34akbm1C2tOfuVhR3PHl47uQiVwNjVLWHc/k5AFWdkMv+fsBWVQ0SkY7Aq6p6nXPbPUBHYDiQANRQ1Yzz3yM3oaGhev5Mglu2bKFZs2aX1caiYv/+/XTt2pWtW7dSqlRh+MxwaZKTk2nZsiXr16/H39/f7e9Xkn5GTOGjqnwdEcfYb2JIy8jiqe6NebBLfUp7FdzvsIisU9XQ89e7O4JwoJHz7ihv4A5g4XmBBYrI2TieA6ZlO7aSiJztMnQDYtSR7VYAtzvX3wcscGMbirwZM2bQoUMHxo8fX6QTx7Jly2jWrBkjRowokMRhjCftO5bMPZ+s5dmwSJrVrMi3T1zDI9ddUaCJ40Lc2vMAEJGewNuAFzBNVceLyCtAhKouFJHbcdxhpcAqYJiqnnEe2x34LyDAOuBhVU0TkQY4Lr5XATYAd589JjclvedxvunTp/POO+/8ZV3nzp2ZPHmyhyIqnEryz4jxjMws5bM1e3j9u214lRJG3dyUu9rXpVQpzxTqzK3n4fbkUVhY8jCXwn5GTEHacegUI8MiWb/3BNc3qcr4vi2pVenCz065W27Jw26SN8YYD0vLyOKjn3bx7o87KV/Wi7cHXkmfK2sV6mkBLHkYY4wHRcad4Nk5kWw9eIpbW9fipVtDCPQr6+mwLsqShzHGeEBqeiZv/bCdqatjqVqhLFPvDaV7SHVPh5VnljyMMaaA/RZ7lFFhkew5msyd7esw6uZm+Pu6vx5Vfioc93yZPPHz88t128qVK8/Vvzpfz549OXHihJuiMsbk1anUdJ6fF8UdU34jS+HLhzowoV+rIpc4wHoeJcKSJUvy5TyFdW4QVUVVi/QzLKb4W7H1MKPnRXHoZCoPdanPUzc2ppx34ft9yquiG3l++3YUHIzK33PWaAk3T8x186hRo6hTpw7Dhg0DHOVGSpcuzYoVKzh+/Djp6emMGzcuz/NVnDx5kl69erFz506uv/563n//fUqVKkVwcDAREREkJSVx880306VLF9asWUPt2rVZsGABvr6+TJ06lSlTppCWlkbDhg35/PPPKVeuHIMHD8bHx4cNGzbQuXNnFi1axJo1a6hatSpZWVk0btyYX3/9lZzKv+Q230hSUhIjRowgIiICEeGll16if//+LF26lNGjR5OZmUlgYCDLly9nzJgx+Pn58fTTTwPQokWLc2Xbe/ToQYcOHVi3bh1Llixh4sSJhIeHk5KSwu23387LL78MkOOcI7169crzHCfGXI5jp9N4ZdFm5m/cT+Pqfrw/qBNt6nqmHlV+so9qHjRw4EBmz559bnn27Nncd999zJs3j/Xr17NixQr+/e9/53ny+rVr1/Luu+8SExPDrl27mDt37t/22bFjB8OGDWPz5s1UqlSJsLAwAPr160d4eDibNm2iWbNmfPLJJ+eOiYuLY82aNbz55pvcfffdzJw5E3A88d26descEwfkPt/I2LFj8ff3JyoqisjISLp160ZCQgJDhgwhLCyMTZs28fXXX1+0vTt27GDo0KFs3ryZevXqMX78eCIiIoiMjOSnn34iMjIy1zlHXJnjxJhLoaos3LSfG978iW+iDvDEPxqxeMQ1xSJxgPU8/nSBHoK7tGnThsOHD7N//34SEhKoXLkyNWrU4Mknn2TVqlWUKlWK+Ph4Dh06RI0aF59Osn379jRo0ACAO++8k59//pnbb7/9L/vUr1//3Kfttm3bsmfPHgCio6N54YUXOHHiBElJSfTo8WepsAEDBpwro/7AAw/Qp08f/vWvfzFt2rQLFijMbb6RZcuWMWvWrHP7Va5cmUWLFnHttdee26dKlSoXbW+9evXo2LHjueXZs2czZcoUMjIyOHDgADExMYhIjnOOuDLHiTGuOpiYygvzo1m25RCtg/x59fYONK1x8fluihJLHh42YMAA5syZw8GDBxk4cCAzZ84kISGBdevWUaZMGYKDg/M8p8bF5gUBKFv2z/vHvby8SElJARylzufPn0/r1q359NNPWbly5bn9ss8NUqdOHapXr86PP/7I2rVrz/VCcjJixAieeuopevfuzcqVKxkzZkye2pHdheYGyR7X7t27eeONNwgPD6dy5coMHjz4gv9v589xsm7dOpdjM+Z8qsqs8H383zdbSM/K4vmezXigS328PFRaxJ1s2MrDBg4cyKxZs5gzZw4DBgwgMTGRatWqUaZMGVasWMEff/yR53OtXbuW3bt3k5WVxVdffZXneUHAMRdHzZo1SU9Pv2BCAEd59rvvvvsvPZKc5DbfSPfu3f9SQ+v48eN07NiRVatWsXv3bsAxXwhAcHAw69evB2D9+vXntp/v5MmTlC9fHn9/fw4dOsS3334LkOucI2fbkZc5TozJiz+Onuauqb/z3NwomteuyNInrmXItQ2KZeIASx4e17x5c06dOkXt2rWpWbMmgwYNIiIigpYtWzJjxgyaNm2a53O1a9eO4cOH06xZM+rXr39uGte8GDt2LB06dKBz584Xfc/evXuTlJR00Tk1cptv5IUXXuD48eO0aNGC1q1bs2LFCqpWrcqUKVPo168frVu3ZuDAgQD079//3PS07733Ho0bN87xvVq3bk2bNm1o2rQpd91117lpfXObcwTyPseJMReSmaV8vDqWHm+vIjo+kQn9WvLlQx0JDix/8YOLMCuMaEXvXBYREcGTTz7J6tWrPR3KZcnLHCf2M2IuZNvBUzwbFsmmfSe4oVk1xt3Wkhr+Pp4OK19ZYUSTLyZOnMgHH3xw0aGtwm7GjBk8//zzvPnmm/Z8iHFZWkYW76/cyeQVO6ngU4ZJd7bh1lY1C3Uhw/xmPY8i9qkyKiqKe+655y/rypYty++//+6hiGD8+PF/u7V2wIABPP/88x6KKP8UxZ8R414b951g5JxIth06RZ8ra/HSrc2pUt7b02G5jc3nkUvyaNq0aYn6tGDyTlXZunWrJQ8DQEpaJm/+sI1Pft5NtQo+jO/bgn80KzqFDC+VDVvlwMfHh6NHjxIQEGAJxPyFqnL06FF8fIrX+LW5NGt2HWFUWBR7jyUzqENdRt7clIo+Ra8eVX4q0ckjKCiIuLg4EhISPB2KKYR8fHwICgrydBjGg06mpjNhyVb+t3YvwQHlmPVwRzo2CPB0WIVCiU4eZcqUOfdEszHGZLcs5hDPz48i4dQZHrm2Af+6oTG+3rk/11TSlOjkYYwx5zuadIYxi2JYtGk/TWtUYOq9obQKquTpsAodSx7GGMOfhQzHLNxM0pkMnuremEevuwLv0nYrd04seRhjSrz9J1J4YX40P249zJV1KvHa7a1oXL2Cp8Mq1Cx5GGNKrKws5cu1e5n47VYys5QXbwlhcKfgYluPKj9Z8jDGlEi7j5xmVFgkv+8+RueGAUzo24q6AeU8HVaRYcnDGFOiZGRm8cnPu3nzh+14ly7Fa/1bMSA0yJ71cpElD2NMibHlwElGhkUSGZdI95DqjLutBdUr2oOgl8KShzGm2DuTkcnkH3fy/spdVCpXhsl3XUXPljWst3EZLHkYY4q19XuPM3JOJDsOJ9GvTW1evCWEysW4kOFfpKdA0iGoVA/yOVFa8jDGFEvJaRm88d12pq/ZTc2KPky/vx3XN6nm6bAuX2YGJB9xJIWkw86v2V9n+3rmpOOY5+KgbP7eemzJwxhT7Py84wij5kYSdzyFe6+ux7M3NcWvbCH+c6cKqSdySAA5JIXTR4AcqqGXrQh+1cCvOtRo6fh6dlnyv6xKIf7fNMYY1ySmpDP+mxhmR8RRP7A8sx+5mvb1q3guoLPDRhfrISQdgsy0vx/v5f1nEqhUF4JC/5oUzr4uXw28C/Y2Y0sexphi4bvNB3lxfjRHT6fxWNcreOIfjfAp44ZChpcybPQXAuUD//zDH9j478ng7FefSvl+rSK/WPIwxhRpCafOMGbhZr6JOkCzmhX55L52tAzyd+0k7h42yv66XCB4Ff0/vUW/BcaYEklVmbchnlcWx5B8JpNnejTh4WsbUMYrWyHDfBk2quEcNqoHQe1ySQrVoIxvwTW+ELDkYYwpWjIzOHAwjvcXrWHf3t08FJjOne3KEpC6EubmddioarZhoyZ/HSrKnhR8/AvtsJGnWfIwxnheHoeN1DlsVBNlLIA3cBL4HSjrn23YqNUFho0CisWwkafZ/6Axxn3ycdjodLna/HKyLjEZPvhXDeLWTlcSWKNuiR028jS3Jw8RuQl4B/ACPlbViedtrwdMA6oCx4C7VTXOuS0TiHLuuldVezvXfwpcByQ6tw1W1Y3ubYkxBsinu43yPmyUkaVMWR3L28t24FO6FC/eFsLtba2Qoae5NXmIiBcwGegOxAHhIrJQVWOy7fYGMENVPxORbsAE4B7nthRVvTKX0z+jqnPcFLoxJUu+3G2U/8NGm/cnMjIskuj4k9zUvAav3NacahWskGFh4O6eR3tgp6rGAojILKAPkD15hABPOV+vAOa7OSZjSo4ierdRanom7/64gw9/iqVyOW8+GHQVN7esmW/nN5fP3cmjNrAv23Ic0OG8fTYB/XAMbfUFKohIgKoeBXxEJALIACaq6vxsx40Xkf8Ay4FRqnrm/DcXkYeBhwHq1q2bPy0yxtMKeNiooO82ithzjGfDIolNOM3tbYN4oVczKpUrIYUMi5DCcMH8aeA9ERkMrALigUzntnqqGi8iDYAfRSRKVXcBzwEHcdxrMQUYCbxy/olVdYpzO6GhoTn0s40pJArpsFFBOn0mg9e/28Znv+6hlr8vMx5oz7WNq3o6LJMLd/8ExQN1si0HOdedo6r7cfQ8EBE/oL+qnnBui3d+jRWRlUAbYJeqHnAefkZEpuNIQMYUPmnJcPrwZQwblc1W26j4PqS2ansCz82NYn9iCvddHcwzPZpQvjAXMjRuTx7hQCMRqY8jadwB3JV9BxEJBI6pahaOHsU05/rKQLKqnnHu0xl4zbmtpqoeEMftFrcB0W5uhzF/KubDRgXpRHIa477Zwpx1cTSoWp6vH7ma0GAPFjI0eebW5KGqGSIyHPgOx62601R1s4i8AkSo6kKgKzBBRBTHsNUw5+HNgI9EJAsoheOax9kL7TNFpCogwEbgUXe2w5QANmxU4L6NOsCLCzZzPDmNYddfwYhubipkaNxCVEvGpYDQ0FCNiIjwdBimoOXnsFGOX4vHsFFBOnwqlZcWbObb6IM0r1WR125vRfNaLhYyNAVGRNapauj56+3jjyl6bNioSFJV5qyLY+ziGFIzshh5U1OGXFOf0tkLGZoiw5KHKRxs2KhY23csmdHzoli94wjtgiszsX8rrqjq5+mwzGXI82+QiNwKfOO8sG1M3tjdRiVaVpYy49c9vPbdNgQY26c5gzrUo1Qp680Vda58/BoIvC0iYTgufG91U0ymsLNhI5MHOw+fYmRYFOv+OM51jasyvm8LgioX7FSpxn3ynDxU9W4RqQjcCXzqvDtqOvA/VT3lrgBNAbFhI5NP0jOzmLIqlneW7aBcWS/e/Gdr+rapbYUMixmXfoNV9aSIzAF8gX/hKCfyjIhMUtV33RCfuVw2bGQKUHR8Is/MiWTLgZP0alWTMbc2p2qFsp4Oy7iBK9c8egP3Aw2BGUB7VT0sIuVwFDq05FFQbNjIFDKp6Zm8vWwHU1fHUqW8Nx/d05YezWt4OizjRq70PPoDb6nqquwrVTVZRB7M37BKIBs2MkXU2t3HGBUWSeyR0wwMrcPons3wL1fG02EZN3PlL8gY4GxNKUTEF6iuqntUdXl+B1Zs2LCRKaaSzmTw6rdb+fy3Pwiq7MsXD3agS6NAT4dlCogryeNroFO25Uznunb5GlFRYMNGpoRbse0wz8+N4sDJVB7oXJ+nezSmnLf1ZksSV77bpVX13EdjVU0TkeJfZP/ntyFhqw0bGQMcP53G2MUxzN0QT8Nqfsx5tBNt61X2dFjGA1z5C5YgIr2dxQwRkT7AEfeEVYjsXAbH/7BhI1OiqSrfRB3gpQWbSUxJ5/FuDRnWrSFlS1shw5LKleTxKI5qtu/hqGa7D7jXLVEVJoMXezoCYzzq0MlUXpwfzfcxh2hZ258vHupAs5oVPR2W8TBXHhLcBXR0TtiEqia5LSpjjMepKrMj9jHumy2kZWTx3M1NebCLFTI0Di4NvItIL6A5jrnFAVDVv03/aowp2vYeTea5eZH8svMo7etX4dX+ragfWN7TYZlCxJWHBD8EygHXAx8DtwNr3RSXMcYDMrOUT9fs4Y3vtuFVShh3Wwvual/XChmav3Gl59FJVVuJSKSqviwi/wW+dVdgxpiCtf3QKZ6dE8nGfSfo1rQa425rQa1KdhOIyZkrySPV+TVZRGoBR4Ga+R+SMaYgpWVk8eFPu3j3xx34lS3NO3dcSe/WtayQobkgV5LHIhGpBLwOrMfxoMNUdwRljCkYm/adYGRYJFsPnuLW1rUYc2sIAX5WyNBcXJ6Sh4iUApar6gkgTEQWAz6qmujO4Iwx7pGSlsnby7YzdXUsVSuUZeq9oXQPqe7psEwRkqfkoapZIjIZaONcPgOccWdgxhj3+C32KKPCItlzNJk729fhuZ7NqOhjhQyNa1wZtlouIv2BuaqaQ20OY0xhdjI1nYnfbuXL3/dSt0o5vnyoA50aWiFDc2lcSR6PAE8BGSKSiuMpc1VVe9TUmELux62HGD03msOnUhlyTX2e6t4EX28rLWIunStPmFdwZyDGmPx3NOkMryyOYcHG/TSpXoEP72nLlXUqeTosUwy48pDgtTmtP39yKGOM56kqiyIPMGbhZk6lpvOvGxoxtGtDvEtbaRGTP1wZtnom22sfoD2wDuiWrxEZYy7LwcRUXpgfxbIth2ldpxKv9W9Fkxo2cGDylyvDVrdmXxaROsDb+R2QMebSZGUps8L3MWHJFtKzsnihVzPu71wfLystYtzgcmYkigOa5VcgxphLt+fIaUbNjeS32GNc3SCAif1bUi/AChka93Hlmse7/Dl9XingShxPmhtjPCQzS5n2827++8M2ypQqxYR+LbmjXR0rLWLczpWeR0S21xnA/1T1l3yOxxiTR9sOnuLZOZvYFJfIDc2qMe62ltTw9/F0WKaEcCV5zAFSVTUTQES8RKScqia7JzRjTE7SMrKYvGIn76/cSUWfMrx7ZxtuaVXTehumQLn0hDlwA3B2BkFf4HugU34HZYzJ2cZ9J3h2zia2H0rititr8Z9bm1OlvLenwzIlkCvJwyf71LOqmiQi5dwQkzHmPMlpGbz5/Xam/bKb6hV9mDY4lG5NrZCh8RxXksdpEblKVdcDiEhbIMU9YRljzlqz8wij5kax91gygzrUZdTNTalghQyNh7mSPP4FfC0i+3HUtaoBDHRHUMYYSExJZ8KSLcwK30dwQDlmPdyRjg0CPB2WMYBrDwmGi0hToIlz1TZVTXdPWMaUbD/EHOKF+VEknDrDI9c14MkbGuNTxgoZmsLDlec8hgEzVTXauVxZRO5U1ffdFp0xJcyRpDOMWbiZxZEHaFqjAlPvDaVVUCVPh2XM37hSJW2IcyZBAFT1ODDkYgeJyE0isk1EdorIqBy21xOR5SISKSIrRSQo27ZMEdno/Lcw2/r6IvK785xfiYjdbmKKNFVl/oZ4ur/5E99vPsS/uzdm4fAuljhMoeXKNQ8vEZGzE0GJiBdwwT/azn0mA91xlDMJF5GFqhqTbbc3gBmq+pmIdAMmAPc4t6Wo6pU5nPpV4C1VnSUiHwIPAh+40BZjCo39J1J4fl4UK7Yl0Kauo5Bho+pWyNAUbq4kj6XAVyLykXP5Eee6C2kP7FTVWAARmQX0AbInjxAck0wBrADmX+iE4ngSqhtwl3PVZ8AYLHmYIiYrS5m5di+vfruVzCzlP7eEcF+nYCtkaIoEV5LHSBwJ4zHn8g/Axxc5pjawL9tyHNDhvH02Af2Ad4C+QAURCVDVo4CPiETgKIcyUVXnAwHACVXNyHbO2jm9uYg8DDwMULdu3Yu1z5gCs/vIaUaGRbJ29zG6NAxkQr+W1Klij02ZosOVu62ycHy6z+9P+E8D74nIYGAVEA9kOrfVU9V4EWkA/CgiUUBiXk+sqlOAKQChoaE277rxuIzMLD7+eTdv/bAd79KleK1/KwaEBllpEVPkuHK3VSMc1yNCcEwGBYCqNrjAYfFAnWzLQc5156jqfhw9D0TED+h/9sK8qsY7v8aKyEqgDRAGVBKR0s7ex9/OaUxhFLP/JCPDIomKT+TGkOqMva0F1StaIUNTNLlyt9V0HL2ODOB6YAbwxUWOCQcaOe+O8gbuABZm30FEAkXkbBzPAdOc6yuLSNmz+wCdgRjnBfsVwO3OY+4DFrjQDmMK1JmMTP77/TZ6v/czBxJTmHzXVXx0T1tLHKZIc+Wah6+qLnfecfUHMEZE1gH/ye0AVc0QkeHAd4AXME1VN4vIK0CEqi4EugITRERxDFsNcx7eDPhIRLJwJLmJ2e7SGgnMEpFxwAbgExfaYUyBWffHcUaGRbLzcBL9rqrNi71CqGyFDE0x4EryOOPsIexwJoR4wO9iB6nqEmDJeev+k+31HBzl3s8/bg3QMpdzxuK4k8uYQun0mQze+H4bn67ZQy1/Xz69vx1dm1TzdFjG5BtXkscTQDngcWAsjqGr+9wRlDFF2eodCTw3N4q44ynce3U9nr2pKX5lL2fGZ2MKH5dqWzlfJgH3n79dRN5V1RH5FZgxRU1icjrjl8QwOyKOBoHlmf3I1bSvX8XTYRnjFvn5cahzPp7LmCJlafRBXlwQzbHTaTzW9Qqe+EcjK2RoijXrSxtzGRJOOQoZfhN1gJCaFZk+uB0tavt7Oixj3M6ShzGXQFWZuz6eVxbHkJKWyTM9mvDwtQ0o4+XK3e/GFF35mTzsEVlTIsQdT2b0vGhWbU+gbb3KvNq/FQ2rXfTGQ2OKFVeeMG+pqlEX2OWdfIjHmEIrK0v54vc/ePXbrSjwcu/m3NOxHqWskKEpgVzpebzvfOL7UxyTQv2lxpSqfpqPcRlTqOxKSGJUWCThe45zTaNA/q+vFTI0JZsrt+pe46xv9QCwTkTWAtNV9Qe3RWeMh6VnZjF1dSxvL9uBbxkv3hjQmv5X1bZChqbEc+mah6ruEJEXgAhgEtDGOb/GaFWd644AjfGU6PhERoZFsnn/SW5uUYOX+zSnWgWrR2UMuHbNoxWOhwN74ZjL41ZVXS8itYBfAUseplhITc9k0vIdfLQqlsrlvPlg0FXc3LKmp8MyplBxpefxLo7Jn0arasrZlaq639kbMabIi9hzjGfDIolNOM2AtkG80CsE/3JlPB2WMYVOnpKHcy7yeFX9PKftua03pqhIOpPB60u3MuO3P6jl78uMB9pzbeOqng7LmEIrT8lDVTNFpI6IeKtqmruDMqYg/bQ9gdFzo9ifmMJ9VwfzTI8mlLdChsZckCu/IbuBX0RkIXD67EpVfTPfozKmAJxITmPs4i2ErY/jiqrl+fqRqwkNtkKGxuSFK8ljl/NfKaCCe8IxpmAsiTrAfxZEczw5neHXN2R4t4ZWyNAYF7jynMfL7gzEmIJw+GQq/1mwmaWbD9KidkU+e6A9zWtZIUNjXOXKrbpVgWeB5sC5m91VtZsb4jImX6kqX6+LY9ziGFIzshh5U1OGXFOf0lbI0JhL4sqw1UzgK+AW4FEcswgmuCMoY/LTvmPJjJ4XxeodR2gfXIWJ/VvSoKoVMjTmcriSPAJU9RMReUJVfwJ+EpHwix5ljIdkZikzft3D699tQ4CxfZozqIMVMjQmP7iSPNKdXw+ISC9gP2C3pphCaefhUzw7J5L1e09wXeOq/F+/ltSu5OvpsIwpNlxJHuNExB/4N46nzSsCT7olKmMuUXpmFh/9tItJy3dSrqwXb/6zNX3bWCFDY/KbK3dbLXa+TASud084xly6qLhEnpmzia0HT9GrVU3G3NqcqhXKejosY4olV++2GgIEZz9OVR/I/7CMybvU9EzeXraDqatjCSjvzUf3tKVH8xqeDsuYYs2VYasFwGpgGZDpnnCMcc3vsUcZNTeK3UdOMzC0DqN7NcPf1woZGuNuriSPcqo60m2RGOOCU6npvLp0K1/8tpc6VXyZ+VAHOjcM9HRYxpQYriSPxSLSU1WXuC0aY/JgxdbDPD8vigMnU3mgc32e7tGYct5WyNCYguTKb9wTwGgROYPjtl0BVFUruiUyY85z7HQaYxfHMG9DPI2q+RH2WCeuqlvZ02EZUyK5creVFUM0HqGqfBN1gJcWbCYxJZ3H/9GIYddfQdnSVsjQGE+5aPIQkaaqulVErsppu6quz/+wjHE4dDKVF+ZH80PMIVoF+fPFQx1oVtM6u8Z4Wl56Hk8BDwP/BTTbenEuW2FEk+9UldkR+xj3zRbSMrIY3bMpD3S2QobGFBYXTR6q+rDzZU9gKNAFR9JYDXzgvtBMSbX3aDKj5kayZtdROtSvwqv9WxEcWN7TYRljsnHlgvlnwElgknP5LmAG8M/8DsqUTJlZyvRfdvPG99soXaoU4/u24M52da2QoTGFkCvJo4WqhmRbXiEiMfkdkCmZth9yFDLcuO8E3ZpWY3zfFtT0t0KGxhRWriSP9SLSUVV/AxCRDkCEe8IyJUVaRhYfrNzFeyt2UMGnDO/ccSW9W9eyQobGFHJ5udsqCsc1jjLAGhHZ61yuB2x1b3imONu07wQjwyLZevAUvVvX4qVbQwjws0KGxhQFeel53HI5byAiNwHvAF7Ax6o68bzt9YBpQFXgGHC3qsZl214RiAHmq+pw57qVQE0gxbnbjap6+HLiNAUnJS2Tt5Zt5+PVsVSr4MPH94ZyQ0h1T4dljHFBXu62+uNSTy4iXsBkoDsQB4SLyEJVzX6t5A1ghqp+JiLdgAnAPdm2jwVW5XD6Qapqw2ZFzK+7jjJqbiR/HE3mzvZ1ea5nUyr6WCFDY4oadxcEag/sVNVYABGZBfTB0ZM4KwTHsyQAK4D5ZzeISFugOrAUCHVzrMaNTqamM2HJVv63di/1Asrx5ZAOdLrCChkaU1S5+4mr2sC+bMtxznXZbQL6OV/3BSqISICIlMLxYOLTuZx7uohsFJEXJZerqyLysIhEiEhEQkLCpbfCXJblWw5x45ur+Cp8L0Ouqc/SJ661xGFMEVcYHtd9GrhORDYA1wHxOOYLGQosyX79I5tBqtoSuMb5754c9kFVp6hqqKqGVq1a1T3Rm1wdTTrD4//bwIOfReDvW4a5QzvzfK8QfL2tJpUxRZ27h63igTrZloOc685R1f04ex4i4gf0V9UTInI1cI2IDAX8AG8RSVLVUaoa7zz2lIh8iWN4bIab22LySFVZuGk/Ly+K4VRqOk/e0JjHul6Bd+nC8FnFGJMf3J08woFGIlIfR9K4A8eT6eeISCBwTFWzgOdw3HmFqg7Kts9gIFRVR4lIaaCSqh4RkTI47gZb5uZ2mDw6kJjCC/OiWb71MK3rVOK1/q1oUsMKMhtT3Lg1eahqhogMB77DcavuNFXdLCKvABGquhDoCkwQEcVxV9Wwi5y2LPCdM3F44UgcU93VBpM3WVnK/8L3MmHJVjKysnihVzPu71wfLystYkyxJKp68b2KgdDQUI2IsDt73WHPkdOMmhvJb7HHuLpBABP7t6RegBUyNKY4EJF1qvq3u11t7k5zyTIys5j2y27++/12vL1KMbFfSwa2q2OlRYwpASx5mEuy9eBJRs6JZFNcIjc0q86421pQw9/H02EZYwqIJQ/jkjMZmUxesYv3V+zE37cM797Zhlta1bTehjEljCUPk2fr9x5n5JxIdhxOom+b2rx4SwhVynt7OixjjAdY8jAXlZyWwX+/3860X3ZTo6IP0waH0q2pFTI0piSz5GEu6JedRxg1N5J9x1K4u2NdRt7UlApWyNCYEs+Sh8lRYko6E5ZsYVb4PuoHlmfWwx3p2CDA02EZYwoJSx7mb77ffJAX5kdzJOkMj1zXgCdvaIxPGatHZYz5kyUPc86RpDOMWbiZxZEHaFqjAh/fF0qroEqeDssYUwhZ8jCoKvM3xvPyohiSz2Ty7+6NebTrFZTxskKGxpicWfIo4eJPpPD8vChWbkugTV1HIcNG1a2QoTHmwix5lFBZWcrMtXuZuGQLWQov3RrCvVcHWyFDY0yeWPIogWITkhgVFsXaPcfo0jCQCf1aUqdKOU+HZYwpQix5lCAZmVl8/PNu3vphO2VLl+K121sxoG2QlRYxxrjMkkcJEbP/JM+GbSI6/iQ9mldnbJ8WVKtohQyNMZfGkkcxl5qeyXs/7uTDn3ZRqVwZ3h90FTe3qGG9DWPMZbHkUYyt++MYz86JZFfCafpdVZsXe4VQ2QoZGmPygSWPYuj0mQxe/24bn/26h1r+vnx6fzu6Nqnm6bCMMcWIJY9iZvWOBJ6bG0Xc8RTuu7oez9zUFL+y9m02xuQv+6tSTCQmpzPumxi+XhdHg6rl+frRq2kXXMXTYRljiilLHsXA0ugDvLhgM8dOpzG06xU8/o9GVsjQGONWljyKsMOnUnlpwWa+jT5ISM2KTB/cjha1/T0dljGmBLDkUQSpKmHr4xm7OIaU9Eye6dGEh69tYIUMjTEFxpJHERN3PJnR86JZtT2B0HqVmdi/FQ2r+Xk6LGNMCWPJo4jIylI+/+0PXl26FYCXezfnno71KGWFDI0xHmDJowjYeTiJUWGRRPxxnGsbV+X/+rYgqLIVMjTGeI4lj0IsPTOLKatieWfZDny9vXhjQGv6X1XbSosYYzzOkkchFR2fyLNzIok5cJKeLWswpndzqlWwQobGmMLBkkchk5qeyTvLdzBlVSyVy3nz4d1XcVOLmp4Oyxhj/sKSRyESvucYI+dEEnvkNAPaBvFCrxD8y5XxdFjGGPM3ljwKgaQzGby2dCszfv2DoMq+fP5ge65pVNXTYRljTK4seXjYym2HeX5eNPsTUxjcKZhnejShvBUyNMYUcvZXykOOn05j7DcxzF0fzxVVyzPn0atpW88KGRpjigZLHgVMVfk2+iD/WRDNieR0hl/fkOHdGlohQ2NMkWLJowAdPpnKiwui+W7zIVrUrshnD7SneS0rZGiMKXoseRQAVeXrdXGMWxzDmYwsRt3clIe61Ke0FTI0xhRRljzcbN+xZJ6bG8XPO4/QPrgKE/u3pEFVK2RojCna3P7RV0RuEpFtIrJTREblsL2eiCwXkUgRWSkiQedtrygicSLyXrZ1bUUkynnOSVII63VkZinTft7NjW+tYsPe44y9rQWzHu5oicMYUyy4techIl7AZKA7EAeEi8hCVY3JttsbwAxV/UxEugETgHuybR8LrDrv1B8AQ4DfgSXATcC37mmF63YcOsXIsEjW7z1B1yZVGd+3JbUr+Xo6LGOMyTfuHrZqD+xU1VgAEZkF9AGyJ48Q4Cnn6xXA/LMbRKQtUB1YCoQ619UEKqrqb87lGcBtFILkkZ6ZxYcrd/HujzspV9aLtwa25rYrrZChMab4cXfyqA3sy7YcB3Q4b59NQD/gHaAvUEFEAoDjwH+Bu4Ebzjtn3HnnrJ3Tm4vIw8DDAHXr1r3kRuRFVFwiz8zZxNaDp7ilVU3G9G5OoF9Zt76nMcZ4SmG4YP408J6IDMYxPBUPZAJDgSWqGnepn9xVdQowBSA0NFTzJdrzpKZn8tay7UxdFUugX1mm3NOWG5vXcMdbGWNMoeHu5BEP1Mm2HORcd46q7sfR80BE/ID+qnpCRK4GrhGRoYAf4C0iSTh6KEEXOmdB+S32KKPCItlzNJk72tXhuZ7N8Pe1QobGmOLP3ckjHGgkIvVx/IG/A7gr+w4iEggcU9Us4DlgGoCqDsq2z2AgVFVHOZdPikhHHBfM7wXedXM7/uJUajoTv93KzN/3UqeKLzMf6kDnhoEFGYIxxniUW5OHqmaIyHDgO8ALmKaqm0XkFSBCVRcCXYEJIqI4hq2G5eHUQ4FPAV8cF8oL7GL5iq2HGT0vioMnU3mwS33+fWNjynkXhtE/Y4wpOKLqlksBhU5oaKhGRERc8vHHTqfxyqLNzN+4n0bV/Hj19lZcVbdyPkZojDGFj4isU9XQ89fbR+aLUFUWRx5gzMLNJKak8/g/GjHs+isoW9oKGRpjSi5LHheQmaUMnbmO7zYfolWQPzOHdKBpjYqeDssYYzzOkscFeJUS6gf6MbpnZR7obIUMjTHmLEseFzHq5qaeDsEYYwod+yhtjDHGZZY8jDHGuMyShzHGGJdZ8jDGGOMySx7GGGNcZsnDGGOMyyx5GGOMcZklD2OMMS4rMYURRSQB+OMSDw8EjuRjOEWBtblkKGltLmnthctvcz1VrXr+yhKTPC6HiETkVFWyOLM2lwwlrc0lrb3gvjbbsJUxxhiXWfIwxhjjMkseeTPF0wF4gLW5ZChpbS5p7QU3tdmueRhjjHGZ9TyMMca4zJKHMcYYl1nyyEZEbhKRbSKyU0RG5bC9rIh85dz+u4gEeyDMfJWHNj8lIjEiEikiy0WknifizC8Xa2+2/fqLiIpIkb+tMy9tFpF/Or/Pm0Xky4KOMb/l4ee6roisEJENzp/tnp6IMz+JyDQROSwi0blsFxGZ5Pw/iRSRqy7rDVXV/jmu+3gBu4AGgDewCQg5b5+hwIfO13cAX3k67gJo8/VAOefrx4pym/PSXud+FYBVwG9AqKfjLoDvcSNgA1DZuVzN03EXQJunAI85X4cAezwddz60+1rgKiA6l+09gW8BAToCv1/O+1nP40/tgZ2qGquqacAsoM95+/QBPnO+ngP8Q0SkAGPMbxdts6quUNVk5+JvQFABx5if8vI9BhgLvAqkFmRwbpKXNg8BJqvqcQBVPVzAMea3vLRZgYrO1/7A/gKMzy1UdRVw7AK79AFmqMNvQCURqXmp72fJ40+1gX3ZluOc63LcR1UzgEQgoECic4+8tDm7B3F8cimqLtpeZ1e+jqp+U5CBuVFevseNgcYi8ouI/CYiNxVYdO6RlzaPAe4WkThgCTCiYELzKFd/3y+o9GWHY0oEEbkbCAWu83Qs7iIipYA3gcEeDqWglcYxdNUVR89ylYi0VNUTngzKze4EPlXV/4rI1cDnItJCVbM8HVhRYT2PP8UDdbItBznX5biPiJTG0d09WiDRuUde2oyI3AA8D/RW1TMFFJs7XKy9FYAWwEoR2YNjXHhhEb9onpfvcRywUFXTVXU3sB1HMimq8tLmB4HZAKr6K+CDo4BgcZan3/e8suTxp3CgkYjUFxFvHBfEF563z0LgPufr24Ef1Xklqoi6aJtFpA3wEY7EUdTHwi/YXlVNVNVAVQ1W1WAc13h6q2qEZ8LNF3n5uZ6Po9eBiATiGMaKLcAY81te2rwX+AeAiDTDkTwSCjTKgrcQuNd511VHIFFVD1zqyWzYyklVM0RkOPAdjrs1pqnqZhF5BYhQ1YXAJzi6tztxXJi6w3MRX748tvl1wA/42nlvwF5V7e2xoC9DHttbrOSxzd8BN4pIDJAJPKOqRbZHncc2/xuYKiJP4rh4PriIfxBERP6H40NAoPNazktAGQBV/RDHtZ2ewE4gGbj/st6viP9/GWOM8QAbtjLGGOMySx7GGGNcZsnDGGOMyyx5GGOMcZklD2OMMS6z5GHMZRCRTBHZmO1frpV6L+HcwblVSDXG0+w5D2MuT4qqXunpIIwpaNbzMMYNRGSPiLwmIlEislZEGjrXB4vIj9nmR6nrXF9dROaJyCbnv07OU3mJyFTnPBvfi4ivc//Hs82zMstDzTQlmCUPYy6P73nDVgOzbUtU1ZbAe8DbznXvAp+paitgJjDJuX4S8JOqtsYxJ8Nm5/pGOMqlNwdOAP2d60cBbZznedQ9TTMmd/aEuTGXQUSSVNUvh/V7gG6qGisiZYCDqhogIkeAmqqa7lx/QFUDRSQBCMpeeFIcM1X+oKqNnMsjgTKqOk5ElgJJOOpSzVfVJDc31Zi/sJ6HMe6jubx2RfYqxpn8eZ2yFzAZRy8l3Fnl2ZgCY8nDGPcZmO3rr87Xa/izoOYgYLXz9XIc0/wiIl4i4p/bSZ3zjtRR1RXASBxTA/yt92OMO9mnFWMuj6+IbMy2vFRVz96uW1lEInH0Hu50rhsBTBeRZ3CUAD9b2fQJYIqIPIijh/EYkFu5bC/gC2eCEWBSMZ+4yRRCds3DGDdwXvMIVdUjno7FGHewYStjjDEus56HMcYYl1nPwxhjjMsseRhjjHGZJQ9jjDEus+RhjDHGZZY8jDHGuOz/AcaeC7PYapppAAAAAElFTkSuQmCC\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "tags": [], + "needs_background": "light" + } + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "opu9neBA_98R", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 280 + }, + "outputId": "b62c34d5-566f-4bcc-b153-574a5ad33465" + }, + "source": [ + "plot_graphs(history, 'loss')" + ], + "execution_count": null, + "outputs": [ + { + "output_type": "display_data", + "data": { + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEHCAYAAAC0pdErAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAA1/ElEQVR4nO3dd3hVVdbH8e9Kp4QeICGBhF6SUAydBAtdBRUQ7KhjxwKK4FgG2yvqDGDBgmMvg4iAqFQVCU0gIKTQIQESWug1pO33j3OZBCZgILk5N/euz/PwmHvOLeuA8Lv7nLPXFmMMSiml1Pm87C5AKaWUa9KAUEopVSQNCKWUUkXSgFBKKVUkDQillFJF0oBQSilVJB9nvrmI9AHeAryBfxtjxp23Pw6YCEQDQ40x0xzbrwImFHpqc8f+mRf6rFq1apnw8PDSLF8ppdze6tWrDxhjgora57SAEBFvYBLQE0gHVonILGPM+kJP2wkMA54q/FpjzEKgjeN9agBbgfkX+7zw8HASEhJKq3yllPIIIrLjQvucOYLoAGw1xmx3FDEFGAD8NyCMMWmOffkXeZ9BwBxjzCnnlaqUUup8zrwGUQ/YVehxumPbpRoK/KdUKlJKKVVsLn2RWkSCgShg3gX23y8iCSKSkJmZWbbFKaWUm3NmQGQAYYUehzq2XYqbgRnGmJyidhpjJhtjYowxMUFBRV5jUUopdZmcGRCrgCYiEiEiflinimZd4nvcgp5eUkopWzgtIIwxucBwrNNDG4CpxpgUEXlJRPoDiEh7EUkHBgMfikjK2deLSDjWCGSRs2pUSil1YeIu7b5jYmKM3uaqlFKXRkRWG2Niitrn0hepy4Ixhv+bvYHtmSfsLkUppVyKxwdE6oGTTFm5k75vLeaDRdvIzbvYlAyllPIcHh8QDYMqs2Bkd7o3DWLcnI3c8N5S1u8+ZndZSillO48PCIA6VQL48I4reO+2duw9mkX/d5fwr/mbOJObZ3dpSillGw0IBxGhX1QwC0Z0p3+bEN75bSv93lrM6h2H7C5NKaVsoQFxnuqV/Bh/cxs+u7s9WTn5DPpgOWNnpXDyTK7dpSmlVJnSgLiAK5vVZt6IOO7o1IDPlqXRe2I8i7doOw+llOfQgLiIyv4+vDQgkqkPdMbP24s7Pl7JqO/WcfRUkZ0/lFLKrWhAFEOHiBrMfjyWh69sxPQ/M+gxYRFzk/faXZZSSjmVBkQxBfh683Sf5vzwSFeCKvvz4Ferefjr1ew/nmV3aUop5RQaEJcosl5VfhjelVG9m/HLhv30HB/PtNXpuEvLEqWUOksD4jL4envxyFWNmf1YLI1rV+ap79Zx16erSD+si94ppdyHBkQJNK5dme8e6MyL/VuRkHaIXhPi+XxZGvn5OppQSpV/GhAl5OUl3NUlnPkj4ogJr8E/ZqVw84fL2abN/5RS5ZwGRCkJrV6Rz+9uzz8Ht2bL/hP0fWsxkxZuJUeb/ymlyikNiFIkIgy6IpQFI+Po0aI2b87bxIB3l5KccdTu0pRS6pJpQDhB7cAA3rvtCj64vR37j59hwKSlvD53I1k52vxPKVV+aEA4UZ/IYH4d2Z2b2tbj/d+30e+txaxK0+Z/SqnyQQPCyapW9OXNwa354p4OnMnNZ/AHy3nhh2ROaPM/pZSL04AoI3FNg5g/Io5hXcL58o8d9J4Qz6LN2vxPKeW6NCDKUCV/H8b2b8W0BzsT4OvFXZ+sZOTUtRw+mW13aUop9T80IGxwRYMa/PxYLMOvasystbvpOWERs5P2aLsOpZRLcWpAiEgfEdkkIltFZEwR++NEZI2I5IrIoPP21ReR+SKyQUTWi0i4M2stawG+3jzVuxk/DO9K3aoBPPz1Gh78ajX7j2nzP6WUa3BaQIiINzAJ6Au0BG4RkZbnPW0nMAz4poi3+AJ40xjTAugA7HdWrXZqFVKVmQ93ZXSf5izclEmP8YuYmrBLRxNKKds5cwTRAdhqjNlujMkGpgADCj/BGJNmjEkEzplu7AgSH2PMAsfzThhj3LYTno+3Fw9d2Yi5j8fSvG4Vnp6WyB0fr2TXIbc9ZKVUacnJgsM7nPLWzgyIesCuQo/THduKoylwRESmi8ifIvKmY0Ti1hoGVWbK/Z14+YZI/tx5mF4T4vl0aSp52vxPKXVWXg7sXAGL3oTProNx9WH6/U75KB+nvGvJ+QCxQFus01DfYp2K+rjwk0TkfuB+gPr165dthU7i5SXc0akBVzevzbMzknjxx/X8uG43rw+MpkmdQLvLU0qVtfw82JsIqfHWrx3LIeekta9uFLT/GzS6yikf7cyAyADCCj0OdWwrjnRgrTFmO4CIzAQ6cV5AGGMmA5MBYmJi3Oprdr1qFfh0WHtmrs3gxR/Xc+3bS3j06sY8eGUjfL315jOl3JYxsH+DFQZpi61fWY5+brWaQZtbICIOGnSDSjWdWoozA2IV0EREIrCCYShw6yW8tpqIBBljMoGrgQTnlOm6RIQb24YS2ySIf8xK4V8LNvNz0h7eHNSaqNCqdpenlCoNxsCh7ZC6yBEKS+CkYxJt9XBo0R8iukNELATWLdPSxJl3y4hIP2Ai4A18Yox5VUReAhKMMbNEpD0wA6gOZAF7jTGtHK/tCfwLEGA1cL/jYneRYmJiTEKCe2fIvJS9PD8zmQMnznBfXENG9GhKgK/bX5pRyv0c2VUwQkiNh2OOkyuBwdboICIOwmOhegOnlyIiq40xMUXuc5fbKT0hIACOns7htdkbmLJqFxG1KjHupig6NnTuMFMpVULH9xWEQWo8HE61tlesaQVBRJw1SqjZCETKtDQNCDe0dOsBxkxPZNeh09zeqT6j+zQnMMDX7rKUUgCnDlmnis6GQuZGa7t/VQjvWjBKCGoBXvZeU9SAcFOnsnP51/zNfLI0lbpVAvi/G6O4qnltu8tSyvNkHYOdywtGCHuTAAO+FaF+54JACG4NXq51WlgDws2t2XmY0dMS2bL/BDe0CeGF61tRo5Kf3WUp5b6yT8GuFQUjhIw1YPLA2w/COhYEQkg78HHtv4saEB7gTG4ekxZu472FW6lawZex/VtxXXQwUsbnM5VyS7nZkLG6YISQvhLyskG8od4VBYEQ1gF8K9hd7SXRgPAgG/YcY/T3iSSmH6VHizq8emMkdaoE2F2WUuVLXi7sWQdpjkDY+QfknAIEgqMLLirX7wT+5XsCqwaEh8nNy+fTpWn8c/4m/Hy8eLZfC4a0D9PRhFIXkp8P+1Mg1XHKaMdSOHPM2hfUomCE0KALVKxhb62lTAPCQ6UdOMno7xNZkXqILo1q8tpNUTSoWcnuspSynzFwcGvB5LTUxXDasV58jYbnzkWo7N43fmhAeLD8fMOUVbv4v9kbyM3P56lezbi7awTeXjqaUB7m8I6Cawip8XBir7W9SmhBIETEQtVQe+ssYxoQij1HT/PsjGR+27if1mHVeGNgNM3qlu9zp0pd1LE9jruMHKOEIzut7ZWCzh0h1GhY5pPTXIkGhALAGMOsdbt58cf1HM/K4ZGrGvPwlY3x89Hmf8oNnDx47mzlg1us7QHVILxboclpzT06EM53sYBw1XbfyglEhAFt6tGtcS1e+mk9E3/ZwpykvbwxKJrWYdXsLk+pS5N1FHYsKwiEfcnWdr/K1sXkdndagVA3yuUmp5UXOoLwYL+s38dzM5PZfzyLe7tFMLJnMyr46V8k5aKyT1q3m54NhD1rweSDT4BjclqsdetpSFvw1rYzxaUjCFWkHi3r0KFhDcbN2chHi1OZv34fr90URZdGtewuTSnIPQPpqwruMkpfBfk54OUDoe0h9ilrhBDaHnx1ro8z6AhCAbBs2wGemZ7EjoOnuKVDfZ7p15wq2vxPlaW8XNj9p3VROW2xNVrIzQLxguA2jhFCHIR1Av/KdlfrNvQitSqW09l5TPhlM/9evJ2gQH9evSGKHi3r2F2Wclf5+bAvqWCEsGMZZB+39tWJLGiD3aALVKhma6nuTANCXZJ1u44w+vtENu49Tv/WIfzj+pbUrOxvd1mqvDMGMjc5FspxrJx2+rC1r2aTghFCeCxU0tOcZUUDQl2y7Nx83v99G+8u3EJlfx/G9m9F/9Yh2q5DFZ8x1sI4Z0cIqfFwcr+1r2r9cyenVQmxt1YPpgGhLtvmfcd5eloia3cd4ZrmtXnlxkiCq5avbpWqDB3NOHcpzaO7rO2V6xaMECLirLWWlUvQgFAlkpdv+HRpKv+cvwkfLy+e6decW9rXx0vbdagTmY6Op45AOLTN2l6h+rlLadZqopPTXJQGhCoVOw+eYsz0RJZtO0jHiBqMGxhNRC1t/udRTh+GtKUFI4T9663tfoHnLqVZu5XtS2mq4tGAUKXGGMO3q3bx6s8byM7L58leTbmnawQ+3vqPgVs6c+LcpTT3rAMM+FSw1kI4O0IIbg3eOq2qPNKAUKVu79EsnpuZzC8b9hEdWpXXB0bTIriK3WWpksrJslZLOxsIGashPxe8fK3V0s6OEOpdAT56Z5s7sC0gRKQP8BbgDfzbGDPuvP1xwEQgGhhqjJlWaF8ekOR4uNMY0/9in6UBUfaMMfyctId//JDC0dM5PHxVYx65qhH+Ptquo9zIy7HWU06Ntyao7VoJeWesyWkh7QotpdkR/CraXa1yAltabYiINzAJ6AmkA6tEZJYxZn2hp+0EhgFPFfEWp40xbZxVnyo5EeG66BC6NrKa/7396xbmJO3h9UHRtKtf3e7yVFHy82BvYsEIYcdyyDlp7asbBR3uswKhfmcI0BGhp3PmScMOwFZjzHYAEZkCDAD+GxDGmDTHvnwn1qGcrHolPyYMaUP/1iH8fUYSA99fxj1dI3iyV1Mq+ul5aVsZA/s3FAqEJVYXVIBazaDNrdbtpw26QaWa9taqXI4z//bWA3YVepwOdLyE1weISAKQC4wzxswsxdqUE1zVvDbzR8Tx+tyNfLwklfnr9zLupmi6NtZZsWXGGDi0/dylNE8dsPZVD4eWAyDcMTktsK6tpSrX58pf7xoYYzJEpCHwm4gkGWO2FX6CiNwP3A9Qv359O2pU5wkM8OWVG6K4PjqEMdOTuO3fKxgSE8bfr21B1Qra/M8pjuwsmIeQGg/Hd1vbA4OhcQ8rDMJjoXoDe+tU5Y4zAyIDCCv0ONSxrViMMRmO/24Xkd+BtsC2854zGZgM1kXqEtarSlHHhjWZ83gsE3/ZwkeLt7Nw035euSGSXq30W2uJHd937lKah9Os7RVrFvQyiugONRvp5DRVIs4MiFVAExGJwAqGocCtxXmhiFQHThljzohILaAr8IbTKlVOEeDrzZi+zbk2Kpinv0/k/i9Xc210MGOvb0VQoN4iWWynDlmN7c62sMjcaG33r2otpdnxQcdSmi10cpoqVc6+zbUf1m2s3sAnxphXReQlIMEYM0tE2gMzgOpAFrDXGNNKRLoAHwL5gBcw0Rjz8cU+S29zdW05efl8uGgbb/+6lYr+3rxwXUtubFtPm/8VJevYuZPT9iYBBnwrWq2vz7awCG6tS2mqEtOJcsplbN1vNf9bs/MIVzYL4tUbo6hXzcOb/2Wfgl0rCkYIGWvA5IG3/7mT00LagY+f3dUqN6MBoVxKXr7hi+VpvDF3E14CY/o257aODTyn+V9uNmQkFFpKcyXkZYN4WzOU/zs5rQP4enh4KqfTgFAuadehU/x9RhKLtxygQ3gNxg2MomGQGy4lmZdr9TAqvJRmzilAIDi6oJ9R/U7gH2h3tcrDaEAol2WMYdrqdF7+aT1ZufmM6NGU+2LLefO//HzYn1JoKc2lcOaYta92y4JrCOFdrbbYStlIA0K5vP3Hsnj+h2Tmpewjsl4V3hjYmpYh5aTVgzFwYItjXQRHKJw+ZO2r0bDglFF4LFSubW+tSp1HA0KVG3OS9vD8DykcOZXNg90bMfzqxgT4uuCdOofTzp2cdmKvtb1K6LlLaVYNtbVMpf6KLc36lLocfaOC6dyoJi//tIF3F25lTvIe3hgUzRUNathb2LHdViCcHSUc2WltrxRUKBDioHqETk5TbkNHEMplLdqcyd+nJ7H76Gnu6hzOqN7NqORfRt9pTh4sWDUtNR4ObrG2B1SzJqdFdLdGCEHNNRBUuaanmFS5deJMLm/O3cgXf+wgpGoFXrspirimQaX/QVlHYceygkDYl2xt96tsTU47O0KoE6mT05Rb0YBQ5d6qtEOM/j6R7ZknGXRFKM9f25KqFUvQ/C/7pHW76X+X0lwLJh98AqzFcc7eehrSBry1yaByXxoQyi1k5eTx9q9b+DB+OzUq+fHygFb0iQwu3otzz0D6qoJASE+A/Bzw8oHQ9gV3GYW2B98A5x6IUi5EA0K5leSMozw9LZH1e47RN7IuLw5oRe3A8/5Rz8uF3X8WdDzdtQJys6ylNIPbFNxlVL8z+FWy5TiUcgUaEMrt5OTlMzl+O2/9uoUKvt48168Zg+odRs5eWN6xDLJPWE+uE1lwDaF+Z6hQzdbalXIlepurcju+XsIjrXK5OX8r21fOpulPiYg4AqFmE4geUrBQTiVd0U6py6EBocoHY+BwasE1hNTFcHI/QUCtqmFsrdWDcbtCWUErhrXrzJ2dwz2n+Z9STqIBoVzX0YyCQEhbDEcdS5xXrgsNr7RGCBFxSPVwmgCPHj7FnhnJjP1xPT8l7mHcwGga13bD5n9KlRG9BqFcx4nMc/sZHXKsMFuhRsHpoojuUKvJBSenGWOYviaDl35az+nsPB7v0YT74xriW56b/ynlRHqRWrmm04chbWnBCGH/emu7fxVo0PW/IwRqt7rkpTQzj5/hH7OSmZ20l5bBVXhjUDSR9ao64SCUKt80IJRrOHPCsZTmImuEsGcdYMCngrUWwtnJacGtwbt0zn7OTbaa/x06mc39cQ15/Jomrtn8Tymb6F1Myh45p2HXykJLaa6G/Fzw9oPQDnDlGCsU6l0BPv5OKaFPZDCdG9bi1dnref/3bcxL3svrg6JpH25z8z+lygEdQajSk5djhUDqYmuUsGsl5J2xltIMaVtoKc2O4FexzMtbsuUAY6Ynkn74NHd2bsDTfZpTuaya/ynlovQUk3KO/DzrNNF/J6cth5yTgEDdSEfHU8fktADXWPzn5Jlc/jl/E58tSyOkagVevTGSK5vpIj7Kc2lAqNKRnw+ZGwruMkpbAmeOWvtqNSu0clo3qOjap3BW7zjM6O8T2br/BDe1q8fz17akeiU/u8tSqszZFhAi0gd4C/AG/m2MGXfe/jhgIhANDDXGTDtvfxVgPTDTGDP8Yp+lAeEExsDBbefeenrqgLWvenjBReXwbhBY19ZSL8eZ3Dze/W0r7/++jWoVfXmxfyT9ouoiur6D8iC2XKQWEW9gEtATSAdWicgsY8z6Qk/bCQwDnrrA27wMxDurRlWEIzvPXUrz+G5re2AINO5R0OSuWn176ywF/j7ePNmrGX0jgxn9fSKPfLOGXi3r8MoNkdSuoh1dlXLmFboOwFZjzHYAEZkCDMAaEQBgjElz7Ms//8UicgVQB5gLFJluqhQc3+e4huDoeno4zdpesVbBPISI7lCjoduunNYypAozHu7Cx0tSGb9gM9eMX8Tz17ZkcEyojiaUR3NmQNQDdhV6nA50LM4LRcQL+BdwO9Cj9EvzYKcOWdcOzo4QDmyytvtXtU4VdXzIsZRmi0uenFae+Xh78UD3RvRsWYcx05N4+vtEZq3bzWs3RRFWo+zvuFLKFbjqPX4PA7ONMekX+wYnIvcD9wPUr1/+T3k4RdYxx+S0eGuUsDcZMOBbCRp0hra3WS0sglvrUppAw6DKTLmvE9+s3Mm4ORvpNSGeUb2bcVeXcLy1+Z/yMM4MiAwgrNDjUMe24ugMxIrIw0BlwE9EThhjxhR+kjFmMjAZrIvUJS/ZDWSfshbHOTtC2P0nmDzw9oewDnDVs9YIIaQd+OhdO0Xx8hJu79SAq5vX5u8zknjpp/X8mLibNwZG06ROoN3lKVVmnHYXk4j4AJuBa7CCYRVwqzEmpYjnfgb8dP5dTI59w4AYvYvpAnKzISOh0FKaqyAv21pKs94VBUtphnUA3wp2V1vuGGP4Ye1uXvwxhZNn8hh+dWMe7N4IPx/POf2m3JstdzEZY3JFZDgwD+s210+MMSki8hKQYIyZJSLtgRlAdeB6EXnRGNPKWTW5hbxca3La2YvKO/+A3NOAWKeJOj7omJzWCfz1225JiQg3tK1Htya1ePHH9YxfsJnZSXt4Y1A00aHV7C5PKafSiXKuLj8f9qcUjBB2LIMzx6x9tVsWjBDCu0KF6vbW6gEWrN/HczOTyDx+hvtiGzKiZ1Nt/qfKtRKPIETkceBT4Djwb6AtMMYYM7/UqlQWY+DAloIRQtoSOH3I2lejEUQOLFgbobK2iChrPVvWoUNEDcbN2cCH8duZl7KXcQOj6dSwpt2lKVXqijWCEJF1xpjWItIbeAB4HvjSGNPO2QUWV7keQRxOO3cpzRN7re1VQqFhd8dCObFQNdTWMtW5lm09wJjpSew8dIrbOtZnTN/mBAb42l2WUpekNK5BnL2/rx9WMKSIziC6fMd2F8xWTou3Zi8DVKpdaHJaHFSPcNvJae6gS+NazH0ilvHzN/PJ0lR+27ifV2+M5OrmdewuTalSUdwRxKdYE98igNZYF51/N8Zc4dzyis+lRxAnDxR0PE1dDAe3WNsDqlmT0852PQ1qpoFQTv258zBPT0tky/4T3NAmhBeub0UNbf6nyoESN+tzzGxuA2w3xhwRkRpAqDEmsVQrLQGXCojTR6yLyWcXytmXbG33qwwNuhSMEOpE6uQ0N3ImN4/3Fm7jvd+3Ehjgy9j+rbg+OljbdSiXVhoB0RVYa4w5KSK3A+2At4wxO0q31Mtna0Bkn3TMVnaMEvasBZMPPgHW4jhn+xmFtAFvPUft7jbuPcboaYmsSz9KjxZW87+6VbX5n3JNpREQiVinlqKBz7DuZLrZGNO9FOsskTINiJwsa0La2dNG6QmQnwNevhAaUzBCCG3vtKU0lWvLyzd8siSVfy3YhK+XF3+/tgVD24fpaEK5nNIIiDXGmHYi8gKQYYz5+Oy20i72cjk1IPJyrJYVZ+802rUCcrNAvCC4TUEg1O8EfpWcU4Mql9IOnGTM9ET+2H6Izg1rMm5gFA1q6v8jynWURkAswmq7fQ8QC+wH1hljokqz0JIo1YDIz4O9SYWW0lwG2SesfXUiCwKhQRcIqFo6n6ncVn6+YcqqXbw2ewM5+fk81asZd3eN0OZ/yiWURkDUBW4FVhljFotIfeBKY8wXpVvq5StRQBgDmZsKOp6mLYGsI9a+mk3OXUqzUq1Sq1l5lj1HT/PcjGR+3bif1mHVeGNgNM3qajsUZa9SWXJUROoA7R0PVxpj9pdSfaXisgPiyE746Bo46TicavULLaUZC1WCS7dQ5dGMMfyYuIexs1I4npXDI1c15uErG2vzP2Wb0mi1cTPwJvA71qS5d0RkVFHdV8udKqHQtLfjbqNYa61lpZxEROjfOoRujWvx4o8pTPxlC3OS9vL6oGjahFWzuzylzlHsVhtAz7OjBhEJAn4xxrR2cn3F5lLzIJQqpl837OPZGcnsP57Fvd0iGNmzGRX8dG6MKjsXG0EUd1zrdd4ppYOX8Fql1AVc06IO80fGMbRDfT5anErvifEs23bA7rKUAor/j/xcEZknIsMcC/j8DMx2XllKeY4qAb78341R/Oe+TojArR+t4JnpiRzLyrG7NOXhLuUi9UCgq+PhYmPMDKdVdRn0FJNyB6ez85j4y2Y+WrydoEB/Xr0hih4ttfmfcp5SuYvJ1WlAKHeSmH6Ep6clsnHvca5vHcLY61tSs7LOylel77KvQYjIcRE5VsSv4yJyzDnlKqWiQ6sxa3g3RvZsytzkPfQYv4gf1mbgLl/oVPlw0YAwxgQaY6oU8SvQGFOlrIpUyhP5+Xjx2DVN+PmxWBrUrMTjU9Zy7+cJ7D5y2u7SlIfQO5GUcnFN6wTy/UNdeP66lizfdpBeE+L56o8d5OfraEI5lwaEUuWAt5dwb7cI5j0RR+uwqjw3M5lbPvqD1AMn7S5NuTENCKXKkfo1K/LVvR15Y2A06/cco8/EeD5ctI3cvHy7S1NuyKkBISJ9RGSTiGwVkTFF7I8TkTUikisigwptb+DYvlZEUkTkQWfWqVR5IiLc3D6MX0Z2J65pEK/N2chN7y9jwx69b0SVLqcFhIh4A5OAvkBL4BYRaXne03YCw4Bvztu+B+hsjGkDdATGiEiIs2pVqjyqUyWAyXdcwaRb27H7yGmuf2cJ4+dv4kxunt2lKTfhzBFEB2CrMWa7MSYbmAIMKPwEY0yaY13r/PO2Zxtjzjge+ju5TqXKLRHh2uhgFozoTv/WIbz921aue3sJa3Yetrs05Qac+Q9vPWBXocfpjm3FIiJhjqVOdwGvG2N2l3J9SrmN6pX8GD+kDZ/e3Z6TZ3IZ+P4yXvpxPaeyc+0uTZVjLvvN3BizyxgTDTQG7nKsR3EOEblfRBJEJCEzM7Psi1TKxVzVrDbzRsRxe8cGfLI0lV4T4lmyRZv/qcvjzIDIAMIKPQ51bLskjpFDMtZSp+fvm2yMiTHGxAQFBV12oUq5k8AAX16+IZKpD3TG19uL2z9ewdPT1nH0tDb/U5fGmQGxCmgiIhEi4gcMBWYV54UiEioiFRw/Vwe6AZucVqlSbqhDRA3mPB7LQ1c24vs1GfQcv4h5KXvtLkuVI04LCGNMLjAcmAdsAKYaY1JE5CUR6Q8gIu1FJB0YDHwoIimOl7cAVjgWKloE/NMYk+SsWpVyVwG+3ozu05yZD3elZmV/HvhyNY98vYbM42f++sXK42k3V6U8RE5ePpPjt/PWL1uo6O/NC9e15Ma29RARu0tTNiqNFeWUUuWcr7cXj1zVmNmPd6NRUGVGTl3HsE9XkaHN/9QFaEAo5WEa1w7kuwc6M/b6lqxKO0Sv8Yv4YnmaNv9T/0MDQikP5OUlDOtqNf9r16A6L/yQwpDJy9mWecLu0pQL0YBQyoOF1ajIF/d04M1B0Wzae5y+by3mvd+3avM/BWhAKOXxRITBMWH88mR3rm5WmzfmbuKG95aSsvuo3aUpm2lAKKUAqB0YwAd3XMH7t7Vj79Ez9H93KW/O20hWjjb/81QaEEqpc/SNCuaXkXHc2LYekxZuo9/bi0lIO2R3WcoGGhBKqf9RraIf/xzcmi/u6cCZnHwGf7icsbNSOHlGm/95Eg0IpdQFxTUNYv6IOO7qHM7ny9PoNSGe+M3aGNNTaEAopS6qkr8PY/u34rsHOuPv68Wdn6zkqe/WceRUtt2lKSfTgFBKFUtMeA1mPxbLI1c1YsafGfQYH8+cpD12l6WcSANCKVVsAb7ejOrdnFnDu1Knij8Pfb2Gh75azf7jWXaXppxAA0IpdclahVRl5iNdGd2nOb9u3E+Pfy3iu4RduEvzT2XRgFBKXRZfby8eurIRcx6PpVndQEZNS+TOT1ay69Apu0tTpUQDQilVIo2CKvPt/Z15eUAr1uw4TO+J8Xy2NFWb/7kBDQilVIl5eQl3dA5n3og42ofXYOyP6xn84XK27j9ud2mqBDQglFKlJrR6RT67uz3jb27NtswT9HtrCZMWbiVHm/+VSxoQSqlSJSLc1C6UBSO607NVHd6ct4n+7y4lOUOb/5U3GhBKKacICvRn0q3t+PCOKzhw4gwDJi1l3Bxt/leeaEAopZyqd6u6/DKiO4PahfLBom30e2sxK1O1+V95oAGhlHK6qhV9eX1QNF/d25HsvHxu/nA5z89M5oQ2/3NpGhBKqTLTrUkt5o+I456uEXy1Yge9xi9i4ab9dpelLsCpASEifURkk4hsFZExReyPE5E1IpIrIoMKbW8jIstFJEVEEkVkiDPrVEqVnYp+PrxwfUumPdiFiv4+3P3pKkZ+u5bDJ7X5n6txWkCIiDcwCegLtARuEZGW5z1tJzAM+Oa87aeAO40xrYA+wEQRqeasWpVSZe+KBtX5+bFuPHZ1Y2at202P8Yv4KXG3tutwIc4cQXQAthpjthtjsoEpwIDCTzDGpBljEoH887ZvNsZscfy8G9gPBDmxVqWUDfx9vBnZqxk/PtqNkGoVGP7Nnzzw5Wr2HdPmf67AmQFRD9hV6HG6Y9slEZEOgB+wrZTqUkq5mBbBVZjxcBee6ducRZsz6TF+Ed+u2qmjCZu59EVqEQkGvgTuNsb8z1RMEblfRBJEJCEzU1e5Uqo88/H24oHujZj7RBwtgqsw+vskbv94BTsPavM/uzgzIDKAsEKPQx3bikVEqgA/A88aY/4o6jnGmMnGmBhjTExQkJ6BUsodRNSqxJT7OvHKDZGs23WU3hPj+XhJKnna/K/MOTMgVgFNRCRCRPyAocCs4rzQ8fwZwBfGmGlOrFEp5YK8vITbOzVg/og4Ojeqycs/rWfQB8vYsk+b/5UlpwWEMSYXGA7MAzYAU40xKSLykoj0BxCR9iKSDgwGPhSRFMfLbwbigGEistbxq42zalVKuaaQahX4+K4Y3hrahrQDJ+n39mLe/nUL2bna/K8siLtcBIqJiTEJCQl2l6GUcpKDJ84w9sf1/LhuN83rBvL6wGhah1Wzu6xyT0RWG2Niitrn0heplVLqrJqV/XnnlrZ8dGcMh09lc+N7S3lt9gZOZ2vzP2fRgFBKlSs9W9ZhwcjuDGkfxofx2+n7Vjx/bD9od1luSQNCKVXuVAnw5bWbovnmbx3JNzB08h88OyOJ41k5dpfmVjQglFLlVpfGtZj3RBz3xUbwn5U76TUhnt827rO7LLehAaGUKtcq+Hnz7LUtmf5wV6oE+HLPZwk8PuVPDp44Y3dp5Z4GhFLKLbQJq8aPj3bjiR5NmJ20h54T4pm1Tpv/lYQGhFLKbfj5ePFEj6b89GgsYTUq8th//uS+LxLYe1Sb/10ODQillNtpVjeQ6Q914blrW7Bk6wF6jl/Ef1Zq879LpQGhlHJL3l7C32IbMu+JOCLrVeWZ6Unc+tEKdhw8aXdp5YYGhFLKrTWoWYlv7uvIuJuiSM6wmv99FL9dm/8VgwaEUsrtiQhDO9RnwcjudGtci1dnb+Cm95ayaa82/7sYDQillMeoWzWAj+6M4Z1b2pJ++DTXvbOYCQs2a/O/C9CAUEp5FBHh+tYhLBjZnWujgnnr1y1c985i1u46YndpLkcDQinlkWpU8mPi0LZ8MiyG41m53PTeUl75ab02/ytEA0Ip5dGubl6H+SPiuKVDff69JJXeE+NZtvWA3WW5BLdeDyInJ4f09HSysnSSzF8JCAggNDQUX19fu0tRyjZ/bD/ImO8TSTt4iqHtw3imXwuqVnDvvxMXWw/CrQMiNTWVwMBAatasiYjYVJnrM8Zw8OBBjh8/TkREhN3lKGWrrJw8JvyymY/itxMU6M8rN0TRs2Udu8tyGo9dMCgrK0vDoRhEhJo1a+pISykgwNebZ/q2YOYjXale0Y/7vkhg+DdrOOCBzf/cOiAADYdi0t8npc4VHVqNWcO78WTPpsxP2UfP8YuY+WeGR7XrcPuAsFvlypXtLkEpdZn8fLx49Jom/PxYN8JrVeKJb9dy7+cJ7D5y2u7SyoQGhFJK/YUmdQKZ9mAXXriuJcu3HaTXhHi++mMH+W7erkMDoowYYxg1ahSRkZFERUXx7bffArBnzx7i4uJo06YNkZGRLF68mLy8PIYNG/bf506YMMHm6pVS3l7CPd0imD8ijjZh1XhuZjJDP/qD1APu2/zPx5lvLiJ9gLcAb+Dfxphx5+2PAyYC0cBQY8y0QvvmAp2AJcaY60pay4s/prB+97GSvs05WoZU4R/XtyrWc6dPn87atWtZt24dBw4coH379sTFxfHNN9/Qu3dvnn32WfLy8jh16hRr164lIyOD5ORkAI4cOVKqdSulLl9YjYp8eW8HvktI5+Wf19NnYjwjejblb90i8PF2r+/cTjsaEfEGJgF9gZbALSLS8ryn7QSGAd8U8RZvAnc4q76ytmTJEm655Ra8vb2pU6cO3bt3Z9WqVbRv355PP/2UsWPHkpSURGBgIA0bNmT79u08+uijzJ07lypVqthdvlKqEBHh5vZh/DKyO92bBjFuzkZufG9ZqX8JtZszRxAdgK3GmO0AIjIFGACsP/sEY0yaY9//dMoyxvwqIleWVjHF/aZf1uLi4oiPj+fnn39m2LBhjBw5kjvvvJN169Yxb948PvjgA6ZOnconn3xid6lKqfPUqRLAh3dcweykvfxjVjL9313CQ1c2YvjVjfH38ba7vBJz5nioHrCr0ON0x7ZSIyL3i0iCiCRkZmaW5luXutjYWL799lvy8vLIzMwkPj6eDh06sGPHDurUqcN9993H3/72N9asWcOBAwfIz89n4MCBvPLKK6xZs8bu8pVSFyAiXBsdzIIR3enfJoR3ftvKtW8vYfWOw3aXVmJOvQbhbMaYycBksGZS21zORd14440sX76c1q1bIyK88cYb1K1bl88//5w333wTX19fKleuzBdffEFGRgZ33303+fnWwOq1116zuXql1F+pXsmP8Te3oX/rEJ6dkcygD5YxrEs4T/VqRiX/8vlPrdNabYhIZ2CsMaa34/EzAMaY//nXTkQ+A34qfJHasf1K4KniXKQuqtXGhg0baNGixWUegefR3y+lSseJM7m8MXcjXyzfQWj1Crx2UxSxTYLsLqtIdrXaWAU0EZEIEfEDhgKznPh5SinlEir7+/DSgEimPtAZP28v7vh4JU9PW8fRUzl2l3ZJnBYQxphcYDgwD9gATDXGpIjISyLSH0BE2otIOjAY+FBEUs6+XkQWA98B14hIuoj0dlatSinlDB0iajD78VgeurIR36/JoMeERcxN3mt3WcXm1BNjxpjZwOzztr1Q6OdVQOgFXhvrzNqUUqosBPh6M7pPc66NCubpaYk8+NVqro0KZmz/VgQF+ttd3kW516wOpZRyUZH1qvLD8K6M6t2MBRv20WP8Ir5fne7Szf80IJRSqoz4envxyFWNmf1YLI1rV+bJ79Zx16erSD98yu7SiqQBoZRSZaxx7cp890BnXuzfioS0Q/SeEM8Xy9NcrvmfBoRSStnAy0u4q0s4856Io12D6rzwQwpDJi9nW+YJu0v7Lw0IF3Ox9SPS0tKIjIwsw2qUUs4WVqMiX9zTgX8Obs3mfSfo+9Zi3vt9Kzl5/9OBqMxpQCillM1EhEFXhLJgZBw9WtTmjbmbuGHSUpIzjtpaV/mc/3055oyBvUml+551o6DvuIs+ZcyYMYSFhfHII48AMHbsWHx8fFi4cCGHDx8mJyeHV155hQEDBlzSR2dlZfHQQw+RkJCAj48P48eP56qrriIlJYW7776b7Oxs8vPz+f777wkJCeHmm28mPT2dvLw8nn/+eYYMGXLZh62Uco7agQG8d9sVzE3ew3MzUxgwaSkPdm/Io1c3IcC37Jv/eU5A2GTIkCE88cQT/w2IqVOnMm/ePB577DGqVKnCgQMH6NSpE/3797+kdaEnTZqEiJCUlMTGjRvp1asXmzdv5oMPPuDxxx/ntttuIzs7m7y8PGbPnk1ISAg///wzAEeP2vutRCl1cX0ig+ncsBav/LyeSQu3MSd5L28MjCYmvEaZ1uE5AfEX3/SdpW3btuzfv5/du3eTmZlJ9erVqVu3LiNGjCA+Ph4vLy8yMjLYt28fdevWLfb7LlmyhEcffRSA5s2b06BBAzZv3kznzp159dVXSU9P56abbqJJkyZERUXx5JNPMnr0aK677jpiY3UOolKurmpFX94c3JrrW4fwzPQkBn+4nDs7NWBUn+ZULqPmf3oNogwMHjyYadOm8e233zJkyBC+/vprMjMzWb16NWvXrqVOnTpkZWWVymfdeuutzJo1iwoVKtCvXz9+++03mjZtypo1a4iKiuK5557jpZdeKpXPUko5X1zTIOaPiOOuzuF88ccOek+IZ9HmslneQAOiDAwZMoQpU6Ywbdo0Bg8ezNGjR6lduza+vr4sXLiQHTt2XPJ7xsbG8vXXXwOwefNmdu7cSbNmzdi+fTsNGzbkscceY8CAASQmJrJ7924qVqzI7bffzqhRo3R9CaXKmUr+Pozt34rvHuhMgK8Xd32ykienruPIqWynfq7nnGKyUatWrTh+/Dj16tUjODiY2267jeuvv56oqChiYmJo3rz5Jb/nww8/zEMPPURUVBQ+Pj589tln+Pv7M3XqVL788kt8fX2pW7cuf//731m1ahWjRo3Cy8sLX19f3n//fSccpVLK2WLCa/DzY7G8+9tWPli0jUWbM3l5QCv6RgU75fOcth5EWdP1IEpOf7+UKj9Sdh9l9PeJJGcc49qoYN65pS1eXsW/0eWsi60HoSMIpZQqh1qFVGXmw13595JUTmTlXlY4/BUNCBeUlJTEHXfccc42f39/VqxYYVNFSilX5OPtxYPdGznv/Z32zuqyRUVFsXbtWrvLUEp5OLe/i8ldrrE4m/4+KaXO59YBERAQwMGDB/Ufv79gjOHgwYMEBATYXYpSyoW49Smm0NBQ0tPTycwsm0kl5VlAQAChoUWu/qqU8lBuHRC+vr5ERETYXYZSSpVLbn2KSSml1OXTgFBKKVUkDQillFJFcptWGyKSCVx617sCtYADpVROeeFpx+xpxwt6zJ6iJMfcwBgTVNQOtwmIkhKRhAv1I3FXnnbMnna8oMfsKZx1zHqKSSmlVJE0IJRSShVJA6LAZLsLsIGnHbOnHS/oMXsKpxyzXoNQSilVJB1BKKWUKpJHBYSI9BGRTSKyVUTGFLHfX0S+dexfISLhNpRZqopxzCNFZL2IJIrIryLSwI46S9NfHXOh5w0UESMi5f6Ol+Ics4jc7PizThGRb8q6xtJWjP+364vIQhH50/H/dz876iwtIvKJiOwXkeQL7BcRedvx+5EoIu1K/KHGGI/4BXgD24CGgB+wDmh53nMeBj5w/DwU+NbuusvgmK8CKjp+fsgTjtnxvEAgHvgDiLG77jL4c24C/AlUdzyubXfdZXDMk4GHHD+3BNLsrruExxwHtAOSL7C/HzAHEKATsKKkn+lJI4gOwFZjzHZjTDYwBRhw3nMGAJ87fp4GXCMipb+OX9n5y2M2xiw0xpxyPPwDKO8tXYvz5wzwMvA6kFWWxTlJcY75PmCSMeYwgDFmfxnXWNqKc8wGqOL4uSqwuwzrK3XGmHjg0EWeMgD4wlj+AKqJSHBJPtOTAqIesKvQ43THtiKfY4zJBY4CNcukOucozjEXdi/WN5Dy7C+P2TH0DjPG/FyWhTlRcf6cmwJNRWSpiPwhIn3KrDrnKM4xjwVuF5F0YDbwaNmUZptL/fv+l9y63bcqPhG5HYgButtdizOJiBcwHhhmcyllzQfrNNOVWKPEeBGJMsYcsbMoJ7sF+MwY8y8R6Qx8KSKRxph8uwsrLzxpBJEBhBV6HOrYVuRzRMQHa1h6sEyqc47iHDMi0gN4FuhvjDlTRrU5y18dcyAQCfwuImlY52pnlfML1cX5c04HZhljcowxqcBmrMAor4pzzPcCUwGMMcuBAKyeRe6qWH/fL4UnBcQqoImIRIiIH9ZF6FnnPWcWcJfj50HAb8Zx9aec+stjFpG2wIdY4VDez0vDXxyzMeaoMaaWMSbcGBOOdd2lvzEmwZ5yS0Vx/t+eiTV6QERqYZ1y2l6GNZa24hzzTuAaABFpgRUQ7ry85CzgTsfdTJ2Ao8aYPSV5Q485xWSMyRWR4cA8rDsgPjHGpIjIS0CCMWYW8DHWMHQr1sWgofZVXHLFPOY3gcrAd47r8TuNMf1tK7qEinnMbqWYxzwP6CUi64E8YJQxptyOjot5zE8CH4nICKwL1sPK8xc+EfkPVsjXclxX+QfgC2CM+QDrOks/YCtwCri7xJ9Zjn+/lFJKOZEnnWJSSil1CTQglFJKFUkDQimlVJE0IJRSShVJA0IppVSRNCCU+gsikiciawv9umCH2Mt47/ALdedUym4eMw9CqRI4bYxpY3cRSpU1HUEodZlEJE1E3hCRJBFZKSKNHdvDReS3Qmts1HdsryMiM0RkneNXF8dbeYvIR451GuaLSAXH8x8rtFbHFJsOU3kwDQil/lqF804xDSm076gxJgp4F5jo2PYO8LkxJhr4Gnjbsf1tYJExpjVWX/8Ux/YmWK24WwFHgIGO7WOAto73edA5h6bUhelMaqX+goicMMZULmJ7GnC1MWa7iPgCe40xNUXkABBsjMlxbN9jjKklIplAaOGGiGKtWrjAGNPE8Xg04GuMeUVE5gInsPoozTTGnHDyoSp1Dh1BKFUy5gI/X4rCHXTzKLg2eC0wCWu0scrRYVipMqMBoVTJDCn03+WOn5dR0OjxNmCx4+dfsZZ1RUS8RaTqhd7UsW5FmDFmITAaq/X8/4xilHIm/Uai1F+rICJrCz2ea4w5e6trdRFJxBoF3OLY9ijwqYiMwmovfbar5uPAZBG5F2uk8BBwoXbM3sBXjhAR4G03X9xHuSC9BqHUZXJcg4gxxhywuxalnEFPMSmllCqSjiCUUkoVSUcQSimliqQBoZRSqkgaEEoppYqkAaGUUqpIGhBKKaWKpAGhlFKqSP8PUBD0IOLKLfwAAAAASUVORK5CYII=\n", + "text/plain": [ + "
" + ] + }, + "metadata": { + "tags": [], + "needs_background": "light" + } + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "hkhtCCgnUbY6", + "colab": { + "base_uri": "https://localhost:8080/", + "height": 110 + }, + "outputId": "9a82488a-2427-433e-a5eb-5741ccfc3bcf" + }, + "source": [ + "model.evaluate(valid_data, verbose=1)" + ], + "execution_count": null, + "outputs": [ + { + "output_type": "stream", + "text": [ + " 2/30 [=>............................] - ETA: 4s - loss: 0.1791 - binary_accuracy: 0.9375WARNING:tensorflow:Callbacks method `on_test_batch_end` is slow compared to the batch time (batch time: 0.0145s vs `on_test_batch_end` time: 0.1386s). Check your callbacks.\n" + ], + "name": "stdout" + }, + { + "output_type": "stream", + "text": [ + "WARNING:tensorflow:Callbacks method `on_test_batch_end` is slow compared to the batch time (batch time: 0.0145s vs `on_test_batch_end` time: 0.1386s). Check your callbacks.\n" + ], + "name": "stderr" + }, + { + "output_type": "stream", + "text": [ + "30/30 [==============================] - 5s 154ms/step - loss: 0.1600 - binary_accuracy: 0.9563\n" + ], + "name": "stdout" + }, + { + "output_type": "execute_result", + "data": { + "text/plain": [ + "[0.1600438952445984, 0.956250011920929]" + ] + }, + "metadata": { + "tags": [] + }, + "execution_count": 23 + } + ] + }, + { + "cell_type": "code", + "metadata": { + "id": "K4B8NQBLd9rN" + }, + "source": [ + "sample_example = [\" \",\\\n", + " \" \",\\\n", + " \" \",\\\n", + " \" \",\\\n", + " \" \",\\\n", + " \" \"]\n", + "test_data = tf.data.Dataset.from_tensor_slices((sample_example, [0]*len(sample_example)))\n", + "test_data = (test_data.map(to_feature_map).batch(1))\n", + "preds = model.predict(test_data)\n", + "#['Toxic' if pred >=0.5 else 'Sincere' for pred in preds]" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "FeVNOGfFJT9O" + }, + "source": [ + "preds" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "I_YWudFRJT__" + }, + "source": [ + "" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "hENB__IlJUCk" + }, + "source": [ + "" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "wkYpiGrhJUFK" + }, + "source": [ + "" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "iYqbQZJnJUHw" + }, + "source": [ + "" + ], + "execution_count": null, + "outputs": [] + }, + { + "cell_type": "code", + "metadata": { + "id": "aiKuBGgfJUKv" + }, + "source": [ + "" + ], + "execution_count": null, + "outputs": [] + } + ] +} \ No newline at end of file