{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "\"Dask\n", "\n", "# Embarrassingly parallel Workloads\n", "\n", "This notebook shows how to use Dask to parallelize embarrassingly parallel workloads where you want to apply one function to many pieces of data independently. It will show three different ways of doing this with Dask:\n", "\n", "1. [dask.delayed](http://dask.pydata.org/en/latest/delayed.html) \n", "2. [concurrent.Futures](https://dask.pydata.org/en/latest/futures.html) \n", "3. [dask.bag](https://dask.pydata.org/en/latest/bag.html)\n", "\n", "This example focuses on using Dask for building large embarrassingly parallel computation as often seen in scientific communities and on High Performance Computing facilities, for example with Monte Carlo methods. This kind of simulation assume the following:\n", "\n", " - We have a function that runs a heavy computation given some parameters.\n", " - We need to compute this function on many different input parameters, each function call being independent.\n", " - We want to gather all the results in one place for further analysis." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Start Dask Client for Dashboard\n", "\n", "Starting the Dask Client will provide a dashboard which \n", "is useful to gain insight on the computation. We will also need it for the\n", "Futures API part of this example. Moreover, as this kind of computation\n", "is often launched on super computer or in the Cloud, you will probably end\n", "up having to start a cluster and connect a client to scale. See \n", "[dask-jobqueue](https://github.com/dask/dask-jobqueue),\n", "[dask-kubernetes](https://github.com/dask/dask-kubernetes) or \n", "[dask-yarn](https://github.com/dask/dask-yarn) for easy ways to achieve this\n", "on respectively an HPC, Cloud or Big Data infrastructure.\n", "\n", "The link to the dashboard will become visible when you create the client below. We recommend having it open on one side of your screen while using your notebook on the other side. This can take some effort to arrange your windows, but seeing them both at the same time is very useful when learning." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "execution": { "iopub.execute_input": "2022-07-27T19:15:04.996604Z", "iopub.status.busy": "2022-07-27T19:15:04.996351Z", "iopub.status.idle": "2022-07-27T19:15:07.092104Z", "shell.execute_reply": "2022-07-27T19:15:07.091198Z" } }, "outputs": [ { "data": { "text/html": [ "
\n", "
\n", "
\n", "

Client

\n", "

Client-6c01af35-0de0-11ed-9e28-000d3a8f7959

\n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "
Connection method: Cluster objectCluster type: distributed.LocalCluster
\n", " Dashboard: http://127.0.0.1:8787/status\n", "
\n", "\n", " \n", "
\n", "

Cluster Info

\n", "
\n", "
\n", "
\n", "
\n", "

LocalCluster

\n", "

6c9279a2

\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", " \n", "
\n", " Dashboard: http://127.0.0.1:8787/status\n", " \n", " Workers: 1\n", "
\n", " Total threads: 4\n", " \n", " Total memory: 6.78 GiB\n", "
Status: runningUsing processes: True
\n", "\n", "
\n", " \n", "

Scheduler Info

\n", " \n", "\n", "
\n", "
\n", "
\n", "
\n", "

Scheduler

\n", "

Scheduler-859f8895-4dc5-45e4-945c-593b2390b28a

\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", " Comm: tcp://127.0.0.1:46295\n", " \n", " Workers: 1\n", "
\n", " Dashboard: http://127.0.0.1:8787/status\n", " \n", " Total threads: 4\n", "
\n", " Started: Just now\n", " \n", " Total memory: 6.78 GiB\n", "
\n", "
\n", "
\n", "\n", "
\n", " \n", "

Workers

\n", " \n", "\n", " \n", "
\n", "
\n", "
\n", "
\n", " \n", "

Worker: 0

\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", "\n", " \n", "\n", "
\n", " Comm: tcp://127.0.0.1:33017\n", " \n", " Total threads: 4\n", "
\n", " Dashboard: http://127.0.0.1:45931/status\n", " \n", " Memory: 6.78 GiB\n", "
\n", " Nanny: tcp://127.0.0.1:46881\n", "
\n", " Local directory: /home/runner/work/dask-examples/dask-examples/applications/dask-worker-space/worker-ii4pk6hj\n", "
\n", "
\n", "
\n", "
\n", " \n", "\n", "
\n", "
\n", "\n", "
\n", "
\n", "
\n", "
\n", " \n", "\n", "
\n", "
" ], "text/plain": [ "" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from dask.distributed import Client, progress\n", "client = Client(threads_per_worker=4, n_workers=1)\n", "client" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Define your computation calling function\n", "\n", "This function does a simple operation: add all numbers of a list/array together, but it also sleeps for a random amount of time to simulate real work. In real use cases, this could call another python module, or even run an executable using subprocess module." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "execution": { "iopub.execute_input": "2022-07-27T19:15:07.095386Z", "iopub.status.busy": "2022-07-27T19:15:07.095185Z", "iopub.status.idle": "2022-07-27T19:15:07.101280Z", "shell.execute_reply": "2022-07-27T19:15:07.100721Z" } }, "outputs": [], "source": [ "import time\n", "import random\n", "\n", "def costly_simulation(list_param):\n", " time.sleep(random.random())\n", " return sum(list_param)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We try it locally below" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "execution": { "iopub.execute_input": "2022-07-27T19:15:07.104106Z", "iopub.status.busy": "2022-07-27T19:15:07.103909Z", "iopub.status.idle": "2022-07-27T19:15:07.257811Z", "shell.execute_reply": "2022-07-27T19:15:07.256957Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "CPU times: user 2.24 ms, sys: 1.62 ms, total: 3.85 ms\n", "Wall time: 146 ms\n" ] }, { "data": { "text/plain": [ "10" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "%time costly_simulation([1, 2, 3, 4])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Define the set of input parameters to call the function\n", "\n", "We will generate a set of inputs on which we want to run our simulation function. Here we use Pandas dataframe, but we could also use a simple list. Lets say that our simulation is run with four parameters called param_[a-d]." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "execution": { "iopub.execute_input": "2022-07-27T19:15:07.261431Z", "iopub.status.busy": "2022-07-27T19:15:07.260853Z", "iopub.status.idle": "2022-07-27T19:15:07.276055Z", "shell.execute_reply": "2022-07-27T19:15:07.275412Z" } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
param_aparam_bparam_cparam_d
00.3121240.0710360.7999190.766664
10.9565770.4711110.5544420.263298
20.5832020.7894350.1356100.622723
30.9900320.2530900.2250350.190979
40.1309920.3032750.5145550.420719
\n", "
" ], "text/plain": [ " param_a param_b param_c param_d\n", "0 0.312124 0.071036 0.799919 0.766664\n", "1 0.956577 0.471111 0.554442 0.263298\n", "2 0.583202 0.789435 0.135610 0.622723\n", "3 0.990032 0.253090 0.225035 0.190979\n", "4 0.130992 0.303275 0.514555 0.420719" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import pandas as pd\n", "import numpy as np\n", "\n", "input_params = pd.DataFrame(np.random.random(size=(500, 4)),\n", " columns=['param_a', 'param_b', 'param_c', 'param_d'])\n", "input_params.head()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Without using Dask, we could call our simulation on all of these parameters using normal Python for loops.\n", "\n", "Let's only do this on a sample of our parameters as it would be quite long otherwise." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "execution": { "iopub.execute_input": "2022-07-27T19:15:07.278842Z", "iopub.status.busy": "2022-07-27T19:15:07.278495Z", "iopub.status.idle": "2022-07-27T19:15:07.281759Z", "shell.execute_reply": "2022-07-27T19:15:07.281094Z" } }, "outputs": [], "source": [ "results = []" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "execution": { "iopub.execute_input": "2022-07-27T19:15:07.284811Z", "iopub.status.busy": "2022-07-27T19:15:07.284274Z", "iopub.status.idle": "2022-07-27T19:15:11.544196Z", "shell.execute_reply": "2022-07-27T19:15:11.543206Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "CPU times: user 123 ms, sys: 15.1 ms, total: 138 ms\n", "Wall time: 4.25 s\n" ] } ], "source": [ "%%time\n", "for parameters in input_params.values[:10]:\n", " result = costly_simulation(parameters)\n", " results.append(result)" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "execution": { "iopub.execute_input": "2022-07-27T19:15:11.548233Z", "iopub.status.busy": "2022-07-27T19:15:11.547673Z", "iopub.status.idle": "2022-07-27T19:15:11.553431Z", "shell.execute_reply": "2022-07-27T19:15:11.552765Z" } }, "outputs": [ { "data": { "text/plain": [ "[1.9497434903818136,\n", " 2.245428056748925,\n", " 2.1309710532899184,\n", " 1.65913554017504,\n", " 1.3695413820396394,\n", " 1.5063310042417761,\n", " 1.2444777543824386,\n", " 1.531794781158632,\n", " 1.3147033688719252,\n", " 2.925532773375028]" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "results" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that this is not very clever as we can easily parallelize code. \n", "\n", "There are many ways to parallelize this function in Python with libraries like `multiprocessing`, `concurrent.futures`, `joblib` or others. These are good first steps. Dask is a good second step, especially when you want to scale across many machines.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Use [Dask Delayed](http://dask.pydata.org/en/latest/delayed.html) to make our function lazy\n", "\n", "We can call `dask.delayed` on our funtion to make it lazy. Rather than compute its result immediately, it records what we want to compute as a task into a graph that we'll run later on parallel hardware. Using `dask.delayed` is a relatively straightforward way to parallelize an existing code base, even if the computation isn't embarrassingly parallel like this one. \n", "\n", "Calling these lazy functions is now almost free. In the cell below we only construct a simple graph." ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "execution": { "iopub.execute_input": "2022-07-27T19:15:11.556361Z", "iopub.status.busy": "2022-07-27T19:15:11.555938Z", "iopub.status.idle": "2022-07-27T19:15:11.559125Z", "shell.execute_reply": "2022-07-27T19:15:11.558495Z" } }, "outputs": [], "source": [ "import dask\n", "lazy_results = []" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "execution": { "iopub.execute_input": "2022-07-27T19:15:11.561979Z", "iopub.status.busy": "2022-07-27T19:15:11.561557Z", "iopub.status.idle": "2022-07-27T19:15:11.566831Z", "shell.execute_reply": "2022-07-27T19:15:11.566378Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "CPU times: user 1.43 ms, sys: 0 ns, total: 1.43 ms\n", "Wall time: 1.08 ms\n" ] } ], "source": [ "%%time\n", "\n", "for parameters in input_params.values[:10]:\n", " lazy_result = dask.delayed(costly_simulation)(parameters)\n", " lazy_results.append(lazy_result)" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "execution": { "iopub.execute_input": "2022-07-27T19:15:11.569729Z", "iopub.status.busy": "2022-07-27T19:15:11.569539Z", "iopub.status.idle": "2022-07-27T19:15:11.573795Z", "shell.execute_reply": "2022-07-27T19:15:11.573161Z" } }, "outputs": [ { "data": { "text/plain": [ "Delayed('costly_simulation-ae2bb390-5058-4db9-a033-6498fb822270')" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "lazy_results[0]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Run in parallel\n", "\n", "The `lazy_results` list contains information about ten calls to `costly_simulation` that have not yet been run. Call `.compute()` when you want your result as normal Python objects.\n", "\n", "If you started `Client()` above then you may want to watch the status page during computation." ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "execution": { "iopub.execute_input": "2022-07-27T19:15:11.577476Z", "iopub.status.busy": "2022-07-27T19:15:11.576934Z", "iopub.status.idle": "2022-07-27T19:15:13.027453Z", "shell.execute_reply": "2022-07-27T19:15:13.026640Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "CPU times: user 261 ms, sys: 43.6 ms, total: 304 ms\n", "Wall time: 1.44 s\n" ] }, { "data": { "text/plain": [ "(1.9497434903818136,\n", " 2.245428056748925,\n", " 2.1309710532899184,\n", " 1.65913554017504,\n", " 1.3695413820396394,\n", " 1.5063310042417761,\n", " 1.2444777543824386,\n", " 1.531794781158632,\n", " 1.3147033688719252,\n", " 2.925532773375028)" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "%time dask.compute(*lazy_results)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Notice that this was faster than running these same computations sequentially with a for loop. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can now run this on all of our input parameters:" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "execution": { "iopub.execute_input": "2022-07-27T19:15:13.033877Z", "iopub.status.busy": "2022-07-27T19:15:13.030023Z", "iopub.status.idle": "2022-07-27T19:15:13.131932Z", "shell.execute_reply": "2022-07-27T19:15:13.131161Z" } }, "outputs": [], "source": [ "import dask\n", "lazy_results = []\n", "\n", "for parameters in input_params.values:\n", " lazy_result = dask.delayed(costly_simulation)(parameters)\n", " lazy_results.append(lazy_result)\n", " \n", "futures = dask.persist(*lazy_results) # trigger computation in the background" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To make this go faster, we can add additional workers.\n", "\n", "(although we're still only working on our local machine, this is more practical when using an actual cluster)" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "execution": { "iopub.execute_input": "2022-07-27T19:15:13.139972Z", "iopub.status.busy": "2022-07-27T19:15:13.139603Z", "iopub.status.idle": "2022-07-27T19:15:13.175478Z", "shell.execute_reply": "2022-07-27T19:15:13.174264Z" } }, "outputs": [], "source": [ "client.cluster.scale(10) # ask for ten 4-thread workers" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "By looking at the Dask dashboard we can see that Dask spreads this work around our cluster, managing load balancing, dependencies, etc..\n", "\n", "Then get the result:" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "execution": { "iopub.execute_input": "2022-07-27T19:15:13.179156Z", "iopub.status.busy": "2022-07-27T19:15:13.178716Z", "iopub.status.idle": "2022-07-27T19:15:25.032623Z", "shell.execute_reply": "2022-07-27T19:15:25.031631Z" } }, "outputs": [ { "data": { "text/plain": [ "(1.9497434903818136,\n", " 2.245428056748925,\n", " 2.1309710532899184,\n", " 1.65913554017504,\n", " 1.3695413820396394)" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "results = dask.compute(*futures)\n", "results[:5]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Using the [Futures API](http://dask.pydata.org/en/latest/futures.html)\n", "\n", "The same example can be implemented using Dask's Futures API by using the `client` object itself. For our use case of applying a function across many inputs both Dask delayed and Dask Futures are equally useful. The Futures API is a little bit different because it starts work immediately rather than being completely lazy.\n", "\n", "For example, notice that work starts immediately in the cell below as we submit work to the cluster:" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "execution": { "iopub.execute_input": "2022-07-27T19:15:25.036850Z", "iopub.status.busy": "2022-07-27T19:15:25.036236Z", "iopub.status.idle": "2022-07-27T19:15:27.016527Z", "shell.execute_reply": "2022-07-27T19:15:27.015362Z" } }, "outputs": [], "source": [ "futures = []\n", "for parameters in input_params.values:\n", " future = client.submit(costly_simulation, parameters)\n", " futures.append(future)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can explicitly wait until this work is done and gather the results to our local process by calling `client.gather`:" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "execution": { "iopub.execute_input": "2022-07-27T19:15:27.022676Z", "iopub.status.busy": "2022-07-27T19:15:27.022015Z", "iopub.status.idle": "2022-07-27T19:15:33.986227Z", "shell.execute_reply": "2022-07-27T19:15:33.985591Z" } }, "outputs": [ { "data": { "text/plain": [ "[1.9497434903818136,\n", " 2.245428056748925,\n", " 2.1309710532899184,\n", " 1.65913554017504,\n", " 1.3695413820396394]" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "results = client.gather(futures)\n", "results[:5]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "But the code above can be run in fewer lines with `client.map()` function, allowing to call a given function on a list of parameters.\n", "\n", "As for delayed, we can only start the computation and not wait for results by not calling `client.gather()` right now.\n", "\n", "It shall be noted that as Dask cluster has already performed tasks launching `costly_simulation` with Futures API on the given input parameters, the call to `client.map()` won't actually trigger any computation, and just retrieve already computed results." ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "execution": { "iopub.execute_input": "2022-07-27T19:15:33.990936Z", "iopub.status.busy": "2022-07-27T19:15:33.989509Z", "iopub.status.idle": "2022-07-27T19:15:34.059272Z", "shell.execute_reply": "2022-07-27T19:15:34.058593Z" } }, "outputs": [], "source": [ "futures = client.map(costly_simulation, input_params.values)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Then just get the results later:" ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "execution": { "iopub.execute_input": "2022-07-27T19:15:34.063512Z", "iopub.status.busy": "2022-07-27T19:15:34.063311Z", "iopub.status.idle": "2022-07-27T19:15:34.122496Z", "shell.execute_reply": "2022-07-27T19:15:34.121974Z" } }, "outputs": [ { "data": { "text/plain": [ "500" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "results = client.gather(futures)\n", "len(results)" ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "execution": { "iopub.execute_input": "2022-07-27T19:15:34.126247Z", "iopub.status.busy": "2022-07-27T19:15:34.126046Z", "iopub.status.idle": "2022-07-27T19:15:34.129614Z", "shell.execute_reply": "2022-07-27T19:15:34.128883Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1.9497434903818136\n" ] } ], "source": [ "print(results[0])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We encourage you to watch the [dashboard's status page](http://127.0.0.1:8787) to watch on going computation." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Doing some analysis on the results\n", "\n", "One of the interests of Dask here, outside from API simplicity, is that you are able to gather the result for all your simulations in one call. There is no need to implement a complex mechanism or to write individual results in a shared file system or object store.\n", "\n", "Just get your result, and do some computation." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Here, we will just get the results and expand our initial dataframe to have a nice view of parameters vs results for our computation" ] }, { "cell_type": "code", "execution_count": 20, "metadata": { "execution": { "iopub.execute_input": "2022-07-27T19:15:34.133587Z", "iopub.status.busy": "2022-07-27T19:15:34.132840Z", "iopub.status.idle": "2022-07-27T19:15:34.147269Z", "shell.execute_reply": "2022-07-27T19:15:34.146511Z" } }, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
param_aparam_bparam_cparam_dresult
940.3982140.4723490.3669290.8227202.060212
1140.8087150.8632100.8180880.3961122.886125
3420.0725360.8142480.3090660.6317321.827582
4270.8222340.9476080.4230600.2008212.393723
3220.3581920.2283150.8649030.1956841.647095
\n", "
" ], "text/plain": [ " param_a param_b param_c param_d result\n", "94 0.398214 0.472349 0.366929 0.822720 2.060212\n", "114 0.808715 0.863210 0.818088 0.396112 2.886125\n", "342 0.072536 0.814248 0.309066 0.631732 1.827582\n", "427 0.822234 0.947608 0.423060 0.200821 2.393723\n", "322 0.358192 0.228315 0.864903 0.195684 1.647095" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "output = input_params.copy()\n", "output['result'] = pd.Series(results, index=output.index)\n", "output.sample(5)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Then we can do some nice statistical plots or save result locally with pandas interface here" ] }, { "cell_type": "code", "execution_count": 21, "metadata": { "execution": { "iopub.execute_input": "2022-07-27T19:15:34.150066Z", "iopub.status.busy": "2022-07-27T19:15:34.149882Z", "iopub.status.idle": "2022-07-27T19:15:35.150042Z", "shell.execute_reply": "2022-07-27T19:15:35.149147Z" } }, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8qNh9FAAAACXBIWXMAAAsTAAALEwEAmpwYAABsm0lEQVR4nO29d7wlRZk+/rx9zrlhch6GmWGGIQcJOmRQkiRZ07q7uoY1LYKYXV0wZ/n5W3ddFxVBXF0FwwoiS1AEyZKGYRjCoAxDGhiYHO7MDeecru8f3dVdVV1VXSfde8659Xw+MPd0qK7qrnrrrecNRYwxeHh4eHh0PoKxroCHh4eHR3PgBbqHh4dHl8ALdA8PD48ugRfoHh4eHl0CL9A9PDw8ugTFsXrwrFmz2OLFi8fq8R4eHh4diQcffHAjY2y27tyYCfTFixdj2bJlY/V4Dw8Pj44EET1rOucpFw8PD48ugRfoHh4eHl0CL9A9PDw8ugReoHt4eHh0CbxA9/Dw8OgSeIHu4eHh0SXwAt3Dw8OjS+AFepNwzUMvYGC4MtbV8PDwGMfwAr0JePj5rfjYr1bgc799ZKyr4uHhMY7hBXoTsHMk0szXbRsa45p4eHiMZ3iB3gQQaKyr4OHh4eEFejPhN/Pz8PAYS3iB3gSQV9A9PDzaAF6gNxNeRffw8BhDeIHeBHgF3cPDox3gBXoTwbyK7uHhMYbwAr0JIE+ie3h4tAG8QG8imFfQPTw8xhBeoDcBXkH38PBoB3iB3kR4Bd3Dw2Ms4QV6E+AVdA8Pj3aAF+hNBPMkuoeHxxgiV6ATUR8R3U9EDxPRY0T0Zc01JxLRNiJaEf/3hdZUtz3hOXQPD492QNHhmmEAJzPGBoioBOAuIrqRMXavct2djLGzm1/FzoHXzz08PMYSuRo6izAQ/yzF/3nZ5eHh0XVgjGGoXB3ratQNJw6diApEtALAegB/ZIzdp7nsmJiWuZGIDjKUcw4RLSOiZRs2bKi/1m0GTp17Ct3Do7Nx9+pNOOwrN2HrrpGxrkpdcBLojLEqY+wwAAsAHElEByuXLAewiDF2KID/AnCNoZxLGWNLGWNLZ8+eXX+t2wxejnt4dAde3DaIoXKIbYPlsa5KXajJy4UxthXAbQDOUI5v57QMY+wGACUimtWkOrY9wjAS6V6wjx42DgzjyvueG+tqeHQZ+FgOO3Qwu3i5zCaiafHf/QBOBfCEcs1uFCc0IaIj43I3Nb22bYoO/fYdjRsffQmf+e0j2DgwPNZV8egiVGPetFNdkF28XOYB+CkRFRAJ6l8zxq4jonMBgDF2CYC3ADiPiCoABgG8lXXqG6kDoSfRRx3VaggAKMf/eng0A52uoecKdMbYSgCHa45fIvx9MYCLm1u1DkKHfvxOBh9wlap/+R7NQzXsbA3dR4o2AVy4dGYXaB2uW/kiPn/Noy0pm6+KKp2qSnm0JTp9LHuB3gT4jS30+NCVD+Fn9z7bkrK5AlUNPeXi0TxwRSH0Gvr4RdjlFHoYMiy+4Hr86M41Y12VBHzAlT3l4tFEpJTLGFekTniB3gRwvq1bNfVyrAV/6/d/GeOapAgTDb0737kOmwaG8etlz491Nboa1Q7X0F28XDxy0KHf3hnt2L5UQx8/lMtHf7kCd63eiCMWz8CesyaOdXW6Ep3usOY19CaAa+ad2gk6EXxVNJ40dO5zv2ukMsY16V54ysWjYz++K9qxfVyOjycOvVSIhqt31WwdqmFnUy5eoDcBXW8UbcOGheNQQy8VosT7Fe/Z0zKEiT2sM+EFeoMIQ4bV6wfyL2wDPLJ2G25Z9XLN9yUCvY028kg09HEk3Iqxhj5S6VRx0/7odA3dG0UbxCV3PJV4f7R7F/ibi+8CADxz0etquq8dleCEQx9H9IPX0FuPdLXdmf3Ka+gNYvmzW8e6Ci1HO3buNFJ0/Ai3YuA59FYjoVw69BV7gd4w0i/fjoKvGWjHZnFvxfEU+l8MIg19PLlqjjaqHZ6cywv0FmOkEuIvL+0Y62o0hHbkE/nk2Sptdc2GAdzzVHtlgE68XDpV2nQAfHIuDyu+eO1jOP07d+ClbUNjXRVn7BgqY7iS7qvYjvKj1cm5Tv727XjbZeo+6GOLYsFr6K1GmstljCtSJ7xAbxB5E/myZzYDQEdtafWKL92Et/zgnuQ311bayMlFSJ9bn3Bb8fzWjnN59H7orUfKoXfmO/YCvYno0D6gxSMvbEv+bseUoo1o6A8+uwVv/N7d+P6tq5tdrZaCc+jjyRA82uD6QTv19VrgBbpHLmpJOvaTu5/Gv/5mZQtrE4E1oKFz+mvVS9ubWaWWI/VD9wK9VQg73A/dC/QmQif4qJ14ijrBlWCXpnzp/x7Hr0YhI2AjGnqnZsXsiTn0YS/Qa8LiC67H5655xOnaqndb9BCxfajcdrnDG0XYhlzzaO1Y1E5caqKhe6Nozfj5vc85Xec19HEO8bMzBmzcEWXEu+I+tw7UCWjHvj1a+dDbyUWQe7l4yqV18IFFHlqoml2nLvOBxrWVVmi5vMxWu/C1o4ugF+itA3cg6tTxmivQiaiPiO4nooeJ6DEi+rLmGiKi7xLRaiJaSUSvbE112xsMAMWkOe8O1FbOfvWh0a5di5L7fw+/iJsfz08gxh09Wq2hl9soERafF2sR6Ffe9xw+/qsVralQh+HWJ9ZjqFy1XpNQLh06Z7po6MMATmaMHQrgMABnENHRyjVnAtgn/u8cAD9oZiU7CVx8d+qSTYfR1NA//IuH8P7/WZZ7XTP2FHWZbNuJr+bCppY6PfTcFvz5qY2tqlJH4T0/eQCfv+ZR6zVJpOhoVKgFyBXoLALPD1uK/1Pb+wYA/xNfey+AaUQ0r7lVbU+Iwooxlni1dOqSTYcksKjOxUYr3kTKoY8fyoW/x1o09CpjXaVc1ApVmXjsRburahop2pkvzYlDJ6ICEa0AsB7AHxlj9ymXzAcg+qqtjY+p5ZxDRMuIaNmGDRvqrHJnQO0PHdo/ADQeBt2KwcEa0NBrqU47CXT+HmsR6Ix1bhh7M6B+65052/flRYpedscaPL95V1Pq1go4CXTGWJUxdhiABQCOJKKDlUt0ulvmjTDGLmWMLWWMLZ09e3bNlW13MHTn7kWNUy5NqoiA0dqxqJ0EOn+PwzXUKXpPXdQZa4Tad3cO2wW6bU/RjQPD+PoNq/DOy1V9tn1Qk5cLY2wrgNsAnKGcWgtgofB7AYAXG6lYp0D97p2WH8QF7Tg5Jblc6qBcaqGO2mnP0jwNfcdQGY8KKRuAqD92YZd0htr2geEKhitVnPzt23DHX7MsAf/ctne2Y6h9N+l28XKZTUTT4r/7AZwK4AnlsmsBvCv2djkawDbG2LpmV7btwbJLte6IFG1fDb2eRFWdTrmYlIZzf/4gzv6vuySBzzR9cjxB7btD5RADQxWs2bBTu3WkLbAoiAdztY3fp4uGPg/ArUS0EsADiDj064joXCI6N77mBgBrAKwGcBmAD7akthZ8/frHcaVDMM8V9z2LxRdcX3eWvjxUDRxcG/eBXDRa99Zw6NG/DQX+OEy27STQ89r80HNbAQCDZTH18fjW0HVdzxaUZtskmo/pdoyc5nDxclnJGDucMXYIY+xgxthX4uOXMMYuif9mjLHzGWN7McZewRjL9ztrMi6782l85rf5+Rq+cf0qAHKnbxYYUv/VsfzkYchw8Z+exJadI80pL0mfW99yozVeLtnQ/4ee29J0AZy3IfPWXSM4/8rlo5IeOc+zh6fXHRyRBbrX0GXc9PhLAPSatm2Di06wj427SFEe+NMqrrvVocPrtw9pDTur1m3Hmg3REvLeNZvwbzf9FRde7ZaQKA9p+lz3RolaTCsESkq5RMLt9r9uwJu+/2f8/N5nc++tpTZ5E8TKtdtw/cp1eDzHHa4ZyNulKRHogrJSDdtbALUaOoH+2d9GvuhWDV3zzpKNydv4hY4/gR7/6xqcUamG+M2Da43LLPHbMsaEJZt8fbNohyO/cQtef/FdmeNn/uedOPnbtwNI27arSauQeuou3tNKP3Suof9uxQsAmk+R5JXHMx+aFITtQ2V895Ynm6JA5PlI9xaj4bxLcM1jrJsiImqH7bXrvknVwqFXc95/O2DcCXQOV1/eH931NP7lfx/GVcvXOl0/Gl4uT23YaT3Pa2AiSGrVmPnltVAuohbDWkBDM0VDX7k28u6Y2FvMvbcW4ihPoPN+ZPK2+fp1q/Dvf/wr/uiQziAP6iSmohQn71Ipl3YWQK2GThFbMnsiAINAt3i5hJZz7YLxJ9Dj0ezqjsazJ27dlc+R2vzQR3VQ5Tyq1qrUQ5mI8q0VOqIq3JJUAA4TdS21GcnpJ3zvVVE4HHfRn/DL+yMD/UCsLTdj5ZDn5dKTaOgC5cI85aKiJ6amdOdYQrlo6Jgkz0v7vtBxJ9ATyqVFGetUKznn7EezD3ABanKZrHVyqafukobegrarbot8e7Za/MZdNPW8CWI40dCj545UQrywdRAXcPtFM9vOJ7EcDl0U6Gy8a+iapo9YaDJbYBE/1s7vc9wJdI5WuKMxls7e2dD/5neC0/7jdut2b0bKxbH8RpL9jxaHzgcg9xF2sY3U8i1yOfSyrKFzg3V/qeD8DFe4auiD5Yp0T/uKHzc8v3kXFl9wfSZoygW6bz1Uzq6qONLkXDYOveZqjBrGnUDnGjPXrAZHqrkpNQEzbaAeTTVT1ShaUzWd8NeXB7TbveXJK5uAFgdAOeZN6hLoQoNbmsslrmMh0dDtAvhdP74ff3jsJefn5HLo8Xl+3UAs0Cf0yAK9GQFmedGxOg29Gna+2yL/Xq52LBHquAso9QKyebnoOfT2f4/5FqQuAyUcejQoDvjC7zFvah/uufCUhstmSDPbZb/96HWGxIhpkCK2fimeq1QZeouoq+riYBmNXC78cTYqrVINteHeOgQUlZnLoZfl5TtP/tQ3Fhq61g+98zl03h51knSBKoT7SoXUM8nia27j1wFg22AZO4bKWDB9Qs11aiXGn0CP/xU1r3XxLvD2+0zCUf7waj5lftfocujyszPnbQJd+Jtztckm0TVomRKH3gqjaPz5OGfusoNRLXnECwEhrDJnt8WKQrn0lZq/+M2LFOVeLt3GoXP323poLJ1A37orCrjTaugWDl3sCn/7gz8nqQMe/uJpmNpfqrlurcC4o1w4WmEUZcycfnNUnVx4ZGcdRlHxXCOUi3SLpPWHuOCqlUkQVL1ItVVZQ7YZRWv55nx1k28UVTl0rk3m60phyGrymOATY55rbIZycSz/1w88jy9d+5hzfUYLXEPvd3inKtSu21cMrKH/ptQdgDwOxDwwG2JPuHbAuBPoyUBtklFUpTVMuSDGRkvSS3Q7h57+nWrotde9KnHo6fHHXtyOXz7wPD76yxU1lyki0VbjOvKBaNPChxXhbKKkogdE/+TJ2xGDhs61SZs4/af/vh9LPnOD/QEC+KrEpKHz44MjolHUXZn49FUr8ZM/P+Ncn5seewlLv3azkw2qESQCvRkaukDbaIOHLDsWmcYBXxm1A8Yt5aIO7lpx4dUrMaW/pOxYJORyUb49/33LqpcREOGk/ec09Hwb8sav7bwogPikV89cJHHomic2SsOouVx4HUWN+pZVL2P3af04YN4UADVuDAEm/WtCwscqRtE+B773zidr2xouj0Pnk5tKufB/rRNYHXhu8y5sHBjGjqFKS2wGHAnl0lO7/qm+qr5iWk8b5aJbOYm26Mm9ReyIv3VDCeKajHGnoXM0muf6F/c/jx/evkY6xhjLGFqSLeni4+/76TK85ycPNPTsPKRGUcP5MMoJY7sXsLtw5SFUJjqO9H3UXKS2fB4pmlIu6ah730+X4cz/vDP5rU7iNu8P16hAE4feXwOHfusT652uS7xcDKsQ7v0i9sFWutrp3nkrIBp5a0WWQ0+/i04Q2767WNbsyb3pcS/Qxw5coLSCQ4+WtwYOvelPy4dJH7vv6U048hu34P8ezu5BIlEunEM3vKqf3fssjvnmLdpzYh8X286Ny40LdF5HWWu1TdSc784DY8yadU9EhnIx0AM23vs9P3kATznYFFiOhs7bLrmMJivG5vfAhOYyjKUbHlmXu0OQC7hffT3bx6rt7hcpFxuHrilLFOizBIHuNfQxRXM5dBEhY4klfGw5dLtR9JE4QGP5c1s0d4qUi8yhq8V9/ppHsW7bkFZYSBy68HezMtUluVyU4Ccbh64KHhMFIY7PvO+mGkU55VII5KGVa8gcdomFiGASIPwZVU39WyFz+HfVvfMnX96BD16xHJ++yhz45gquodfTd6yUi04Lt6bPTY9NFnIGtdMuZV0n0BdfcD0uveMp43nVD72ZCJnw0Q0c+mggoVwMOjofgDxcXncvkHVbNMEWoKEi4eWV49+8cRVO+4/b7Q+Syud1lD1xbN/V1W5iMujaykz461igqwIhT4tzobXyOHTedl1Ql1r+rpFKw55G/FXrNHQufFe/bH/G1cvX5q5OuE2grpxCGrfF5JzVy0VXVvY6wAv0luOiG9Ud8rKo1SjqMuDk9LkR+AQymhp63pPK8aYNqhYJ6N0W8waSjuYwBRalhlb5nh/evgZ/zRn8unqmlItcvg6uNJv4DnI19CSwKE5ZrGiT/PbmpM+FVLaKREPXTEjqLe/9yQNJuuXMc5S6vrh1UL8Ks6yKemNNOG8jmU/8+mGc8Z07rNfwMup5hSpNw1MMA/qIW1v6XGml2YJVZzPQlQLd9uF1gUXNe66Zex1VwoVr6AbKhbdd524l1tOw2EiQ2CPi8k7/j3RgmrxcmrXpssqhp9kWzeXXo6HnjdXhqsyh82eofVDV0OvRNpkwSei0y2Ry00xI6uPuXbM5Oh/fs+yZzcm5siDoVq/fgWMv+hMuu1N2AADSSUw3UfK67lIMmowxvP+ny6SI3bw+kVAudUh0m9uiTgTwy3WPMmnlXkMfBVSqodY/lr/8Wo2i5kjR9O+IcomPZ65r/KPXWkaeQC/kUC7JHoqG53LKhtMef3l5h7Ycsb+7pLd1gZoPvR4O3QRJIOYMVjU5F+9z6n1VpV5Zj5v8eonfQUfhVDQrqlRA6R/Ay3nLJfckx0QB+8zGXQDSCUCEuiraumsk2YqPP0/0iQeidt+86mW89ycPOHuHuBqodVBvETl0ux86w19e2iFt4yg7DHiBPqo452cPYv/P/z5zXE2m1EyELI382zVSxVf+7/FEuzD1xWsffhEn/9ttTp3VtT/n0UNlJeWscnOCPNe9giVlrZw+V9TQ3Xzb8wa7aOwLw3RlZA/9d/NykTlo+7Wql4tpByNVAA/U4f2hcymVnsGDrDTCxtQMfTnpO+T11k3+/BuseG4rrlv5Ig77yh9x6JdvisuNrlEpF5HSKDu6reTZDlzu5chzWxSpstO/cwfO+m7q9hoqEz3PnaPW6x0/ug+X3ZFd0YwGujaw6E8a3957ntqEHUPRQGqW26IoPBmTP/qP7346+dvUFz/56xUoVxkqIcuNOHPtznlG0VRDt3PoahoD1SukYIm6lSmXFCOJUdTemipjCOL637tmE+ZP68fCGWkiJPF9VsJ0IrUaRcuKl4vp2RqjorFMRYAnqVmTd5YtE0Bd7nyyhh4CkF0jy0nULPDC1kHc+Mg6YeIzaejmcsR66yZ/fu7bf/yrsa4Z6kkwtLsKaH5VXRx6jUZR1ctFzPMk1rcSMvQUA4xUw0w77lq9EXet3oh/fvUSbZ1e2jaEqf0lyYWyWcjV0IloIRHdSkSriOgxIvqo5poTiWgbEa2I//tC02vaBLztsnuTv/Oy6LlCVDJClg0s4jBp4HlLYulZjip6clUO5aL1ctE8z5SfxpayVg4sEjV0tzaI97/10ntxwrduNZ6vhiwZ7NZcLo6rslo0QS7Q+Tvgv/O8XLhiUQskL4uQ4ehv3ILjLvqTcCz1cnnX5ffha9evwvo4z4ip62g1dKFT8/4cWAS6a7mAzM+79gWboVKH869cjld86Q/xPfI5UUO35XLRVV9dIfH887WuHE78t1vxnZuzk2Az4KKhVwB8kjG2nIgmA3iQiP7IGHtcue5OxtjZza9ia9AsykXVaE3f1nQ80T4cqmPqz9lEYLF2aLiGDyQ9hy6vOGzPLcZLTt3AFLUfvZeLvsz0/rzzQnvCMDfIBchq6EYaohYvl4qeQ1d3vlEHvUpFuIgE8dtUQoaXlGhfkXJ5cWt0jn9ik0Kh+3aiYZm/Z93k75rkTYSk5TqOwVqUHgC4fuU64V75nqKwKlWVL8aY9VmqEmGiXEzYtquMt1zyZwyVQ6e9b+tBrobOGFvHGFse/70DwCoA81tSmzpRj7GkXA2bErIrC3Qb92vX3F1cn1wGiQiRIhFvTfzQNRSPWJRq8Vcpl0CgXDJGQHFiEI6b/NBV5L0P8XG/Xf6CE+Wiaugu79OVchG3oAPSoBXVvZKjHuOwqqGrED1+Elc/Q24hWzmiFp1w6BoLu02QmU6J38eZcrFozXlQ7xH7vPp8Xd8XodomuIbuGil621/X48k4S+OksRLoIohoMYDDAdynOX0MET1MRDcS0UHNqJwr6nEgqYZmeqQWyJ3AnHs6T3NvxE/dpT+J5Y/EWqWOkRG57Tz+tShQLhlhKfyUJpM6XAd1EOv0xWsfS75lLYFFJkVAptHMdWCMZfanVCmX1Ddcfrbr5GKqr83LRdIknTh05ZjEoZs9ouqhXMSyy46CkF/mMgGonm1qu8V28PKe2bgTX73ucewSPHL0kaLp35UwTHzaXceumFK5VQLduVQimgTgKgAfY4xtV04vB7CIMTZARGcBuAbAPpoyzgFwDgDsscce9dY5g3pEoegz7vaMfG2uGprLVL95GDKJl3RZLZgnC3XpGP1L0jXp3zpviPTmbLkmwSd6uaiC1ERbqBtSDJWr+PZNf8lWI2eQqKf5b7V8EWouFxOtY/LQUTGi8QZRKZeK8i+HSnW4fX+hjhqqRM1rI/5tKl234TRfcQUBJWXqVnN2DT1/AnGmXGD+pkPlKjbtHMH8af0AgLVbdlnrURIoF37u3J8/iCde2oHL73paeKamHgrl0tcbGTVdNfSJghF0Ut8YauhEVEIkzK9gjF2tnmeMbWeMDcR/3wCgRESzNNddyhhbyhhbOnv27AarnsJ1hjwwTqMKcENa4xo6kwSXu9BVXbZc+oSpumrZefwfF746m5R4SKVcVIhG0YyA0nDx4rP5od88uBaX3fk0VNg28OXl6/zsuZDV1VldHThRLhaZI2r8XJNVvV6Sf5X3o06Abt9f1NCVCTRM+V9tjhKjgM0ef3zddiz5zA246bGXkolGq6HbOHSh3B1D5eRvsZ84G8jjpure0YVXP4LjLvpTMllv2VWWzqtVFNvBJ7M8oz6H2N7hSphQLnIyNHObSkKU6phx6BQRp5cDWMUY+3fDNbvF14GIjozL3dTMitrgKphLBcKJ+83GgfOmWLXp2p4t/zbN1upRVTNyqYsrncN/BxQJgX/7w18kzYV7+GjdtjSaten1igI9IyxFo6gmxzpv77ptg9qydcKCB63wunGjlATL8lylXMw0mH51kSmvLGqbsobOb+PHsxp6ViDnwcahi+VphYqh+P/605O48OpHpGMPPbcVAHD9I+usHLpNeInf7+XtkafN5p0j0jd0HX82P/T7n44Cnl7YEvUj1Tahfj9xpWHajCY6Zz82XAmT/icFGVn6i1j/saRcjgPwTgCPENGK+NhnAOwBAIyxSwC8BcB5RFQBMAjgrWwUtxp3fVIlZCgGhEJAkYZeg13K5NOdGViG0PM/PPYSVj6/Na2LRaM1wXRF1rgT/SYiPL5uOy6+dTVuXvWy8GyuoWdLFA/l8ftWysXg5ZIEdsUDT9WodPdwbN45ghkTe5K69RQCSUgT2Xn/vMGuq7tN5owoBr5KNcyE3yc5ypWCXFcLpmvUCSLPkKvaejh+tyKbQpnPk8Pl1Mda67ZomhCVsfXy9iHsPWcSXvnVP0rXuXqa2baFmz+tHy9sHcRzm3dhyexJGV4+YxQNNG6Ljisa8flD5Sp6Y592U46XTJntINAZY3fBHIPBr7kYwMXNqlStcBXo1ZChEBCCgFBlbp4lHL95cC2+fsMqPPHVM+TgBKUMnZEJAP60ar2kHWQpFweBbuj/JrdFIOsnLf6t63ziEVNudw4xsMhm5JMol3jC49dvMwh0Xd22x0v3W59Yjw07hjFzYg8gbOdYDCgZwLoqq0LQRaDb9JJhwQBXDplCwciaeS6H7iTQ9XUEVEOm7rtm+4QJXOgNVarWwCKThj5SDaX2vGTYhN2Few4lKil7/YLp/bj/GeD5zdEKVOXlrUZRi4YuUmRX3Pcszjx4nvReRwwauq1NYv0n9rZmh6euCP133VEn0tCDaODXSLnwPCVbFQGU4cYNaku5GkqDrh7KRY5KNQ9g/pOQaqVPbdiZqaNOWKk2AbU8EVbKRdQINZTLcKKhj0CHJJhEKIjTGXzHp5JCuRSDIF2e6zR0R97a1W1R5dDF33leLvVQLoyl0cSq4KhoXA1FiIfy9gDl7MpwOV1x6KKKTcrLcDmU3r/qL8+hS/ilQlR8dK+Ir9ie3RQJdPW9Zv3Qs14utkySAPDZ3z6Kj/1qheLlwhIvFylpl8UuIN4/ubdkvK4RdIVAd5XLXEMvEKEShk5akQp18shw6MbtwZhWS07KcVh9ystmcx10XiUiRhINPfsMJj1Dpi/UkviKoxbKhV/H/82jXCRDVDmUPFV6iopALxAYy7qPvvpbt+KOv27ICE2T9m3yoVchCvBKlUmCMqFcBA79W79/Ar9/NAp6yU4uLgI9ncRsuWIGNFGoYlvzUtrySXS4UrUGFpkYk+FKVXreywaBLgYAmeBKJfFJI+s9JF9f0PihazV05f1u3jmceX4SKSp5O5kHsqiceA3dAttgUD0DigEhCKIP3RyjqKqhmz+obWkWMoZL73gKiy+43pi4SdaezR09KZr09UmDX3QaerYc0+sVA4tsAkq8nU8mvA7bTBq6xhA2VK5iv8+lCdfU3Ddc6DAmD57nNu/C925d7UxzuCbnGlEoFlljT4/zf79/21M49+fLAeQLHm29GNMKdMaY1J90W+1JdgwD5fL6Q3eP709XUbxczqFvHyqnGykb3t9wJZSE/U6H3ZhMENtlM+JvjrMiqgLV6raYaOj255qen0SKCoddffOLOoN+E9AVAt2m3IjnqtVIQy8GAaqsPi8X9VnqR3YNllA1+Spj+Nm9zwIANg0M625RaAzxuCocot8E0gp0PuB1A8QWWKTqaGL6XJWXNWlW6bZ20TUmDV0NnQciTleEyttzWoAhK4iP2nMGKmEIUdE0+qE7Uy5RfSb0FFAJZQ093SJPjiTlsPntA5GAuvDqldIGyaFEucgUi9if9PnJxWfr28Q1Tt6OoXI1aX+BCBsHhnHIl27Cxbeujups6OvDAveua6sOxtVSVfwWmvOhLNBFp4RopSZfLzJH/JvoKFv95CH/TnO52OmupK6j4CfSJQLd/KJUz4BigZKACfHc+Vcsd9pE2BYuDLiHdOsCS7gxyuz6qBc0Zi8X/eAdKus19K9e9zg+9Zt0D0h+1vR6OYc+UmUa7VeotyhMhPczVK4al/86LlxNaKUaVLmwC1k2xqCnGKBSZdLmzaYBJr5/u1E0asuEniKqYRo12lMMMkbRXD905Rteesca/OL+53HFfc8KddFTLlcvX5u8/1KBtEnIbP2FozcR6FkNvVggrI/dD294ZF2mTBFD5VB6byOV0PoebXUSOXRbMq1EoAvXc/pNRFETWOSioRMo096EQxdet6kdK9duxfbYZfPGj56gvaYZ6BKBbjkn/B0yzqEjYxS9/pF12O9zvzf6RYtl2AySrlFj2aVhKiBdok1Nf/OygNgoatGORCGycWAYl9/1dOLXG5VrX1pLgUUWP3RojKJAdjcbuQ0pVcHBE05xbFcEPK9PyFhGQIYsena/EH5tEjLyMt9YxWRVMrG3ICkIPYVAy6GLUIXu05t24hbBtXRyHEm4YUe6WmNIBbpY3r9e9UjynnqLBe0kLh4x9QnuvcUVm4g6Se9UXdHNGnpqFO2NU8zmjQtTJsw8jyM+UW7ZNRJRT6K3j0ZD1+Vy0XWDjL0FeiUhutauoQ+Vq3j9xXcnypIuSKtZ6AqB7pr1rRIyFEjwQ9fc99wmNXRYPh8N3PS32slcU7Su3TIo5Y6ohiyhMEwcp0mI24x9tvqI7V/6tZuN501jsSBQLupz5PD59LhISdkMSDovlxe2ypOtyqFzYceY3lBcCRkm9Igup/pni/SFrW/xDTMm9BRRqYZJedw4K7Yj4+WixCt86/d/wft+uiz5PXtSLwBgg0C/SZSLQeNXDcUc7/vJAzj/yoi/NwnXjIZeTqmTJ17agTP/807pehfKpb+ngJFK1saiwtTnK3mUS/yiy1WG7UMVOcBKs1LTeblon+vCoWs19Gw7VK+iwLSVWBPQJQLdfC7LoQeJQNf1MTWAQu0QaoSpumx3zU/xwSuW4++Ebb9CxhLtwUT9mAyhptB/MhhFk7rnVDXJ1Gfw9eAdcyTHy0X8PlLaW8v+n/zZ4vt9QcnTccxes3CAkM6hIBhF1e8Ssujb9FtiCDgqhrqr4JTLxJhD5xNpMe5fYll5HLqKCbEXxKaB1GgcMpZsiqAKwFRD1w/pNRt3Jl4lOqET3RuVzW0VQ4KG/tT6dANvdbJSMVwJk2v6igWMVEPrtwbcNHQdRSb2p807R6TVSRjm+KEnGrpG81feUUS5yNek9JddQ8/aerxAt8Lmhy5+K86hFwIyGkXVV53ZqKDKFGGKzHlXPPZimuMsooNkDUmFWLKLlwuBrPXJc5XLC/3nh/V+6PqlshSc4bB6EMtRNfThchVnHzIv+V20UC7cE6TPSUNPqRMrhx63ub+nEGuDvB5BRjMX2/3I2m2JAVyFmip248AwXtg6iFd/61as3TKY7ImpTvojORq6CJNRlG/+wCeqEUGg61ZTJhuEGGHaVwowUsmu4FTc85Q+U0hF4sQ1glc4tmXXSGZ1pd4ixi5YA4u0RtGstl+MZUlSX827VfPw69IoNAsdKdCvffhF/PqB55PfNrmkGoMKASGgKLBIJ9DUnN+6pbtMfSialyOHrkKkXEyBH6YQY/WR/Fy+hm6vK28af+6O4Qoeem6LcD4e7ALl8vuPnRAfEwS6+EzhfVnrxgWbcAnPCcIxVAml5WtRMIpm7QoRvzpB2YJsqFzFP/zwHiwX2sWFSKmQNYRdcvtTOPALkeskn8T6SwWJwosoFyZdI7b1g1c+aGx3klwsfucbB4bxv8uex3Obd4GxlOdWJ1DeZ7T5bRSYJnmu3YuTRbLCyNnERG0D/359pYKTQP/W77MZN8XnAwajqHBsqFyV861rKBdRO7blincR6DzqPK+OqgeYJkaraehIgf6RXzyET18leGPYKBfhb+6HXrRp6MrkmQ3tl/OoZ3O5uFEuKkSjaF5oNiBrWWod0z1A7VpwnhsVX/mITXzT9/+ceW65ypJ2c0rDFFgkCoHfPLjW+Gx+na2Ow+Wq9L24B0OooVyqYeQB0d8jUy6r1w/gvqc34yO/eCg5zoVXqRhkJsuLbnwCu0aqGK5Uk3fb31OQFATev4YrVewcSQ2MHBN7zBk3+HW8rI0DI1IeoUSLVvoI1wJ7S/YhXQ2Z0XZRKmZXiHyFIfYj3i9s2y0yQaDrjOYiigFht6l9AIB3/fh+nPGdO5JzMuWUvVfs+8OVUFKoWJi9R+TQ+XvQrfC1G0irQUqxLJEULc07URW0VlIuXbFJtItRNIyNmcmsWtVvcKEaLLLarxxhmjWa1ivQU4OXUUM3TCQ2V0o7T51HuWSfK98f/TsipM/lGqTJ9U8sS8w/bXy2pY5D5arkV54MVqaf5CpVpgh0JEFca7ekdA7/hj2FwEjmbRssJ++9FHu18EcWCwGqw1VJGIkePbbETOqGGdVQThPca9LQK24a+vbBslFDLxChVCBZQ7ekmDXpCqJ9qq8UYMsuu1H0gHlTkj5yx183JMeHylW8/UfpXjq6fij2/eFyKFEuuRo645OPvg0iHnlhGx55YZt0rJhEnTPsHK7gf5c9j+lxKgIR6uTbSsqlowU6p1DskaLxtYL2VCDS8qyAG4cuJslSy6h38+moLTkculB0RfG3FZG+D31gkfjM6H6DwGZu58OQJUIgEeiSRic80/H1iALNhKGyTLnY3Raj4BvOQQNRu7ZqIlUrgqA2tX37YDlpf6lAqAoGuGIQUS5coBcCkgKEbJsbcEEtPlb01e8v6Tn0REMv2kPKtw6WjUpHEAfdiZwv13hroVyqwiq234Fy6SkGWo04z1c/elZ0/0glSguhrlzF70eU1Y5D5s6hqwgCQqEQGcC/dv3j+MX9z2uvG1a9XLxRVA+e2MlKuTBZMMheLtkb1SNaLxeLQbIsZGGrBSFLOXR1sDLGsGGHnEvCll1P0tCthsfs9brzutMPP78VD8XpgKshi4VrysPKGrpQpqONwZZki2OoUpVsHtzgJRooxbaUQ4aeopwPe/PONDgptQlE//YWA6OysG0wDYHnSkViFC1ElMvGnRHnv/u0PuwUXFRtGnqyQYbw3I2CLzo3eqoaOr8vzyi6ZdeI0W2xQIRigaSI3LLGBsCrZpoYGINEuYxUskZzESbjs+4bZq9JXVGHFfdIvirnIGRX4NXQTUPXgSuH1ZBhy059xDOvlwhvFDVgY+yjaxfo0b9iGtBEoOcs4QAN5aIs41QjaCUM60q8E4ap9qBq6L9e9jyO+PrNkleM7FqXpRcA4Bf3P4ebHn8ZJuiCd3Tl6ITaG753t5QTZqhcRV+pkAwYiUMXpgTds1R/cvE6m5fJp0/fX6ZcEqOo3pWTr+jSY3K2xzARVKmGfvfqTfj4r1Zknr1tsJxQeHzFx+taiL1cuCCeP62/ZspFrP96QaDzSTPDoXPKJUegb901YqZcAkJPIZC0XD4R5WVwFCHap/pKBYmS06EnnjhXrt0ql688wDReJ5RSga76rYvvkYgMwVH5ckAHUTm0IeOH7jV0PfiS1ua2yD9omgY0dVvUKRh8ub4i1j51RlHxmKp5VKpM2gzWFVXGkplb7QAPPrsFAJI6RfUwB7+IHezpjTuhA5EQvGPkyPl5+fhGJddMGDIMVSKBnkSPGtKe6p6lpsEVrzMtME7efw7+8ag9ZC8X7oeO7OorDCNf+aKSnGnLTtnPG0jpIi4cf/vQC5nnb4spl4AiQSFqeqUgCizaFJe9+9R+qZ84CXSh/hskgU7o0Qj0oYRysQ/pwREznx0ElNk7lE9Eeg7dQrmIAj0nsKhUIIQh8PqL75aOq5OIXotP7SLDWi8X+XpVOxZtH7Zn6yAqhzrw+mY0dC/Q9eCCxfbuX/W1m7FpYDjV0AuR26JJQw9Dhh/c/hTe+L278eCzm7NaQtW+01E5DLUaZx7EqDY1CRU3tGzeqTeyuSxNVUzpK+XuCM+PqufVqNJKTLn0FYNEYxZzl9hSJQB5Al1fNy7IxbEhBhapTfrpPc9ix1BF+jYhY9i8SyPQEw3d/B0//quH8f3bngLFkceilwsf5Jt3jqCnGGDaBNlQVrII3SRxmlB/8bsHREk4vYjEbTFHoNuS0hWIpAkPEAW6uxYruo32laK62iiXgPR2sMwqS1NENUwVKNXLJdocQy5DS7lo6uTMoVsEOj+c8XLxlIsePC1nXoDMA89sSTTagFJXI13EXJWl2vmGHSMaLxb75tLlCqtrSbV9sJwIEjUQYfqErEB/s+A+aAv9N2Fi7GoHOHDoOcWFAuVCRAhI4dCFa3WTqE6gX7dyHZ7ZuDN514tmTpDO81tEDr0ocOgm7l1MWxoyecOShBvmgUWKcNR5HwWxoU3UBrnXy1C5igk9hcTVMHmuRVioXi4ApBQRRJHhU+0jXAvMM4qGITPGSgSUbbP4bA7TRC8+Q/RDZ8yegz0gcuKxTZRLv0S5yCtXtQw9h54t11VDVwOL5DLCpF5SHbwfegrxgyU7yOe8+5FqGrnGl0mVUB/6H4bpoNIZxUw5YJL6hWFduRo++ssV+PNTGwGkGwTwpfaUvmh3E9PuPqbQfxv64mAYIJ9Dd8mUN1QOE5c6vgJKCxLqpnlWj0YTvnr5C3jtf9yelHPAblOk81wb11EuOg6doxTIGrqanjZqTwgieaIJQ4bH16U2jKQexAPVBC+XAmGkEmLTwAh6i0FGSNq0P9UPHZCFIadcRqqy8Z1r9nmUSzVkxvQUQSygRNgSqBkpF5ae415Fphz/0XOj9vKJb0rsBSSWXwjISLkUAko8XeRcLlllRRWmJg3dxXivCyxSywa8hm7FUEUn0O0vf7hcTbQu/hF0szcQaQFiGlS1aBP3zjFSCVEgwtfeeLBLcyRwHnSoHOKbNz6BI75+M7bsHElsBKKGLkKtjwvl0iNMVrluiTnlVcNIoPABGXVyPcdfjXlnESYKolxleOCZzQCAPWdPlM6RlXLRa12ArKEzlt3oGYgM3apgu/TONdKqiCMgQiHgXGyqNABRBs/eYiGjNfP3eck7XpkpTxXoxYAkuoMQG0XjXOVLZk2U7muYclFWS4MOGTEzxwWtt78nKm+nRaCTQrnoFI2eQiD9/u+7n8aK57cmhu7IUCy7Leq072yciYlDz48nKVA2sEjEll1l3PPUpoyTg+fQBUgb8/K9MXPukTT0QupqpOXtQia5DWY1dPvWdZUwCgR5x9GLsN/cyXnN0WKoXMWld6wBEOUA5x3GKNCF+qjbr5lQKqRRkHkcoEvOl6FyNVn6FkjOISPeHobZ3Vp025txfO36VQCAwxdOw6ELpmK3KX3JMwB5gPLdaBgzG1PV5EyqRsePF4NAKvu+NfpcI0RI4xoYb0/avp5ikNGaQ8Ywpa+IKf3ZfSWT0P+4rMmKz3oQpBp6FIwWlc09RPI0dMbS3PWqYIm8XFQNPSuIV68fwA9vfyrKXqrbmo6JgUVRn+AC/btvOxzvPnZxcu3Nn3h1QrlwGZrkkBc6To8Stfvl/3scb/ze3aiy6J30Fgtx/nbRvZJlxoYr5eLm5RKtzkwa+vt+8gDedtm92LQzNWoTZdOLNBMdJ9BFDb1SzS5PdVA3u438hk2uWCzdJq2aFd5qcq7MsyphMsjqnYnFqMVymE5GpuWv7BefT0EB0SCoagaOiLzkXByJUZQLdMVQJN4eMibRHoCeQ1cxoaeI333oeBy1ZEbyDEBO1VC0bHCRPkumXERjHR/YkTcMSdq/iUYLAgLFAklUGjhMlAsXBiq4wsLLmtwnC33ibovlKF0vf9ajL0R0UF4MRDVMx02fUq/Iy0WdfPTlfPPGJyT+Wr4nff8p5RK16+glM/DmV85Prp03tR8Bye7AifeV8PBIAdErYAXhnYiR0TrKRee2qGuiy+5CPcUAxYJZQ3/ipWhj+SGFMmslckcSES0koluJaBURPUZEH9VcQ0T0XSJaTUQriSi7lmwSxJdTTj68/Z4/P7URX7nucQAphw7oU91WQ5YYnCLhnT2vdqyPnLIPfvD2Vybn+SCrV6CLIcZRigL79Wpo/fod+k15RRQoX2Dz47kaesgwWE4pFyKzn3z0fuRu55IhkCu9qmZu5NANg0zesQZmDb0gC1yToZvn1wfSZbq44tBp6NWY99X1jxGFRsxo6InbYjQO1MkwN5cLS/cf7VOEMacQXBGGLFMGPx6yaKXKvy3X0HsLBandoh9/silIrDWLQtUU5JWhXJQdjmyh//waVaITIbPDlA6lQpCE/tvktEi5tJI/B9w09AqATzLGDgBwNIDziehA5ZozAewT/3cOgB80tZYCpLBk3vlzSJebV61PckSIA0nnGxuydLPfcjW7dZbOt3XBtH7Mn96f/OZL/2ZwZeVqmGugEZtx+V1P44ZHXsotV9Si842i9rLuWr0Rq9cPJNpYQeHQJcqFMePGFNb6cgGeGEMR/xauSSiX7DdKn0W46eOvxrF7zQQTVmO8bkBE5RWCQFoamz4liQI9oTLSSvUWg8RYzMGX/royUy+X6HdWoEda+GA5zTcjIk9DD8MoOZduQgkCt2/BURUMmdLxMBa08eQDpAK9WCDpGYV4hROGUT8Rd+1SA6BMcSOib34lE/qfXsug8XJhWelh81wREWnoASphaB0j4sq6lR4ugINAZ4ytY4wtj//eAWAVgPnKZW8A8D8swr0AphHRPLQAQ1LiIDcvFxGihq7zja2GcspTFw1dXT5zDb0WbceEspCK1ASxPuI2ZjbwYBj1frncbPk29Aocuuy3LGvoqtBw8dvn34wLdp2XS8kSKSrWZN+5k7H3nEkIGZOX6MkEF8USOFEulJ7jmq94X2+xkBGy1TASQjouVTWKTupVKBfEfHG8Ui0V5TIKuZRLJCiLOoEeJ+dyRRjCQrlEky//1txTR51ICombqzxBqe7BfaXA6LZYCAi9pUIS+m/L6aMOSV0a7WIQOO1r0FMIUCpQTZt3tFpDrymkkYgWAzgcwH3KqfkAxMw0a+Nj65T7z0GkwWOPPfaosaoRhnRG0RoEOl/iAfpEWnybLl6+jkNXNVq+aUb6O/X2aBSRe2U+5cHhYswBYsNlyKMSDeU6erlwcGpB5OeB7HZ5ajSii1ZIqoaecOjy8j16ntkPnU/WUTBLypdXQoZbnlgf/V3NctxmgU6JTzxXMMSVX08xyNAg1VDuh7r6ccpikpJGgvuK83Ggvru8LhfGlEtRw+EXNBy6DVUmZ6/k+P2jL2HRzAmSX7so0EVFJ+CGRcH3fzDe+o73oU++dl/cs2aTMVo1oVziSNHeYoBdI1UDhy63WRdXwvtDHnqKAXqKhWTzZxPEeJdWhv0DNRhFiWgSgKsAfIwxpjrk6mqZeSOMsUsZY0sZY0tnz55dW01j6CgXVw0SiGZf/lLztgGrVMOMgNQl8ynGhlYObvRrioZeydZBRagITBNEY5REueRo6DovgLlTejPHuIubGlgkVilkKSXF4ZLMLNHQM1x6ek0a+m92xeQaMFE0EY5UUmPuFfc9i8vuXINyvIoQq2lSrETBz9ssamS9Gg6dMYYg0E8SnBvnlMUEJU0AxZGigwaBnqcB8qyTxULQsIZeNXDof3l5B256/GWJk+cTlWhzSJ4bZPdEFQXtoQunJRMwoCgwTOTQQwyVwyRyNC9mhF+jU9Bc/NB7iwF64pgDG8TVaitdFgFHgU5EJUTC/ArG2NWaS9YCWCj8XgDgxcarl4XoUliPQA8CZDqZCRGHLh/TzuiKAS0VPk0Q6C4aOpM7uAmqZhQy4PnNu/DClkHt9bbkXLqIRB74FLmFChy6QrnUo6FnjaGQfgNKpKjh0/I+M2NCD3YMV7BjuJLu1FMOY2+VMOF2k+cbviWJlEs8cMV+VYyFjYgq45SLXFZvHBwTtSEqV837EhBhYm8RW2OtUJ0M8zTAaKMPlqGUeBtr4dABPeUi1iXNIBoFa/EUvSJIcP1LKJdqKMWOiLmH1A1mCpS6LQ6Vq0lyPFs8AgdfsYgoBPp0vip6Yg+mcjW0GkUro0i5uHi5EIDLAaxijP274bJrAbwr9nY5GsA2xtg6w7UNYe6UPrz58PnoLaY8Vw2Mi6Shu8yspqx9ItTB0ajbIpBqhBWN66QK2YvEfJ3MXUb3nfCtW/GOy1UGLQIvVvd43UDmYfRBIHPoqlFUHdC23CYc/JZEoCvGUUBwWwzNkzz/5vsLm0tzgcvdVMvVLCVhGoiBoHGWDaHe6uTHhZDaP0qFIPl+YazFT+jJUi57zJiQJgJTJkcXDb1aZdI4EO9Vv00eDl0wFee8eon2XEBpsq/hSjUR7tmVQbq64Rq6mGuJv2M1FTYQaetBQOgtBRipVON0C6KGbq+/TmlzNYqWCgFKhWxeHRXi5NAOlMtxAN4J4GQiWhH/dxYRnUtE58bX3ABgDYDVAC4D8MHWVBc4fI/p+Pd/OAzzp/dnXLxcIHKX67bp3fu+/PqDAHCjaJZDVztJQaFcErfFBmZjLjDLGs5ehVhH21JRXUW4lqvTVnTeDfvMnaQtW7xb6+Xi0MnVVQ9/t1IuF8EYZuoTXEM/YF4a9MVpg1RDj1YRUq2EH+JnDSgdpFzBEAV6yLLBPpGwznLYAQl2C065KAKdCNhzVho1m6FccjV0hjL3clGfH6QThIsrKQD09RTwmbMO0J4rCNr4sLAZiUpFRhy6bBTdOlhObDuc1tIlk+NZSnuLAYbKIQbLVUyM35nNOM6xYUc2WM9lbACxhl4IaqNcxtooyhi7C3qOXLyGATi/WZVyQUmwRNfu5RL9vXrDgPaaPWZEiaAqmhleFylaUr1cmuC2OKGngF0jkZEnr3OJCoJNsxDrk8m3ogE/rQv/Vl3x5kzuxefPjrxZuW8uh5ptUTW8uSzz+ftVBbs8SYm5TfSD7OT95wKIcpQTRX2nV9gFqL+nkEmzGzUi/bOnkKavDQTByIWSFKyErHAMQ57US36EatcIiDIroSAgLBYFelGlL7TNTvDspl24evkLmDWpN+MSKVIufcV8QQXYBZSsoYephl7ICnTeXbgB+bT/uAM/etfSuF5I8uUAsoZeDeNI0aCAgeEKQobE7rBxYBjfv+0pa/1veCRLJJQKjgK9EKAUUy4ifnf+cXjD99JUwBLl0gYaeluiVCSBQ3e/L8rZHTV7zXq9QOedSueHXgmzrlCqKxbXclSuuBake3OGcSi6uSxdHgwd1ICOPO2Ft11N5yvWj+PwPaYlxyK+U/ZDZzFVFbIsTaC63nEcumBq8rcaSKTL5cLD1jcODOOjv1yRKe9DJ+2NY/aamdzP/ebT3e7DhFIrBpRZWXCIGrfo5cID3YaV6FPxXRGJHHrWy4RPyIzFgkp5z7Mm9mKBEPOgcuh5AuOq5dHG3BsHhjUrhFQA67xXdODP0/XPgFLlZrhSTVYyOg2dQ2zPrsQzJpBXLwqFVwiib7IttitwDV3cP8CEPz+1CfOn9UvHXIVuqUDS5M4xZ0ov5kxOnQbEMTnmfujtimIQpJGiNajo86b1JQNQNIiJ6Ik9ALR+6JqgFdVjgA8KlzBf0yUS5cKyvtsiXN0WRW3KTUOPzus0dDVsXNRoC4GSywUM//K/K7HXZ24AkNXITbytKFQyXi4Kpy6W+8JWvZFX5S+55qwK9Eo1olyklYWooQuceEDp5MLf5zFLZibnGUuzZfL6hqE+9J+f42UFlLVVzJ3aJ70/VTjWsqRXX3vk5RIdtBk7ddAJQYZ0LAyVQyuHziG2jbtmJhktBTqKgyfDE8cx59DVFYhOTGwcGMYcxWPLNtYWC2mcKQ5oUjX0AhEOEGw0YvTqmBtF2xU9hSDZ77AWymVKX0kaSLsrszPAl56kzdtS1RxTDWhcQLm4Lb7z6EXa45w75ZGiNneye4XEUa4aOvdysYE3U5cHXNXQxdVIQPLm1IylmiGQfS+mtolCJQkoUv4VycA8H2p1MCX0Qvycahhl0qyEEeUiaejCy8po6IqXywdP2gufe13EKzPI0ZTV2N9dx2GLlAtPC6u+Z9VdVKVc8oxusyal9+uez7+F+LcNJl4cAAaGKtI+uak2b66zSE/x4Kkg4CvK6LhIK+4aqcTJudL7uJeLC2UEANOUJGkmgf6Zs/bHbZ86STqm49ALAeHQhdOS36Jy0w5G0bZEsZAGxtRiFAXkDqXzoy0VApQCntFOPmdyW9RRLi4fT/d8INVOK7GGbhNW16xIPUSdKRfKX9mEiUDPDgzVKKrSOZIngmZFI9+br6EnlySCPLsK4u/dmB5WeQynaNS2VEKNhi6U2ZOhXGKBLuRymTc1UhTCUJ9dT7e/pWr4Iw2HPndyn/S7Vj90sU02LxdRW7eBF6HT0AfL1eRbD1fSCE71UrHK4rsVg5F43AAgT64hi+otUlNcQ3cW6OqOUoaJTLfi5llLxf5RDAIctLugoY+iUbRjBXrkLhTzjTXeK/bTfo23RiEglGK3yEwuF42htBgEUiflndhFQ1epi7ReqYZeDd2ztA1r+G4OVUN3plxcNHTF4GoyigJZ3tf0nsRnJNkV49/8X91y3ZTDWxWsKeWSPiehXAJ5Fx0platQ/yBIBSMfuKKwNuUZKpBGoAYkCK1o8KuTzbQJsjapprvN43/5ymmv2ROtXi46Y64OCS9uEP7cg4kxvWcSoHDookAfSSdIbvN5cesgjvzGLdL9PLCIg3PoJsO4iqmOGrru3fL6iivSIACO2nNG8lvMa+SNogZElEhtgUWc2xQ7kM74UypQvLFA1sNEZxQtFkganLwTu3w806BJNPQw2ipPFHpXnXdM5vol8QYQW3aZw5BVv+q8aDhWi0CXhJzZbTG6VhUk+ZSLLfQ+KTcux5RmWOf3DcgUShRoElMuQsXFSSmjoSteLgHlh+AbKReBVggo+55VYSgKnz9fcHKyz+5iZcs+Dj7R/vz9R2knFDE4yyWCl9fH1NellZtR803/7i1kNfQg4dCBu57cmL1fEejcy4Vr6LqNRERkJknDmNS1kU+Aw1IwWbSPbJKBVaRcvIauR6mQui26eLkcsXg6fvreIwHIH6ZPE/FYCKKAgbLG51yfzEcenHxQuAh0lW7gQqy/FHXKKPQ/63Koopbwef53XjRcQrmMVPEPSxfix+9empzLGkVlOkdKzqU8Rn0vJg3dJtB5keJhLtx0nH9UL72GLgrNkMU57ZV0rTevWp+5L3q+kMslfmEBEU7afw7+fukCfCmOaVDB914VEYi0QuyrLtbt/s+ekilHnEh3n9aPE/aZjaWLpuNzr1MTosZ1rKZ1VJ9fEGgWxlzdSaN/TbEEYv1swVkc4rsdUiiXasiwSbPJC48U5Ug19Oj+GROzaSpETM9QLvp268Ydn0hEpSehlvjKzWvo+SgWUuuyC4c+tb8n6SySQNdw2MXYIGR0W3T0cnGJulP7zsRYu+BL7XI8gYhFmbi8/Gel9y2cMQHrdwxL53UBMEDktjhzUk/iww1k/dCzHHo2LW1SV+W9mDq5zstFxztz8Pdu2pBY1UhTo2han3I1xAtbBrFger+RypMEU8x1A7JA7y0W8K23HJpw6boytG6LAk8cEEnvYI7CnwNZymW3qX34zXnHYg+Dhs6FC1H2vROlOxbxJFd5SOIDDNp3UekXOojvQeflwldBjDFs3jmcuZ9HinIkGnosH6b028NtVA3dpGDojutoPjX4reyNovkoFSjpnC6Mi6ljaQV6nLO5ogkiqoZZGkb1cikJhqU8qNdwCz1RnPUtpn1EIaIrtlaPhNfsm02O9onX7iv9ZogGdrmaTcKk/hYHopibg5cjXWvwO1chCXQnyiWqg4lyUccSX9WIWuTWXWVUQoa9Z08yKgpqVVTKxWVVrcYuRMfSHCIhi64x2VgSzdioTeqfy5vEaQy1TjNjL5htu8puHHpchEl5Eek1k0AXD+uMojxXUpXZNHQNhx4b8/OUHXUrQGM9LRy6vJE3vz76V9rsurXyvIMFehAkeYjzNrgA5I4le7nojaLFQoCRSlYbL4dZQ6maWzrR0B2+nqqlTexJtYloUom8XMTOVK+GLvoBHzhvSmbQq2UMDFUSzlL1trB6uSg+7vc8Je/H6ZovRKJc4ltUGSsbo2MN3ZFD54NRZ4OJ8qXr66UThIBoFNXfJ5UR6JJjRXX5+b3PYutgGUTmAB9eB/N3t1eioBPoRIlb5I7hSk1pjU1CUFyNmRQccbKWjaKCH3pAqIb6fXWjwCKNl0s8waorQhUTe4pY/fUz03qa7AGWcSf2Of5OdN5NraZcasqH3k4oFfPzeUvXC51TzI+hyxpYCqK0mFoNXcOrm7xc3DR0+XeioSOK2huphEleDxvEhGAm7xVeRDGgZPMB0Zij+jRf+/CLuPbhyCWyr8eskQMyh6oGFvEy0nsdNXQHo6g4aPhfRsolY1CMfuu2G1s8c6JRQ88E5MRt5zSTrq5qOHhA2VzuBSLc//Rm/OmJiK/fZ84krY0nfUY2c6VYvg26bI9BINM6bhp6uurTno/5b8bMCo7kh64zigbpFnNbNALd5IfO+3aeYsV3HuIwjTWbl4uuz+n6gTeKGlAMAmkzgDyIL3KisGkAX/7/zaG7J8cKBUo4erXoEc0OQsWCzIfWkg9d7SScQydCQvuou6vrNXSZt9OBV5uXpQpllY8VoS79sx4aspeLLQOd6yYKbhx6+jf/Bq4aOs/Zo+YcB6L0D6ZeJb5/BoFyETh0FYcunIZ3HZMGkanZFld84bUIAsJAvFUbL8ekLSZ7rDrw0trzQVaHD4ikqEkXQzt/vE3zTLZkNE7K6d9yYFGaQ31CTwGDI1UtnRb5oaf3cTdEZ4HumD7BRrnoDPG6YryGbkBPMeUbXZwWRVpmgkBrcDcp8TWXgiB2W8x6tOgMpRmvjRq8XNTBL2pkxYBw6xMbsPecSVJZeg49febP3n0k3nn5/ZlrJsQT2buPXRzXU09B6KAu/bNtFiccYzFSXfNQq9sif+4uRy+XC886AIcunIb50/rx3VuezJRrolwyftRxc8qC22JeXYlE/pkwua+k8Qs3v0g1t032vPHW5D7ejjcfPh97zJyACT0FWTGpRUNXvum8qX04bu9ZAOK+UrVx6OnxXg2HXihEAr0SMq1veSEg9BSEcRPXhSt8pSDAXf96ktHWpu4o5UINcfDJoKxZ5emub7WG3rECnfuJA25eLuIlIk/NhZj4ngtBlKMhyt4ml83zfYgw7ZFZj0AXhSr3Qnlh6yAO0SSqkp4pePCcsM9sTJ9Qyvik9xULWPONs5K2mkLhdeDC9fyT9sKvl621epvk0kMZ7wrDMzUaugrdNnFDxsAi+XdfqYA3v3IBlj2zWVOuuV9l3Q055ZIGFuU9P2TZ76gKcFv3UTf6yDwrh0MPKFVi9p47CR88cW/pfDEgRz/06N+zXzEPDwvJsL755lfgxP3mRGUVCCjXbxSNNPRozO4UVjDp/bKGzvsfd1ssFggLpuu9fgCdhq6/jh+/+ROvSWhbmxKk6wfey8UA7ice7UqSf70ohCcIlIvugxQDSvaWFPn5npiGUdkE9RvVkj5XvYRrKLrlMIdWQxcMnoB+1cKjGk1GG5tA57aGT52+Px747KkZYST+zOu0zpSLpKErJ1nWAMn/NnHotSylbZqUzSjqIoSBdLd6qRxDuQBwfKztcvBLXSYPU134NWo97r3wFNz3mVNqclt8/wl7YuWXTkuOi4bvvA1fdJ5KgGAUDdK88CIlJSbfkvPrRP8mGnpOf1PbmVfPvedMSnJA2crWUi6tleedq6HzGTnSmPOvF68RP0KPhnLhBsPIZS/qFAtn9GPfOZPx0vahjIauDqpiEzR0nY9yek+2HHXQ6CY5VWtTB71NI1MnPhuXnaehu+61KqedjSchpQ06Db1mga6pL5HZNiNx6IKmXQnDnIkg/bsasqz7Y2blEv1++ptnGftDQITD95iGvzlkd+VeYzWENui1/N2mRobRWlxhiUjKKqnGJajH5LoK71NQRbiGLfrji5TLYQun4c4nN+KZTTsl5wY+QSccek5/4317nzmT8OT6AeM31HHxtjGjUxRq2YS7HnSshs655qFy1ckoarqGp0LVaatD5So2DgzjIyfvjTs/fXKSKtNkdOPgHchFcJlSuqqQhaSZcuHFcbrgv99zROKXq/ZTtRSbtqHWK6OhIzuATVAFhYkecIpUFC5JNHSjH3r+kp+XQ5Tmcjn1gDnW65PkXNWs1m16PmNR//jAa5bg6g8eG7dF1dh5fcwriICA337wOLz3+D2VNuRTLrp6iajFD12FKPzS1WNa3oVn7o+L3vwKa134lobFgCS7Fwfn6Cf3FZNvMG1Cmk012ZjaUaD/9vzjcM+FJ9c08S+aOQG7TckGfKnt4mjGxvE2dK5AL3GBHroZRQ0CnQuXrHAjvLB1ECED5gnLq7++PIAvXvuY9Vm1uC0esXi69FvnRgk4aOgGykWmaszaLZA1DolQNRFby/Ka7aql6DQiNeZAp6HvGqlg7zmT8MET95KudR2o/DfvMuo3MVEleZsFq5ooEeHCMw/AIQumxeXY6yWfs19Ti9uiSfi7CHTTvZJNJfHASs9/4DV74a1H7hEdFyq71+xJ+Mgp+wAQvIYCSpQSEYcumIarzjsGHz55H8yc2IO3HrEQP3/fURKHXipkI3JV8L49qbeIeVP7zUZRzUud2FvEPxyxUHu9rhwf+m8AD2wZKledjKK5vurKey7GHD2Q5kznHXxiTwFfe+PBAKCdnRO3xZwl6yXveBUWzZwopdoUB9GHT04NVflui4prGHdRlASefI/62065qHQAjL9r1dBN4FeduF82qpVDdluM/uX7eKq9wsX7RLyOr+rU+mb90KN/KxoaxfR8XX/MbPxgeY+pHcRwPscoSqTPWCmillwuKkRNNHFbNAT4yP74wMdP3SdzjS7AqhAQXrVoBvpKBQQB4aK/PQQHz58Kih8TbXvnEHDn6LZoohLV1AEcunHaag29Yzn0REOvVGs2iorQabKA3Jl3TzjF6NhuU/vw9qP2wLypfcmWZiJc3Rb56es/cgIWX3A9ANlA88nT9sPTG3fiupXrpMGtNYpyLxch/Sl/hqmNquZi19DtWf+kQZnLoafP0e0Yf/E/Ho5p/T1YMnsSvv13h+L0g3dLn5OxA+gnuv5SQRNVatAmVQGtaOhiX3jNvrMzmnbCoddAuej6YyYVgKUsfq4eDT0gOcVvI5SLSfvVRU6b5geVctGVqaNcTG0sCN+vnm0gazGeA9nkXmn9dBq659C1SDX0rBuhDibDKdfudZQLB6dcuLDtj/11TzlgrtTRDt9jGgDZ3coGXcftVdwoZ07sictKr3HR0JN2iZRL5vnqs83bjqn5VzJlCX/nebmI/s2fOeuATD0WTJ+A4/eJ+NG/fdUCTNIE/iTPMqxA+nsKNXsvqL85vcPrO7m3iJ++90ijl0s1zBPo6d+6LpvV0I1F5VIuNgXdtCJRwWM0bCsqFw6dCzGTtqyjzVRM0GjoxqAroQwX10vb/SJM/cekoeuKGXMOnYh+TETriehRw/kTiWgbEa2I//tC86uZhWgUvezOp3OvN9Ey/LD68kWvES5QuMZi2m/xX8/YHwCw39zJyb026E6rnWly7DmQS2NwrweFQ5cDkuy0ic1NTR0YVrfFnD5rSrWalGW/HYC8AknrkP6Y0FPAua/ZCx8+eW/sOSvKFW8WAMrzBeoGSNvOFYcsdRVr6KE7h65TQkwTi60sVxpJd46vdvI4dBttYfQIESmXnB28JA8pwzU6gW5M2CZUtz4NXX/c1FZ1xyPb9aaslM2Cy/T1EwBn5FxzJ2PssPi/rzRerXzw9K1Pb9yJVeu2516vDiC+o3ySV1sRI7wjiIKMd0zTtnFHL5mJZy56XeL2pXPdEqHrHwfPj+rFdzyZmKQCtafgTCgXi/alPs8W1KQiz21R/Klr66FCYJTKWapX2wWZ+Vrx1ISeIvp7CvjkafulAsVQrkmgcSWgqEySdj/0fEEKIJM+QixHd32mrMBed5vYSI2h9ueUHDR0UxVFaoG3y6SdmmgzETrKpVa/dleYJgpT/acbNHRd/cacQ2eM3UFEi1taizrAKZcLr37E6Xo1Mveq845FlTFcvfwF7fVckIuCjPPINmpChPhBewoBBkPZnU43GI/ccwaWfe7UZDPfSXEQlBj9qOsSJj9022DJC2zRlZ/WXT6vywZJJHiKCJNgntaUw1QZrxXbI066SQ4b01LaJNDjf/lkqVvNiVurVaqhdVKUjKI6ysUwUejLql9DF33YAfP75m2xB884aOg56aQljxhDg/pKgdSfbOWJ79HVAC/Vx2QUNVIuJg3dvYxmoVkc+jFE9DAR3UhE+i1aABDROUS0jIiWbdiwoaEH6rTkb7zpFbjwzP2112dS3hYC9BYLRsqFz+xih+A8sussK2oHuoFuKkXcmZ1rJkMVMd9y9s6exPAU/ctXJBLloj4/R2OXy1cFunytTkMX7xHpnLx0pjbYDJ3i33rOVV9m1qc+Ahe6yb6Y4PYWZWKMyy3nceg5JHpdof9GrTf/Xt4M03MaEehivWozikb/LvvcqdI1ug2zbcZLXq16Anlccu+LmCzYeI5cPCP5W6ewjbmG7oDlABYxxgaI6CwA1wDI+h0BYIxdCuBSAFi6dKmL+7gROr735P3nYM7kXnzzxicy597yqgXacpJBqrxnLrzFj9Lj6L3CIfYlXX1d/NQ55SJmc9PdZgqvDkhwT1OqkKE6bEY4pdxMFTSCNUp5GsZ/mzX0WiYWy2Olv0WBntgTjJSLoXBOuSQcuv4yMZeLzedZPOWiobtQT7XSSGo9bNcmlItlQxIXt8U0UtTBKBpfKyo1HBN6ilK2RdswLBWibKz1CNBaNfQgIJx34l44YZ9ZOHavNEWDXkNvrR9KwwKdMbZd+PsGIvo+Ec1ijGV3c20idBo6kX5wPnPR64zlpIqSfGO6TEyP6baws4F/vAXT+/Ge4/bEV697PFNfjove/Ao8vXFnpgxukB3MEehqMJPI9yZ2gowh012AqDBpteK53lIBGIpyb4gbYuQtg23VeNcxi/Dw2q14z3F7Zuoh3qfrH7W6oyWUC5+ZDQJd9nIx1z3PbTGrodsEKadN9Od1t3LKIjWK2p9z5OIZOP2gudg+WMHzmwcNz8kXfqmyoa+r2G4b7aeuumzeVD2xQHfN7CnC6IduKYo7ROSV0/YaOhHtBuBlxhgjoiMR0Tibcm5rGGaBXtsL48u4KX3yq0gjSLMauutH4R3g1APm4uglMzLnxYHEo+ZU8NztgyNpJAoh3TRArW/KoWdzc2c0cuVALfxeVssTy4n+FYW4qKFn+HhLfhYV0yf24MfvPkL7XBPlonPhFKEKEX6dGlikE8KigFTroyKXQw/U3+bC8v3Qs8dLQYCRapjUI89TZvGsifjhO5fiPf+dTcWcPkd/XBwjaSoMk4ZurzeHKtBtwr+WXcNU1Eq5mDAWkaK5Ap2IfgHgRACziGgtgC8CKAEAY+wSAG8BcB4RVQAMAngrcwndbBC6vRbzouN0eOPh87FxYBjvOmYxfnjHmuR4qvGm1+bxlioKgjFI93Fd+sckDeXCaRTxJSd0kKp1C6/Jln9Fd94G9d6pwr6M/P2IQlwU7nkRkbWMGxcOXefCKZch/+Y/1cCidKUjX58XxZuWm57TDZHsxGIsKp9y0RwrFQgjQl5yfk3e+7YpSS70BFe+XCYfGyORl5NfREIX1UFx1Eq5mKBdSY+1QGeMvS3n/MUALm5ajRyhM3bU864KAeEDr9krczzJwih8larixpZbtuBFoF0CO0xAeg49jqYThALvhKqXi017zHLXudVJcNSSGfi7Vy3AuSfuhWXPbMbrXjFPKIcLdL2Grg6MfzxyDzyzcSd+du+zNdfDHFikcXOrkW8OVYFu0FMKhklFhZiHRrdNYCY5l4OXi+lx2rDzQgCgmkkb4JI7vdZzojbOJ3uX7fLENt/+qROlPUQnKt80j3IxPfOa84/DjqGydhMYIJtTR1c3F4yFH3rHhv7rUOuSSMVesycm2kQxCYZIz1eFZEEuEDUhXd1ciuGduCIIANHQycF/25Jz5bot1tBhS4UA///fHQogSqgklROXK3r29EocujwZ9/cU8NU3HpwIdLfQoghSlUWBLlJyfHJz8LIQ8apF07Bq3XbMmhS5pRmNokK5ti4ozgc6+kZVFFxcD01R0jbtUG1v/ibrllWHA4c+pT/dVjGvDLHNi2ZOxKKZE5PfGQ3diXLJfvTDFk4z3gfYOPTGBfqYa+idhAblOW755InJ37wjiNoLF+jObouchye90HDh+8X9T5P7kN3gN+FDMxy6eJ96T+31cQGvg+i2KG6tl7cMrqXPm4SB1m3RQfiI+MLZB+EdRy+yPp/BPmmq1+r+Tu6tySga/WveJi97rKgEWPFL8ghSV0Ov9CzhpmSPz7I+Q57r+8saRc314kqDLfT/p+89ErM13jRGyqVWDl3zaJ/LxQKe8ZCjWQIJSP26xW9btWwCrEPqiaBPOOQiuPjEIvq6Rhv8yjcnGnr8Rz3pc5sF3vFFTVzS0A1ucKqxzgUmg1p/DRx65nnxz55igP13m5I7gYvttF0qatOhRhLbonxVqIbbzPm4EVKSLCXAh5eRL9DzJ5bMcVFDj9NXDGi2j1PLsGnBGYFuqRf/JraEc6/ZdzYOFDKdJnUwcf0doKF3tEBXtadmyife+cXBzjUcnfanL4NLKP2i1bW+l71rKa7/yAnpfcn/UqSBRHJUo21z6RbJ88RbQ3y2yKebvB043VVLtUxePOKKQOfxI5dhf0aeVhXl3M7WR4VMueie406BJRq6QUXn50XhxCmwJG0Ar5fxKfF1dWjoIibFHmQDQyaBLmro5nLU8H97ZHN0zmUbvUx9mqWhj4GXS0cLdBUsL+d5DSgpWQ8B4O+XLsQHXrME55+0t+EuGSmHbvJycfu4rz1wLvaYmW5yG3nNyNfw1YO6whQfkU2o1ZrOxcsVc1yIRlHTMpgPvnr94cW/dZqZUZvMeV6eVkVEyQTiWnUdb13L90k5dHOdxOsAkUNPFQ0X1JJbRwfuqbVzRC/QpT5ai4bu4OVSj0AXBffF/3i40/N00F3uNfQaoEuCXy9KaudHpEFeeOYBiedJHtKcGSajaH0flyhLuVQ1of7qM9S+1Kq+VYkT5/C0w4DstmjyduitUSiq15LQm3WThosBD8jKOZsvM9f++QRi+6aiNq2zI6j1sA3+PMqF36qL2FTrmOtlXIOGvmT2xMw1XKCbKBcpiZ3l/dXmtsg19Nplglju2YfsXnNAIYc29L/Fe4p2jVHUFg1aD9Q9OutBoqGTyW2xPkScvHzMxO9bKZc6n5+Hl7YPAQDmTe1Df6mAwXJVCgQzCvRYKOpc+kwwauiCZlYrpWDboNsENY+9DmI9LheCo0zPsU0kqVHU5OWS9j2OUkGuI1cK8t5PLRr61ecdi5e3D0vHuAK008ihZycdHVS3RZdkci6bdGTqk3UDyn2eDmMRKdpVGnozkVme1gGRctEV00jZ6p26ZFyAosGOklF03bYoTHze1H4snBFp6eVqyoeZvFy4UBR3ds+DyYtHHMhJAjZjGXmUS/4wyQueEevxESFHu60etrD198apD/aNc++boE2SlRjrecWsRVgnfrXO0yb0YL/d5DrxLRzPPmR3ffmk/1vF3Cl9kkC0fRZ+XSOUi+oaWquDik52ew59jFDSBBbViqKgodcbKaqDLvKUy0sb5aI+7l8NmSlFXP5PS/G784+rqX7cB3zRzAn49t8dhqOXzJAEj4mLnDM5yiPvsgNVUpZBQ5cEuiEBm+4+HWrT0C2Uiym1Z/Ic+bdNmzvtoN3wzEWv0yax4ghISSWb8XJR6mUpp55zHFP7S3jiq2dIe+TKZbhRLqcdOBe3fepE7X2mMuvZsYjfwt8/32ayGUZR74eeg5kTe7BJiCZrNhp5/6J7WDMFOhEy0vnUA+Zgv7mT8cET5UFji2I8YvEM/ODtr8R5Vyw3PuuUA+bWXL+vv+kVOP2g3RIh/stzjsGzm7KJx1T851sPwzUrXsT+u9m1ThEm7U7LoRt0zbxv7DIIOVdruzTdTEUPlWlqdPAnEcW8vIIsyF1TZdgpF7cyTJvCqOVbPXsCwoLpqXOATcCGim2jFvD68Pf/w3e+Cs9vHqyZ/9ZVb8xzubQ7bv/0SahUm+jeEoN3iEZYiWQAGcqpJ/cMoB9g0yb04A8ff7X1Wi3tE5DxXL2YMbEHbzx8vrEeJsyc1Iv3Hb9nTc8yBRbVMvj4hskmBdplEPY5GEWR06fE9A5A4wa0yBif/uarzoxRNK8gq4beeMeRaLNaPJws34VPjo0YRfn7n9BTzNBITvXTGkU95WLFpN6icceQRsA7REMcurC01QvT+sqlGuolen5o/WLjYypfu0TD8TYCVSgunNGPdx+7uKnPML2S3aZEVI49r7f5XDY7ZBa1aOimZ6m2g0YHP4HkXbNUP/T4VK2BRY9++XThXENVjOvRfCHHFbJGjKKNrpB0K4hW2a04Ol5DbxV2j/cF/RuDIccFeX7oDbktCre6pFnl92XqEPd3laK44aMnNFVrV+t456dPbl7hMLuGAsAl73gV7nhyA+ZN7dee5/dXLefyUAuHbrpCDY1vZGenqC7RO+krBRgqp9vjiYoGkJ/LRa2vazIyV7SChUi2PmzAKNrwhOopl/bBnCl9ePwrp2e2vaoFRWGTDC2HXme5RDJZYzP8yEZR86SibjpRKgRN7XytVEy+/qaDceTiGUbBMHNSL950uH7HKo7oPegFrk1Ic8GRermYn5FkwDRcNFyRp5RG3z+f+Cf0FDFUHkmEWyq+SaqXCdmAJ/3f9aIVQq4RDT3d1LrRCXX0NfSOp1xaiQk9xYaWg+LSVldKrWV/4rX7Jn9/922HJ3/bOq34CH2yIJkvTO6rqWb5qNVDoBa8/ahF2Gfu5Ma+lbSS0Zdjo4hcolxDu5MLhlQNvUENkXtDcaWE15H7+bu+LptNoTkaem1lXPn+o/DPJ9htLYlRtA4OPWiShq5Di+W519BbCTEfTDO8XD5yyj74yCnRdq0n7DMbP3znq/CBnz1oFejyJtFmDp1r+ZP7itgxVGl6x2v1UrNR5FUvL3Ctt5Qf5ZrsnGSYLlUNvVGjKCF677x/8AheNXArP/BKrm+zKZdaizh271k4du9Z1mt4ExvT0JvfZz2H3sHoLQY4cvEMHDhvimSc5Gj04/K7bTxhXuIjPli5Nnjth47H3as3Nt1QVWsejNFGo9/ChUM37XjEMUlJKdGoQOERxYfMn4rV6weS0PlMoFUO56LWV1r1NeGztkLI8cmzMT/05hMYXqB3MIKA8OtzjwEA7BgqZ843+mmT4AmrQBeeZzHScG1wz1kTtVGMjaLVHblRNFq9NJeL+RpdjnoRn33dAdhvt8m47I41eHHbUONL/th28403vwJvPHx+EgvA8/7wZFd5KwG1vuKk1YyJ32THaQR8jqrnHbaScmm1XuM59FFCM71cOPjtNi0kb/DxW+vZHb0WtJJDbwYKAQkbg7vjdYdEW+/1OXC1CYdueMLkvhLec9yeiSbfqIYYEKFAhL5SAa/ed3YiuHmSsI+dui8+dNLeeMurXAzGpnMNVVEqQ12hNII8jyIXtIJyaVWGUw6voY8Smhkpqt6f55ql2180LYNz6C22vre56hAQoacQoFw1OS9m8fmzD8R7YkMp19A5T61DTuR/As5xN6qxqu6tXEBxDX1ibxH/cvp++eVYzjWTQ+d505uB1ABde/2SnclaoOR4Db0OXHjm/vjqGw4a62pI0PWrxgV6PuUiXa85xgXQeNfQiahmA9rk3mJiG+Aa+lDFPCG89/jFOGj3KZkoWhVcGDVDQ9flt6klmyVgF4rNEOjcu2dSbynnSnfk0Vs2VJK9BTrPKJrbY4jox0S0nogeNZwnIvouEa0mopVE9MrmV7M2fOA1e+Efj7LvBTna0Ab1NM0oml3uX/fh4/GZs+TkW7rn8bQJrQ5Jbncvl1mTejA3jiit57PMmhwlydq4w5xXaMH0Cbj+IydYE2pFiAVKw5Gi+i3oTLscGcuxVENn7K8Vu0aiSXBKEzX098YpJOqxB6VKTucJdJc3+BMAFwP4H8P5MwHsE/93FIAfxP+OKdpNfLSCcuEdT6dZHjx/Kg6ePzX3eSOxQG+1ht5q7rBR/OqcY3DPmk049+cPop7ewyOLeS74RpBQLk0IbBHnUT5pV2vIZgmkfffK9x+FYSVvUjME1Cv3mIZ3Hr0I5524V8NlcbzhsPl4w2H2lZAJlTBqYys2dG71MMitMWPsDgCbLZe8AcD/sAj3AphGRPOaVcF60W7yoxVGUS6MXV2zdI8rV+t37+omTJ1QqnnHKzFkXtydqVGEDXhoiIg49LQMrnHWmsuOl3D0kpk4ab852nONoFgI8NU3HpzkTR9rHL5wOqb2l/DRU9y2mnTB+SdFk9XCGRNyrmwMzVjjzAfwvPB7bXxsnXohEZ0D4BwA2GOPPZrwaDPaTSPU+oA3WOZInMzJmUPXvJNF8V6lr9lvdoO16Xy4skK6rjV3ch6N4g7uodF4pKjcpoRyqVVDDyJNXxdL0O7uqPVg6oQSHv7iaU0t833HL8GnTs/ff6BRNEOg676otscwxi4FcCkALF26tLZe1eHQCdNGJx0eWegq0HUCa9+5k3H/Z0/B7FxetzOweOaEXKOjCY0Ip2Z6RHB52+iSX822yDX+WgU6kdkG0oXyvCWY2t88g68NzRDoawEsFH4vAPBiE8rtejQ6GLiG7ppRzuT/zHcK6gbc9qmT6r630e/xydfui/nTG6cNuMBtPFJUpVzq83J5/aG7G/tIN2rorcBoOQU0Q6BfC+BDRPRLRMbQbYyxDN3ikUWjg2G4RsqlzR1Nxhx8wqv3s3w4zrPTKFLKpblG0XoF+kG7T8VBu0/VnvN9qr2QK9CJ6BcATgQwi4jWAvgigBIAMMYuAXADgLMArAawC8B7WlXZbsDpB83FHx57GUDzBoO6G7oJY21XOHTBVPzNofXnl2812kU4NcsoCqhui9wo2jy202vo7YVcScAYe1vOeQbg/KbVqMk4YR97VrbRxEOffy0m9hax7+duBGCmQFzx9qMWYd22IWd3r7Eee7/70PFjW4EcJNvxjXE9WLMol0AWuFxDr5VDt2Gs+5SHjK4O/X/gs6dichODFRrF9InyVnmNBmX09xTw+bMPdL7ea1N2tJ+G3rhRVLdJdBMV9DFf9bU7vvW3h2DetNGzUbWPtGsBZjfRlawVGO2h4IdeHhrj0JuFhENvglFU4tCD+jh0j/rx90cszL+oiRjf0SRjjNHWmL2Gbke7aOisWRp6nG2Ro9XpHTzGHl6gjyFGXb768WyF64T3vuOXYNakXpy8/9yW1qc5e4p6gT6e0NWUS7tj9DX0UX1cx8H1e+y322Qs+9ypLa5NE9LnQv7m4z29w3iA/8LjCJ5ysYO/nka9j5qFRtPnlgqB5MveivzeHu0Fr6GPIUZbwHp5bke7vZ9G3Ra/cPaBmDohDTlvxQ48Hu0FL9DHEH58tRf4BDvWgn1yXxE7hioNb6x97N5yDEarUyR7jD28QB9DeB/e9kK7UFLXffh4LH9uS9PLbfdNRjwahxfoY4jRGl+7Te3D0xt3tg033K5IOfSxxaKZE7FoZu077Xh4eIE+hhgtDf3Kfz4K963ZXPMGDuMN40WBPfPg3ca6Ch4tghfo4wDzpvbXnSN8PGE8UGCrvnJGS/bK9GgPeIHu4RFjPIg5v0rrbnizt4dHjNTLZTyIdo9uhBfoHh4x2sXLxcOjXniB7uERw8tzj06HF+hjgH85bV/0lzyX6eHh0Vx4o+gY4EMn74MPndyc/Sc9mg+vqXt0KryG7uHh4dEl8ALdw8PDo0vgKRcPjxhN3Du563HzJ16NLbvKY10NDwVOGjoRnUFEfyGi1UR0geb8iUS0jYhWxP99oflV9fAYHXgOPR97z5mMIxbPGOtqeCjI1dCJqADgewBeC2AtgAeI6FrG2OPKpXcyxs5uQR09PEYFDF5F9+hsuGjoRwJYzRhbwxgbAfBLAG9obbU8PEYfPLDIu5R6dCpcBPp8AM8Lv9fGx1QcQ0QPE9GNRHSQriAiOoeIlhHRsg0bNtRRXQ+P1mHB9H588rX74vJ/OmKsq+LhURdcjKI6RlFdmy4HsIgxNkBEZwG4BkDG0ZoxdimASwFg6dKlfn3r0VYgInz4FB8f4NG5cNHQ1wJYKPxeAOBF8QLG2HbG2ED89w0ASkQk73/l4eHh4dFSuAj0BwDsQ0R7ElEPgLcCuFa8gIh2ozhFHREdGZe7qdmV9fDw8PAwI5dyYYxViOhDAP4AoADgx4yxx4jo3Pj8JQDeAuA8IqoAGATwVsa8V6+Hh4fHaILGSu4uXbqULVu2bEye7eHh4dGpIKIHGWNLded86L+Hh4dHl8ALdA8PD48ugRfoHh4eHl0CL9A9PDw8ugRjZhQlog0Anq3z9lkANjaxOp0A3+bxAd/m8YFG2ryIMTZbd2LMBHojIKJlJitvt8K3eXzAt3l8oFVt9pSLh4eHR5fAC3QPDw+PLkGnCvRLx7oCYwDf5vEB3+bxgZa0uSM5dA8PDw+PLDpVQ/fw8PDwUOAFuoeHh0eXoOMEet6G1Z0KIvoxEa0nokeFYzOI6I9E9GT873Th3IXxO/gLEZ0+NrVuDES0kIhuJaJVRPQYEX00Pt617SaiPiK6P97d6zEi+nJ8vGvbDER7ExPRQ0R0Xfy7q9sLAET0DBE9QkQriGhZfKy17WaMdcx/iNL3PgVgCYAeAA8DOHCs69Wktr0awCsBPCoc+xaAC+K/LwDw/8V/Hxi3vRfAnvE7KYx1G+po8zwAr4z/ngzgr3HburbdiHYAmxT/XQJwH4Cju7nNcTs+AeBKANfFv7u6vXFbngEwSznW0nZ3mobetRtWM8buALBZOfwGAD+N//4pgDcKx3/JGBtmjD0NYDWid9NRYIytY4wtj//eAWAVov1qu7bdLMJA/LMU/8fQxW0mogUAXgfgR8Lhrm1vDlra7k4T6K4bVncL5jLG1gGR8AMwJz7ede+BiBYDOByRxtrV7Y7phxUA1gP4I2Os29v8HQCfBhAKx7q5vRwMwE1E9CARnRMfa2m7XTaJbie4bFg9HtBV74GIJgG4CsDHGGPb490MtZdqjnVcuxljVQCHEdE0AL8looMtl3d0m4nobADrGWMPEtGJLrdojnVMexUcxxh7kYjmAPgjET1hubYp7e40DT13w+ouw8tENA8A4n/Xx8e75j0QUQmRML+CMXZ1fLjr2w0AjLGtAG4DcAa6t83HAXg9ET2DiCI9mYh+ju5tbwLG2Ivxv+sB/BYRhdLSdneaQM/dsLrLcC2Af4r//icAvxOOv5WIeoloTwD7ALh/DOrXEOKNxS8HsIox9u/Cqa5tNxHNjjVzEFE/gFMBPIEubTNj7ELG2ALG2GJE4/VPjLF3oEvby0FEE4loMv8bwGkAHkWr2z3WluA6LMdnIfKGeArAZ8e6Pk1s1y8ArANQRjRbvw/ATAC3AHgy/neGcP1n43fwFwBnjnX962zz8YiWlSsBrIj/O6ub2w3gEAAPxW1+FMAX4uNd22ahHSci9XLp6vYi8sR7OP7vMS6rWt1uH/rv4eHh0SXoNMrFw8PDw8MAL9A9PDw8ugReoHt4eHh0CbxA9/Dw8OgSeIHu4eHh0SXwAt3Dw8OjS+AFuoeHh0eX4P8BckWeWLofeXgAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "%matplotlib inline\n", "output['result'].plot()" ] }, { "cell_type": "code", "execution_count": 22, "metadata": { "execution": { "iopub.execute_input": "2022-07-27T19:15:35.153720Z", "iopub.status.busy": "2022-07-27T19:15:35.153390Z", "iopub.status.idle": "2022-07-27T19:15:35.160720Z", "shell.execute_reply": "2022-07-27T19:15:35.160093Z" } }, "outputs": [ { "data": { "text/plain": [ "1.9844499213466507" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "output['result'].mean()" ] }, { "cell_type": "code", "execution_count": 23, "metadata": { "execution": { "iopub.execute_input": "2022-07-27T19:15:35.164483Z", "iopub.status.busy": "2022-07-27T19:15:35.164056Z", "iopub.status.idle": "2022-07-27T19:15:35.177119Z", "shell.execute_reply": "2022-07-27T19:15:35.176496Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "244\n" ] } ], "source": [ "filtered_output = output[output['result'] > 2]\n", "print(len(filtered_output))\n", "filtered_output.to_csv('/tmp/simulation_result.csv')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Handling very large simulation with [Bags](http://dask.pydata.org/en/latest/bag.html)\n", "\n", "The methods above work well for a size of input parameters up to about 100,000. Above that, the Dask scheduler has trouble handling the amount of tasks to schedule to workers. The solution to this problem is to bundle many parameters into a single task.\n", "You could do this either by making a new function that operated on a batch of parameters and using the delayed or futures APIs on that function. You could also use the Dask Bag API. This is described more in the documentation about [avoiding too many tasks](http://dask.pydata.org/en/latest/delayed-best-practices.html#avoid-too-many-tasks).\n", "\n", "Dask Bags hold onto large sequences in a few partitions. We can convert our `input_params` sequence into a `dask.bag` collection, asking for fewer partitions (so at most 100,000, which is already huge), and apply our function on every item of the bag." ] }, { "cell_type": "code", "execution_count": 24, "metadata": { "execution": { "iopub.execute_input": "2022-07-27T19:15:35.181606Z", "iopub.status.busy": "2022-07-27T19:15:35.181401Z", "iopub.status.idle": "2022-07-27T19:15:35.197220Z", "shell.execute_reply": "2022-07-27T19:15:35.196611Z" } }, "outputs": [], "source": [ "import dask.bag as db\n", "b = db.from_sequence(list(input_params.values), npartitions=100)\n", "b = b.map(costly_simulation)" ] }, { "cell_type": "code", "execution_count": 25, "metadata": { "execution": { "iopub.execute_input": "2022-07-27T19:15:35.200194Z", "iopub.status.busy": "2022-07-27T19:15:35.199987Z", "iopub.status.idle": "2022-07-27T19:15:42.994308Z", "shell.execute_reply": "2022-07-27T19:15:42.992667Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "CPU times: user 926 ms, sys: 121 ms, total: 1.05 s\n", "Wall time: 7.79 s\n" ] } ], "source": [ "%time results_bag = b.compute()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Looking on Dashboard here, you should see only 100 tasks to run instead of 500, each taking 5x more time in average, because each one is actually calling our function 5 times." ] }, { "cell_type": "code", "execution_count": 26, "metadata": { "execution": { "iopub.execute_input": "2022-07-27T19:15:42.998106Z", "iopub.status.busy": "2022-07-27T19:15:42.997641Z", "iopub.status.idle": "2022-07-27T19:15:43.016245Z", "shell.execute_reply": "2022-07-27T19:15:43.015548Z" } }, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.all(results) == np.all(results_bag)" ] } ], "metadata": { "anaconda-cloud": {}, "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.12" } }, "nbformat": 4, "nbformat_minor": 4 }