{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "Scale XGBoost\n", "=============\n", "\n", "Dask and XGBoost can work together to train gradient boosted trees in parallel. This notebook shows how to use Dask and XGBoost together.\n", "\n", "XGBoost provides a powerful prediction framework, and it works well in practice. It wins Kaggle contests and is popular in industry because it has good performance and can be easily interpreted (i.e., it's easy to find the important features from a XGBoost model).\n", "\n", "\"Dask \"Dask" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Setup Dask\n", "We setup a Dask client, which provides performance and progress metrics via the dashboard.\n", "\n", "You can view the dashboard by clicking the link after running the cell." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "execution": { "iopub.execute_input": "2021-10-14T16:51:30.376271Z", "iopub.status.busy": "2021-10-14T16:51:30.375512Z", "iopub.status.idle": "2021-10-14T16:51:33.357724Z", "shell.execute_reply": "2021-10-14T16:51:33.358312Z" } }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
\n", "

Client

\n", "\n", "		\n", "

Cluster

\n", "
    \n", "
  • Workers: 4
    • \n", "
  • Cores: 4
    • \n", "
  • Memory: 7.29 GB
    • \n", "
\n", "
" ], "text/plain": [ "" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from dask.distributed import Client\n", "\n", "client = Client(n_workers=4, threads_per_worker=1)\n", "client" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Create data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "First we create a bunch of synthetic data, with 100,000 examples and 20 features." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "execution": { "iopub.execute_input": "2021-10-14T16:51:33.367680Z", "iopub.status.busy": "2021-10-14T16:51:33.367108Z", "iopub.status.idle": "2021-10-14T16:51:34.191724Z", "shell.execute_reply": "2021-10-14T16:51:34.192579Z" } }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Array Chunk
Bytes 16.00 MB 160.00 kB
Shape (100000, 20) (1000, 20)
Count 100 Tasks 100 Chunks
Type float64 numpy.ndarray
\n", "		\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", "\n", " \n", " \n", "\n", " \n", " 20\n", " 100000\n", "\n", "
" ], "text/plain": [ "dask.array" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from dask_ml.datasets import make_classification\n", "\n", "X, y = make_classification(n_samples=100000, n_features=20,\n", " chunks=1000, n_informative=4,\n", " random_state=0)\n", "X" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Dask-XGBoost works with both arrays and dataframes. For more information on creating dask arrays and dataframes from real data, see documentation on [Dask arrays](https://dask.pydata.org/en/latest/array-creation.html) or [Dask dataframes](https://dask.pydata.org/en/latest/dataframe-create.html)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Split data for training and testing\n", "We split our dataset into training and testing data to aid evaluation by making sure we have a fair test:" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "execution": { "iopub.execute_input": "2021-10-14T16:51:34.197061Z", "iopub.status.busy": "2021-10-14T16:51:34.195651Z", "iopub.status.idle": "2021-10-14T16:51:34.457756Z", "shell.execute_reply": "2021-10-14T16:51:34.458587Z" } }, "outputs": [], "source": [ "from dask_ml.model_selection import train_test_split\n", "\n", "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.15)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now, let's try to do something with this data using [dask-xgboost][dxgb].\n", "\n", "[dxgb]:https://github.com/dask/dask-xgboost" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Train Dask-XGBoost" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "execution": { "iopub.execute_input": "2021-10-14T16:51:34.462520Z", "iopub.status.busy": "2021-10-14T16:51:34.461240Z", "iopub.status.idle": "2021-10-14T16:51:34.506813Z", "shell.execute_reply": "2021-10-14T16:51:34.507665Z" } }, "outputs": [], "source": [ "import dask\n", "import xgboost\n", "import dask_xgboost" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "dask-xgboost is a small wrapper around xgboost. Dask sets XGBoost up, gives XGBoost data and lets XGBoost do it's training in the background using all the workers Dask has available." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's do some training:" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "execution": { "iopub.execute_input": "2021-10-14T16:51:34.511699Z", "iopub.status.busy": "2021-10-14T16:51:34.510205Z", "iopub.status.idle": "2021-10-14T16:51:42.660539Z", "shell.execute_reply": "2021-10-14T16:51:42.657222Z" } }, "outputs": [], "source": [ "params = {'objective': 'binary:logistic',\n", " 'max_depth': 4, 'eta': 0.01, 'subsample': 0.5, \n", " 'min_child_weight': 0.5}\n", "\n", "bst = dask_xgboost.train(client, params, X_train, y_train, num_boost_round=10)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Visualize results" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The `bst` object is a regular `xgboost.Booster` object. " ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "execution": { "iopub.execute_input": "2021-10-14T16:51:42.665535Z", "iopub.status.busy": "2021-10-14T16:51:42.665012Z", "iopub.status.idle": "2021-10-14T16:51:42.673555Z", "shell.execute_reply": "2021-10-14T16:51:42.674215Z" } }, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "bst" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This means all the methods mentioned in the [XGBoost documentation][2] are available. We show two examples to expand on this, but these examples are of XGBoost instead of Dask.\n", "\n", "[2]:https://xgboost.readthedocs.io/en/latest/python/python_intro.html#" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Plot feature importance" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "execution": { "iopub.execute_input": "2021-10-14T16:51:42.676997Z", "iopub.status.busy": "2021-10-14T16:51:42.676355Z", "iopub.status.idle": "2021-10-14T16:51:43.109645Z", "shell.execute_reply": "2021-10-14T16:51:43.110403Z" } }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAZR0lEQVR4nO3de5wU9Z3u8c8zgEgUUCO4XOSieBRBAwYlboiOidFE9ARRyVHcxEvkeEyCJF5i4rqrOVHcJComevS46nrBXTeKAqvuJt4mGhUjKsEEFnWFhFuiEpSrkcHv/lEFNuPM0MB018z8nvfr1S+66ldT9e1f0/V0XbpKEYGZmaWrpugCzMysWA4CM7PEOQjMzBLnIDAzS5yDwMwscQ4CM7PEOQhsm0j6jKQFRdfRGEm1kpY00/5pSa9JWiNpTBVL22GSvifp1qLrsPbJQZAISYskrc9XgpseN5TxdyFp0KbhiHg6IvavUI13SPpBJead+z5wQ0TsGhHTd2RGeX8e3TJlbV1EXBURX6vW8poj6XJJU4uuw1pOx6ILsKo6ISIeK7qIAvUHfld0EQCSOkZEfdF1bCtJXme0RxHhRwIPYBFwdBNtg4BfAu8CbwP/mo9/CghgLbAG+DJQCyxpMN+LgLn5dLcBewH/DqwGHgN2L5n+PuCP+bKeAobk4ycAG4D382X9Wz6+NzANeAtYCEwsmVcX4A5gJTAvr2NJE6/xv4APgPX5/DsD3fN6lwNLgR8AHfLp9wWeAFbkfXIPsFvedneDeV3csF8a9jlwOXA/MBVYBXytueU3Uv/lwNT8+YD8fTkTWJy//nOBQ/P34R2yLZ9Nf3sG8Azw07zf/xP4XEl7b2Am8GfgdeCcBsstrfsb+Xu0IX/tv8mnOxOYn7/nbwD/u2QetcAS4ALgzfz1ntngfbwG+H1e36+ALnnbp4Bn89f0G6C26M9Se3wUXoAfVXqjmw+CfwEuJdtVuDMwqqQtgEElw1us8PL5ziJb+ffJP+gvAcPJVrZPAH9fMv1ZQNe8bQowp6TtDuAHJcM1wIvA3wE7AfvkK5lj8/argaeBPYC9gd/SRBA01gfAdOD/A7sAPYFfb1qBkYXj5/M6e5CF1pRm5lXbcNl8NAg2AGPy19WlueU3UvvlfDQIbs7fr2OA9/L59Sx5H47Mpz8DqAe+BXQiC/R3gT3y9l8C/y+f1zCy0P1cM3VvrqWkvtFk4SngSGAdcEhJ39ST7ZrrBByXt++et98I1OV1dwD+Ou/3PmRBfFy+7M/nwz2K/jy1t0fhBfhRpTc6WymtIftmtelxTt52F3AL0LeRvysnCMaXDE8DbioZ/iYwvYmadsvn3z0fvoMtg2Ak8IcGf/Nd4J/y528AXyhpm0CZQUAWXH8h/+aZjzsVeLKJvx0DvNzYvBrrl0aWdznwVEnbti5/88qXD4OgT0n7CuDLDd6HSfnzM4BlgErafw38DVmAbgS6lrRNBu5orO6GtTTT19OB80v6Zj3QsaT9TbJv+zV52ycamcd3gLsbjPs58NUiP0vt8eH9fWkZE40fI7gY+L/AryWtBK6JiNu3Yb5/Knm+vpHhXQEkdQCuBE4h+5b9QT7NnmTfUBvqD/SW9E7JuA5kWwGQ7dJYXNL2+22ouT/Zt9PlkjaNq9k0P0k9gZ8AnyHbgqkh2wWzI0prbXb5ZSqr33NLI1+T5n5P1n+9gT9HxOoGbSOaqLtRkr4I/D3wP8hex8eAV0omWRFbHhNZl9e3J9mWyH81Mtv+wCmSTigZ1wl4cmv12LZxEBgR8UfgHABJo4DHJD0VEa+38KJOA74EHE32bbk72cp105qw4aVwFwMLI2K/Jua3nOwb7aYDwP22oZbFZN/I94zGD9pOzus5OCJW5Keblp5l1bDWtWQrP2Bz6PVoME3p32xt+S2tjySVhEE/suMCy4A9JHUtCYN+ZMcsNmn4WrcYltSZbAvkK8CMiNggaTofvq/NeZtst9a+ZMcASi0m2yI4p4z52A7w6aOGpFMk9c0HV5J90Dfmw38i2zffErqSrfxWkK00r2rQ3nBZvwZWSfqOpC6SOkgaKunQvP1nwHcl7Z7X/81yC4mI5cAvgGskdZNUI2lfSUeW1LoGeEdSH7ID0c3V+iqws6TRkjoBf0u2n3t7l9/SegITJXWSdAowGHgkIhaTHYydLGlnSQcDZ5MdHG/Kn4ABkjatP3Yie61vAfX51sEx5RQVER8AtwPXSuqdv8eH5+EyFThB0rH5+J3z34r0bX6utq0cBGn5twa/I3gwH38o8LykNWTfEs+PiIV52+XAnZLekTRuB5d/F9luh6VkZ/nMatB+G3BgvqzpEbEROIHsAOZCsm+Pt5JtSQBckc9vIdlK9e5trOcrZCuxeWQBeD/Qq2Teh5DtsnoYeKDB304G/jav9cKIeBc4L69vKdkWQpM/bitj+S3teWA/sj68Ejg5IlbkbaeSHXdYBjxIdnD/0WbmdV/+7wpJL+VbEhPJgnkl2ZbfzG2o7UKy3UgvkJ259A9ATR5SXwK+RxYyi8kC2eutFqYtdxuaWXsj6QzgaxExquharHVyspqZJc5BYGaWOO8aMjNLnLcIzMwS1yZ/R7DbbrvFoEGDtj5hYtauXcsuu+xSdBmtkvumae6bprWnvnnxxRffjoiGv20B2mgQ7LXXXsyePbvoMlqduro6amtriy6jVXLfNM1907T21DeSmvzlvXcNmZklzkFgZpY4B4GZWeIcBGZmiXMQmJklzkFgZpY4B4GZWeIcBGZmiXMQmJklzkFgZpY4B4GZWeIcBGZmiXMQmJklzkFgZpY4B4GZWeIcBGZmiXMQmJklzkFgZpY4B4GZWeIcBGZmiXMQmJklzkFgZpY4B4GZWeIcBGZmiXMQmJklzkFgZpY4B4GZWeIcBGZmiXMQmJklzkFgZpY4B4GZWeIUEUXXsM367TMoasZdX3QZrc4FB9VzzSsdiy6jVXLfNM1907RK9M2iq0e36PzKJenFiBjRWJu3CMzMEucgMDNLnIPAzCxxDgIzs8Q5CMzMEucgMDNLnIPAzCxxDgIzs8Q5CMzMEucgMDNLnIPAzCxxDgIzs8Q5CMzMEucgMDNLnIPAzCxxDgIzs8Q5CMzMEucgMDMr2HXXXceQIUMYOnQop556Ku+99x4AP/3pT9l///0ZMmQIF198ccWWX8j96SRNBP4P8BKwAjgOWAecEREvFVGTmVkRli5dyk9+8hPmzZtHly5dGDduHPfeey/9+/dnxowZzJ07l86dO/Pmm29WrIaitgjOI1v53wPslz8mADcVVI+ZWWHq6+tZv3499fX1rFu3jt69e3PTTTdxySWX0LlzZwB69uxZseVXPQgk3QzsA8wEHgTuiswsYDdJvapdk5lZUfr06cOFF15Iv3796NWrF927d+eYY47h1Vdf5emnn2bkyJEceeSRvPDCCxWroepBEBHnAsuAo4BHgcUlzUuAPo39naQJkmZLmr1m1arKF2pmVgUrV65kxowZLFy4kGXLlrF27VqmTp1KfX09K1euZNasWfzoRz9i3LhxRERFaij6YLEaGdfoK42IWyJiRESM2LVbtwqXZWZWHY899hgDBw6kR48edOrUibFjx/Lss8/St29fxo4diyQOO+wwampqePvttytSQ9FBsATYu2S4L9nWgplZEvr168esWbNYt24dEcHjjz/O4MGDGTNmDE888QQAr776Ku+//z577rlnRWoo5KyhEjOBb0i6FxgJvBsRywuuycysakaOHMnJJ5/MIYccQseOHRk+fDgTJkxAEmeddRZDhw5lp5124s4770RqbCfKjis6CB4hO3vodbLTR88sthwzs+q74ooruOKKKz4yfurUqVVZfiFBEBEDSga/XkQNZmaWKfoYgZmZFcxBYGaWOAeBmVniHARmZolzEJiZJc5BYGaWOAeBmVniHARmZolzEJiZJc5BYGaWOAeBmVniHARmZolzEJiZJc5BYGaWOAeBmVniir4xzXbp0qkDC64eXXQZrU5dXR2LxtcWXUar5L5pmvumaan0jbcIzMwS5yAwM0ucg8DMLHEOAjOzxDkIzMwS5yAwM0ucg8DMLHEOAjOzxDkIzMwS5yAwM0tcm7zExPoNGxlwycNFl9HqXHBQPWe4XxrVsG8W+RIlZpt5i8DMLHEOAjOzxJUVBJL2ldQ5f14raaKk3SpamZmZVUW5WwTTgI2SBgG3AQOBf65YVWZmVjXlBsEHEVEPnAhMiYhvAb0qV5aZmVVLuUGwQdKpwFeBh/JxnSpTkpmZVVO5QXAmcDhwZUQslDQQmFq5sszMrFrK+h1BRMyT9B2gXz68ELi6koWZmVl1lHvW0AnAHOA/8uFhkmZWsC4zM6uScncNXQ4cBrwDEBFzyM4cMjOzNq7cIKiPiHcbjIuWLsbMzKqv3GsN/VbSaUAHSfsBE4FnK1eWmZlVS7lbBN8EhgB/Ifsh2bvApArVZGZmVbTVLQJJHYCZEXE0cGnlSzIzs2ra6hZBRGwE1knqXoV6zMysyso9RvAe8IqkR4G1m0ZGxMSKVGVmZlVTbhA8nD/MzKydKetgcUTc2dij0sWZVdvGjRsZPnw4xx9/PAD33XcfQ4YMoaamhtmzZxdcnVlllPvL4oWS3mj42N6F5vczmC/pnnz4UEkbJZ28vfM0awnXX389gwcP3jw8dOhQHnjgAY444ogCqzKrrHJ3DY0oeb4zcAqwxw4s9zzgi/kF7DoA/wD8fAfmZ7bDlixZwsMPP8yll17KtddeC7BFKJi1V+VedG5Fg1FTJP0K+LttXaCkm4F9gJmSbif7hfI04NBtnZdZS5o0aRI//OEPWb16ddGlmFVVWUEg6ZCSwRqyLYSu27PAiDhX0heAo4DOZD9Q+yxbCQJJE4AJALt/vAfdtmfhZk146KGH6NmzJ5/85Cepq6sruhyzqip319A1Jc/rgYXAuBZY/hTgOxGxUVKzE0bELcAtAP32GeTrHFmLeuaZZ5g5cyaPPPII7733HqtWreL0009n6lTfdsPav3KD4OyI2OLgcH5zmh01Arg3D4E9geMk1UfE9BaYt1nZJk+ezOTJkwGoq6vjxz/+sUPAklHutYbuL3PcNomIgRExICIG5PM7zyFgrcmDDz5I3759ee655xg9ejTHHnts0SWZtbhmtwgkHUB2sbnuksaWNHUjO3vIrN2pra2ltrYWgBNPPJETTzyx2ILMKmxru4b2B44HdgNOKBm/GjhnexeabwE0HHfG9s7PzMy2X7NBEBEzgBmSDo+I56pUk5mZVVG5B4tflvR1st1Em3cJRcRZFanKzMyqptyDxXcDfwUcC/wS6Eu2e8jMzNq4coNgUERcBqzNLzY3GjiocmWZmVm1lBsEG/J/35E0FOgODKhIRWZmVlXlHiO4RdLuwGXATGBXtuM6Q2Zm1vqUe9G5W/OnvyS7YJyZmbUT5d6PYC9Jt0n693z4QElnV7Y0MzOrhnKPEdxBdr+A3vnwq8CkCtRjZmZVVm4Q7BkRPwM+AIiIemBjxaoyM7OqKTcI1kr6ONlNZJD0KeDdilVlZmZVU+5ZQ98mO1toX0nPAD0A31/YzKwd2NrVR/tFxB8i4iVJR5JdhE7AgojY0NzfmplZ27C1XUPTS57/a0T8LiJ+6xAwM2s/thYEpfeP9O8HzMzaoa0FQTTx3MzM2omtHSz+hKRVZFsGXfLn5MMREd0qWl0TunTqwIKrRxex6Fatrq6OReNriy6jVXLfmDVtazem6VCtQszMrBjl/o7AzMzaKQeBmVniHARmZolzEJiZJc5BYGaWOAeBmVniHARmZolzEJiZJc5BYGaWuHLvR9CqrN+wkQGXPFx0Ga3OBQfVc0aZ/bLIl+gws5y3CMzMEucgMDNLnIPAzCxxDgIzs8Q5CMzMEucgMDNLnIPAzCxxDgIzs8Q5CMzMEucgMDNLnIPAzCxxDgIzs8Q5CMzMEucgMDNLnIPAzCxxDgIzs8Q5CAyAxYsXc9RRRzF48GCGDBnC9ddfD8Bll13GwQcfzLBhwzjmmGNYtmxZwZWaWUurWBBImihpvqRpkp6T9BdJF5a07y3pyXya30k6v1K12NZ17NiRa665hvnz5zNr1ixuvPFG5s2bx0UXXcTcuXOZM2cOxx9/PN///veLLtXMWlglb1V5HvBFYC3QHxjToL0euCAiXpLUFXhR0qMRMa+CNVkTevXqRa9evQDo2rUrgwcPZunSpRx44IGbp1m7di2SiirRzCqkIkEg6WZgH2AmcHtEXCdpi5vkRsRyYHn+fLWk+UAfwEFQsEWLFvHyyy8zcuRIAC699FLuuusuunfvzpNPPllwdWbW0iqyaygizgWWAUdFxHVbm17SAGA48Hwz00yQNFvS7DWrVrVYrbalNWvWcNJJJzFlyhS6desGwJVXXsnixYsZP348N9xwQ8EVmllLK/xgsaRdgWnApIhocg0fEbdExIiIGLFrvoKylrVhwwZOOukkxo8fz9ixYz/SftpppzFt2rQCKjOzSio0CCR1IguBeyLigSJrSV1EcPbZZzN48GC+/e1vbx7/2muvbX4+c+ZMDjjggCLKM7MKquTB4mYpO+p4GzA/Iq4tqg7LPPPMM9x9990cdNBBDBs2DICrrrqK2267jQULFlBTU0P//v25+eabiy3UzFpcxYNA0l8Bs4FuwAeSJgEHAgcDfwO8ImlOPvn3IuKRStdkHzVq1Cgi4iPjjzvuuAKqMbNqqlgQRMSAksG+jUzyK8DnIpqZFazwg8VmZlYsB4GZWeIcBGZmiXMQmJklzkFgZpY4B4GZWeIcBGZmiXMQmJklzkFgZpY4B4GZWeIcBGZmiXMQmJklzkFgZpY4B4GZWeIcBGZmiXMQmJklrrBbVe6ILp06sODq0UWX0erU1dWxaHxt0WWYWRvjLQIzs8Q5CMzMEucgMDNLnIPAzCxxDgIzs8Q5CMzMEucgMDNLnIPAzCxxDgIzs8Q5CMzMEucgMDNLnIPAzCxxDgIzs8Q5CMzMEucgMDNLnIPAzCxxDgIzs8Q5CMzMEucgMDNLnIPAzCxxDgIzs8Q5CMzMEucgMDNLnIPAzCxxDgIzs8Q5CMzMEucgMDNLnIPAzCxxDgIzs8Q5CMzMEucgMDNLnIPAzCxxioiia9hmklYDC4quoxXaE3i76CJaKfdN09w3TWtPfdM/Ino01tCx2pW0kAURMaLoIlobSbPdL41z3zTNfdO0VPrGu4bMzBLnIDAzS1xbDYJbii6glXK/NM190zT3TdOS6Js2ebDYzMxaTlvdIjAzsxbiIDAzS1ybCgJJX5C0QNLrki4pup4iSdpb0pOS5kv6naTz8/F7SHpU0mv5v7sXXWsRJHWQ9LKkh/Jh9wsgaTdJ90v6z/z/zuHum4ykb+Wfpd9K+hdJO6fSN20mCCR1AG4EvggcCJwq6cBiqypUPXBBRAwGPgV8Pe+PS4DHI2I/4PF8OEXnA/NLht0vmeuB/4iIA4BPkPVR8n0jqQ8wERgREUOBDsD/IpG+aTNBABwGvB4Rb0TE+8C9wJcKrqkwEbE8Il7Kn68m+0D3IeuTO/PJ7gTGFFJggST1BUYDt5aMdr9I3YAjgNsAIuL9iHgH980mHYEukjoCHwOWkUjftKUg6AMsLhleko9LnqQBwHDgeWCviFgOWVgAPQssrShTgIuBD0rGuV9gH+At4J/y3Wa3StoF9w0RsRT4MfAHYDnwbkT8gkT6pi0FgRoZl/y5r5J2BaYBkyJiVdH1FE3S8cCbEfFi0bW0Qh2BQ4CbImI4sJZ2uqtjW+X7/r8EDAR6A7tIOr3YqqqnLQXBEmDvkuG+ZJtuyZLUiSwE7omIB/LRf5LUK2/vBbxZVH0F+TTwPyUtItt9+FlJU3G/QPYZWhIRz+fD95MFg/sGjgYWRsRbEbEBeAD4axLpm7YUBC8A+0kaKGknsgM5MwuuqTCSRLavd35EXFvSNBP4av78q8CMatdWpIj4bkT0jYgBZP9HnoiI00m8XwAi4o/AYkn756M+B8zDfQPZLqFPSfpY/tn6HNlxtyT6pk39sljScWT7fzsAt0fElcVWVBxJo4CngVf4cF/498iOE/wM6Ef2n/uUiPhzIUUWTFItcGFEHC/p47hfkDSM7CD6TsAbwJlkXwjdN9IVwJfJzsh7GfgasCsJ9E2bCgIzM2t5bWnXkJmZVYCDwMwscQ4CM7PEOQjMzBLnIDAzS1xbvXm9WYuTtJHsdNxNxkTEooLKMasanz5qlpO0JiJ2reLyOkZEfbWWZ9YU7xoyK5OkXpKekjQnv2b9Z/LxX5D0kqTfSHo8H7eHpOmS5kqaJengfPzlkm6R9AvgLkk9JE2T9EL++HSBL9ES5V1DZh/qImlO/nxhRJzYoP004OcRcWV+f4yPSeoB/CNwREQslLRHPu0VwMsRMUbSZ4G7gGF52yeBURGxXtI/A9dFxK8k9QN+Dgyu2Cs0a4SDwOxD6yNiWDPtLwC35xf7mx4Rc/LLWDwVEQsBSi4/MAo4KR/3hKSPS+qet82MiPX586OBA7PL2wDQTVLX/B4TZlXhIDArU0Q8JekIspve3C3pR8A7NH459OYum762ZFwNcHhJMJhVnY8RmJVJUn+yex38I9mVXw8BngOOlDQwn2bTrqGngPH5uFrg7SbuF/EL4BslyxhWofLNmuQtArPy1QIXSdoArAG+EhFvSZoAPCCphux69Z8HLie7E9hcYB0fXsq4oYnAjfl0HckC5NyKvgqzBnz6qJlZ4rxryMwscQ4CM7PEOQjMzBLnIDAzS5yDwMwscQ4CM7PEOQjMzBL338liXpStFZ/1AAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "%matplotlib inline\n", "import matplotlib.pyplot as plt\n", "\n", "ax = xgboost.plot_importance(bst, height=0.8, max_num_features=9)\n", "ax.grid(False, axis=\"y\")\n", "ax.set_title('Estimated feature importance')\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We specified that only 4 features were informative while creating our data, and only 3 features show up as important." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Plot the Receiver Operating Characteristic curve\n", "We can use a fancier metric to determine how well our classifier is doing by plotting the [Receiver Operating Characteristic (ROC) curve](https://en.wikipedia.org/wiki/Receiver_operating_characteristic):" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "execution": { "iopub.execute_input": "2021-10-14T16:51:43.113061Z", "iopub.status.busy": "2021-10-14T16:51:43.112379Z", "iopub.status.idle": "2021-10-14T16:51:43.283417Z", "shell.execute_reply": "2021-10-14T16:51:43.283929Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[16:51:43] WARNING: /home/conda/feedstock_root/build_artifacts/xgboost-split_1631904754241/work/src/learner.cc:1238: Empty dataset at worker: 0\n" ] }, { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Array Chunk
Bytes 60.00 kB 600 B
Shape (15000,) (150,)
Count 100 Tasks 100 Chunks
Type float32 numpy.ndarray
\n", "		\n", "\n", "\n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", "\n", " \n", " 15000\n", " 1\n", "\n", "
" ], "text/plain": [ "dask.array<_predict_part, shape=(15000,), dtype=float32, chunksize=(150,), chunktype=numpy.ndarray>" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "y_hat = dask_xgboost.predict(client, bst, X_test).persist()\n", "y_hat" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "execution": { "iopub.execute_input": "2021-10-14T16:51:43.288169Z", "iopub.status.busy": "2021-10-14T16:51:43.287501Z", "iopub.status.idle": "2021-10-14T16:51:45.488577Z", "shell.execute_reply": "2021-10-14T16:51:45.488127Z" } }, "outputs": [], "source": [ "from sklearn.metrics import roc_curve\n", "\n", "y_test, y_hat = dask.compute(y_test, y_hat)\n", "fpr, tpr, _ = roc_curve(y_test, y_hat)" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "execution": { "iopub.execute_input": "2021-10-14T16:51:45.501021Z", "iopub.status.busy": "2021-10-14T16:51:45.500196Z", "iopub.status.idle": "2021-10-14T16:51:45.679317Z", "shell.execute_reply": "2021-10-14T16:51:45.680280Z" } }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVIAAAFNCAYAAABSVeehAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAABMlklEQVR4nO3dd3hUZfbA8e9JIQkQQu+9S0dCE6WXKE1EECxY1rYu2Fh72V1ldVdZC4I/RGVtFCuCSG/iCgoovXcIoYaehNTz+2OGIUAIA8nkZibn8zzz5N47d+49N+Xkfe99i6gqxhhjrl6Q0wEYY4y/s0RqjDE5ZInUGGNyyBKpMcbkkCVSY4zJIUukxhiTQ5ZIjTEmhyyRGp8SkV0ikiQip0XkgIh8IiJFL9jnOhFZICKnROSEiPwgIg0u2KeYiLwjInvcx9rmXi99ifOKiDwqIutEJEFEYkXkaxFp7MvrNQWTJVKTF3qralGgGdAceO7sGyLSFpgDTAUqAjWA1cAvIlLTvU8hYD7QEIgBigHXAfFAq0uc813gMeBRoCRQF/ge6HmlwYtIyJV+xhQwqmove/nsBewCumZafwP4MdP6z8D7WXxuJvCZe/l+4CBQ1Mtz1gHSgVbZ7LMIuD/T+j3A/zKtK/AXYCuwExgLjLzgGFOBJ93LFYFvgcPu/R91+ntvr7x7WYnU5BkRqQzcCGxzrxfGVbL8OovdvwK6uZe7ArNU9bSXp+oCxKrqspxFzM1Aa6ABMBG4TUQEQERKAN2BySISBPyAqyRdyX3+x0WkRw7Pb/yEJVKTF74XkVPAXuAQ8Df39pK4fgf3Z/GZ/cDZ+5+lLrHPpVzp/pfyuqoeVdUkXCVnBW5wv3crsFRV44CWQBlVfUVVU1R1B/AhMCgXYjB+wBKpyQs3q2ok0BGoz7kEeQzIACpk8ZkKwBH3cvwl9rmUK93/UvaeXVBVBSYDg92bbgcmuJerARVF5PjZF/A8UC4XYjB+wBKpyTOq+hPwCTDSvZ4ALAUGZLH7QFwPmADmAT1EpIiXp5oPVBaR6Gz2SQAKZ1ovn1XIF6xPAm4VkWq4qvzfurfvBXaqavFMr0hVvcnLeI2fs0Rq8to7QDcRaeZefxa4291UKVJESojICKAt8A/3Pp/jSlbfikh9EQkSkVIi8ryIXJSsVHUr8D4wSUQ6ikghEQkXkUEi8qx7t1XALSJSWERqA3+6XOCquhLXw6SPgNmqetz91jLgpIg8IyIRIhIsIo1EpOWVfnOMf7JEavKUqh4GPgNecq//D+gB3ILrvuZuXE2krncnRFQ1GdcDp03AXOAkruRVGvjtEqd6FBgNjAGOA9uBfrgeCgG8DaTgag3wKeeq6ZczyR3LxEzXlA70xtW8ayeuWxIfAVFeHtP4OXHd+jHGGHO1rERqjDE55LNEKiLjReSQiKy7xPsiIqPcXf3WiMi1vorFGGN8yZcl0k9wdee7lBtx9UCpAzwI/J8PYzHGGJ/xWSJV1cXA0Wx26YurC6Cq6q9AcRHJjbZ/xhiTp5y8R1qJTA2egVj3NmOM8StOjmojWWzLsgmBiDyIq/pPkSJFWtSvX9+XcRlj8gFVOJOaTnJaOmkZSmq6kpqe4XqlKakZGblwDiXt2H40PQXS046oapmrOY6TiTQWqJJpvTIQl9WOqjoOGAcQHR2tK1as8H10xhifS0xJI/ZYEnviE9lzNJG9xxLZe9S1vONwAmkZ58pWoe5XTkSGhxAVEUqx8FCKBKWxZOwzxB3cTpGokiScOLr7ao/rZCKdBgwVkcm4utudUNXcGGjCGJPPpKZnsGLXMZZsP8K+Y0nsOZrIrvhEjpxOztFxSxcNo0JUuOdVLiqcEoULERUR6kmYZ5eLhocQHOSqCJ86dYqePXsSt2E55cuXZ/78+TRs2PCq4/BZIhWRSbgGqSgtIrG4RvwJBVDVscAM4CZcQ6olAvf6KhZjTN47cjqZnzYfZsHmQyzecphTZ9Ku+BjVSxWmQcVilCxSiApREVQqHuFOmhGUiwojLCT4io954sQJbrzxRpYuXUqlSpVYsGABdevWveLjZOazRKqqgy/z/tmBc40xfi4hOY2th05z8OQZVu45ztwNB9hxJIHLdZwUgWolC1PF/apasjBVSri+VitdmGLhOa3Mny85OZlu3bqxfPlyqlatyoIFC6hVq1aOjxsQUyikpqYSGxvLmTNnnA7FFDDh4eFUrlyZ0NDc/YPPz9LSM9h7LInZ6w+wYNMhth86zdHElMsmzUrFI+hUvwyNK0VRtlg4VUpEUKN0UU91Oy+EhYXRp08fjhw5wsKFC6lWrVquHNfv+tpn9bBp586dREZGUqpUKdwDmBvjc6pKfHw8p06dokaNGk6Hk6uS09LZezSJ3fEJ7IpPPO/rvmNJ5z0Eyk6zKsXp0bA8neuXpW65ovnm7/PEiRNERZ0/poyI/K6q2Q29eEkBUSI9c+YM1atXzzc/JFMwiAilSpXi8OHDToeSI+kZyh97jvHr9njWxZ1gfdxJ9h1PumwJ80K1yxalasnClCsWRoe6Zbi+ThmKhjmfYvbv3899993H2LFjPSXQC5NoTjl/lbnEkqhxgr/+3qWmZ7B0ezyz1h9gzvqDV/z0vHyxcKqXLkyvJhXpVL8sZSPDCA3Of2MgxcbG0rlzZ7Zu3cqjjz7K1KlTfXKegEmkxpjsnUhK5dcd8cxed4B5Gw9yMpun6CJQMSqC6qULU61UEaqXOvu1CFVLFiai0JU/Lc9ru3btonPnzuzcuZNmzZrx8ccf++xclkhzSXBwMI0bNyYtLY0aNWrw+eefU7x4cQDWr1/PsGHDiI2NRVUZMmQIL774oqc0M3PmTF566SUSEhJQVXr16sXIkSMvOoe3+/mKqtKlSxe+//57ihUrlmfnvRKffvopI0aMAODFF1/k7rvvvmifJ554goULFwKQmJjIoUOHOH78OABPP/00P/74IxkZGXTr1o13330XEWHQoEG8+uqr1KlTJ8+uJSdUld3xiazYfYzfdx/j991H2XLw0pOwli5aiK7XlOPaaiVoVDGKWmWLXFXTovxi+/btdO7cmT179tCyZUtmz55NiRIlfHdCp+eDvtJXixYt9EIbNmy4aFteK1KkiGd5yJAhOmLECFVVTUxM1Jo1a+rs2bNVVTUhIUFjYmJ09OjRqqq6du1arVmzpm7cuFFVVVNTU3XMmDEXHd/b/S4lLS3t6i4sk+nTp+vjjz9+RZ/JjfN6Kz4+XmvUqKHx8fF69OhRrVGjhh49ejTbz4waNUrvvfdeVVX95Zdf9LrrrtO0tDRNS0vTNm3a6MKFC1VVddGiRXr//fdneYz88Pt3JjVNl+2M17GLtukDny7XFq/O0WrPTM/2dd3r8/Uf09brbzviNS09w+lLyDWbNm3SSpUqKaBt27bV48ePe/U5YIXavPb5R9u2bdm3bx8AEydOpF27dnTv3h2AwoULM3r0aP71r38B8MYbb/DCCy9wdvyAkJAQHnnkkYuOmd1+99xzD998841n36JFiwKwaNEiOnXqxO23307jxo155plneP/99z37/f3vf+c///kPAG+++SYtW7akSZMm/O1vfyMrEyZMoG/fvp71m2++mRYtWtCwYUPGjRt33vlffvllWrduzdKlS/niiy9o1aoVzZo146GHHiI9PR2AP//5z0RHR9OwYcNLnvNKzJ49m27dulGyZElKlChBt27dmDVrVrafmTRpEoMHu5o8iwhnzpwhJSWF5ORkUlNTKVfONRHoDTfcwLx580hLu/JG5b607dBpRkzfQJvX5jNg7FJen7mJORsOcuR0ykX7BgcJTSpH8UjHWvww9Hr+90wnXu7dgFY1SuZpEyRfmzt3Lvv27aN9+/bMnj071x8sZSXgqvbVn/3RZ8fe9a+el90nPT2d+fPn86c/ueZSW79+PS1atDhvn1q1anH69GlOnjzJunXrGD58+GWP6+1+F1q2bBnr1q2jRo0arFy5kscff9yTgL/66itmzZrFnDlz2Lp1K8uWLUNV6dOnD4sXL6Z9+/bnHeuXX37hgw8+8KyPHz+ekiVLkpSURMuWLenfvz+lSpUiISGBRo0a8corr7Bx40b+/e9/88svvxAaGsojjzzChAkTGDJkCP/85z8pWbIk6enpdOnShTVr1tCkSZPzzvnmm28yYcLF0ym1b9+eUaNGnbdt3759VKlybviGypUre/6hZWX37t3s3LmTzp07A65/gJ06daJChQqoKkOHDuWaa64BICgoiNq1a7N69eqLfp55JSklnd92xrPt0Gm2HDzFmtgTbDpw6pL7FwsP4dpqJYiuVoIW1UrStEoUhQsF3J/8RYYOHUpUVBS33HILRYp4O/FszgT+dzWPJCUl0axZM3bt2kWLFi3o1q0b4Lp1cqknu3nxxLdVq1aeNo7Nmzfn0KFDxMXFcfjwYUqUKEHVqlUZNWoUc+bMoXnz5gCcPn2arVu3XpRIjx49SmRkpGd91KhRTJkyBYC9e/eydetWSpUqRXBwMP379wdg/vz5/P7777Rs6ZpQMykpibJlywKuRD5u3DjS0tLYv38/GzZsuCiRPvXUUzz11FNeXatm0V4nu+/x5MmTufXWWwkOdt0L3LZtGxs3biQ2NhaAbt26nfcPpWzZssTFxeVpIk1Nz2DJ9nhmrNnPD2viSExJv+S+lYpH0LpmSaKrlSS6eglqlylKUACVNLPzxx9/EBUV5emldNddd+Xp+S2R5pKIiAhWrVrFiRMn6NWrF2PGjOHRRx+lYcOGLF68+Lx9d+zYQdGiRYmMjKRhw4b8/vvvNG3aNNvjZ7dfSEgIGe4hxVSVlJRz1boL/yPfeuutfPPNNxw4cIBBgwZ5PvPcc8/x0EMPZRvD2fMEBQWxaNEi5s2bx9KlSylcuDAdO3b09CwLDw/3JCdV5e677+b1118/71g7d+5k5MiRLF++nBIlSnDPPfdk2TPtSkqklStXZtGiRZ712NhYOnbseMnrmTx5MmPGjPGsT5kyhTZt2nhujdx44438+uuvnkR65swZIiIisvkO5Z41scf5fOluZq8/kO3TdYDuDcpxe+uq3FCnTEBV0b3122+/0aNHD6Kioli6dCkVK1bM+yCu9uaqUy9/eNj0xx9/aJUqVTQlJUUTExO1Ro0aOnfuXFV1PXzq2bOnjho1SlVVV69erbVq1dLNmzerqmp6err+5z//uej42e336quv6tNPP62qqlOmTFHXj1V14cKF2rNnz/OOs27dOm3btq3WqVNH4+LiVFV19uzZ2qpVKz116pSqqsbGxurBgwcviqF169a6detWVVX9/vvvtVevXqqqunHjRg0LC/M8mMn8vVi/fr3Wrl3bc7z4+HjdtWuXrlq1Sps0aaLp6el64MABLVu2rP73v/+9/Dc6G/Hx8Vq9enU9evSoHj16VKtXr67x8fFZ7rtp0yatVq2aZmSce8gyefJk7dKli6ampmpKSop27txZp02b5nm/UaNGnu9ZZrn1+7f90Cn9+Ocd2m/M/y75gKjdv+brc9+t0Q8Xb9d5Gw7osYTkXDm3v/r55581MjJSAe3fv78mJ1/994McPGyyEqkPNG/enKZNmzJ58mTuuusupk6dyrBhw/jLX/5Ceno6d911F0OHDgWgSZMmvPPOOwwePJjExEREhJ49L74Xm91+DzzwAH379qVVq1Z06dIl2/tCDRs25NSpU1SqVIkKFVwzu3Tv3p2NGzfStm1bwPWw6IsvvvBUwc/q2bMnixYtonbt2sTExDB27FiaNGlCvXr1aNOmTZbna9CgASNGjKB79+5kZGQQGhrKmDFjaNOmDc2bN6dhw4bUrFmTdu3aXfk3+gIlS5bkpZde8txGePnllylZsqRnOTo6mj59+gCuh0yDBg06r+p/6623smDBAho3boyIEBMTQ+/evQE4ePAgERERnu9Zblq99zivTt/Ait3Hsny/UvEIYhqVJ6ZRea6tWqJAljqzsmjRInr16kVCQgKDBg3i888/JyTEmZQWEH3tN27c6HkoYHxn//79DBkyhLlz5zodSp57++23KVasmOchYmZX8/t36NQZFm85wpfL97B818UJNDRY6N2kIne2rUbzKsX9tgeVr8ydO5e+ffuSlJTEkCFDGD9+vOd20tUq8H3tTd6oUKECDzzwACdPnsy3DfJ9pXjx4jl6gJGWnsEfe47z05ZDLNp8mPVxJ7Pcr3KJCB5qX5MejcpTNjL8qs8XyHbv3k3v3r1JTk7m/vvv54MPPiAoyNmWnJZIzRUZOHCg0yE44t57r3zc8TOp6Uxfs5/5Gw/yv21Hsh3YuGnlKP7ZrzGNKvm+zaO/q1atGn//+9/Zu3cv7733nuNJFAIokWo2zYyM8ZULb43FHU9iwaZDbNh/ktnrDhCfcHHDeICQIKFFtRJ0rFeWDnXLcE2FSPv9vYwzZ84QHu4qpT/77LP56m8+IBJpeHg48fHxNh6pyVOqrvFIw8PD2XzgFP+3aBvT1+y/5FidFaPC6eBOnO1qlyIyl0d/D2QTJ07khRdeYMGCBZ520fnpbz0gEmnlypWJjY31+3Ehjf8JKVSIn/ZlMGLG4izfr1Q8gjvaVKXbNeWoXTb/DGzsTz799FPuvfdeVJUpU6bw5JNPOh3SRQIikYaGhgbcCOUm/1JVVseeYNqqOCYv33pRb6NW1UvS5Zqy1K9QjOtqlcqX43T6iw8//JCHHnoIVeWVV17Jl0kUAiSRGuNrqsrmg6f4YXUcP6zez56jiRftU6JwKO8Oak77umUciDDwjBkzxtPe+t///jdPP/20wxFdmiVSY7Kx80iCO3nGsfVQ1uN5lo0M4/4banBnm2oFYlCQvPD22297Sp9vv/02jz/+uLMBXYb91I25QNzxJKavcZU81+47keU+keEhxDQsT++mFbmuVilCrPqeq86OHfH+++/z5z//2eFoLs8SqTG4kuec9Qf4ce3+LHsaAUSEBtO1QTl6N6lAh3pl/HoE+fxu+PDhdO3a9bKD+eQXlkhNgbb3aCLvLdjKN7/HklWrpULBQXSoV4beTSvS9ZqyVnX3EVXlzTffpF+/fp7pXPwliYIlUlNAHU1IYfSCbXzx625S0jPOey84SLiuVin6NK1I94bliYqw9p6+pKo8/fTTjBw5krFjx7Jx40bCwsKcDuuKWCI1Bc6KXUd5bPIq9h1POm97y+ol6NusEjGNylO6qH/9IfsrVeXxxx9n1KhRhIaGMnLkSL9LomCJ1BQg8aeTGf/LTsb+tIP0TPX4ZlWK82iX2nSqV9YazOehjIwMHnnkET744AMKFSrEN9984xm20N9YIjUB70xqOv+Zs5nPlu4mOe1cNb5YeAgv9LyGgdFVLIHmsfT0dB544AH++9//Eh4ezpQpU4iJiXE6rKtmidQEtPVxJ3h88qqL2oBGVyvBO4OaUblEYYciK9jmzZvHf//7XyIiIvjhhx/o0qWL0yHliCVSE5D2HU/iw8U7mPDbblLTz1XjG1UqxgM31KRn4wrW9tNBPXr04J133qF58+YXTbLojyyRmoCRnJbOil3HmLx8LzPW7j/vPmhEaDAv9WrA4FZWjXdKSkoKcXFxVK9eHYDHHnvM2YBykSVS4/cOnjzDh4t3MGnZHhKymK64SeUo3rmtGTXLFHUgOgOusURvvfVWVq5cyU8//UTt2rWdDilXWSI1fis1PYP3F25nzMJtF7UFBWhTsyT3XFeDrteUtWq8gxITE+nXrx9z5syhVKlSnD6d9ZgF/swSqfE7qsrs9QcZOWcz2y54iFS1ZGE61ivDwOgqNm1HPpCQkEDv3r1ZuHAhZcuWZd68eTRu3NjpsHKdJVLjV5ZsO8K/Z29m9d7j521vXCmKx7vWoXN9awuaX5w6dYqePXvy888/U758eRYsWBCws/1aIjX5XkaG8uWKvXy+dDcb9p8/+2bRsBAe71qHe9vVsPne85HU1FR69OjB0qVLqVSpEgsWLKBu3bpOh+UzlkhNvnUmNZ1v/4jlv7/suqgKXygkiCFtqvFIp9qULFLIoQjNpYSGhnLbbbcRFxfHggULqFmzptMh+ZRcOAtifhcdHa0rVqxwOgzjY3viE2n/5sKLtoeFBHFzs0o81rUOFYtHOBCZuRKnTp0iMjLS6TC8IiK/q2r01XzWHmWafEVVmbIy9qIkGhYSxAM31GD5i135961NLInmQwcPHqRbt25s2bLFs81fkmhOWdXe5Bup6Rm8PHU9k5bt8WwTgSe71mVI2+pEFbbh7PKruLg4unTpwqZNmxg2bBizZ892OqQ8ZYnU5Asb4k7ywGcrzhvarkihYP4zsCkxjSo4GJm5nL1799K5c2e2bdtGkyZN+OKLL5wOKc9ZIjWOWh93gs+X7mby8r3nbe/ZuAL/6NvQxgXN53bt2kWnTp3YtWsX1157rafRfUFjidQ45svle3juu7UXTfFxQ53SjL69ubUHzee2bdtG586d2bt3L61bt2bWrFkUL17c6bAcYYnU5LnElDQen7yKORsOnre9Xe1SvNyrIfXKF4wHFP5u8eLF7N27l3bt2jFjxgyKFSvmdEiOsURq8tSKXUd5ZMIfHDqV7NlWrVRhXuvXmOtqlbJSqB+57777iIyM5MYbb6Ro0YI9IIxPmz+JSIyIbBaRbSLybBbvR4nIDyKyWkTWi8i9vozHOCc1PYO35mxm4AdLz0uijSoVY/KDbWhXu7QlUT+wevVqNmzY4FkfMGBAgU+i4MMSqYgEA2OAbkAssFxEpqnqhky7/QXYoKq9RaQMsFlEJqhqiq/iMnlv4aZD/G3aevYcTfRsK1womGdi6jOkbTVLoH5ixYoVdO/enbCwMJYuXeoZV9T4tmrfCtimqjsARGQy0BfInEgViBTXX1JR4CiQ5sOYTB46kZTKm7M38cWve87b3qZmSd4a2Mwa1fuRX3/9lR49enDy5En69OlDhQrWJC0zXybSSkDmNi2xQOsL9hkNTAPigEjgNlW9aGBJEXkQeBCgatWqPgnW5J6UtAy++yOWt+ZuOa8aHyTwdEx9Hrihpg0w4kd+/vlnbrrpJk6fPk3//v2ZOHEihQrZ+AaZ+TKRZvWXcmHH/h7AKqAzUAuYKyI/q+p5Q/yo6jhgHLj62ud+qCY3nElN56sVexm7aDtxJ86c916bmiV5+7ZmVIiyUqg/WbBgAb179yYxMZHBgwfz2WefERJiz6gv5MvvSCxQJdN6ZVwlz8zuBf6lrpFTtonITqA+sMyHcZlcdiY1nS9+3c0Hi3dwOFMJFKBMZBiv9GlITKPydi/Uz8TFxdGrVy+SkpK4++67+fjjjwkODnY6rHzJl4l0OVBHRGoA+4BBwO0X7LMH6AL8LCLlgHrADh/GZHLZil1HefqbNew4knDe9tJFC3H/DTW5s001ioZZCcYfVaxYkTfffJPVq1czduxYgoJsjKNL8dlvuKqmichQYDYQDIxX1fUi8rD7/bHAq8AnIrIW162AZ1T1iK9iMrknI0P5z9zNvL9oO5lHYixXLIyH2tdicKuqRBSy0os/SkpKIiLCdQvmL3/5C6pqtYnL8GlRQVVnADMu2DY203Ic0N2XMRjfeH3mRj78eadnvWhYCMO712Vwq6qEh1oC9Vdff/01TzzxBPPmzaN+/foAlkS9YGV1c0USU9K4/9Pl5yXR1jVKMueJ9tzbroYlUT82YcIEBg0axL59+5gyZYrT4fgVu3llrshrMzYyb+Mhz3q3BuUYe2cLa87k5z755BPuu+8+VJW//e1vPPvsRR0RTTYskRqvzVp34LzG9Tc1Ls9/BjSzJOrnxo0bx0MPPQTAP//5T55//nmHI/I/lkiNV1btPc7T36z2rMc0LM+Y26+1+2d+bvTo0QwbNgyAkSNHMnz4cIcj8k+WSM1l7T+RxN3jl3HyjKv3bumiYfz71iaWRANAoUKFEBHeffddT0I1V84SqclWeoYybOJKTiSlAhAVEcon97YkKsLmTwoEDz74IO3ataNhw4ZOh+LX7Km9uaQf1+yn9WvzWLH7GODqK//hkGgaVYpyODJztVSVN954g3Xr1nm2WRLNOUukJktzNxzk0ckrOXL63IiGQzvVplWNkg5GZXJCVXn++ed55pln6NGjBwkJCZf/kPGKVe3NeVSVu/+7nMVbDp+3fUjbajzeta5DUZmcUlX++te/8tZbbxEcHMzbb79NkSJFnA4rYFgiNR6HTyXz0vfrzkuiJQqH8tVDbalTzuZR8lcZGRk89thjjB49mtDQUL788kv69evndFgBxRKpAeDj/+3knXlbOHXm3Lja1UoV5uuH21I2MtzByExOZGRk8PDDD/Phhx9SqFAhvv32W3r16uV0WAHHEqmh1T/nnTcAM8CgllV4pW8jCoXYbXR/9r///Y8PP/yQ8PBwpk6dSvfuNrSFL1giLeDmbjh4URIdd1cLujcs71BEJje1b9+ecePGUatWLTp37ux0OAHLEmkBtvNIAn/9evV52+YP70CtMjYrpD9LTU1l9+7d1K5dG4AHHnjA4YgCn9XbCqgTSan86dPlnob2pYuG8cuznS2J+rnk5GQGDBhA27ZtWb9+vdPhFBiWSAuoZ79dw47DrnaEYSFBfHx3NJVsVk+/dubMGW655RamTp1KWloaSUlJTodUYFjVvgD66OcdzFx3wLP+5oCmNK1S3LmATI4lJibSt29f5s2bR6lSpZg3bx7NmjVzOqwCwxJpATN11T5G/LjRsz6gRWX6NK3oYEQmp06fPk3v3r1ZtGgRZcuWZf78+TRq1MjpsAoUS6QFyKdLdvG3aefum5UqUojnbrrGwYhMTqWnp9OzZ08WL15MhQoVWLBggWeKEJN3LJEWAKrKqPnbeHveFs+2yiUi+O7P11GySCEHIzM5FRwczJAhQ9i5cyfz58+nTp06TodUIIlmngLSD0RHR+uKFSucDsNvqCojftzIx/87N8dSpeIR/PfeltS1bp8BIyEhwfrO55CI/K6q0VfzWXtqH+Bem3F+Er2hTmnmPdnBkqgfO3z4MJ07d2b16nNtgC2JOssSaYBKTc/g9RnnT5ncpX5ZPhwSbfPN+7EDBw7QsWNHFi5cyNChQ/G3GmWgsnukASj2WCKPTlrJH3uOe7bVLluU/7uzhfWd92P79u2jc+fObNmyhQYNGvD111/bdC/5hCXSADN7/QGe+nq1Z34lgE71yvD+HZZE/dmePXvo3Lkz27dvp2nTpsydO5cyZco4HZZxs0QaIJLT0nl9xiY+WbLLsy04SHiqRz0evKEmQTZlst/auXMnnTp1Yvfu3bRo0YI5c+ZQsqTNVJCfWCINALuOJDB00h+s23fSs61S8QhGDW5Oi2olHIzM5Ibly5ezZ88eWrduzaxZsyhevLjTIZkLWCL1c1NX7eOFKes4nXyuKt+9QTnevLUpUYVtps9AMHDgQMLDw+nYsSPFihVzOhyTBUukfupMajp/n7aeycv3erYVCg7i+Zvqc/d11e0hhJ9bt24dqampNG/eHIA+ffo4HJHJjiVSP5SUks4Dn63gf9uOeLZVL1WY0bdfa1MlB4BVq1bRtWtXVJWlS5dSt65NOpjfWSL1M4kpafzpkxUs3RHv2da7aUVe69eIyHCryvu7FStW0L17d44dO8ZNN91E1apVnQ7JeMESqR9JSE7j3k+Ws2znUc+24d3qMrRzbavKB4ClS5cSExPDyZMn6du3L19++SVhYWFOh2W84HXDQhGxPmgOOp2cxt3jl52XRJ+OqcewLnUsiQaAxYsX0717d06ePMmAAQP4+uuvLYn6kcsmUhG5TkQ2ABvd601F5H2fR2Y8Tp5JZcjHv7Fi9zHPtudvqs8jHWs7GJXJLUeOHKFXr16cPn2aO+64g4kTJxIaardp/Ik3Vfu3gR7ANABVXS0i7X0alfE4kZTKkPHLWL33uGfbS70a8KfrazgXlMlVpUuX5r333mPx4sWMGzeO4GAbC8HfeFW1V9W9F2xK90Es5gLHE1O486Pfzkui/+jT0JJogEhISPAs33333Xz88ceWRP2UN4l0r4hcB6iIFBKRv+Ku5hvfOZaQwh0f/cbafSc82169uRF3X1fduaBMrpkyZQq1atVi5cqVTodicoE3ifRh4C9AJSAWaAY84sOYCjxV5dHJK1kfd67L5+u3NOauNtUcjMrkli+//JIBAwZw8OBBpk2b5nQ4Jhd4c4+0nqrekXmDiLQDfvFNSAVbeobyxuxN/Lz1XGP7N/o3YWDLKg5GZXLLF198wd13301GRgYvvPACL7/8stMhmVzgTYn0PS+3mVzw+oyNfPDTDs/6kLbVLIkGiPHjxzNkyBAyMjL4xz/+wYgRI6zpWoC4ZIlURNoC1wFlROTJTG8VA+yOuA+siT3Ox7+cG9G+fd0yvNSrgYMRmdzywQcf8PDDDwPw+uuv8+yzzzockclN2VXtCwFF3ftknuDnJHCrL4MqqEZM38jZmSPa1CzJx3dHExpsgzEHgqJFixIUFMTIkSN54oknnA7H5LJLJlJV/Qn4SUQ+UdXdeRhTgbRk+xGW7XL1WgoNFv7dv4kl0QByxx130KJFC5tzPkB585eaKCJvisgMEVlw9uXzyAqQI6eTeeLLVZ71Pk0rUa2U9cj1dyNHjiTz1OGWRAOXN4l0ArAJqAH8A9gFLPdhTAVKeoby6KSVHDyZDECJwqEM727DpvkzVeXll1/mqaeeIiYmhhMnTlz+Q8aveZNIS6nqx0Cqqv6kqvcBbbw5uIjEiMhmEdkmIlneXReRjiKySkTWi8hPVxB7QHhn3haWbHcNiScCb9/WjIrFIxyOylwtVeXZZ5/l1VdfJSgoiHfffZeoKBsjNtB504401f11v4j0BOKAypf7kIgEA2OAbrga8i8XkWmquiHTPsWB94EYVd0jImWvMH6/tu3QacYs3OZZH9a5Dh3rFahvQUBRVZ588kneeecdQkJCmDhxIgMGDHA6LJMHvEmkI0QkChiOq/1oMeBxLz7XCtimqjsARGQy0BfYkGmf24HvVHUPgKoe8j50/zd6wVYy3E/pW9UoyWNd6jgbkLlqGRkZDBs2jPfff5/Q0FC++uorbr75ZqfDMnnkslV7VZ2uqidUdZ2qdlLVFsDRy30OV5fSzIOdxLq3ZVYXKCEii0TkdxEZ4nXkfm774dNMWx3nWX8mpj7BNmWy31qxYgVjx44lLCyMKVOmWBItYLJrkB8MDMSV/Gap6joR6QU8D0QAzS9z7KyygmZx/hZAF/cxl4rIr6q65YJYHgQeBAJi6oX408n8ZcIfntJo+7plbNpkP9eqVSs+++wzypQpQ/fu3Z0Ox+Sx7Kr2HwNVgGXAKBHZDbQFnlXV7704dqz782dVxnV/9cJ9jqhqApAgIouBpsB5iVRVxwHjAKKjoy9Mxn4lLT2Dh7/4nU0HTnm2WZXeP6WlpbFt2zZPs6Y77rjjMp8wgSq7qn000E1VnwNuAgYAHb1MouBqIlVHRGqISCFgEO7BoTOZCtwgIiEiUhhoTYAP0ffOvK0s33VupPsnuta10qgfSk1NZfDgwbRp0+a8tqKmYMquRJqiqhkAqnpGRLao6gFvD6yqaSIyFJiNq2/+eFVdLyIPu98fq6obRWQWsAbIAD5S1XVXfTX53MLNhxiz6NxT+uHd6jLMSqN+Jzk5mYEDBzJt2jSioqJIT7dxzgs6Uc26piwiicDZv3oBarnXBVBVbZInEV4gOjpa/bEEkJKWQZe3FrH3aBIA19cuzaf3tbIHTH4mKSmJ/v37M3PmTEqWLMmcOXNo0aKF02GZXCAiv6tq9NV8NrsS6TVXGY/JwsTfdnuSaGR4CG8NbGpJ1M8kJibSt29f5s2bR+nSpZk3bx5NmzZ1OiyTD2Q3aIkNVJJLTiSl8u78rZ71x7rUoWyxcAcjMldKVT1JtFy5csyfP5+GDRs6HZbJJ2x4oTzwf4u2cyzR1UGscokI7rQpQ/yOiHD//fdTtWpVfvrpJ0ui5jyWSH1s3/EkPlu6y7P+TEx9wkNtXGx/kfkZwm233camTZuoV6+egxGZ/MirRCoiESJivz1XYdS8rSSmuJ7q1i5blJ6NKzgckfFWfHw8nTt35rfffvNsi4iwAWXMxS6bSEWkN7AKmOVebyYiNvWhF+JPJzNl1T7P+su9GhBkD5j8wqFDh+jUqROLFi1i2LBhXKp1izHgXYn077gGIDkOoKqrgOq+CiiQTFq2h5S0DACaVI7ihjqlHY7IeGP//v107NiRtWvXUr9+fb7//nubpM5ky5tEmqaqNjLtFUpOS2fknHM9Xe9rV8P+GP1AbGwsHTp0YOPGjTRq1IhFixZRsWJFp8My+Zw3iXSdiNwOBItIHRF5D1ji47j83pQ/zlXpI8NCuMnujeZ7u3fvpkOHDmzdupVmzZqxcOFCypUr53RYxg94k0iHAQ2BZGAicALvxiMtsBJT0nh95ibPeq2yRSkUYg0k8rs1a9awe/duoqOjmT9/PqVL260Y4x1vBnaup6ovAC/4OphA8e3vsZxIcrUbLRQSxIT7WzsckfFG7969mT59Om3btrXpQcwV8aaY9JaIbBKRV0XEWiFfRkaG8vH/dnrWn42pT5Ewb/5fGSds2LCBpUuXetZjYmIsiZor5s0I+Z2AjsBhYJyIrBWRF30dmL/6/Nfd7IpPBFx96ge2rHKZTxinrF27lo4dOxITE8PatWudDsf4Ma9u3KnqAVUdBTyMq03py74Myl+lZ+h5k9nde111ilppNF9auXIlnTp14vDhw7Rp04batWs7HZLxY940yL9GRP4uIuuA0bie2F92FtGC6Ns/Yjl0yjU/fakihXikk/1x5kfLli2jc+fOxMfH07NnT6ZOnWo9lkyOeFNc+i8wCeiuqhdOFWLcjiWk8PQ3azzrvZtWtD71+dCSJUuIiYnh1KlT9OvXj8mTJ1OoUCGnwzJ+7rKJVFXb5EUg/u6N2eeaOxULD+HJ7nUdjMZk5cSJE/Tq1YtTp04xcOBAvvjiC0JDQ50OywSA7GYR/UpVB4rIWs6f/dPREfLzo2U7jzJp2bmZp1+9uRHFwu0PNL+Jioriww8/5IcffuCjjz4iJMTuX5vckd1v0mPur73yIhB/dSY1nWe/PVel73pNWfo0tS6F+cmpU6eIjIwEoH///vTv39/hiEygueTDJlXd7158RFV3Z34Bj+RNePnfpGV72HEkAYCiYSG8enMj61Ofj0ybNo0aNWqwZIn1aja+403zp25ZbLsxtwPxRylpGYxbvMOz/mS3ulSIsqe/+cW3335L//79iY+P54cffnA6HBPAsrtH+mdcJc+aIrIm01uRwC++DswffL9yH/tPnAGgdNEwbm9d1eGIzFmTJk3irrvuIj09naeeeorXXnvN6ZBMAMvuHulEYCbwOvBspu2nVPWoT6PyA+kZyv/9tN2zfv8NNay5Uz7x2Wefce+995KRkcELL7zAq6++ardbjE9ll0hVVXeJyF8ufENEShb0ZDpt9T52uu+NFgsP4Q4rjeYL48eP5/7770dVeeWVV3jppZecDskUAJcrkfYCfsfV/Cnzv3QFavowrnzteGIK//xxo2d9SNvqRFpzp3yhVKlSBAcHM2LECJ555hmnwzEFRHbz2vdyf62Rd+H4h9dnbOLI6RQAyhcL56EOBfZ/Sr7Tt29fNm7caH3nTZ7ypq99OxEp4l6+U0TeEpECW49duj2eL1eca3z/St+GVhp12FtvvcXixYs965ZETV7zpvnT/wGJItIUeBrYDXzu06jyKVXllekbPOsxDcvTvWF5ByMyr776KsOHD6d3794cOXLE6XBMAeXt5HcK9AXeVdV3cTWBKnB+23mUjftPAhAeGsTf+9g4105RVV566SVefvllgoKCGDVqlE0NYhzjTWfjUyLyHHAXcIOIBAMFsi776ZJdnuV+zStTPircuWAKMFXl2Wef5Y033iA4OJjPP/+cwYMHOx2WKcC8KZHehmviu/tU9QBQCXjTp1HlQ3HHk5iz4aBn/Z7rqjsXTAGmqjzxxBO88cYbhISEMHnyZEuixnHeTDVyAJgARIlIL+CMqn7m88jymS9+3U16hmsQrLY1S1GvfIG8u+G4devW8f777xMaGso333zDrbfe6nRIxly+ai8iA3GVQBfhakv6nog8parf+Di2fONMajqTl597Un+3lUYd07hxY7766isKFSrETTfd5HQ4xgDe3SN9AWipqocARKQMMA8oMIn0h9VxHE1wtRutVDyCrteUdTiigiU9PZ2NGzfSqFEjAG6++WZnAzLmAt7cIw06m0Td4r38XEBQVT5dusuzflfbaoQEF5jLd1xqaip33nknrVu35ueff3Y6HGOy5E2JdJaIzMY1bxO4Hj7N8F1I+csfe46xbp+ryVNYSBC3Rdv0ynklJSWFwYMH89133xEZGUlQkP0DM/mTN3M2PSUitwDX47pHOk5Vp/g8snxi/C+7PMs3N6tEiSI2UVpeSE5OZsCAAfzwww8UL16c2bNn06pVK6fDMiZL2Y1HWgcYCdQC1gJ/VdV9eRVYfrBu3wl+XLPfs24PmfJGUlIS/fr1Y/bs2ZQsWZK5c+dy7bXXOh2WMZeUXV1pPDAd6I9rBKj38iSifEJVGfHjue6gXa8pR4OKxRyMqGBQVW699VZmz55NmTJlWLRokSVRk+9lV7WPVNUP3cubReSPvAgov5i/8RC/7nANuRocJDx3U32HIyoYRIQ///nPrFu3jpkzZ9KgQQOnQzLmsrJLpOEi0pxz45BGZF5X1YBNrKnpGfxzxrnxRu9oXZVaZYo6GFHgU1XPKPa9evWia9euhIdbF1zjH7Kr2u8H3gL+434dyLQ+0vehOWfV3uOe0e8jw0N4rEsdhyMKbMeOHaNTp04sWrTIs82SqPEn2Q3s3CkvA8lP1u074Vnu1qAcpYqGORhNYDty5AjdunVj1apVPProo6xcuZLgYJv7yvgXb9qRFjhrYs8l0oYVoxyMJLAdPHiQrl27sm7dOurUqcOMGTMsiRq/ZIn0AqnpGfy05bBnvUW1Eg5GE7ji4uLo0qULmzZton79+ixYsIAKFSo4HZYxV8WnXUVEJEZENovINhF5Npv9WopIuog4PpTPz1sPe/rVV4gKp0klK5Hmtr1799KhQwc2bdpEo0aNWLRokSVR49e8mbNJ3HM1vexeryoil+1i4h4AegxwI9AAGCwiF7Vlce/3b2D2lQbvC3M3nBtWoE/TigQF2XzouW3z5s3s2bOHZs2asXDhQsqVK+d0SMbkiDcl0veBtsDZ0XNP4UqQl9MK2KaqO1Q1BZiMa7qSCw0DvgUOZfFenluw6dzgzTfUKeNgJIGra9euzJw5kwULFtj0ICYgeJNIW6vqX4AzAKp6DPCmw3klYG+m9Vj3Ng8RqQT0A8Z6Fa2PbT5wioMnkwHXACWta5Z0OKLAsWXLFhYsWOBZ79y5MyVK2P1nExi8SaSp7uq3gmc80gwvPpdVnVgvWH8HeEZV07M9kMiDIrJCRFYcPnw4u11z5K25mz3LHeuVIdSGy8sVGzZsoH379vTq1Yvly5c7HY4xuc6bTDEKmAKUFZF/Av8DXvPic7FA5jHnKgNxF+wTDUwWkV3ArcD7InLzhQdS1XGqGq2q0WXK+Ka6vXLPMWavP1etf7B9TZ+cp6BZs2YNHTt25ODBg1x33XXW5dMEJG+G0ZsgIr8DXXCVMm9W1Y2X+RjAcqCOiNQA9gGDgNsvOHaNs8si8gkwXVW/9zr6XJR5lKfuDcrRoppV63Pqjz/+oFu3bhw9epSYmBi+++47IiIinA7LmFznzZxNVYFE4IfM21R1T3afU9U0ERmK62l8MDBeVdeLyMPu9/PFfdGzFm4+96zr9tZVHYwkMCxbtowePXpw/Phxevfuzddff01YmPUQM4HJmwb5P+K6tylAOFAD2Aw0vNwHVXUGF4ymf6kEqqr3eBGLT+w9msj2w66+9eGhQbSpWcqpUAJCQkICvXv35vjx4/Tv35+JEydSqJANiG0ClzdV+8aZ10XkWuAhn0XkgB/XnqvWX1u1BOGh1k0xJ4oUKcKnn37K5MmT+eijjwgJsQ50JrBd8W+4qv4hIi19EYwT0jOUL37d7Vnv1sAah1+tEydOEBXl6gkWExNDTEyMwxEZkze86dn0ZKbXX0VkIuC7Nkh57Pfdx4g9lgRA8cKh3NbSJre7GjNmzKB69erMnz/f6VCMyXPeNH+KzPQKw3XPNKseSn5p1roDnuWejStQuJBVQ6/U1KlTufnmmzl+/DgzZhSYCWaN8cg2a7gb4hdV1afyKJ48parMXn8ukcY0Ku9gNP7pm2++YfDgwaSlpfH4448zcmRAj/ltTJYuWSIVkRB3j6OAnXls7b4T7DvuqtYXCw+xp/VXaOLEiQwaNIi0tDSeeeYZ3nrrLc90IcYUJNmVSJfhSqKrRGQa8DWQcPZNVf3Ox7H5XOZqfdcG5axL6BX4/PPPueeee8jIyOCll17iH//4hyVRU2B5c0OwJBAPdOZce1IF/DqRqup5zZ5iGlq1/kqUL1+e0NBQXnzxRV588UWnwzHGUdkl0rIi8iSwjnMJ9KwLBx/xO8t2HmV3fCIAkWEhtK9rQ+ZdiW7durFhwwZq1rQxCYzJri4bDBR1vyIzLZ99+bWvVsR6lvs0q2iN8L3w7rvvMnv2ufG3LYka45JdiXS/qr6SZ5HkoVNnUpmRqVpvbUcv71//+hfPPfccERERbN++3aYGMSaT7EqkAfvkYNKyPSSluoZArV8+ksY2L9MlqSqvvPIKzz33HCLCe++9Z0nUmAtkVyLtkmdR5KH0DOX9Rds96wOjq9jT5ktQVV588UVee+01goKC+OSTT7jrrrucDsuYfOeSiVRVj+ZlIHll7oYDHE9MBSAiNNiGzLsEVeXpp59m5MiRBAcHM2HCBG677TanwzImXypw/SHHLDxXGr21RWV7yHQJW7duZfTo0YSGhjJ58mRuueUWp0MyJt8qUIn0WEIKa/edACA4SHisax2HI8q/6taty9SpU0lOTqZ3795Oh2NMvlagEulvO+M9y00qR1G6qI3Ynll6ejpr1qyhefPmAHTv3t3hiIzxDwWqT+SvO87d9rV+9edLS0tjyJAhtGnT5ry2osaYyytQiXTp9nMl0raWSD1SU1O5/fbbmThxIqGhoTZBnTFXqMBU7eNPJ7P54CkAQoKE6OolHI4of0hOTmbQoEF8//33FCtWjJkzZ3Ldddc5HZYxfqXAJNLfdp6r1jetUtwGcAbOnDlD//79mTFjBsWLF2fOnDm0bBkws8gYk2cKTDaxav3Fbr/9dmbMmEGpUqWYO3eu5yGTMebKFJh7pL/uyJRIa1kiBRg6dCjVqlVj4cKFlkSNyYECUSI9fCqZrYdOAxAaLFxbteDeH1VVT5fYzp07s3nzZsLCrBmYMTlRIEqkS7Yf8Sw3r1KCiEIFszfT8ePH6dSp03kT1FkSNSbnCkSJ9Ns/9nmW29Uu7WAkzjl69Cg9evRgxYoVHDhwgO7duxMSUiB+/Mb4XMD/Je09msjPWw8DIAL9W1RyOKK8d+TIEbp168aqVauoWbMms2fPtiRqTC4K+L+mScv2oO6JUTrULUPlEoWdDSiPHTx4kC5durB+/Xrq1q3L/PnzqVy5stNhGRNQAvoeaWp6xnlTitzeqmANmRcXF0fHjh1Zv349DRo0YNGiRZZEjfGBgE6kczcc5MjpZADKFQujc/2yDkeUt3bu3Mnu3btp3LgxCxcutJHtjfGRgK7aT1q2x7N8W3QVQgrYvPXt2rVj7ty51K9fn1KlrO2sMb4SsJlld3wCP291NXsKEritgFTrt23bdl7zpnbt2lkSNcbHAjaRfr8yzrPcsV5ZKhUP/BGNNm3aRPv27bn55pv5+eefnQ7HmAIjYBPpj2vPJdJ+zQO/ydO6devo0KED+/fvp127dtbl05g8FJCJdMvBU2w56OoSGh4aRJdrAvsh0+rVq+nUqROHDh2iW7du/PjjjxQtWtTpsIwpMAIykU5ffa402qV+uYAeMm/FihV06tSJI0eOcNNNNzFt2jQKFy5YbWWNcVrAJVJVZfqa/Z71Xk0Ct8lPcnIyN998M8eOHaNv37589913hIeHOx2WMQVOwCXSHUcS2HEkAYDChYLpWC9wq/VhYWFMmDCBIUOG8PXXX9sAJMY4JODqvMszjYR/Xa1SATnS07FjxyhRwjUUYIcOHejQoYPDERlTsAVciXTF7mOe5ejqJR2MxDfmzJlD9erV+eGHH5wOxRjjFniJdNe5EmnLAJvg7scff6R3796cPHmSWbNmOR2OMcYtoBLp4VPJ7IpPBKBQSBCNKkU5HFHumTJlCv369SMlJYVHHnmE9957z+mQjDFuAZVIf9+daabQylGEhQTG/dGvvvqKAQMGkJqayhNPPMHo0aMJCgqoH50xfi2g/hqX7wq8+6OTJk1i8ODBpKen88wzz/Cf//zHM+eSMSZ/CKin9oF4f7RKlSpEREQwfPhw/v73v1sSNSYfCphEmpiSxvq4k571QJkp9Prrr2f9+vVUq1bN6VCMMZfg06q9iMSIyGYR2SYiz2bx/h0issb9WiIiTa/2XKv2HictwzWnSN1yRSleuFAOInfWmDFj+O677zzrlkSNyd98ViIVkWBgDNANiAWWi8g0Vd2QabedQAdVPSYiNwLjgNZXc75lmRri+/P90bfeeovhw4dTqFAhtmzZYknUGD/gyxJpK2Cbqu5Q1RRgMtA38w6qukRVzz4h+hW4qgmFVJUfM/Wvb1PTPwcyfv311xk+fDgAo0aNsiRqjJ/wZSKtBOzNtB7r3nYpfwJmZvWGiDwoIitEZMXhw4cven/D/pNsPeQaNi8iNJgufjY3k6ryj3/8g+effx4RYfz48Tz00ENOh2WM8ZIvHzZl9XhZs9xRpBOuRHp9Vu+r6jhc1X6io6MvOsZvO85V67s2KEeRMP95hqaqvPjii7z22msEBQXx6aefcueddzodljHmCvgy48QCVTKtVwbiLtxJRJoAHwE3qmr81ZwoPiHZs1y7jH8NaLxnzx5GjRpFcHAwEydOZODAgU6HZIy5Qr5MpMuBOiJSA9gHDAJuz7yDiFQFvgPuUtUtV3uiY4mpnuWSRUKv9jCOqFatGjNnzuTw4cP069fP6XCMMVfBZ4lUVdNEZCgwGwgGxqvqehF52P3+WOBloBTwvruheZqqRl/pufYeTfQslyqa/8fkzMjI4Pfff6dly5aAq62oMcZ/+bQdqarOUNW6qlpLVf/p3jbWnURR1ftVtYSqNnO/rjiJAmw6cMqzfE2FYrkSu6+kp6fzpz/9ibZt2/L99987HY4xJhf4z1OZSziemMLhU657pGEhQVQtmX/nK0pLS+Puu+9m4sSJFC5cmGLF8nfSN8Z4x+8T6TZ3syeAWmWKEhyUP/uip6amcscdd/D1119TtGhRZsyYwQ033OB0WMaYXOD3iXR3/Ln7ozXKFHEwkktLTk7mtttuY+rUqRQrVoxZs2bRtm1bp8MyxuSSAEikCZ7lKiXyZ7X+3nvvZerUqZQoUYI5c+YQHX1Vt4KNMfmU349Huv3wuURau2z+bEM6bNgwatSowYIFCyyJGhOA/L5Euv1w5nuk+adqr6qesUPbtm3L5s2bCQ31rzauxhjv+HWJ9GhCClsOupo+iUCtfFIiPXnyJJ06deLrr7/2bLMkakzg8usS6aq9x3APQUqzKsUpFu58sjp+/DgxMTH89ttv7N27lz59+hAWlv87CRhjrp5fJ9ItB89V6+vkg9Lo0aNH6d69O7///jvVq1dn/vz5lkSNKQD8OpH+uuPcGCfR1ZwdzPnw4cN07dqVNWvWUKtWLRYuXEiVKlUu/0FjjN/z63uk6/adm6OpZQ3nEumBAwfo2LEja9asoV69eixevNiSqDEFiN8m0jOp6Rw57eoaGhwkVCkR4Vgs+/btIzY2loYNG/LTTz9RsWJFx2IxxuQ9v63a7zue5FkuXyyckGDn/ie0aNGC+fPnU61aNcqUKeNYHMYYZ/htiXRHpob41UrlfY+mHTt2nDfTZ3R0tCVRYwoov02kWw+dGzqvbrnIvD331q106NCBgQMHMmfOnDw9tzEm//HbRHrwxBnPcuU8vD+6ceNG2rdvT2xsLG3btqVNmzZ5dm5jTP7kt4n08Olz8zSVicybtppr166lQ4cOHDhwgE6dOjFz5kwbU9QY47+JNPPweXmRSFeuXEmnTp04fPgw3bp1Y/r06RQt6nwnAGOM8/wykR4+lcz6OFcb0uAgoWHFKJ+eLzU1lf79+xMfH89NN93EtGnTKFw4fw7ZZ4zJe36ZSH/actizHF2tBFERvu1jHxoayuTJk7nrrrv47rvvCA8P9+n5jDH+xS8T6cLNhzzLneqX9dl54uPPdUFt1aoVn332mfWdN8ZcxC8T6ZJtRzzLHev5pu3mggULqFGjBpMnT/bJ8Y0xgcPvEmmGKscSUwHXrKH1fNCGdPbs2fTs2ZNTp04xf/78XD++MSaw+F0iTU1Xz3L5qHDPKPS5Zfr06fTp04czZ87w4IMP8sEHH+Tq8Y0xgccPE2mGZ7l8sdx96DNlyhRuueUWUlJSGDp0KGPHjiUoyO++RcaYPOZ3WSJzIq0QlXuJ9Ntvv2XAgAGkpqYyfPhwRo0aleulXWNMYPK70Z9S09WT/cvlYiKtUaMGkZGRPPLII4wYMcKSqDHGa36YSDM42wCpQi5W7a+99lrWrVtHxYoVLYkaY66IX1fty+ewRDp27Fg+//xzz3qlSpUsiRpjrpgflkgzP7W/+lGfRo0axWOPPUZwcDCtW7embt26uRGeMaYA8usS6dU+bHrzzTd57LHHAHjnnXcsiRpjcsTvEmm6eyL74CChdNEr7645YsQInn76aUSEDz74gKFDh+Z2iMaYAsbvqvZnlY0MIzjI+/uZqsrf/vY3Xn31VUSE8ePHc8899/guQGNMgeG3ifRKHzTt37+f0aNHExwczGeffcbtt9/uo8iMMQWN/ybSK2z6VLFiRebOncuuXbvo37+/j6IyxhREfneP9Cxv7o9mZGSwZMkSz3qLFi0siRpjcp3fJtLLDeackZHBQw89xPXXX8+ECRPyKCpjTEHkt1X77BJpeno69913H5999hnh4eGULeu7wZ+NMSbgEmlaWhpDhgxh0qRJFC5cmOnTp9OpU6c8js4YU5D4bSItlkUiTUlJ4fbbb+fbb78lMjKSGTNmcP311zsQnTGmIPHbRJpVifThhx/m22+/JSoqilmzZtGmTRsHIjPGFDR++7CpWMTF/wMeffRRatWqxfz58y2JGmPyjN+XSDMyMjyj2Ddr1oxNmzYREuK3l2WM8UN+WyKNigjl9OnTdO3alU8++cSz3ZKoMSav+WXWCQ4S0s8k0LNnT5YsWcK2bdsYMGAARYoUcTo0Y0wB5NMSqYjEiMhmEdkmIs9m8b6IyCj3+2tE5FpvjltYk+jevTtLliyhSpUqLFiwwJKoMcYxoqqX3+tqDiwSDGwBugGxwHJgsKpuyLTPTcAw4CagNfCuqrbO7riFytXUsJBgTsdto3r16ixcuJDq1av75BqMMQWHiPyuqtFX81lflkhbAdtUdYeqpgCTgb4X7NMX+ExdfgWKi0iF7A6adnQfp+O2Ubt2bRYvXmxJ1BjjOF8m0krA3kzrse5tV7rPeTQ9jWLlq/HTTz9RpUqVXAnUGGNywpcPm7IadfnC+wje7IOIPAg86F5NPnlg97pKlbLNt/6sNHDE6SB8yK7PfwXytQHUu9oP+jKRxgKZi4yVgbir2AdVHQeMAxCRFVd7H8Mf2PX5t0C+vkC+NnBd39V+1pdV++VAHRGpISKFgEHAtAv2mQYMcT+9bwOcUNX9PozJGGNync9KpKqaJiJDgdlAMDBeVdeLyMPu98cCM3A9sd8GJAL3+ioeY4zxFZ82yFfVGbiSZeZtYzMtK/CXKzzsuFwILT+z6/NvgXx9gXxtkIPr81k7UmOMKSj8tq+9McbkF/k2kfqqe2l+4cX13eG+rjUiskREmjoR59W43LVl2q+liKSLyK15GV9OeXN9ItJRRFaJyHoR+SmvY8wJL343o0TkBxFZ7b4+v3m2ISLjReSQiKy7xPtXl1dUNd+9cD2c2g7UBAoBq4EGF+xzEzATV1vUNsBvTsedy9d3HVDCvXyjv1yfN9eWab8FuO6h3+p03Ln8sysObACqutfLOh13Ll/f88C/3ctlgKNAIadj9/L62gPXAusu8f5V5ZX8WiL1SffSfOSy16eqS1T1mHv1V1xtbP2BNz87cI2x8C1wKC+DywXeXN/twHequgdAVf3pGr25PgUiRUSAorgSaVrehnl1VHUxrngv5arySn5NpD7pXpqPXGnsf8L1X9IfXPbaRKQS0A8Yi//x5mdXFyghIotE5HcRGZJn0eWcN9c3GrgGV+eZtcBjqpqRN+H53FXllfw6HmmudS/Np7yOXUQ64Uqk/jKLnzfX9g7wjKqmuwo1fsWb6wsBWgBdgAhgqYj8qqpbfB1cLvDm+noAq4DOQC1groj8rKonfRxbXriqvJJfE2mudS/Np7yKXUSaAB8BN6pqfB7FllPeXFs0MNmdREsDN4lImqp+nycR5oy3v5tHVDUBSBCRxUBTXMNK5nfeXN+9wL/UdVNxm4jsBOoDy/ImRJ+6urzi9M3fS9zwDQF2ADU4d8O74QX79OT8m8LLnI47l6+vKq4eX9c5HW9uX9sF+3+Cfz1s8uZndw0w371vYWAd0Mjp2HPx+v4P+Lt7uRywDyjtdOxXcI3VufTDpqvKK/myRKoB3r3Uy+t7GSgFvO8uuaWpHwwY4eW1+S1vrk9VN4rILGANkAF8pKpZNrfJb7z8+b0KfCIia3ElnGdU1S9GhRKRSUBHoLSIxAJ/A0IhZ3nFejYZY0wO5den9sYY4zcskRpjTA5ZIjXGmByyRGqMMTlkidQYY3LIEqnxinuUplWZXtWz2fd0LpzvExHZ6T7XHyLS9iqO8ZGINHAvP3/Be0tyGqP7OGe/L+vcIyIVv8z+zUTkptw4t8k/rPmT8YqInFbVorm9bzbH+ASYrqrfiEh3YKSqNsnB8XIc0+WOKyKfAltU9Z/Z7H8PEK2qQ3M7FuMcK5GaqyIiRUVkvru0uFZELhrhSUQqiMjiTCW2G9zbu4vIUvdnvxaRyyW4xUBt92efdB9rnYg87t5WRER+dI+PuU5EbnNvXyQi0SLyLyDCHccE93un3V+/zFxCdJeE+4tIsIi8KSLL3eNSPuTFt2Up7gEuRKSVuMaRXen+Wk9ck0C+AtzmjuU2d+zj3edZmdX30fgBp7tr2cs/XkA6roEqVgFTcHUlLOZ+rzSuniBnazin3V+HAy+4l4OBSPe+i4Ei7u3PAC9ncb5PcHcdBQYAv+EaCGQtUATX8G3rgeZAf+DDTJ+Ncn9dhKv054kp0z5nY+wHfOpeLoRr5J8I4EHgRff2MGAFUCOLOE9nur6vgRj3ejEgxL3cFfjWvXwPMDrT518D7nQvF8fVH7+I0z9ve13ZK192ETX5UpKqNju7IiKhwGsi0h5XN8hKuPpdH8j0meXAePe+36vqKhHpADQAfnF3fS2EqySXlTdF5EXgMK4RsLoAU9Q1GAgi8h1wAzALGCki/8Z1O+DnK7iumcAoEQkDYoDFqprkvp3QRM6N3h8F1AF2XvD5CBFZhav/9u/A3Ez7fyoidXCNHhR6ifN3B/qIyF/d6+G4xlnYeAXXYBxmidRcrTtwjY7eQlVTRWQXriTgoaqL3Ym2J/C5iLwJHAPmqupgL87xlKp+c3ZFRLpmtZOqbhGRFrj6SL8uInNU9RVvLkJVz4jIIlxDw90GTDp7OmCYqs6+zCGSVLWZiEQB03HNijsKV3/0haraz/1gbtElPi9Af1Xd7E28Jn+ye6TmakUBh9xJtBNQ7cIdRKSae58PgY9xTfHwK9BORM7e8ywsInW9POdi4Gb3Z4rgqpb/LCIVgURV/QIY6T7PhVLdJeOsTMY1OMUNuAbrwP31z2c/IyJ13efMkqqeAB4F/ur+TBSuUZHAVZ0/6xSuWxxnzQaGibt4LiLNL3UOk39ZIjVXawIQLSIrcJVON2WxT0dglYisxHUf811VPYwrsUwSkTW4Emt9b06oqn/gune6DNc9049UdSXQGFjmrmK/AIzI4uPjgDVnHzZdYA6uuXzmqWt6DXCNA7sB+ENcE6V9wGVqcO5YVgODgDdwlY5/wXX/9KyFQIOzD5twlVxD3bGtc68bP2PNn4wxJoesRGqMMTlkidQYY3LIEqkxxuSQJVJjjMkhS6TGGJNDlkiNMSaHLJEaY0wOWSI1xpgc+n+v2GnI3LaDCwAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "from sklearn.metrics import auc\n", "\n", "fig, ax = plt.subplots(figsize=(5, 5))\n", "ax.plot(fpr, tpr, lw=3,\n", " label='ROC Curve (area = {:.2f})'.format(auc(fpr, tpr)))\n", "ax.plot([0, 1], [0, 1], 'k--', lw=2)\n", "ax.set(\n", " xlim=(0, 1),\n", " ylim=(0, 1),\n", " title=\"ROC Curve\",\n", " xlabel=\"False Positive Rate\",\n", " ylabel=\"True Positive Rate\",\n", ")\n", "ax.legend();\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This Receiver Operating Characteristic (ROC) curve tells how well our classifier is doing. We can tell it's doing well by how far it bends the upper-left. A perfect classifier would be in the upper-left corner, and a random classifier would follow the diagonal line.\n", "\n", "The area under this curve is `area = 0.76`. This tells us the probability that our classifier will predict correctly for a randomly chosen instance." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Learn more\n", "* Similar example that uses DataFrames for a real world dataset: http://ml.dask.org/examples/xgboost.html\n", "* Recorded screencast stepping through the real world example above:\n", "* A blogpost on dask-xgboost http://matthewrocklin.com/blog/work/2017/03/28/dask-xgboost\n", "* XGBoost documentation: https://xgboost.readthedocs.io/en/latest/python/python_intro.html#\n", "* Dask-XGBoost documentation: http://ml.dask.org/xgboost.html" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.12" } }, "nbformat": 4, "nbformat_minor": 4 }