{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "Scale XGBoost\n", "=============\n", "\n", "Dask and XGBoost can work together to train gradient boosted trees in parallel. This notebook shows how to use Dask and XGBoost together.\n", "\n", "XGBoost provides a powerful prediction framework, and it works well in practice. It wins Kaggle contests and is popular in industry because it has good performance and can be easily interpreted (i.e., it's easy to find the important features from a XGBoost model).\n", "\n", "\"Dask \"Dask" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Setup Dask\n", "We setup a Dask client, which provides performance and progress metrics via the dashboard.\n", "\n", "You can view the dashboard by clicking the link after running the cell." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "execution": { "iopub.execute_input": "2021-07-26T20:14:05.589304Z", "iopub.status.busy": "2021-07-26T20:14:05.588710Z", "iopub.status.idle": "2021-07-26T20:14:08.819098Z", "shell.execute_reply": "2021-07-26T20:14:08.819908Z" } }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
\n", "

Client

\n", "\n", "		\n", "

Cluster

\n", "
    \n", "
  • Workers: 4
    • \n", "
  • Cores: 4
    • \n", "
  • Memory: 7.29 GB
    • \n", "
\n", "
" ], "text/plain": [ "" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from dask.distributed import Client\n", "\n", "client = Client(n_workers=4, threads_per_worker=1)\n", "client" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Create data" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "First we create a bunch of synthetic data, with 100,000 examples and 20 features." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "execution": { "iopub.execute_input": "2021-07-26T20:14:08.822681Z", "iopub.status.busy": "2021-07-26T20:14:08.822216Z", "iopub.status.idle": "2021-07-26T20:14:09.916322Z", "shell.execute_reply": "2021-07-26T20:14:09.917050Z" } }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Array Chunk
Bytes 16.00 MB 160.00 kB
Shape (100000, 20) (1000, 20)
Count 100 Tasks 100 Chunks
Type float64 numpy.ndarray
\n", "		\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", "\n", " \n", " \n", "\n", " \n", " 20\n", " 100000\n", "\n", "
" ], "text/plain": [ "dask.array" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from dask_ml.datasets import make_classification\n", "\n", "X, y = make_classification(n_samples=100000, n_features=20,\n", " chunks=1000, n_informative=4,\n", " random_state=0)\n", "X" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Dask-XGBoost works with both arrays and dataframes. For more information on creating dask arrays and dataframes from real data, see documentation on [Dask arrays](https://dask.pydata.org/en/latest/array-creation.html) or [Dask dataframes](https://dask.pydata.org/en/latest/dataframe-create.html)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Split data for training and testing\n", "We split our dataset into training and testing data to aid evaluation by making sure we have a fair test:" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "execution": { "iopub.execute_input": "2021-07-26T20:14:09.919688Z", "iopub.status.busy": "2021-07-26T20:14:09.919212Z", "iopub.status.idle": "2021-07-26T20:14:10.233459Z", "shell.execute_reply": "2021-07-26T20:14:10.233024Z" } }, "outputs": [], "source": [ "from dask_ml.model_selection import train_test_split\n", "\n", "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.15)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now, let's try to do something with this data using [dask-xgboost][dxgb].\n", "\n", "[dxgb]:https://github.com/dask/dask-xgboost" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Train Dask-XGBoost" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "execution": { "iopub.execute_input": "2021-07-26T20:14:10.237461Z", "iopub.status.busy": "2021-07-26T20:14:10.236954Z", "iopub.status.idle": "2021-07-26T20:14:10.311788Z", "shell.execute_reply": "2021-07-26T20:14:10.312187Z" } }, "outputs": [], "source": [ "import dask\n", "import xgboost\n", "import dask_xgboost" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "dask-xgboost is a small wrapper around xgboost. Dask sets XGBoost up, gives XGBoost data and lets XGBoost do it's training in the background using all the workers Dask has available." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's do some training:" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "execution": { "iopub.execute_input": "2021-07-26T20:14:10.317118Z", "iopub.status.busy": "2021-07-26T20:14:10.316677Z", "iopub.status.idle": "2021-07-26T20:14:20.731487Z", "shell.execute_reply": "2021-07-26T20:14:20.730700Z" } }, "outputs": [], "source": [ "params = {'objective': 'binary:logistic',\n", " 'max_depth': 4, 'eta': 0.01, 'subsample': 0.5, \n", " 'min_child_weight': 0.5}\n", "\n", "bst = dask_xgboost.train(client, params, X_train, y_train, num_boost_round=10)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Visualize results" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The `bst` object is a regular `xgboost.Booster` object. " ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "execution": { "iopub.execute_input": "2021-07-26T20:14:20.735151Z", "iopub.status.busy": "2021-07-26T20:14:20.734772Z", "iopub.status.idle": "2021-07-26T20:14:20.745318Z", "shell.execute_reply": "2021-07-26T20:14:20.744479Z" } }, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "bst" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This means all the methods mentioned in the [XGBoost documentation][2] are available. We show two examples to expand on this, but these examples are of XGBoost instead of Dask.\n", "\n", "[2]:https://xgboost.readthedocs.io/en/latest/python/python_intro.html#" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Plot feature importance" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "execution": { "iopub.execute_input": "2021-07-26T20:14:20.762355Z", "iopub.status.busy": "2021-07-26T20:14:20.761882Z", "iopub.status.idle": "2021-07-26T20:14:21.291980Z", "shell.execute_reply": "2021-07-26T20:14:21.292559Z" } }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAAAauUlEQVR4nO3de5wU9Z3u8c/DRQUR0AjugEG8YBZvQYMRN2YdL3hdo2vUjdENuirryxg1x0tismskJ0aSEwN61pgl0UjEjbteAoiuUdHRjcYYUKIoMZpAAojiFeRiDoPf80f9wHYYhh6ku2b4Pe/Xq1/TVdVT9XT1TD9dVd3VigjMzCxfXcoOYGZm5XIRmJllzkVgZpY5F4GZWeZcBGZmmXMRmJllzkVg7SLp05JeKDtHayQ1SlrQxvRPSXpR0jJJJ9Qx2ocm6WuSflx2Dts8uQgyIWmepJXpSXDN5d+q+L2QtNua4Yj4n4j4WI0y3izpW7WYd/JN4N8ioldETP4wM0rr8/BNE2vDIuLbEXF2vZbXFklXSppUdg7bdLqVHcDq6riIeLDsECXaCXiu7BAAkrpFRHPZOdpLkp8zNkcR4UsGF2AecPh6pu0GPAIsAV4H/jONfxQIYDmwDPgHoBFY0GK+lwLPpNvdCOwA/DfwDvAgsG3F7W8HXknLehTYM40fDawC/l9a1t1p/ADgTuA1YC5wQcW8egA3A28Bz6ccC9ZzH/8AvAesTPPfEuiT8i4CFgLfArqm2+8KPAS8kdbJrUDfNO2WFvO6rOV6abnOgSuBO4BJwFLg7LaW30r+K4FJ6frg9LicCcxP9/9cYP/0OLxNseWz5nfPAB4D/m9a778DDquYPgCYCrwJvASc02K5lbnPT4/RqnTff5tudyYwJz3mfwT+uWIejcAC4GJgcbq/Z7Z4HK8B/pTy/RLokaaNAB5P9+m3QGPZ/0ub46X0AL7U6YFuuwh+BnydYlfhVsBBFdMC2K1i+ANPeGm+T1A8+Q9M/+hPAftSPNk+BHyj4vb/BGyTpo0HZlVMuxn4VsVwF2AmcAWwBbBLepI5Mk0fC/wPsB3wUWA26ymC1tYBMBn4d2BroD/w5JonMIpyHJly9qMorfFtzKux5bJZtwhWASek+9WjreW3kv1K1i2CH6bH6wjg3TS//hWPw8Hp9mcAzcCXge4Uhb4E2C5NfwT4QZrXMIrSPayN3GuzVOQ7lqI8BRwMrAD2q1g3zRS75roDx6Tp26bp1wNNKXdX4G/Seh9IUcTHpGWPTMP9yv5/2twupQfwpU4PdPGktIzildWayzlp2k+BCcCOrfxeNUVwWsXwncANFcNfAiavJ1PfNP8+afhmPlgEBwB/bvE7lwM/Sdf/CBxVMW00VRYBRXH9hfTKM407FXh4Pb97AvB0a/Nqbb20srwrgUcrprV3+WuffHm/CAZWTH8D+IcWj8NF6foZwMuAKqY/CfwjRYGuBrapmHY1cHNruVtmaWNdTwYurFg3K4FuFdMXU7za75KmfbyVeXwFuKXFuF8Ao8r8X9ocL97fl5cTovVjBJcB/xt4UtJbwDURcVM75vtqxfWVrQz3ApDUFbgKOJniVfZ76TbbU7xCbWknYICktyvGdaXYCoBil8b8iml/akfmnShenS6StGZclzXzk9QfuA74NMUWTBeKXTAfRmXWNpdfparWe7Iw0jNp8ieK9TcAeDMi3mkxbfh6crdK0tHAN4DdKe5HT+DZipu8ER88JrIi5dueYkvkD63MdifgZEnHVYzrDjy8oTzWPi4CIyJeAc4BkHQQ8KCkRyPipU28qM8DxwOHU7xa7kPx5LrmmbDlqXDnA3MjYsh65reI4hXtmgPAg9qRZT7FK/Lto/WDtlenPPtExBvp7aaV77JqmXU5xZMfsLb0+rW4TeXvbGj5m9pASaoog0EUxwVeBraTtE1FGQyiOGaxRsv7+oFhSVtSbIF8AZgSEaskTeb9x7Utr1Ps1tqV4hhApfkUWwTnVDEf+xD89lFD0smSdkyDb1H8o69Ow69S7JvfFLahePJ7g+JJ89stprdc1pPAUklfkdRDUldJe0naP03/L+BySdum/F+qNkhELALuB66R1FtSF0m7Sjq4Iusy4G1JAykORLeV9ffAVpKOldQd+BeK/dwbu/xNrT9wgaTukk4GhgL3RsR8ioOxV0vaStI+wFkUB8fX51VgsKQ1zx9bUNzX14DmtHVwRDWhIuI94Cbg+5IGpMf4wFQuk4DjJB2Zxm+VPiuyY9tztfZyEeTl7hafI/h5Gr8/8GtJyyheJV4YEXPTtCuBiZLelnTKh1z+Tyl2OyykeJfPEy2m3wjskZY1OSJWA8dRHMCcS/Hq8ccUWxIAY9L85lI8qd7SzjxfoHgSe56iAO8AGirmvR/FLqt7gLta/O7VwL+krJdExBLgvJRvIcUWwno/3FbF8je1XwNDKNbhVcBJEfFGmnYqxXGHl4GfUxzcf6CNed2efr4h6am0JXEBRTG/RbHlN7Ud2S6h2I30G4p3Ln0H6JJK6njgaxQlM5+ikP28tYnpg7sNzWxzI+kM4OyIOKjsLNYxuVnNzDLnIjAzy5x3DZmZZc5bBGZmmeuUnyPo27dv7Lbbbhu+YZ0tX76crbfeuuwY63Cu9nGu9nGu9ikr18yZM1+PiJafbSmU/dHmjbnsvvvu0RE9/PDDZUdolXO1j3O1j3O1T1m5gBmxnudU7xoyM8uci8DMLHMuAjOzzLkIzMwy5yIwM8uci8DMLHMuAjOzzLkIzMwy5yIwM8uci8DMLHMuAjOzzLkIzMwy5yIwM8uci8DMLHMuAjOzzLkIzMwy5yIwM8uci8DMLHMuAjOzzLkIzMwy5yIwM8uci8DMLHMuAjOzzLkIzMwy5yIwM8uci8DMLHMuAjOzzLkIzMwy5yIwM8uci8DMLHMuAjOzzCkiys7QboN22S26nHJt2THWcfHezVzzbLeyY6zDudrHudrHudrn5qO2prGxse7LlTQzIoa3Ns1bBGZmmXMRmJllzkVgZpY5F4GZWeZcBGZmmXMRmJllzkVgZpY5F4GZWeZcBGZmmXMRmJllzkVgZpY5F4GZWeZcBGZmmXMRmJllzkVgZpY5F4GZWeZcBGZmmXMRmJmVbNy4cey5557stddenHrqqbz77ru8+eabjBw5kiFDhjBy5Ejeeuutmi2/lCKQdIGkOZJulXSdpJckPSNpvzLymJmVZeHChVx33XXMmDGD2bNns3r1am677TbGjh3LYYcdxosvvshhhx3G2LFja5ahrC2C84BjgFuBIekyGrihpDxmZqVpbm5m5cqVNDc3s2LFCgYMGMCUKVMYNWoUAKNGjWLy5Mk1W37di0DSD4FdgKnAz4GfRuEJoK+khnpnMjMry8CBA7nkkksYNGgQDQ0N9OnThyOOOIJXX32Vhobi6bChoYHFixfXLEPdiyAizgVeBg4BHgDmV0xeAAxs7fckjZY0Q9KMZUuX1j6omVkdvPXWW0yZMoW5c+fy8ssvs3z5ciZNmlTXDGUfLFYr46K1G0bEhIgYHhHDe/XuXeNYZmb18eCDD7LzzjvTr18/unfvzoknnsjjjz/ODjvswKJFiwBYtGgR/fv3r1mGsotgAfDRiuEdKbYWzMyyMGjQIJ544glWrFhBRDB9+nSGDh3KZz7zGSZOnAjAxIkTOf7442uWoVvN5lydqcD5km4DDgCWRMSikjOZmdXNAQccwEknncR+++1Ht27d2HfffRk9ejTLli3jlFNO4cYbb2TQoEHcfvvtNctQdhHcS/HuoZeAFcCZ5cYxM6u/MWPGMGbMmA+M23LLLZk+fXpdll9KEUTE4IrBL5aRwczMCmUfIzAzs5K5CMzMMuciMDPLnIvAzCxzLgIzs8y5CMzMMuciMDPLnIvAzCxzLgIzs8y5CMzMMuciMDPLnIvAzCxzLgIzs8y5CMzMMuciMDPLXNlfTLNRenTvygtjjy07xjqampqYd1pj2THW4Vzt41zt41zt09TUVHaEdXiLwMwscy4CM7PMuQjMzDLnIjAzy5yLwMwscy4CM7PMuQjMzDLnIjAzy5yLwMwscy4CM7PMdcpTTKxctZrBX72n7BjruHjvZs5wrqp1xlzzOuCpTcw+LG8RmJllzkVgZpa5qopA0q6StkzXGyVdIKlvTZOZmVldVLtFcCewWtJuwI3AzsB/1CyVmZnVTbVF8F5ENAN/D4yPiC8DDbWLZWZm9VJtEaySdCowCpiWxnWvTSQzM6unaovgTOBA4KqImCtpZ2BS7WKZmVm9VPU5goh4XtJXgEFpeC4wtpbBzMysPqp919BxwCzgvjQ8TNLUGuYyM7M6qXbX0JXAJ4G3ASJiFsU7h8zMrJOrtgiaI2JJi3GxqcOYmVn9VXuuodmSPg90lTQEuAB4vHaxzMysXqrdIvgSsCfwF4oPki0BLqpRJjMzq6MNbhFI6gpMjYjDga/XPpKZmdXTBrcIImI1sEJSnzrkMTOzOqv2GMG7wLOSHgCWrxkZERfUJJWZmdVNtUVwT7qYmdlmptpPFk+sdRCzzm716tUMHz6cgQMHMm3aNC699FLuvvtutthiC3bddVd+8pOf0Ldv37Jjmq2j2k8Wz5X0x5aXjV1o+j6DOZJuTcP7S1ot6aSNnadZ2a699lqGDh26dnjkyJHMnj2bZ555ht13352rr766xHRm61ft20eHA/uny6eB6/hwJ507DzgmIk5L70r6DvCLDzE/s1ItWLCAe+65h7PPPnvtuCOOOIJu3YqN7hEjRrBgwYKy4pm1qaoiiIg3Ki4LI2I8cOjGLFDSD4FdgKmSvkzxGYU7gcUbMz+zjuCiiy7iu9/9Ll26tP4vddNNN3H00UfXOZVZdao6RiBpv4rBLhRbCNtszAIj4lxJRwGHAFtSfEDtUIqtjbYyjAZGA2z7kX703piFm9XAtGnT6N+/P5/4xCdoampaZ/pVV11Ft27dOO200+ofzqwK1b5r6JqK683AXOCUTbD88cBXImK1pDZvGBETgAkAg3bZzec5sg7jscceY+rUqdx77728++67LF26lNNPP51JkyYxceJEpk2bxvTp09nQ37hZWaotgrMi4gMHh9OX03xYw4Hb0j/I9sAxkpojYvImmLdZXVx99dVrDwQ3NTXxve99j0mTJnHffffxne98h0ceeYSePXuWnNJs/ao9WHxHlePaJSJ2jojBETE4ze88l4BtLs4//3zeeecdRo4cybBhwzj33HPLjmTWqja3CCT9NcXJ5vpIOrFiUm9gq1oGM+uMGhsbaWxsBOCll14qN4xZlTa0a+hjwN8BfYHjKsa/A5yzsQtNWwAtx52xsfMzM7ON12YRRMQUYIqkAyPiV3XKZGZmdVTtweKnJX2RYjfR2l1CEfFPNUllZmZ1U+3B4luAvwKOBB4BdqTYPWRmZp1ctUWwW0T8K7A8nYDuWGDv2sUyM7N6qbYIVqWfb0vaC+gDDK5JIjMzq6tqjxFMkLQt8K/AVKAXcEXNUpmZWd1U+30EP05XH6E4YZyZmW0mqv0+gh0k3Sjpv9PwHpLOqm00MzOrh2qPEdxM8X0BA9Lw74GLapDHzMzqrNoi2D4i/gt4DyAimoHVNUtlZmZ1U20RLJf0ESAAJI0AltQslZmZ1U217xr6XxTvFtpV0mNAP8DfL2xmthnY0NlHB0XEnyPiKUkHU5yETsALEbGqrd81M7POYUO7hiZXXP/PiHguIma7BMzMNh8bKoLK79bz5wfMzDZDGyqCWM91MzPbTGzoYPHHJS2l2DLoka6ThiMietc03Xr06N6VF8YeW8ai29TU1MS80xrLjrEO52qfjprLrFY29MU0XesVxMzMylHt5wjMzGwz5SIwM8uci8DMLHMuAjOzzLkIzMwy5yIwM8uci8DMLHMuAjOzzLkIzMwyV+33EXQoK1etZvBX7yk7xjou3ruZM5xrrXkd8DQgZrYubxGYmWXORWBmljkXgZlZ5lwEZmaZcxGYmWXORWBmljkXgZlZ5lwEZmaZcxGYmWXORWBmljkXgZlZ5lwEZmaZcxGYmWXORWBmljkXgZlZ5lwEZmaZcxFYXc2fP59DDjmEoUOHcsYZZ3DttdcCMGvWLEaMGMGwYcMYPnw4Tz75ZMlJzfJRsyKQdIGkOZLulPQrSX+RdEnF9I9Kejjd5jlJF9Yqi3Uc3bp145prrmHOnDn84Ac/4Prrr+f555/nsssu4xvf+AazZs3im9/8JpdddlnZUc2yUcuvqjwPOBpYDuwEnNBiejNwcUQ8JWkbYKakByLi+RpmspI1NDTQ0NAAQM+ePRk6dCgLFy5EEkuXLgVgyZIlDBgwoMyYZlmpSRFI+iGwCzAVuCkixkn6wBfYRsQiYFG6/o6kOcBAwEWQiVdeeYWnn36aAw44gPHjx3PkkUdyySWX8N577/H444+XHc8sGzXZNRQR5wIvA4dExLgN3V7SYGBf4Ndt3Ga0pBmSZixLrxyt81q2bBlXXHEF48ePp3fv3txwww2MGzeO+fPnM27cOM4666yyI5plo/SDxZJ6AXcCF0XEep/hI2JCRAyPiOG9eveuX0Db5FatWsVnP/tZDj/8cE488UQAJk6cuPb6ySef7IPFZnVUahFI6k5RArdGxF1lZrH6iAjOOusshg4dyimnnLJ2/IABA3jkkUcAeOihhxgyZEhZEc2yU8uDxW2SJOBGYE5EfL+sHFZfjz32GLfccgt7770306ZNo1evXnz729/mRz/6ERdeeCHNzc1stdVWTJgwoeyoZtmoeRFI+itgBtAbeE/SRcAewD7APwLPSpqVbv61iLi31pmsPAcddBARAUBTUxONjY1rp82cObOkVGZ5q1kRRMTgisEdW7nJLwHVavlmZlad0g8Wm5lZuVwEZmaZcxGYmWXORWBmljkXgZlZ5lwEZmaZcxGYmWXORWBmljkXgZlZ5lwEZmaZcxGYmWXORWBmljkXgZlZ5lwEZmaZcxGYmWXORWBmlrnSvqryw+jRvSsvjD227BjraGpqYt5pjWXHWEdHzWVmHYO3CMzMMuciMDPLnIvAzCxzLgIzs8y5CMzMMuciMDPLnIvAzCxzLgIzs8y5CMzMMuciMDPLnIvAzCxzLgIzs8y5CMzMMuciMDPLnIvAzCxzLgIzs8y5CMzMMuciMDPLnIvAzCxzLgIzs8y5CMzMMuciMDPLnIvAzCxzLgIzs8y5CMzMMuciMDPLnIvAzCxzLgIzs8y5CMzMMuciMDPLnIvAzCxzLgIzs8wpIsrO0G6S3gFeKDtHK7YHXi87RCucq32cq32cq33KyrVTRPRrbUK3eifZRF6IiOFlh2hJ0gznqp5ztY9ztY9zVc+7hszMMuciMDPLXGctggllB1gP52of52of52of56pSpzxYbGZmm05n3SIwM7NNxEVgZpa5TlUEko6S9IKklyR9tcQcN0laLGl2xbjtJD0g6cX0c9sScn1U0sOS5kh6TtKFHSGbpK0kPSnptynXmI6QqyJfV0lPS5rWUXJJmifpWUmzJM3oQLn6SrpD0u/S39mBZeeS9LG0ntZclkq6qOxcKduX09/8bEk/S/8LpedqqdMUgaSuwPXA0cAewKmS9igpzs3AUS3GfRWYHhFDgOlpuN6agYsjYigwAvhiWkdlZ/sLcGhEfBwYBhwlaUQHyLXGhcCciuGOkuuQiBhW8Z7zjpDrWuC+iPhr4OMU663UXBHxQlpPw4BPACuAn5edS9JA4AJgeETsBXQFPld2rlZFRKe4AAcCv6gYvhy4vMQ8g4HZFcMvAA3pegPFh97KXmdTgJEdKRvQE3gKOKAj5AJ2pPhnPBSY1lEeS2AesH2LcaXmAnoDc0lvMukouVpkOQJ4rCPkAgYC84HtKD68Oy3l6zDra82l02wR8P5KXWNBGtdR7BARiwDSz/5lhpE0GNgX+DUdIFva/TILWAw8EBEdIhcwHrgMeK9iXEfIFcD9kmZKGt1Bcu0CvAb8JO1K+7GkrTtArkqfA36WrpeaKyIWAt8D/gwsApZExP1l52pNZyoCtTLO731thaRewJ3ARRGxtOw8ABGxOopN9x2BT0raq+RISPo7YHFEzCw7Sys+FRH7UewK/aKkvy07EMWr2v2AGyJiX2A5HWG3RiJpC+AzwO1lZwFI+/6PB3YGBgBbSzq93FSt60xFsAD4aMXwjsDLJWVpzauSGgDSz8VlhJDUnaIEbo2IuzpSNoCIeBtoojjGUnauTwGfkTQPuA04VNKkDpCLiHg5/VxMsb/7kx0g1wJgQdqaA7iDohjKzrXG0cBTEfFqGi471+HA3Ih4LSJWAXcBf9MBcq2jMxXBb4AhknZOzf85YGrJmSpNBUal66Mo9s/XlSQBNwJzIuL7HSWbpH6S+qbrPSj+QX5Xdq6IuDwidoyIwRR/Tw9FxOll55K0taRt1lyn2K88u+xcEfEKMF/Sx9Kow4Dny85V4VTe3y0E5ef6MzBCUs/0v3kYxcH1snOtq+yDFO08+HIM8HvgD8DXS8zxM4p9fqsoXiWdBXyE4qDji+nndiXkOohid9kzwKx0OabsbMA+wNMp12zgijS+9HVWkbGR9w8Wl72+dgF+my7PrflbLztXyjAMmJEey8nAth0kV0/gDaBPxbiOkGsMxYue2cAtwJYdIVfLi08xYWaWuc60a8jMzGrARWBmljkXgZlZ5lwEZmaZcxGYmWWus355vdkmJ2k18GzFqBMiYl5Jcczqxm8fNUskLYuIXnVcXreIaK7X8szWx7uGzKokqUHSo+mc97MlfTqNP0rSU+n7FqancdtJmizpGUlPSNonjb9S0gRJ9wM/TZ+6vlPSb9LlUyXeRcuUdw2Zva9HOkMqFOeI+fsW0z9PcSr0q9L3Y/SU1A/4EfC3ETFX0nbptmOApyPiBEmHAj+l+FQuFOfMPygiVkr6D2BcRPxS0iDgF8DQmt1Ds1a4CMzetzKKM6Suz2+Am9KJ/SZHxCxJjcCjETEXICLeTLc9CPhsGveQpI9I6pOmTY2Ilen64cAexaloAOgtaZuIeGdT3SmzDXERmFUpIh5Np4M+FrhF0v8B3qb106G3ddr05RXjugAHVhSDWd35GIFZlSTtRPH9BT+iOMvrfsCvgIMl7Zxus2bX0KPAaWlcI/B6tP7dEPcD51csY1iN4putl7cIzKrXCFwqaRWwDPhCRLyWvkHsLkldKM4tPxK4kuKbvJ6h+A7dUa3PkguA69PtulEUyLk1vRdmLfjto2ZmmfOuITOzzLkIzMwy5yIwM8uci8DMLHMuAjOzzLkIzMwy5yIwM8vc/wfxH3cRw1gX9wAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "%matplotlib inline\n", "import matplotlib.pyplot as plt\n", "\n", "ax = xgboost.plot_importance(bst, height=0.8, max_num_features=9)\n", "ax.grid(False, axis=\"y\")\n", "ax.set_title('Estimated feature importance')\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We specified that only 4 features were informative while creating our data, and only 3 features show up as important." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Plot the Receiver Operating Characteristic curve\n", "We can use a fancier metric to determine how well our classifier is doing by plotting the [Receiver Operating Characteristic (ROC) curve](https://en.wikipedia.org/wiki/Receiver_operating_characteristic):" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "execution": { "iopub.execute_input": "2021-07-26T20:14:21.295636Z", "iopub.status.busy": "2021-07-26T20:14:21.294512Z", "iopub.status.idle": "2021-07-26T20:14:21.602460Z", "shell.execute_reply": "2021-07-26T20:14:21.601661Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[20:14:21] WARNING: ../src/learner.cc:1238: Empty dataset at worker: 0\n" ] }, { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Array Chunk
Bytes 60.00 kB 600 B
Shape (15000,) (150,)
Count 100 Tasks 100 Chunks
Type float32 numpy.ndarray
\n", "		\n", "\n", "\n", " \n", " \n", " \n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", " \n", " \n", "\n", " \n", " 15000\n", " 1\n", "\n", "
" ], "text/plain": [ "dask.array<_predict_part, shape=(15000,), dtype=float32, chunksize=(150,), chunktype=numpy.ndarray>" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "y_hat = dask_xgboost.predict(client, bst, X_test).persist()\n", "y_hat" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "execution": { "iopub.execute_input": "2021-07-26T20:14:21.612983Z", "iopub.status.busy": "2021-07-26T20:14:21.612638Z", "iopub.status.idle": "2021-07-26T20:14:24.625573Z", "shell.execute_reply": "2021-07-26T20:14:24.626399Z" } }, "outputs": [], "source": [ "from sklearn.metrics import roc_curve\n", "\n", "y_test, y_hat = dask.compute(y_test, y_hat)\n", "fpr, tpr, _ = roc_curve(y_test, y_hat)" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "execution": { "iopub.execute_input": "2021-07-26T20:14:24.629645Z", "iopub.status.busy": "2021-07-26T20:14:24.628805Z", "iopub.status.idle": "2021-07-26T20:14:24.922383Z", "shell.execute_reply": "2021-07-26T20:14:24.923022Z" } }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVIAAAFNCAYAAABSVeehAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8rg+JYAAAACXBIWXMAAAsTAAALEwEAmpwYAABMoUlEQVR4nO3dd3hUZfbA8e9JI0BC6NI7SJMikaqUUJUmAgoWLGtbxcray9p2XZW1IPrDxtooKyKCLJ0QsaAUaaGDtNAJNUD6+f0xwxAghCHJ5GYm5/M882TunVvOTTl5732bqCrGGGNyL8jpAIwxxt9ZIjXGmDyyRGqMMXlkidQYY/LIEqkxxuSRJVJjjMkjS6TGGJNHlkiNT4nINhE5JSJJIrJXRD4XkYhztmkvIrEiclxEjorIDyLS+JxtSonIuyKyw32sze7l8hc4r4jIwyISLyInRCRBRCaJyBW+vF5TNFkiNQWhr6pGAC2AlsAzpz8QkXbAHGAqUAWoDawEfhGROu5twoD5QBOgF1AKaA8kAq0vcM73gEeAh4GyQAPge6D3pQYvIiGXuo8pYlTVXvby2QvYBnTLsvwm8L8syz8BH2az30zgS/f7u4F9QISX56wPZACtc9gmDrg7y/IdwM9ZlhV4ENgEbAXGACPPOcZU4HH3+yrAZOCAe/uHnf7e26vgXlYiNQVGRKoB1wKb3cslcJUsJ2Wz+TdAd/f7bsAsVU3y8lRdgQRVXZy3iLkeaAM0BsYDN4mIAIhIGaAHMFFEgoAfcJWkq7rP/6iI9Mzj+Y2fsERqCsL3InIc2AnsB/7uXl8W1+/gnmz22QOcfv5Z7gLbXMilbn8hr6vqIVU9havkrMA17s8GAYtUdTdwFVBBVV9R1VRV/RP4BBiSDzEYP2CJ1BSE61U1EugMNORMgjwMZAKVs9mnMnDQ/T7xAttcyKVufyE7T79RVQUmAkPdq24Gxrnf1wSqiMiR0y/gWeCyfIjB+AFLpKbAqOqPwOfASPfyCWARMDibzW/EVcEEMA/oKSIlvTzVfKCaiETnsM0JoESW5UrZhXzO8gRgkIjUxHXLP9m9fiewVVVLZ3lFqup1XsZr/JwlUlPQ3gW6i0gL9/LTwO3upkqRIlJGRF4D2gEvu7f5CleymiwiDUUkSETKicizInJeslLVTcCHwAQR6SwiYSISLiJDRORp92YrgBtEpISI1AP+crHAVXU5rsqkT4HZqnrE/dFi4JiIPCUixUUkWESaishVl/rNMf7JEqkpUKp6APgSeMG9/DPQE7gB13PN7biaSF3tToioagquCqf1wFzgGK7kVR74/QKnehgYDXwAHAG2AANwVQoBvAOk4moN8AVnbtMvZoI7lvFZrikD6IureddWXI8kPgWivDym8XPievRjjDEmt6xEaowxeeSzRCoiY0Vkv4jEX+BzEZFR7q5+q0TkSl/FYowxvuTLEunnuLrzXci1uHqg1AfuBf7Ph7EYY4zP+CyRqupC4FAOm/TH1QVQVfU3oLSI5EfbP2OMKVBOPiOtSpYGz0CCe50xxvgVJ0e1kWzWZduEQETuxXX7T8mSJVs1bNjQl3EZYwqJjEwlJT2DlLRMktMzSUnLICU9k7SMzOyTxSVSVdIP70EzUiEj/aCqVsjNcZxMpAlA9SzL1YDd2W2oqh8DHwNER0fr0qVLfR+dMabApGdksjLhKD9uPMC6PcfYfzyFXYdPcjgp1bNNqPsVccGjXFjJsGBKFgsholiI52sYacSOGkHCvi2UjCrLiaOHtuc2ficT6TRguIhMxNXd7qiq5sdAE8aYQig9I5O9x5JJOHyKHYdOsnl/EnuPJrPlQBJ/HjjBqbSMSzpepVLhVCkdTuWo4lSOCqdSVDgVS4VTpkQopYuHUbpEKFElQokICyEo6Owb4OPHj9O7d28S1iyhUqVKzJ8/nyZNmuT62nyWSEVkAq5BKsqLSAKuEX9CAVR1DDADuA7XkGongTt9FYsxpmCoKnuOJrNy5xE27U8i4fBJdh46RcKRk+w5kkx65qXdkBcLCaJOhQjqVihJvYoRnlf1MiUoWSx36evo0aNce+21LFq0iKpVqxIbG0uDBg1ydazTfJZIVXXoRT4/PXCuMcZPpWVksmTrIWLX72f1rqOs33uco6fScnWsy0oVo1ODCrSvW55qZYpTuXRxKpUKJzgou+qU3ElJSaF79+4sWbKEGjVqEBsbS926dfN83ICYQiEtLY2EhASSk5OdDsUUMeHh4VSrVo3Q0FCnQykQmZnKZvet+IL1+5mzdi+HT3qfOMtHFKN62eJUK1OC2uVLUq10cWpXKEmd8iUpWzIM97jZPlOsWDH69evHwYMHWbBgATVr1syX4/pdX/vsKpu2bt1KZGQk5cqV8/kPwpjTVJXExESOHz9O7dq1nQ7HpzIyle/+SOCjhX+yeX/OExVEhofQqFIpmlePoka5klQv40qcVUsXp3hYcAFFnLOjR48SFXX2mDIiskxVcxp68YICokSanJxMrVq1LImaAiUilCtXjgMHDjgdSr7JzFR2Hz3F5v1JbN6fxJYDrq/r9x7neHJ6tvtUKhVOjyaXcXW98jSuUoqqpYsXqr/FPXv2cNdddzFmzBhPCfTcJJpXAZFIgUL1gzNFh7/+3qkq2xJPsn7PMVfSdCdMb2rPS4QFc2WNMjSuUopeTSvRolrp82rFC4uEhARiYmLYtGkTDz/8MFOnTvXJeQImkRpjcpaekcnS7YeZu3Yf89btY3viyUvaP6p4KHd1qM2dV9eiVHjhfya8bds2YmJi2Lp1Ky1atOCzzz7z2blsGL18EhwcTIsWLWjatCl9+/blyJEjns/WrFlDTEwMDRo0oH79+rz66qtkfTY9c+ZMoqOjadSoEQ0bNuRvf/tbtufwdjtfUVViYmI4duxYgZ73UnzxxRfUr1+f+vXr88UXX2S7zWOPPUaLFi1o0aIFDRo0oHTp0p7PnnzySZo0aUKjRo14+OGHPT+nIUOGsGnTpoK4hHylqvz2ZyIPT1hO9D/mMeTj3/js5605JtGyJcNoXassQ1vX4IU+jfnirtb88nQMy1/oziPd6vtFEt2yZQudOnVi69atXHXVVcTGxlK+fPmL75hbTs8HfamvVq1a6bnWrl173rqCVrJkSc/7YcOG6WuvvaaqqidPntQ6dero7NmzVVX1xIkT2qtXLx09erSqqq5evVrr1Kmj69atU1XVtLQ0/eCDD847vrfbXUh6enruLiyL6dOn66OPPnpJ++THeb2VmJiotWvX1sTERD106JDWrl1bDx06lOM+o0aN0jvvvFNVVX/55Rdt3769pqena3p6urZt21YXLFigqqpxcXF69913Z3uMwvD7d64Dx5P1i1+36nXvLdSaT03P9tX4hZl622e/68vT1ui437br738mamJSitOh59n69eu1atWqCmi7du30yJEjXu0HLNVc5iW7tfeBdu3asWrVKgDGjx9Phw4d6NGjBwAlSpRg9OjRdO7cmQcffJA333yT5557jtPjB4SEhPDAAw+cd8yctrvjjjvo06cPgwYNAiAiIoKkpCTi4uJ4+eWXqVy5MitWrKBv377UrFnTs99LL71EZGQkI0aM4K233uKbb74hJSWFAQMG8PLLL58Xw7hx47j33ns9y9dffz07d+4kOTmZRx55xPNZREQEjz/+OLNnz+bf//4327ZtY9SoUaSmptKmTRs+/PBDgoOD+etf/8qSJUs4deoUgwYNyvacl2L27Nl0796dsmXLAtC9e3dmzZrF0KEXbtI8YcIEz3lFhOTkZFJTU1FV0tLSuOwy10Sg11xzDXfccQfp6emEhBSuP5uU9Aw27UsiftdRVu866vmaXdv3SqXC6da4It0bV6JtnbIUCykctej5ae7cuezatYuOHTsyffp0IiMjfX7OwvUbkQ9qPf0/nx172796X3SbjIwM5s+fz1/+4ppLbc2aNbRq1eqsberWrUtSUhLHjh0jPj6eESNGXPS43m53rsWLFxMfH0/t2rVZvnw5jz76qCeRfvPNN8yaNYs5c+awadMmFi9ejKrSr18/Fi5cSMeOHc861i+//MJHH33kWR47dixly5bl1KlTXHXVVQwcOJBy5cpx4sQJmjZtyiuvvMK6det44403+OWXXwgNDeWBBx5g3LhxDBs2jH/84x+ULVuWjIwMunbtyqpVq2jWrNlZ53zrrbcYN+786ZQ6duzIqFGjzlq3a9cuqlc/M3xDtWrV2LVr1wW/N9u3b2fr1q3ExMQArn+AXbp0oXLlyqgqw4cPp1GjRgAEBQVRr149Vq5ced7P0wmqStyGA0xftYeZ8Xs4mXrhCqJiIUH0blaZwa2q07ZOWb+tIPPW8OHDiYqK4oYbbqBkSW8nns2bgEukTjl16hQtWrRg27ZttGrViu7duwOuX/gL/eIWxC9069atPW0cW7Zsyf79+9m9ezcHDhygTJky1KhRg1GjRjFnzhxatmwJQFJSEps2bTovkR46dOis/+6jRo1iypQpAOzcuZNNmzZRrlw5goODGThwIADz589n2bJlXHWVa0LNU6dOUbFiRcCVyD/++GPS09PZs2cPa9euPS+RPvHEEzzxxBNeXatm0yY6p+/xxIkTGTRoEMHBrlLZ5s2bWbduHQkJCYCrRJv1H0rFihXZvXu3Y4lUVdmeeJJfthzk++W7WLLtcI7bt65Vlr4tqtCvWRWiShT+55p58ccffxAVFeXppXTbbbcV6PktkeaT4sWLs2LFCo4ePUqfPn344IMPePjhh2nSpAkLFy48a9s///yTiIgIIiMjadKkCcuWLaN58+Y5Hj+n7UJCQsjMzARcf2ypqWdGzDn3P/KgQYP49ttv2bt3L0OGDPHs88wzz3DfffflGMPp8wQFBREXF8e8efNYtGgRJUqUoHPnzp6eZeHh4Z7kpKrcfvvtvP7662cda+vWrYwcOZIlS5ZQpkwZ7rjjjmx7pl1KibRatWrExcV5lhMSEujcufMFr2fixIl88MEHnuUpU6bQtm1bIiJc4wtde+21/Pbbb55EmpycTPHixXP4DuWvtIxM1uw+xvIdh1mx8whLtx1m15FT2W4bUSyETg0q0KRqKZpWieKKqlGUKRlWYLE66ffff6dnz55ERUWxaNEiqlSpUvBB5PbhqlMvf6hs+uOPP7R69eqampqqJ0+e1Nq1a+vcuXNV1VX51Lt3bx01apSqqq5cuVLr1q2rGzZsUFXVjIwM/fe//33e8XPa7tVXX9Unn3xSVVWnTJmirh+r6oIFC7R3795nHSc+Pl7btWun9evX1927d6uq6uzZs7V169Z6/PhxVVVNSEjQffv2nRdDmzZtdNOmTaqq+v3332ufPn1UVXXdunVarFgxT8VM1u/FmjVrtF69ep7jJSYm6rZt23TFihXarFkzzcjI0L1792rFihX1P//5z8W/0TlITEzUWrVq6aFDh/TQoUNaq1YtTUxMzHbb9evXa82aNTUzM9OzbuLEidq1a1dNS0vT1NRUjYmJ0WnTpnk+b9q0qed7llV+/v6lpGXojFW79ZEJf2jTv8+6YEVRzaema+2np+sTk1bozNW7NS09I99i8Cc//fSTRkZGKqADBw7UlJTcV5ZhlU2FS8uWLWnevDkTJ07ktttuY+rUqTz00EM8+OCDZGRkcNtttzF8+HAAmjVrxrvvvsvQoUM5efIkIkLv3uc/i81pu3vuuYf+/fvTunVrunbtmuNzoSZNmnD8+HGqVq1K5cqumV169OjBunXraNeuHeCqLPr66689t+Cn9e7dm7i4OOrVq0evXr0YM2YMzZo14/LLL6dt27bZnq9x48a89tpr9OjRg8zMTEJDQ/nggw9o27YtLVu2pEmTJtSpU4cOHTpc+jf6HGXLluWFF17wPEZ48cUXPRVPL774ItHR0fTr1w9wVTINGTLkrFv/QYMGERsbyxVXXIGI0KtXL/r27QvAvn37KF68uOd7lt+OnExl3O87+OLXbew/nnLB7SKKhdCmdlna1ytPTMOK1C5fMM8AC6O4uDj69OnDiRMnGDJkCF999ZVjFYEB0dd+3bp1nkoB4zt79uxh2LBhzJ071+lQCtw777xDqVKlPJWIWeX29y8tI5Ovf9vO5D8S2LgvidT0zPO2qRIVzlW1y9Kyemla1ChDkyqlCA225t9z586lf//+nDp1imHDhjF27FjP46TcKvJ97U3BqFy5Mvfccw/Hjh2jVKlSTodToEqXLp1vFRiJSSnEbTjA//24JdsBQCpGFmPIVdXp2bQSjSuXCvha9ku1fft2+vbtS0pKCnfffTcfffQRQUHO/nOxRGouyY033uh0CI64887cjzuemanE7z7KgvUHWLBhPysTjpDdjWCjyqX4y9W16de8CmEhVuq8kJo1a/LSSy+xc+dO3n//fceTKARQItUcmhkZ4ysXejSWkp7BgvUHmLN2Lws3HuBglrmHsoooFsLDXesxqFV1yhaRWvbcSk5OJjw8HICnn366UP3NB0QiDQ8PJzEx0cYjNQVK1TUe6ek/blVl+c4jfLNkJ/9bveeCw84FCbSqWYbOl1fkxujqVIgsVpBh+6Xx48fz3HPPERsb62kXXZj+1gMikVarVo2EhISAGhfS+Ifw8HAqV6nK3LX7eGjCHySnnV9hBFA+IoyODSrQ5fKKdKxfIeAbyOenL774gjvvvBNVZcqUKTz++ONOh3SegEikoaGhAT9CuSl8EpNS+O/SnYz778/ZNpSvVqY417eoSo8ml9G0SlShHbOzMPvkk0+47777UFVeeeWVQplEIUASqTEFRVX5Y8cRvv5tO/9btYfUjPNLoHXKl+SNQc2IrlmmUN1++psPPvjA0976jTfe4Mknn3Q4oguzRGqMF06mpjN1xW6+WrSdtXvOH4+1dIlQboquzs1talCzXNFtJJ9f3nnnHU/p85133uHRRx91NqCLsERqTA4270/yNJrPrvKoRfXS3Na2Jr2bVSY8NPCGpHPK6bEjPvzwQ/761786HM3FWSI15hzpGZnMW7ePLxdt59ctied9XiwkiP4tqnBb21pcUS1/J1EzLiNGjKBbt24XHcynsLBEaozb/mPJTFi8kwmLd7D32PkjUdUqV4Jb29ZkcKvqVuuez1SVt956iwEDBlC/fn0Av0miYInUGNIzMvm/uC2Mit1EWsbZDeyDBLo1uozb2tWkQ93yVvPuA6rKk08+yciRIxkzZgzr1q2jWDH/altridQUaTsPneSBcX+wetfRs9aXjwhjyFU1uLlNDaqULrgxSIsaVeXRRx9l1KhRhIaGMnLkSL9LomCJ1BRRSSnpfPbTVsb8uOWsedyvqBrFPR3r0KtJJevv7mOZmZk88MADfPTRR4SFhfHtt996hi30N5ZITZGRmams3nWUT376k9lr9p51Gx8k8HDX+gzvUo8QG6bO5zIyMrjnnnv4z3/+Q3h4OFOmTKFXr15Oh5VrlkhNwEtNz2Rm/B7enbeJrQdPnPd5w0qRvNK/Ka1rl3UguqJp3rx5/Oc//6F48eL88MMPdO3a1emQ8sQSqQloP248wO1jF2f7WaPKpbi9XU0GR1cn2CqRClTPnj159913admy5XmTLPojS6QmICUcPslr09cxa83es9ZHhofQvdFl3N6+Fs2rl3YmuCIqNTWV3bt3U6tWLQAeeeQRZwPKR5ZITUBZsfMIf5+2hpU7j5z3WacGFXj3phZFZnbNwiQ5OZlBgwaxfPlyfvzxR+rVq+d0SPnKEqkJCIdPpPLpz3/ywYIt5302qFU1nurV0Mb9dMjJkycZMGAAc+bMoVy5ciQlnT+9ir+zRGr82vbEE57ZN1POmTyuSZVSvNyvCdG1rBLJKSdOnKBv374sWLCAihUrMm/ePK644gqnw8p3lkiNX0pOy+CduRsZ+8vW83oj1a8YwSv9m9KubjmHojMAx48fp3fv3vz0009UqlSJ2NjYgJ3t1xKp8St7jp5iVvxexv++g03nzMBZq1wJ/tbzcq5rWtm6cjosLS2Nnj17smjRIqpWrUpsbCwNGjRwOiyfsURq/MLSbYf4eOGfzFu3j8xz5pu7skZpbmlTk/4tqlhj+kIiNDSUm266id27dxMbG0udOnWcDsmn5EKzIBZW0dHRunTpUqfDMAXgVGoGS7cf4rOftxK34fz5uMKCg3j2uoYMa1fLSqCF1PHjx4mMjHQ6DK+IyDJVjc7NvlYiNYXSlgNJ3Pbp7+w+ev5wdu3qlOO6KyrRtdFlNqBIIbJv3z5uvfVWPvjgA89tvL8k0byyRGoKnU37jnPLp7+z/3iKZ50IDGhRlQe61KNexQgHozPZ2b17N127dmX9+vU89NBDzJ492+mQCpQlUlNoZGQq43/fzusz13My9cyITINbVeOWtjVpYT2RCqWdO3cSExPD5s2badasGV9//bXTIRU4S6SmUNh95BQPjPuDFVl6JIUGC5/f2ZoO9co7F5jJ0bZt2+jSpQvbtm3jyiuv9DS6L2oskRrHHT6Rym2f/c6WA2dGZqpdviSv33AFbesUvT9Kf7F582ZiYmLYuXMnbdq0YdasWZQuXdrpsBxhidQ4Zt+xZD77eSuTlyWQeCIVgOAg4cEu9Xigc12blbOQW7hwITt37qRDhw7MmDGDUqVKOR2SYyyRmgJ3+EQq/567gW+WJJCacXa3zpGDmzGgZTWHIjOX4q677iIyMpJrr72WiIiiXQHo09bLItJLRDaIyGYReTqbz6NE5AcRWSkia0TkTl/GY5w3ZXkCV78Ry9e/7TgriYaFBPFS38aWRAu5lStXsnbtWs/y4MGDi3wSBR+WSEUkGPgA6A4kAEtEZJqqrs2y2YPAWlXtKyIVgA0iMk5VU30Vl3GGqvLe/E28O2/TWetbVC/N/Z3q0L5eeUqF2xTHhdnSpUvp0aMHxYoVY9GiRZ5xRY1vb+1bA5tV9U8AEZkI9AeyJlIFIkVEgAjgEJDuw5iMA1SVl39Yy+e/bjtr/ds3NmdAy6q4fvymMPvtt9/o2bMnx44do1+/flSuXNnpkAoVXybSqsDOLMsJQJtzthkNTAN2A5HATaqaec42iMi9wL0ANWrU8EmwxjcyM5UXp8Xz9W87POuaV4vi/aFXUqNcCQcjM9766aefuO6660hKSmLgwIGMHz+esDAbHDsrXz4jza6YcW7H/p7ACqAK0AIYLSLnVf2p6seqGq2q0RUqVMjvOI2PHD2Zxl1fLDkrifZuVplv/9rekqifiI2NpVevXiQlJTF06FAmTpxoSTQbvkykCUD1LMvVcJU8s7oT+E5dNgNbgYY+jMkUkMVbDzHgw1/OGmykX/MqvHdTC0JthCa/sHv3bvr06cPJkye5/fbb+eqrrwgJsYY+2fHld2UJUF9EagO7gCHAzedsswPoCvwkIpcBlwN/+jAmUwDW7D7KrZ/+flat/A0tq/LW4OY2W6cfqVKlCm+99RYrV65kzJgxBAXZP8AL8VkiVdV0ERkOzAaCgbGqukZE7nd/PgZ4FfhcRFbjehTwlKoe9FVMxveS0zJ4dkq8J4mGhQTxRI/Lufua2lap5CdOnTpF8eKuUbUefPBBVNV+dhfh03K6qs4AZpyzbkyW97uBHr6MwRScLQeSGD5+Oev2HANc44V+99f2NK0a5XBkxluTJk3iscceY968eTRs6HrKZkn04qysbvLFt8sS6Pv+z54kCvBw13qWRP3IuHHjGDJkCLt27WLKlClOh+NX7MmxyZMTKem88H083y3f5VkXFhLEC30ac2sba6rmLz7//HPuuusuVJW///3vPP30eR0RTQ4skZpcW7v7GMPH/8GfB8+M2lSnQklGD72SxlWK7gAW/ubjjz/mvvvuA+Af//gHzz77rMMR+R9LpCZX5q/bx4Pj/yA57UzN/KBW1XilfxNKhNmvlb8YPXo0Dz30EAAjR45kxIgRDkfkn+w33lyyb5bu5Lkpqz3zyZcIC+YfA5ragCN+KCwsDBHhvffe8yRUc+kskRqvZWYqz32/mgmLz/T8rRBZjIn3tqVuBRsByB/de++9dOjQgSZNmjgdil+zWnvjldUJR+nz/s9nJdGQIGHCPW0sifoRVeXNN98kPj7es86SaN5ZidTkaNn2w7w7byM/bTq7n0Tz6qUZPbQl1ctan3l/oao8++yz/Otf/+K9995j48aNlCxZ0umwAoIlUnNBv/+ZyE0f/3be+uFd6jE8pp5NBeJHVJW//e1vvP322wQHB/POO+9YEs1HlkjNeQ6fSOWV6WuZkqVtKMA19cvzePcGtKxRxqHITG5kZmbyyCOPMHr0aEJDQ/nvf//LgAEDnA4roFgiNWfZvD+JYZ/9zu6jyWetn3hvW5vR0w9lZmZy//3388knnxAWFsbkyZPp06eP02EFHEukxuN4chq3ffY7e85Joq/0b2JJ1E/9/PPPfPLJJ4SHhzN16lR69LChLXzBEqnx+O+SnWcl0Ue61ufBLvUIC7HGHf6qY8eOfPzxx9StW5eYmBinwwlYlkgNAKnpmXyxaJtn+bnrGnFPxzrOBWRyLS0tje3bt1OvXj0A7rnnHocjCnxW1DCkZWTy5Lcr2XnoFAClS4Rya9uaDkdlciMlJYXBgwfTrl071qxZ43Q4RYaVSIu4NbuP0nvUz2eteyimPsXDrGmTv0lOTmbgwIHMmDGD0qVLc+rUKadDKjIskRZh01ftZvj45WetG3hlNe7qUMuZgEyunTx5kv79+zNv3jzKlSvHvHnzaNGihdNhFRmWSIsgVWXsL9t4dfras9b3bHIZbwy8wkZE9zNJSUn07duXuLg4KlasyPz582natKnTYRUplkiLmN1HTvH6zPX8sPLMhK4VIovx+oAr6Nb4MgcjM7mRkZFB7969WbhwIZUrVyY2NtYzRYgpOJZIi5AP4zbz3rxNpKSfGUO0efXSfHRrKypFhTsYmcmt4OBghg0bxtatW5k/fz7169d3OqQiSVTV6RguSXR0tC5dutTpMPzOY/9dcV6Xzxujq/FK/6bWZz4AnDhxwvrO55GILFPV6Nzsa82fioDJyxLOSqL1KkYw/u42vDmouSVRP3TgwAFiYmJYuXKlZ50lUWdZIg1w01ft5olvz/zB1a1Qku8f7ED7euUdjMrk1t69e+ncuTMLFixg+PDh+NsdZaCyZ6QBKj0jk5FzNjLmxy2edQ0rRTLhnrZEFLMfuz/atWsXMTExbNy4kcaNGzNp0iRrYVFI2F9UANp3LJmHJixn8dZDnnV1KpTkq7+0oUzJMAcjM7m1Y8cOYmJi2LJlC82bN2fu3LlUqFDB6bCMmyXSALNw4wEe/2YFB5NSPes6NajAOze1oKwlUb+0detWunTpwvbt22nVqhVz5syhbNmyTodlsrBEGiBS0jN4fcZ6Pv91m2ddkMDj3RvwQOd6BAXZLaC/WrJkCTt27KBNmzbMmjWL0qVLOx2SOYcl0gAQu34fT367moNJKZ515SOKMWpoC9rXtUolf3fjjTcSHh5O586dKVWqlNPhmGxYIvVze48mc9fnZ7er7daoIv8a2IzyEcUcisrkVXx8PGlpabRs2RKAfv36ORyRyYklUj+mqjw7ZfVZ656+tiH3XlPHbuX92IoVK+jWrRuqyqJFi2jQoIHTIZmLsHakfuzbZQnErt/vWX65XxPu71TXkqgfW7p0KTExMSQmJtK2bVtq1KjhdEjGC5ZI/dSeo6d4JcvoTXe0r8Xt7Ws5F5DJs0WLFtG1a1cOHz5M//79+e677wgPtzEQ/IHXiVRErA9aIaGqPD15NceT0wGoWa4ET/a63OGoTF4sXLiQHj16cOzYMQYPHsykSZMoVsyecfuLiyZSEWkvImuBde7l5iLyoc8jMxc0aWkCP248AIAIvDWoOSXC7HG3vzp48CB9+vQhKSmJW265hfHjxxMaGup0WOYSePPX9w7QE5gGoKorRaSjT6MyF7TlQBJPTl7lWb6zfW1a17bG2f6sfPnyvP/++yxcuJCPP/6Y4GAbSMbfeHVrr6o7z1mV4YNYzEWoKs9MPlNLX7t8SZ7oabf0/urEiROe97fffjufffaZJVE/5U0i3Ski7QEVkTAR+Rvu23xTsL5dlsDibWf6z//rhitskjo/NWXKFOrWrcvy5csvvrEp9LxJpPcDDwJVgQSgBfCAD2My2ThwPOWsWvr7OtWhTZ1yDkZkcuu///0vgwcPZt++fUybNs3pcEw+8OYZ6eWqekvWFSLSAfjFNyGZc2VmKg+MW+appa9aujiPdLUpJfzR119/ze23305mZibPPfccL774otMhmXzgTYn0fS/XGR/5ZulOlmw77Fl+pX8Tq6X3Q2PHjmXYsGFkZmby8ssv89prr9l4ogHign+NItIOaA9UEJHHs3xUCrAHcwXkyMlU3pi13rM8tHUNujay2T79zUcffcT9998PwOuvv87TTz/tcEQmP+VUrAkDItzbRGZZfwwY5MugjIuq8tyUeA6fTANct/Qv9mnscFQmNyIiIggKCmLkyJE89thjTodj8tkFE6mq/gj8KCKfq+r2AozJuP13yU7+t3qPZ/mlfk2slt5P3XLLLbRq1crmnA9Q3jwjPSkib4nIDBGJPf3yeWRF3KZ9x3nphzWe5Zvb1KB7Y7ul9ycjR44k69ThlkQDlzeJdBywHqgNvAxsA5b4MKYiLzktg4cmLCc5LROA+hUjeKG33dL7C1XlxRdf5IknnqBXr14cPXrU6ZCMj3mTSMup6mdAmqr+qKp3AW29ObiI9BKRDSKyWUSyfbouIp1FZIWIrBGRHy8h9oD1+ox1rN97HIBiIUG8f3NLu6X3E6rK008/zauvvkpQUBDvvfceUVFRTodlfMybNjRp7q97RKQ3sBuodrGdRCQY+ADojqsh/xIRmaaqa7NsUxr4EOilqjtEpOIlxh9w5q7dxxeLzjySfr5PYxpWsukl/IGq8vjjj/Puu+8SEhLC+PHjGTx4sNNhmQLgTSJ9TUSigBG42o+WAh71Yr/WwGZV/RNARCYC/YG1Wba5GfhOVXcAqOr+845ShOw9mswT3670LPdofBm3trGBff1BZmYmDz30EB9++CGhoaF88803XH/99U6HZQrIRW/tVXW6qh5V1XhV7aKqrYBDF9sPV5fSrIOdJLjXZdUAKCMicSKyTESGeR15AHr++9UccTd1qhwVzpuDmlmDbT+xdOlSxowZQ7FixZgyZYol0SImpwb5wcCNuJLfLFWNF5E+wLNAcaDlRY6dXQbQbM7fCujqPuYiEflNVTeeE8u9wL1AwE698OeBJOatcxXIReCdm1pQuoTNQ+8vWrduzZdffkmFChXo0aOH0+GYApbTrf1nQHVgMTBKRLYD7YCnVfV7L46d4N7/tGq4nq+eu81BVT0BnBCRhUBz4KxEqqofAx8DREdHn5uMA8K433d43ndtWJG2NiBJoZeens7mzZs9zZpuueWWi+xhAlVOt/bRQHdVfQa4DhgMdPYyiYKriVR9EaktImHAENyDQ2cxFbhGREJEpATQhiI4RN+y7Yf57OetnuVb29Z0MBrjjbS0NIYOHUrbtm3PaitqiqacSqSpqpoJoKrJIrJRVfd6e2BVTReR4cBsXH3zx6rqGhG53/35GFVdJyKzgFVAJvCpqsbn+mr80IL1+3lw/B+e5cpR4VxTv4KDEZmLSUlJ4cYbb2TatGlERUWRkWHjnBd1opr9nbKInAQ2n14E6rqXBVBVbVYgEZ4jOjpaA6UEsHLnEW74v1/JyDzzM/jyrtZ0bGCJtLA6deoUAwcOZObMmZQtW5Y5c+bQqlUrp8My+UBElqlqdG72zalE2iiX8Rgv/V/cFk8SrRIVzqihLYmuZfMvFVYnT56kf//+zJs3j/LlyzNv3jyaN2/udFimEMhp0BIbqMSHEg6fZM7aM09K/nNnay6vFJnDHsZJqupJopdddhnz58+nSZMmTodlCgmv57U3+eux/67g9B391fXKWxIt5ESEu+++mxo1avDjjz9aEjVnsUTqAFU9a8T7Ye2slr6wylqHcNNNN7F+/Xouv9xmbjVn8yqRikhxEbHfnnwybeXZzWm7NCzyQwwUSomJicTExPD777971hUvXtzBiExhddFEKiJ9gRXALPdyCxGxqQ9zKSklnTdmnpk6pFujywgNthuDwmb//v106dKFuLg4HnroIS7UusUY8K5E+hKuAUiOAKjqCqCWrwIKdE99u4rdR5MBiCgWwts3Wa1vYbNnzx46d+7M6tWradiwId9//72NeWBy5E0iTVdVG5k2H0xdseusqUNe6d+EUuGhDkZkzpWQkECnTp1Yt24dTZs2JS4ujipVqjgdlinkvEmk8SJyMxAsIvVF5H3gVx/HFXB2HznFc1PO7rR1w5UXHdbVFKDt27fTqVMnNm3aRIsWLViwYAGXXWbTu5iL8yaRPgQ0AVKA8cBRvBuP1LipKs9OWU1SSjoANcuVYPVLNkJQYbNq1Sq2b99OdHQ08+fPp3z58k6HZPyENwM7X66qzwHP+TqYQDVxyU7iNhwAXEPk/XtwcyLtlr7Q6du3L9OnT6ddu3Y2PYi5JN6USN8WkfUi8qqIWCvkS7R02yGe+W61Z/n2drWsG2ghsnbtWhYtWuRZ7tWrlyVRc8m8GSG/C9AZOAB8LCKrReR5XwcWCFLSM7j/6zMjO5UrGcYTPa05bmGxevVqOnfuTK9evVi9evXFdzDmArxqwKiqe1V1FHA/rjalL/oyqEARu24/B5NSPMvDY+pRspg3T1OMry1fvpwuXbpw4MAB2rZtS7169ZwOyfgxbxrkNxKRl0QkHhiNq8beqpsvIjU9k0cmrvAs39uxDnd2qO1cQMZj8eLFxMTEkJiYSO/evZk6dar1WDJ54k3x6D/ABKCHqp47VYi5gM9+3kpqRiYAJcKCua9jHYcjMgC//vorvXr14vjx4wwYMICJEycSFmZzY5m8uWgiVdW2BRFIIElJz+DTn/70LN/atiblIoo5GJEBOHr0KH369OH48ePceOONfP3114SGWusJk3c5zSL6jareKCKrOXv2T0dHyPcHnyz8k8QTqQCULRnG33pYBVNhEBUVxSeffMIPP/zAp59+SkiIPa82+SOn36RH3F/7FEQggWJH4knej93sWX44ph5hITYoiZOOHz9OZKRrvNeBAwcycOBAhyMygeaCf+GqerpT+AOquj3rC3igYMLzL6rK36fFk5LuejbapEopmxHUYdOmTaN27dr8+qv1aja+401RqXs2667N70ACwew1e1mQpQfTPwZcQYgNkeeYyZMnM3DgQBITE/nhhx+cDscEsJyekf4VV8mzjoisyvJRJPCLrwPzN0kp6bz8w1rP8s2ta9CiemnnAiriJkyYwG233UZGRgZPPPEE//znP50OyQSwnJ6RjgdmAq8DT2dZf1xVD/k0Kj/07zkb2OMeZ7RcyTCe7NnQ4YiKri+//JI777yTzMxMnnvuOV599VUbT9T4VE6JVFV1m4g8eO4HIlLWkukZK3Ye4Ytft3mWn+vdiKgS1qzGCWPHjuXuu+9GVXnllVd44YUXnA7JFAEXK5H2AZbhav6U9V+6AtbCHEhOy+D6D8486bi6XnkGtKzqYERFW7ly5QgODua1117jqaeecjocU0TkNK99H/dX69eYg1s+zTIxWmgwr99whd1GOqh///6sW7fO+s6bAuVNX/sOIlLS/f5WEXlbRGr4PrTCb83uo6zYecSzPKJHA6qXLeFcQEXU22+/zcKFCz3LlkRNQfOmbc7/ASdFpDnwJLAd+MqnUfmJv09dQ0amq9NX5ahwbm9fy9mAiqBXX32VESNG0LdvXw4ePOh0OKaI8nbyOwX6A++p6nu4mkAVaVsOJLF0+2EAwoKD+PruNjatcgFSVV544QVefPFFgoKCGDVqlE0NYhzjTWfj4yLyDHAbcI2IBANFukpaVXl7zkbPcscGFahbIcLBiIoWVeXpp5/mzTffJDg4mK+++oqhQ4c6HZYpwrwpQt2Ea+K7u1R1L1AVeMunURVyy3ceOWta5UGtrJa+oKgqjz32GG+++SYhISFMnDjRkqhxnDdTjewFxgFRItIHSFbVL30eWSE2b+0+z/s2tcvSs0klB6MpWuLj4/nwww8JDQ3l22+/ZdCgQU6HZMzFb+1F5EZcJdA4XG1J3xeRJ1T1Wx/HViipKjOylEbvurq2NXcqQFdccQXffPMNYWFhXHfddU6HYwzg3TPS54CrVHU/gIhUAOYBRTKRfvbzVrYlngRcA5NcXc8qOHwtIyODdevW0bRpUwCuv/56ZwMy5hzePCMNOp1E3RK93C/grN97jH/NXO9ZHta2pk1m52NpaWnceuuttGnThp9++snpcIzJljdZYJaIzMY1bxO4Kp9m+C6kwmvk7A2ku9uNNq5ciqeutYFJfCk1NZWhQ4fy3XffERkZSVBQkfz/bfyAN3M2PSEiNwBX43pG+rGqTvF5ZIVMYlIKsevPFMz/NfAKSoRZadRXUlJSGDx4MD/88AOlS5dm9uzZtG7d2umwjMlWTuOR1gdGAnWB1cDfVHVXQQVW2Pxf3BbchVEaVS5Fs2qlHY0nkJ06dYoBAwYwe/ZsypYty9y5c7nyyiudDsuYC8rpXmksMB0YiGsEqPcLJKJCKOHwSb5ctN2z/EjX+g5GE9hUlUGDBjF79mwqVKhAXFycJVFT6OV0bxqpqp+4328QkT8KIqDC6J25mzxz1LesUZqeTS5zOKLAJSL89a9/JT4+npkzZ9K4cWOnQzLmonJKpOEi0pIz45AWz7qsqkUisW7Ye5zvlid4lp/q1dDajfqAqnq+r3369KFbt26Eh4c7HJUx3snp1n4P8Dbwb/drb5blkb4PrXB4a/Z61P1stMvlFWhbp5yzAQWgw4cP06VLF+Li4jzrLIkaf5LTwM5dCjKQwmjJtkPMW+eqqReBJ3tZc6f8dvDgQbp3786KFSt4+OGHWb58OcHBwU6HZcwlsfY7F6CqZzW+v75FVRpVLuVgRIFn3759dOvWjfj4eOrXr8+MGTMsiRq/ZIn0Auat288y93ijocHC490bOBxRYNm9ezddu3Zl/fr1NGzYkNjYWCpXrux0WMbkik+7iohILxHZICKbReTpHLa7SkQyRKRQDOWTkam8OetMafTWtjVtCpF8tHPnTjp16sT69etp2rQpcXFxlkSNX/NmziZxz9X0onu5hohctIuJewDoD4BrgcbAUBE5ry2Le7s3gNmXGryvTP4jgU37kwCIKBbC8C42B1B+2rBhAzt27KBFixYsWLCAyy6z5mTGv3lTIv0QaAecHj33OK4EeTGtgc2q+qeqpgITcU1Xcq6HgMnA/mw+K3DJaRk8NXmVZ/mea+pQLqKYgxEFnm7dujFz5kxiY2NtehATELxJpG1U9UEgGUBVDwNhXuxXFdiZZTnBvc5DRKoCA4AxXkVbAN6P3eRp7lQ+Ioy7r7HZqPPDxo0biY2N9SzHxMRQpkwZByMyJv94k0jT3LffCp7xSDO92C+7Vut6zvK7wFOqmpHjgUTuFZGlIrL0wIEDXpw6d5JS0vn8l22e5b7Nq9gweflg7dq1dOzYkT59+rBkyRKnwzEm33mTSEcBU4CKIvIP4Gfgn17slwBUz7JcDdh9zjbRwEQR2QYMAj4UkevPPZCqfqyq0aoaXaFCBS9OnTs/rNzNidQzOf0pazeaZ6tWraJz587s27eP9u3bW5dPE5C8GUZvnIgsA7riKmVer6rrvDj2EqC+iNQGdgFDgJvPObbnvllEPgemq+r3XkefzyYu3uF5/2KfxoSHWpvGvPjjjz/o3r07hw4dolevXnz33XcUL17c6bCMyXfezNlUAzgJ/JB1naruuPBeoKrpIjIcV218MDBWVdeIyP3uzwvNc1GAdXuOsTLhKOCap35AS5sZNC8WL15Mz549OXLkCH379mXSpEkUK2aVdiYwefMA8H+4nm0KEA7UBjYATS62o6rO4JzR9C+UQFX1Di9i8ZlJS88MTNKjyWWUKelNfZrJzokTJ+jbty9Hjhxh4MCBjB8/nrAw+36awOXNrf0VWZdF5ErgPp9F5ABVZdrKM49vb4yunsPW5mJKlizJF198wcSJE/n0008JCbEKOxPYLvk3XFX/EJGrfBGMU3YeOsXBpBTA1QC/g80MmitHjx4lKioKgF69etGrVy+HIzKmYHjTs+nxLK+/ich4wHdtkBwwI/7MPPVNqpQiOMjGG71UM2bMoFatWsyfP9/pUIwpcN40f4rM8iqG65lpdj2U/NbUFXZbnxdTp07l+uuv58iRI8yYUSQnmDVFXI639u6G+BGq+kQBxVPgNu07zro9xwAoFhJED5tG5JJ8++23DB06lPT0dB599FFGjiwyY34b43HBEqmIhLh7HAX0zGNZK5m6NqpIZHiog9H4l/HjxzNkyBDS09N56qmnePvtt20aFlMk5VQiXYwria4QkWnAJODE6Q9V9Tsfx+Zz59bW92texcFo/MtXX33FHXfcQWZmJi+88AIvv/yyJVFTZHlTa18WSARiONOeVAG/T6SrEo6yPfEkAJHFQuh8eUWHI/IflSpVIjQ0lOeff57nn3/e6XCMcVROibSiiDwOxHMmgZ527uAjfmnikjOds3o0qWRdQi9B9+7dWbt2LXXq1HE6FGMcl1OtfTAQ4X5FZnl/+uX3lmw77Hnft7mN0H4x7733HrNnnxl/25KoMS45lUj3qOorBRZJAcvMVHYcOulZvrKmjY2Zk3/9618888wzFC9enC1bttjUIMZkkVOJNKBrDvYdTyY13TWsatmSYZSy2vpsqSqvvPIKzzzzDCLC+++/b0nUmHPkVCLtWmBROGDOmn2e9zXL2cR22VFVnn/+ef75z38SFBTE559/zm233eZ0WMYUOhdMpKp6qCADKUhHT6Xx7ryNnuXrmloJ61yqypNPPsnIkSMJDg5m3Lhx3HTTTU6HZUyhVCSH5fnvkh0cPpkGQPWyxbmtXU2HIyp8Nm3axOjRowkNDWXixInccMMNTodkTKFV5BKpqjJh8Zk5+YZ3qWfNnrLRoEEDpk6dSkpKCn379nU6HGMKtSKXSBdtSWTrQVcHrchiIfS13kweGRkZrFq1ipYtWwLQo0cPhyMyxj94M/pTQBmXZV6mAVdWpURYkftfkq309HSGDRtG27Ztz2oraoy5uCKVSA8mpTBnzV7P8s1tajgYTeGRlpbGzTffzPjx4wkNDbUJ6oy5REWqODZpaQJpGa7erVfWKE3DSqUcjsh5KSkpDBkyhO+//55SpUoxc+ZM2rdv73RYxviVIpNIVfWsvvU3t7Ga+uTkZAYOHMiMGTMoXbo0c+bM4aqrAmoWGWMKRJFJpBv3JZ010lOfZtZ29Oabb2bGjBmUK1eOuXPneiqZjDGXpsg8I43bsN/zvmODCtbkCRg+fDg1a9ZkwYIFlkSNyYMiUyJdkCWRdr68goOROEtVPQMwx8TEsGHDBooVK+ZwVMb4tyJRIj2enMbSLEPmdSqiifTIkSN06dLlrAnqLIkak3dFokS6cONB0jNdtfVNqpSiYmS4wxEVvEOHDtGzZ0+WLl3K3r176dGjByEhReLHb4zPFYm/pKzz1ndrVPRmCT148CDdu3dnxYoV1KlTh9mzZ1sSNSYfBfxf06nUDGLXnXk+2ruI1dbv27ePrl27smbNGho0aMD8+fOpVq2a02EZE1AC/hlp3Ib9nErLAKBuhZLUrxgQs6R4Zffu3XTu3Jk1a9bQuHFj4uLiLIka4wMBn0hnxJ/pEtr7ispFasrgrVu3sn37dq644goWLFhgI9sb4yMBfWufnJbB/HVnRsK/rojd1nfo0IG5c+fSsGFDypUr53Q4xgSsgC6Rxm04wMlU1219nfIlufyySIcj8r3Nmzef1bypQ4cOlkSN8bGATqQzs9TWX1cEbuvXr19Px44duf766/npp5+cDseYIiNgE2lqeiZTV+z2LF93RWDf1sfHx9OpUyf27NlDhw4drMunMQUoYBPp9yt2ed7XLFeCRpUD97Z+5cqVdOnShf3799O9e3f+97//ERFRdFonGOO0gE2k29zTiQBUKhUesLf1S5cupUuXLhw8eJDrrruOadOmUaKETS9tTEEK2ES6fMcRz/vb29dyLA5fSklJ4frrr+fw4cP079+f7777jvDwotf91RinBWQizchUViYc8Sy3rFHasVh8qVixYowbN45hw4YxadIkG4DEGIcEZDvSjfuOe5o9VSoVTuWowJqD6PDhw5QpUwaATp060alTJ4cjMqZoC8gSadbb+kArjc6ZM4datWrxww8/OB2KMcYtQBPpmbFHAymR/u9//6Nv374cO3aMWbNmOR2OMcYtMBPpziOe9y1rlHEukHw0ZcoUBgwYQGpqKg888ADvv/++0yEZY9wCLpEePZXG5v1JAIQECU2rRDkcUd598803DB48mLS0NB577DFGjx5NUFDA/eiM8VsB99e4MktptFHlUhQP8+9J7iZMmMDQoUPJyMjgqaee4t///nfAtok1xl8FXK19oFU0Va9eneLFizNixAheeuklS6LGFEIBl0hnrTkz/mggJNKrr76aNWvWULNmTadDMcZcgE9v7UWkl4hsEJHNIvJ0Np/fIiKr3K9fRaR5Xs63ad9x1u05Briej15T3z9nC/3ggw/47rvvPMuWRI0p3HxWIhWRYOADoDuQACwRkWmqujbLZluBTqp6WESuBT4G2uT2nHEbDnjet6tbjvIR/tfT5+2332bEiBGEhYWxceNGS6LG+AFflkhbA5tV9U9VTQUmAv2zbqCqv6rq6UafvwF5mlDo1y0HPe/9cdi8119/nREjRgAwatQoS6LG+AlfJtKqwM4sywnudRfyF2Bmdh+IyL0islRElh44cCC7TUjLyGTx1kOe5fZ1/WdUeFXl5Zdf5tlnn0VEGDt2LPfdd5/TYRljvOTLyqbsqpc12w1FuuBKpFdn97mqfozrtp/o6Ohsj7Eq4Qgn3P3rq0SFU6Osfwwlp6o8//zz/POf/yQoKIgvvviCW2+91emwjDGXwJeJNAGonmW5GrD73I1EpBnwKXCtqibm9mS//XmmNNqhXnm/aSa0Y8cORo0aRXBwMOPHj+fGG290OiRjzCXyZSJdAtQXkdrALmAIcHPWDUSkBvAdcJuqbszLyZZtP9O//qraZfNyqAJVs2ZNZs6cyYEDBxgwYIDT4RhjcsFniVRV00VkODAbCAbGquoaEbnf/fkY4EWgHPChuwSZrqrRl3quzEzljywDlUTXLNz96zMzM1m2bBlXXXUV4GoraozxXz5tR6qqM1S1garWVdV/uNeNcSdRVPVuVS2jqi3cr0tOogB/HkziyMk0AMqWDKN2+ZL5dg35LSMjg7/85S+0a9eO77//3ulwjDH5ICB6Ni3ddqY0emWNMoX2+Wh6ejq3334748ePp0SJEpQqVcrpkIwx+SAwEmmW56PRtQrnbX1aWhq33HILkyZNIiIighkzZnDNNdc4HZYxJh8ERCLNWtFUGJ+PpqSkcNNNNzF16lRKlSrFrFmzaNeundNhGWPyid8n0oNJKWx1T70cFhxE06qFb/zRO++8k6lTp1KmTBnmzJlDdHSuHgUbYwopvx+PNGtp9IpqUYSHFr7xRx966CFq165NbGysJVFjApDfl0gL6229qnoqvdq1a8eGDRsIDQ11OCpjjC/4fYl06bYzPZpaFZJEeuzYMbp06cKkSZM86yyJGhO4/LpEmpyWQfyuY57lKwtBIj1y5Ai9evXi999/Z+fOnfTr149ixfxvOD9jjPf8OpGu3nWU1IxMAGqXL+n4+KOHDh2iR48eLFu2jFq1ajF//nxLosYUAX6dSLM2xHf6tv7AgQN069aNVatWUbduXRYsWED16tUvvqMxxu/59TPSZdvPPB91sqJp7969dO7cmVWrVnH55ZezcOFCS6LGFCF+nUhXJRz1vHeyRLpr1y4SEhJo0qQJP/74I1WqVHEsFmNMwfPbW/sTKensP54CuCa6c3KgklatWjF//nxq1qxJhQr+OeGeMSb3/LZEuuPQSc/7amWKExJcsJfy559/njXTZ3R0tCVRY4oov02k2xPPJNKa5Qq2NLpp0yY6derEjTfeyJw5cwr03MaYwsdvE+m2xBOe9zXLFdz8TOvWraNjx44kJCTQrl072rZtW2DnNsYUTn6bSLN2Da1XMaJAzrl69Wo6derE3r176dKlCzNnzrQxRY0x/ptID7grmgAaVvJ9Mlu+fDldunThwIEDdO/enenTpxMRUTAJ3BhTuPltIt1z9JTnffmIMJ+eKy0tjYEDB5KYmMh1113HtGnTKFHCP6Z7Nsb4nl8m0uS0DPYdc5VIg4OE6j6ewz40NJSJEydy22238d133xEeHu7T8xlj/ItfJtL1e4973lctXZxQHzV9SkxM9Lxv3bo1X375pfWdN8acxy8T6ZrdZ3o0Na3qm+ejsbGx1K5dm4kTJ/rk+MaYwOGXifT01MuAT27rZ8+eTe/evTl+/Djz58/P9+MbYwKLXybS/ceSPe/LlMjfiqbp06fTr18/kpOTuffee/noo4/y9fjGmMDjl4l00Z9nnl1efllkvh13ypQp3HDDDaSmpjJ8+HDGjBlDUJBffouMMQXI77KEKmzclwRAkEDbOuXy5biTJ09m8ODBpKWlMWLECEaNGuWZc8kYY3Lid6M/nR4RH6BqmeIUD8ufWUNr165NZGQkDzzwAK+99polUWOM1/wvkaZneN7XLJt/g5VceeWVxMfHU6VKFUuixphL4ne39inpZ0qkea2xHzNmDF999ZVnuWrVqpZEjTGXzA9LpJmcrqfPy6hPo0aN4pFHHiE4OJg2bdrQoEGD/AnQGFPk+F2JNDVLibRmLkukb731Fo888ggA7777riVRY0ye+F8izVLZVCMXJdLXXnuNJ598EhHho48+Yvjw4fkZnjGmCPLLW/vTLmVkfFXl73//O6+++ioiwtixY7njjjt8EKExpqjxu0Sq7q/lSoYRUcz78Pfs2cPo0aMJDg7myy+/5Oabb/ZNgMaYIsfvEulpl1pjX6VKFebOncu2bdsYOHCgj6IyxhRFfveM9LQqpS8+JmhmZia//vqrZ7lVq1aWRI0x+c5vE2npiwxWkpmZyX333cfVV1/NuHHjCigqY0xR5Le39mVKhF7ws4yMDO666y6+/PJLwsPDqVixYgFGZowpavw2kZYunn2JND09nWHDhjFhwgRKlCjB9OnT6dKlSwFHZ4wpSvw3kWZTIk1NTeXmm29m8uTJREZGMmPGDK6++moHojPGFCV+m0izG9D5/vvvZ/LkyURFRTFr1izatm3rQGTGmKLGjyubzi+RPvzww9StW5f58+dbEjXGFBi/LZGerrXPzMz0jGLfokUL1q9fT0iI316WMcYP+XWJNCkpiW7duvH555971lsSNcYUNL/NOpJ6kp59+/Drr7+yefNmBg8eTMmS+TfQszHGeMunJVIR6SUiG0Rks4g8nc3nIiKj3J+vEpErvTluicxTXNurJ7/++ivVq1cnNjbWkqgxxjE+K5GKSDDwAdAdSACWiMg0VV2bZbNrgfruVxvg/9xfL0gzM9gx7hlO7N5MrVq1WLBgAbVq1fLJNRhjjDd8WSJtDWxW1T9VNRWYCPQ/Z5v+wJfq8htQWkQq53TQ9EO7OLF7M/Xq1WPhwoWWRI0xjvNlIq0K7MyynOBed6nbnEUz0ilVqSY//vgj1atXz5dAjTEmL3xZ2ZTdLHKai20QkXuBe92LKcf2bo+vWjXHfOvPygMHnQ7Ch+z6/FcgXxvA5bnd0ZeJNAHIWmSsBuzOxTao6sfAxwAislRVo/M31MLDrs+/BfL1BfK1gev6cruvL2/tlwD1RaS2iIQBQ4Bp52wzDRjmrr1vCxxV1T0+jMkYY/Kdz0qkqpouIsOB2UAwMFZV14jI/e7PxwAzgOuAzcBJ4E5fxWOMMb7i0wb5qjoDV7LMum5MlvcKPHiJh/04H0IrzOz6/FsgX18gXxvk4frElcuMMcbklt/2tTfGmMKi0CZSX3UvLSy8uL5b3Ne1SkR+FZHmTsSZGxe7tizbXSUiGSIyqCDjyytvrk9EOovIChFZIyI/FnSMeeHF72aUiPwgIivd1+c3dRsiMlZE9otI/AU+z11eUdVC98JVObUFqAOEASuBxudscx0wE1db1LbA707Hnc/X1x4o435/rb9cnzfXlmW7WFzP0Ac5HXc+/+xKA2uBGu7lik7Hnc/X9yzwhvt9BeAQEOZ07F5eX0fgSiD+Ap/nKq8U1hKpT7qXFiIXvT5V/VVVD7sXf8PVxtYfePOzA3gImAzsL8jg8oE313cz8J2q7gBQVX+6Rm+uT4FIEREgAlciTS/YMHNHVRfiivdCcpVXCmsi9Un30kLkUmP/C67/kv7gotcmIlWBAcAY/I83P7sGQBkRiRORZSIyrMCiyztvrm800AhX55nVwCOqmlkw4flcrvJKYR2PNN+6lxZSXscuIl1wJVJ/mcXPm2t7F3hKVTNchRq/4s31hQCtgK5AcWCRiPymqht9HVw+8Ob6egIrgBigLjBXRH5S1WM+jq0g5CqvFNZEmm/dSwspr2IXkWbAp8C1qppYQLHllTfXFg1MdCfR8sB1IpKuqt8XSIR54+3v5kFVPQGcEJGFQHPAHxKpN9d3J/AvdT1U3CwiW4GGwOKCCdGncpdXnH74e4EHviHAn0BtzjzwbnLONr05+6HwYqfjzufrq4Grx1d7p+PN72s7Z/vP8a/KJm9+do2A+e5tSwDxQFOnY8/H6/s/4CX3+8uAXUB5p2O/hGusxYUrm3KVVwpliVQDvHupl9f3IlAO+NBdcktXPxgwwstr81veXJ+qrhORWcAqIBP4VFWzbW5T2Hj583sV+FxEVuNKOE+pql+MCiUiE4DOQHkRSQD+DoRC3vKK9Wwyxpg8Kqy19sYY4zcskRpjTB5ZIjXGmDyyRGqMMXlkidQYY/LIEqnxinuUphVZXrVy2DYpH873uYhsdZ/rDxFpl4tjfCoijd3vnz3ns1/zGqP7OKe/L/HuEZFKX2T7FiJyXX6c2xQe1vzJeEVEklQ1Ir+3zeEYnwPTVfVbEekBjFTVZnk4Xp5juthxReQLYKOq/iOH7e8AolV1eH7HYpxjJVKTKyISISLz3aXF1SJy3ghPIlJZRBZmKbFd417fQ0QWufedJCIXS3ALgXrufR93HyteRB51ryspIv9zj48ZLyI3udfHiUi0iPwLKO6OY5z7syT31/9mLSG6S8IDRSRYRN4SkSXucSnv8+Lbsgj3ABci0lpc48gud3+9XFyTQL4C3OSO5SZ37GPd51me3ffR+AGnu2vZyz9eQAaugSpWAFNwdSUs5f6sPK6eIKfvcJLcX0cAz7nfBwOR7m0XAiXd658CXszmfJ/j7joKDAZ+xzUQyGqgJK7h29YALYGBwCdZ9o1yf43DVfrzxJRlm9MxDgC+cL8PwzXyT3HgXuB59/piwFKgdjZxJmW5vklAL/dyKSDE/b4bMNn9/g5gdJb9/wnc6n5fGld//JJO/7ztdWmvQtlF1BRKp1S1xekFEQkF/ikiHXF1g6yKq9/13iz7LAHGurf9XlVXiEgnoDHwi7vraxiuklx23hKR54EDuEbA6gpMUddgIIjId8A1wCxgpIi8getxwE+XcF0zgVEiUgzoBSxU1VPuxwnN5Mzo/VFAfWDrOfsXF5EVuPpvLwPmZtn+CxGpj2v0oNALnL8H0E9E/uZeDsc1zsK6S7gG4zBLpCa3bsE1OnorVU0TkW24koCHqi50J9rewFci8hZwGJirqkO9OMcTqvrt6QUR6ZbdRqq6UURa4eoj/bqIzFHVV7y5CFVNFpE4XEPD3QRMOH064CFVnX2RQ5xS1RYiEgVMxzUr7ihc/dEXqOoAd8Vc3AX2F2Cgqm7wJl5TONkzUpNbUcB+dxLtAtQ8dwMRqene5hPgM1xTPPwGdBCR0888S4hIAy/PuRC43r1PSVy35T+JSBXgpKp+DYx0n+dcae6ScXYm4hqc4hpcg3Xg/vrX0/uISAP3ObOlqkeBh4G/ufeJwjUqErhu5087jusRx2mzgYfEXTwXkZYXOocpvCyRmtwaB0SLyFJcpdP12WzTGVghIstxPcd8T1UP4EosE0RkFa7E2tCbE6rqH7ienS7G9cz0U1VdDlwBLHbfYj8HvJbN7h8Dq05XNp1jDq65fOapa3oNcI0Duxb4Q1wTpX3ERe7g3LGsBIYAb+IqHf+C6/npaQuAxqcrm3CVXEPdscW7l42fseZPxhiTR1YiNcaYPLJEaowxeWSJ1Bhj8sgSqTHG5JElUmOMySNLpMYYk0eWSI0xJo8skRpjTB79P1PyeXYpcOFKAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": { "needs_background": "light" }, "output_type": "display_data" } ], "source": [ "from sklearn.metrics import auc\n", "\n", "fig, ax = plt.subplots(figsize=(5, 5))\n", "ax.plot(fpr, tpr, lw=3,\n", " label='ROC Curve (area = {:.2f})'.format(auc(fpr, tpr)))\n", "ax.plot([0, 1], [0, 1], 'k--', lw=2)\n", "ax.set(\n", " xlim=(0, 1),\n", " ylim=(0, 1),\n", " title=\"ROC Curve\",\n", " xlabel=\"False Positive Rate\",\n", " ylabel=\"True Positive Rate\",\n", ")\n", "ax.legend();\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This Receiver Operating Characteristic (ROC) curve tells how well our classifier is doing. We can tell it's doing well by how far it bends the upper-left. A perfect classifier would be in the upper-left corner, and a random classifier would follow the diagonal line.\n", "\n", "The area under this curve is `area = 0.76`. This tells us the probability that our classifier will predict correctly for a randomly chosen instance." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Learn more\n", "* Similar example that uses DataFrames for a real world dataset: http://ml.dask.org/examples/xgboost.html\n", "* Recorded screencast stepping through the real world example above:\n", "* A blogpost on dask-xgboost http://matthewrocklin.com/blog/work/2017/03/28/dask-xgboost\n", "* XGBoost documentation: https://xgboost.readthedocs.io/en/latest/python/python_intro.html#\n", "* Dask-XGBoost documentation: http://ml.dask.org/xgboost.html" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.10" } }, "nbformat": 4, "nbformat_minor": 4 }