##// END OF EJS Templates
update reference to v3
Matthias BUSSONNIER -
Show More
@@ -1,12 +1,16 b''
1 1 {
2 "metadata": {},
2 "metadata": {
3 "name": "tutorial"
4 },
3 5 "nbformat": 3,
6 "nbformat_minor": 0,
4 7 "worksheets": [
5 8 {
6 9 "cells": [
7 10 {
8 11 "cell_type": "heading",
9 12 "level": 1,
13 "metadata": {},
10 14 "source": [
11 15 "An Introduction to machine learning with scikit-learn"
12 16 ]
@@ -14,121 +18,134 b''
14 18 {
15 19 "cell_type": "heading",
16 20 "level": 1,
21 "metadata": {},
17 22 "source": [
18 23 "Section contents"
19 24 ]
20 25 },
21 26 {
22 27 "cell_type": "markdown",
28 "metadata": {},
23 29 "source": [
24 "In this section, we introduce the machine learning",
25 "vocabulary that we use through-out scikit-learn and give a",
30 "In this section, we introduce the machine learning\n",
31 "vocabulary that we use through-out scikit-learn and give a\n",
26 32 "simple learning example."
27 33 ]
28 34 },
29 35 {
30 36 "cell_type": "heading",
31 37 "level": 2,
38 "metadata": {},
32 39 "source": [
33 40 "Machine learning: the problem setting"
34 41 ]
35 42 },
36 43 {
37 44 "cell_type": "markdown",
38 "source": [
39 "In general, a learning problem considers a set of n",
40 "samples of",
41 "data and try to predict properties of unknown data. If each sample is",
42 "more than a single number, and for instance a multi-dimensional entry",
43 "(aka multivariate",
44 "data), is it said to have several attributes,",
45 "metadata": {},
46 "source": [
47 "In general, a learning problem considers a set of n\n",
48 "samples of\n",
49 "data and try to predict properties of unknown data. If each sample is\n",
50 "more than a single number, and for instance a multi-dimensional entry\n",
51 "(aka multivariate\n",
52 "data), is it said to have several attributes,\n",
45 53 "or features."
46 54 ]
47 55 },
48 56 {
49 57 "cell_type": "markdown",
58 "metadata": {},
50 59 "source": [
51 60 "We can separate learning problems in a few large categories:"
52 61 ]
53 62 },
54 63 {
55 64 "cell_type": "markdown",
65 "metadata": {},
56 66 "source": [
57 "supervised learning,",
58 "in which the data comes with additional attributes that we want to predict",
59 "(:ref:`Click here <supervised-learning>`",
60 "to go to the Scikit-Learn supervised learning page).This problem",
67 "supervised learning,\n",
68 "in which the data comes with additional attributes that we want to predict\n",
69 "(:ref:`Click here <supervised-learning>`\n",
70 "to go to the Scikit-Learn supervised learning page).This problem\n",
61 71 "can be either:"
62 72 ]
63 73 },
64 74 {
65 75 "cell_type": "markdown",
66 "source": [
67 "classification:",
68 "samples belong to two or more classes and we",
69 "want to learn from already labeled data how to predict the class",
70 "of unlabeled data. An example of classification problem would",
71 "be the digit recognition example, in which the aim is to assign",
72 "each input vector to one of a finite number of discrete",
76 "metadata": {},
77 "source": [
78 "classification:\n",
79 "samples belong to two or more classes and we\n",
80 "want to learn from already labeled data how to predict the class\n",
81 "of unlabeled data. An example of classification problem would\n",
82 "be the digit recognition example, in which the aim is to assign\n",
83 "each input vector to one of a finite number of discrete\n",
73 84 "categories."
74 85 ]
75 86 },
76 87 {
77 88 "cell_type": "markdown",
89 "metadata": {},
78 90 "source": [
79 "regression:",
80 "if the desired output consists of one or more",
81 "continuous variables, then the task is called regression. An",
82 "example of a regression problem would be the prediction of the",
91 "regression:\n",
92 "if the desired output consists of one or more\n",
93 "continuous variables, then the task is called regression. An\n",
94 "example of a regression problem would be the prediction of the\n",
83 95 "length of a salmon as a function of its age and weight."
84 96 ]
85 97 },
86 98 {
87 99 "cell_type": "markdown",
88 "source": [
89 "unsupervised learning,",
90 "in which the training data consists of a set of input vectors x",
91 "without any corresponding target values. The goal in such problems",
92 "may be to discover groups of similar examples within the data, where",
93 "it is called clustering,",
94 "or to determine the distribution of data within the input space, known as",
95 "density estimation, or",
96 "to project the data from a high-dimensional space down to two or thee",
97 "dimensions for the purpose of visualization",
98 "(:ref:`Click here <unsupervised-learning>`",
100 "metadata": {},
101 "source": [
102 "unsupervised learning,\n",
103 "in which the training data consists of a set of input vectors x\n",
104 "without any corresponding target values. The goal in such problems\n",
105 "may be to discover groups of similar examples within the data, where\n",
106 "it is called clustering,\n",
107 "or to determine the distribution of data within the input space, known as\n",
108 "density estimation, or\n",
109 "to project the data from a high-dimensional space down to two or thee\n",
110 "dimensions for the purpose of visualization\n",
111 "(:ref:`Click here <unsupervised-learning>`\n",
99 112 "to go to the Scikit-Learn unsupervised learning page)."
100 113 ]
101 114 },
102 115 {
103 116 "cell_type": "heading",
104 117 "level": 2,
118 "metadata": {},
105 119 "source": [
106 120 "Training set and testing set"
107 121 ]
108 122 },
109 123 {
110 124 "cell_type": "markdown",
125 "metadata": {},
111 126 "source": [
112 "Machine learning is about learning some properties of a data set",
113 "and applying them to new data. This is why a common practice in",
114 "machine learning to evaluate an algorithm is to split the data",
115 "at hand in two sets, one that we call a training set on which",
116 "we learn data properties, and one that we call a testing set,",
127 "Machine learning is about learning some properties of a data set\n",
128 "and applying them to new data. This is why a common practice in\n",
129 "machine learning to evaluate an algorithm is to split the data\n",
130 "at hand in two sets, one that we call a training set on which\n",
131 "we learn data properties, and one that we call a testing set,\n",
117 132 "on which we test these properties."
118 133 ]
119 134 },
120 135 {
121 136 "cell_type": "heading",
122 137 "level": 2,
138 "metadata": {},
123 139 "source": [
124 140 "Loading an example dataset"
125 141 ]
126 142 },
127 143 {
128 144 "cell_type": "markdown",
145 "metadata": {},
129 146 "source": [
130 "scikit-learn comes with a few standard datasets, for instance the",
131 "iris and digits",
147 "scikit-learn comes with a few standard datasets, for instance the\n",
148 "iris and digits\n",
132 149 "datasets for classification and the boston house prices dataset for regression.:"
133 150 ]
134 151 },
@@ -136,28 +153,31 b''
136 153 "cell_type": "code",
137 154 "collapsed": false,
138 155 "input": [
139 "from sklearn import datasets",
140 "iris = datasets.load_iris()",
156 "from sklearn import datasets\n",
157 "iris = datasets.load_iris()\n",
141 158 "digits = datasets.load_digits()"
142 159 ],
143 160 "language": "python",
161 "metadata": {},
144 162 "outputs": []
145 163 },
146 164 {
147 165 "cell_type": "markdown",
166 "metadata": {},
148 167 "source": [
149 "A dataset is a dictionary-like object that holds all the data and some",
150 "metadata about the data. This data is stored in the .data member,",
151 "which is a n_samples, n_features array. In the case of supervised",
152 "problem, explanatory variables are stored in the .target member. More",
153 "details on the different datasets can be found in the :ref:`dedicated",
168 "A dataset is a dictionary-like object that holds all the data and some\n",
169 "metadata about the data. This data is stored in the .data member,\n",
170 "which is a n_samples, n_features array. In the case of supervised\n",
171 "problem, explanatory variables are stored in the .target member. More\n",
172 "details on the different datasets can be found in the :ref:`dedicated\n",
154 173 "section <datasets>`."
155 174 ]
156 175 },
157 176 {
158 177 "cell_type": "markdown",
178 "metadata": {},
159 179 "source": [
160 "For instance, in the case of the digits dataset, digits.data gives",
180 "For instance, in the case of the digits dataset, digits.data gives\n",
161 181 "access to the features that can be used to classify the digits samples:"
162 182 ]
163 183 },
@@ -168,13 +188,15 b''
168 188 "print digits.data # doctest: +NORMALIZE_WHITESPACE"
169 189 ],
170 190 "language": "python",
191 "metadata": {},
171 192 "outputs": []
172 193 },
173 194 {
174 195 "cell_type": "markdown",
196 "metadata": {},
175 197 "source": [
176 "and digits.target gives the ground truth for the digit dataset, that",
177 "is the number corresponding to each digit image that we are trying to",
198 "and digits.target gives the ground truth for the digit dataset, that\n",
199 "is the number corresponding to each digit image that we are trying to\n",
178 200 "learn:"
179 201 ]
180 202 },
@@ -185,21 +207,24 b''
185 207 "digits.target"
186 208 ],
187 209 "language": "python",
210 "metadata": {},
188 211 "outputs": []
189 212 },
190 213 {
191 214 "cell_type": "heading",
192 215 "level": 2,
216 "metadata": {},
193 217 "source": [
194 218 "Shape of the data arrays"
195 219 ]
196 220 },
197 221 {
198 222 "cell_type": "markdown",
223 "metadata": {},
199 224 "source": [
200 "The data is always a 2D array, n_samples, n_features, although",
201 "the original data may have had a different shape. In the case of the",
202 "digits, each original sample is an image of shape 8, 8 and can be",
225 "The data is always a 2D array, n_samples, n_features, although\n",
226 "the original data may have had a different shape. In the case of the\n",
227 "digits, each original sample is an image of shape 8, 8 and can be\n",
203 228 "accessed using:"
204 229 ]
205 230 },
@@ -210,48 +235,54 b''
210 235 "digits.images[0]"
211 236 ],
212 237 "language": "python",
238 "metadata": {},
213 239 "outputs": []
214 240 },
215 241 {
216 242 "cell_type": "markdown",
243 "metadata": {},
217 244 "source": [
218 "The :ref:`simple example on this dataset",
219 "<example_plot_digits_classification.py>` illustrates how starting",
220 "from the original problem one can shape the data for consumption in",
245 "The :ref:`simple example on this dataset\n",
246 "<example_plot_digits_classification.py>` illustrates how starting\n",
247 "from the original problem one can shape the data for consumption in\n",
221 248 "the scikit-learn."
222 249 ]
223 250 },
224 251 {
225 252 "cell_type": "heading",
226 253 "level": 2,
254 "metadata": {},
227 255 "source": [
228 256 "Learning and Predicting"
229 257 ]
230 258 },
231 259 {
232 260 "cell_type": "markdown",
261 "metadata": {},
233 262 "source": [
234 "In the case of the digits dataset, the task is to predict the value of a",
235 "hand-written digit from an image. We are given samples of each of the 10",
236 "possible classes on which we fit an",
237 "estimator to be able to predict",
263 "In the case of the digits dataset, the task is to predict the value of a\n",
264 "hand-written digit from an image. We are given samples of each of the 10\n",
265 "possible classes on which we fit an\n",
266 "estimator to be able to predict\n",
238 267 "the labels corresponding to new data."
239 268 ]
240 269 },
241 270 {
242 271 "cell_type": "markdown",
272 "metadata": {},
243 273 "source": [
244 "In scikit-learn, an estimator is just a plain Python class that",
274 "In scikit-learn, an estimator is just a plain Python class that\n",
245 275 "implements the methods fit(X, Y) and predict(T)."
246 276 ]
247 277 },
248 278 {
249 279 "cell_type": "markdown",
280 "metadata": {},
250 281 "source": [
251 "An example of estimator is the class sklearn.svm.SVC that",
252 "implements Support Vector Classification. The",
253 "constructor of an estimator takes as arguments the parameters of the",
254 "model, but for the time being, we will consider the estimator as a black",
282 "An example of estimator is the class sklearn.svm.SVC that\n",
283 "implements Support Vector Classification. The\n",
284 "constructor of an estimator takes as arguments the parameters of the\n",
285 "model, but for the time being, we will consider the estimator as a black\n",
255 286 "box:"
256 287 ]
257 288 },
@@ -259,35 +290,39 b''
259 290 "cell_type": "code",
260 291 "collapsed": false,
261 292 "input": [
262 "from sklearn import svm",
293 "from sklearn import svm\n",
263 294 "clf = svm.SVC(gamma=0.001, C=100.)"
264 295 ],
265 296 "language": "python",
297 "metadata": {},
266 298 "outputs": []
267 299 },
268 300 {
269 301 "cell_type": "heading",
270 302 "level": 2,
303 "metadata": {},
271 304 "source": [
272 305 "Choosing the parameters of the model"
273 306 ]
274 307 },
275 308 {
276 309 "cell_type": "markdown",
310 "metadata": {},
277 311 "source": [
278 "In this example we set the value of gamma manually. It is possible",
279 "to automatically find good values for the parameters by using tools",
280 "such as :ref:`grid search <grid_search>` and :ref:`cross validation",
312 "In this example we set the value of gamma manually. It is possible\n",
313 "to automatically find good values for the parameters by using tools\n",
314 "such as :ref:`grid search <grid_search>` and :ref:`cross validation\n",
281 315 "<cross_validation>`."
282 316 ]
283 317 },
284 318 {
285 319 "cell_type": "markdown",
320 "metadata": {},
286 321 "source": [
287 "We call our estimator instance clf as it is a classifier. It now must",
288 "be fitted to the model, that is, it must learn from the model. This is",
289 "done by passing our training set to the fit method. As a training",
290 "set, let us use all the images of our dataset apart from the last",
322 "We call our estimator instance clf as it is a classifier. It now must\n",
323 "be fitted to the model, that is, it must learn from the model. This is\n",
324 "done by passing our training set to the fit method. As a training\n",
325 "set, let us use all the images of our dataset apart from the last\n",
291 326 "one:"
292 327 ]
293 328 },
@@ -298,13 +333,15 b''
298 333 "clf.fit(digits.data[:-1], digits.target[:-1])"
299 334 ],
300 335 "language": "python",
336 "metadata": {},
301 337 "outputs": []
302 338 },
303 339 {
304 340 "cell_type": "markdown",
341 "metadata": {},
305 342 "source": [
306 "Now you can predict new values, in particular, we can ask to the",
307 "classifier what is the digit of our last image in the digits dataset,",
343 "Now you can predict new values, in particular, we can ask to the\n",
344 "classifier what is the digit of our last image in the digits dataset,\n",
308 345 "which we have not used to train the classifier:"
309 346 ]
310 347 },
@@ -315,40 +352,46 b''
315 352 "clf.predict(digits.data[-1])"
316 353 ],
317 354 "language": "python",
355 "metadata": {},
318 356 "outputs": []
319 357 },
320 358 {
321 359 "cell_type": "markdown",
360 "metadata": {},
322 361 "source": [
323 362 "The corresponding image is the following:"
324 363 ]
325 364 },
326 365 {
327 366 "cell_type": "markdown",
367 "metadata": {},
328 368 "source": [
329 "As you can see, it is a challenging task: the images are of poor",
369 "As you can see, it is a challenging task: the images are of poor\n",
330 370 "resolution. Do you agree with the classifier?"
331 371 ]
332 372 },
333 373 {
334 374 "cell_type": "markdown",
375 "metadata": {},
335 376 "source": [
336 "A complete example of this classification problem is available as an",
337 "example that you can run and study:",
377 "A complete example of this classification problem is available as an\n",
378 "example that you can run and study:\n",
338 379 ":ref:`example_plot_digits_classification.py`."
339 380 ]
340 381 },
341 382 {
342 383 "cell_type": "heading",
343 384 "level": 2,
385 "metadata": {},
344 386 "source": [
345 387 "Model persistence"
346 388 ]
347 389 },
348 390 {
349 391 "cell_type": "markdown",
392 "metadata": {},
350 393 "source": [
351 "It is possible to save a model in the scikit by using Python's built-in",
394 "It is possible to save a model in the scikit by using Python's built-in\n",
352 395 "persistence model, namely pickle:"
353 396 ]
354 397 },
@@ -356,27 +399,29 b''
356 399 "cell_type": "code",
357 400 "collapsed": false,
358 401 "input": [
359 "from sklearn import svm",
360 "from sklearn import datasets",
361 "clf = svm.SVC()",
362 "iris = datasets.load_iris()",
363 "X, y = iris.data, iris.target",
364 "clf.fit(X, y)",
365 "import pickle",
366 "s = pickle.dumps(clf)",
367 "clf2 = pickle.loads(s)",
368 "clf2.predict(X[0])",
402 "from sklearn import svm\n",
403 "from sklearn import datasets\n",
404 "clf = svm.SVC()\n",
405 "iris = datasets.load_iris()\n",
406 "X, y = iris.data, iris.target\n",
407 "clf.fit(X, y)\n",
408 "import pickle\n",
409 "s = pickle.dumps(clf)\n",
410 "clf2 = pickle.loads(s)\n",
411 "clf2.predict(X[0])\n",
369 412 "y[0]"
370 413 ],
371 414 "language": "python",
415 "metadata": {},
372 416 "outputs": []
373 417 },
374 418 {
375 419 "cell_type": "markdown",
420 "metadata": {},
376 421 "source": [
377 "In the specific case of the scikit, it may be more interesting to use",
378 "joblib's replacement of pickle (joblib.dump & joblib.load),",
379 "which is more efficient on big data, but can only pickle to the disk",
422 "In the specific case of the scikit, it may be more interesting to use\n",
423 "joblib's replacement of pickle (joblib.dump & joblib.load),\n",
424 "which is more efficient on big data, but can only pickle to the disk\n",
380 425 "and not to a string:"
381 426 ]
382 427 },
@@ -384,13 +429,15 b''
384 429 "cell_type": "code",
385 430 "collapsed": false,
386 431 "input": [
387 "from sklearn.externals import joblib",
432 "from sklearn.externals import joblib\n",
388 433 "joblib.dump(clf, 'filename.pkl') # doctest: +SKIP"
389 434 ],
390 435 "language": "python",
436 "metadata": {},
391 437 "outputs": []
392 438 }
393 ]
439 ],
440 "metadata": {}
394 441 }
395 442 ]
396 443 } No newline at end of file
General Comments 0
You need to be logged in to leave comments. Login now