神刀安全网

Use Keras Deep Learning Models with Scikit-Learn in Python

Keras is one of the most popular deep learning libraries in Python for research and development because of its simplicity and ease of use.

Thescikit-learn library is the most popular library for general machine learning in Python.

In this post you will discover how you can use deep learning models from Keras with the scikit-learn library in Python.

This will allow you to leverage the power of the scikit-learn library for tasks like model evaluation and model hyper-parameter optimization.

Let’s get started.

Use Keras Deep Learning Models with Scikit-Learn in Python

Use Keras Deep Learning Models with Scikit-Learn in Python

Photo by Alan Levine , some rights reserved.

Overview

Keras is a popular library for deep learning in Python, but the focus of the library is deep learning. In fact it strives for minimalism, focusing on only what you need to quickly and simply define and build deep learning models.

The scikit-learn library in Python is built upon the SciPy stack for efficient numerical computation. It is a fully featured library for general machine learning and provides many utilities that are useful in the development of deep learning models. Not least:

  • Evaluation of models using resampling methods like k-fold cross validation.
  • Efficient search and evaluation of model hyper-parameters.

The Keras library provides a convenient wrapper for deep learning models to be used as classification or regression estimators in scikit-learn.

In the next sections we will work through examples of using the KerasClassifier wrapper for a classification neural network created in Keras and used in the scikit-learn library.

The test problem is the Pima Indians onset of diabetes classification dataset . This is a small dataset with all numerical attributes that is easy to work with. Download the dataset and place it in your currently working directly with the name pima-indians-diabetes.csv .

The following examples assume you have successfully installed Keras and scikit-learn.

Get Started in Deep Learning With Python

Use Keras Deep Learning Models with Scikit-Learn in Python

Deep Learning gets state-of-the-art results and Python hosts the most powerful tools.

Get started now!

PDF Download and Email Course.

FREE 14-Day Mini-Course onDeep Learning With Python

Download Your FREE Mini-Course

Download your PDF containing all 14 lessons.

Get your daily lesson via email with tips and tricks.

Evaluate Deep Learning Models with Cross Validation

The KerasClassifier and KerasRegressor classes in Keras take an argument build_fn which is the name of the function to call to get your model.

You must define a function called whatever you like that defines your model, compiles it and returns it.

In the example below we define a function create_model() that create a simple multi-layer neural network for the problem.

We pass this function name to the KerasClassifier class by the build_fn argument. We also pass in additional arguments of nb_epoch=150 and batch_size=10 . These are automatically bundled up and passed on to the fit() function which is called internally by the KerasClassifier class.

In this example we use the scikit-learn StratifiedKFold to perform 10-fold stratified cross validation. This is a resampling technique that can provide a robust estimate of the performance of a machine learning model on unseen data.

We use the scikit-learn function cross_val_score() to evaluate our model using the cross validation scheme and print the results.

# MLP for Pima Indians Dataset with 10-fold cross validation via sklearn fromkeras.modelsimportSequential fromkeras.layersimportDense fromkeras.wrappers.scikit_learnimportKerasClassifier fromsklearn.cross_validationimportStratifiedKFold fromsklearn.cross_validationimportcross_val_score importnumpy importpandas   # Function to create model, required for KerasClassifier defcreate_model():  # create model  model = Sequential()  model.add(Dense(12, input_dim=8, init='uniform', activation='relu'))  model.add(Dense(8, init='uniform', activation='relu'))  model.add(Dense(1, init='uniform', activation='sigmoid'))  # Compile model  model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])  return model   # fix random seed for reproducibility seed = 7 numpy.random.seed(seed) # load pima indians dataset dataset = numpy.loadtxt("pima-indians-diabetes.csv", delimiter=",") # split into input (X) and output (Y) variables X = dataset[:,0:8] Y = dataset[:,8] # create model model = KerasClassifier(build_fn=create_model, nb_epoch=150, batch_size=10) # evaluate using 10-fold cross validation kfold = StratifiedKFold(y=Y, n_folds=10, shuffle=True, random_state=seed) results = cross_val_score(model, X, Y, cv=kfold) print(results.mean()) 

Running the example displays the skill of the model for each epoch. A total of 10 models are created and evaluated and the final average accuracy is displayed.

... Epoch 145/150 692/692 [==============================] - 0s - loss: 0.4671 - acc: 0.7803 Epoch 146/150 692/692 [==============================] - 0s - loss: 0.4661 - acc: 0.7847 Epoch 147/150 692/692 [==============================] - 0s - loss: 0.4581 - acc: 0.7803 Epoch 148/150 692/692 [==============================] - 0s - loss: 0.4657 - acc: 0.7688 Epoch 149/150 692/692 [==============================] - 0s - loss: 0.4660 - acc: 0.7659 Epoch 150/150 692/692 [==============================] - 0s - loss: 0.4574 - acc: 0.7702 76/76 [==============================] - 0s 0.756442244065 

Grid Search Deep Learning Model Parameters

The previous example showed how easy it is to wrap your deep learning model from Keras and use it in functions from the scikit-learn library.

In this example we go a step further. The function that we specify to the build_fn argument when creating the KerasClassifier wrapper can take arguments. We can uses these arguments to further customize the construction of the model. In addition, we know we can provide arguments to the fit() function.

In this example we use a grid search to evaluate different configurations for our neural network model and report on the combination that provides the best estimated performance.

The create_model() function is defined to take two arguments optimizer and init, both of which must have default values. This will allow us to evaluate the effect of using different optimization algorithms and weight initialization schemes for our network.

After creating our model, we define arrays of values for the parameter we wish to search, specifically:

  • Optimizers for searching different weight values.
  • Initializers for preparing the network weights using different schemes.
  • Epochs for training the model for different number of exposures to the training dataset.
  • Batches for varying the number of samples before a weight update.

The options are specified into a dictionary and passed to the configuration of the GridSearchCV scikit-learn class. This class will evaluate a version of our neural network model for each combination of parameters (2 x 3 x 3 x 3 for the combinations of optimizers, initializations, epochs and batches). Each combination is then evaluated using the default of 3-fold stratified cross validation.

That is a lot of models and a lot of computation. This is not a scheme that you want to use lightly because of the time it will take. It may be useful for you to design small experiments with a smaller subset of your data that will complete in a reasonable time. This is reasonable in this case because of the small network and the small dataset (less than 1000 instances and 9 attributes).

Finally, the performance and combination of configurations for the best model are displayed, followed by the performance of all combinations of parameters.

# MLP for Pima Indians Dataset with grid search via sklearn fromkeras.modelsimportSequential fromkeras.layersimportDense fromkeras.wrappers.scikit_learnimportKerasClassifier fromsklearn.grid_searchimportGridSearchCV importnumpy importpandas   # Function to create model, required for KerasClassifier defcreate_model(optimizer='rmsprop', init='glorot_uniform'):  # create model  model = Sequential()  model.add(Dense(12, input_dim=8, init=init, activation='relu'))  model.add(Dense(8, init=init, activation='relu'))  model.add(Dense(1, init=init, activation='sigmoid'))  # Compile model  model.compile(loss='binary_crossentropy', optimizer=optimizer, metrics=['accuracy'])  return model   # fix random seed for reproducibility seed = 7 numpy.random.seed(seed) # load pima indians dataset dataset = numpy.loadtxt("pima-indians-diabetes.csv", delimiter=",") # split into input (X) and output (Y) variables X = dataset[:,0:8] Y = dataset[:,8] # create model model = KerasClassifier(build_fn=create_model) # grid search epochs, batch size and optimizer optimizers = ['rmsprop', 'adam'] init = ['glorot_uniform', 'normal', 'uniform'] epochs = numpy.array([50, 100, 150]) batches = numpy.array([5, 10, 20]) param_grid = dict(optimizer=optimizers, nb_epoch=epochs, batch_size=batches, init=init) grid = GridSearchCV(estimator=model, param_grid=param_grid) grid_result = grid.fit(X, Y) # summarize results print("Best: %f using %s" % (grid_result.best_score_, grid_result.best_params_)) for params, mean_score, scoresin grid_result.grid_scores_:     print("%f (%f) with: %r" % (scores.mean(), scores.std(), params)) 

This might take about 5 minutes to complete on your workstation executed on the CPU (rather than CPU). running the example shows the results below.

We can see that the grid search discovered that using a uniform initialization scheme, rmsprop optimizer, 150 epochs and a batch size of 5 achieved the best cross validation score of approximately 75% on this problem.

Best: 0.751302 using {'init': 'uniform', 'optimizer': 'rmsprop', 'nb_epoch': 150, 'batch_size': 5} 0.653646 (0.031948) with: {'init': 'glorot_uniform', 'optimizer': 'rmsprop', 'nb_epoch': 50, 'batch_size': 5} 0.665365 (0.004872) with: {'init': 'glorot_uniform', 'optimizer': 'adam', 'nb_epoch': 50, 'batch_size': 5} 0.683594 (0.037603) with: {'init': 'glorot_uniform', 'optimizer': 'rmsprop', 'nb_epoch': 100, 'batch_size': 5} 0.709635 (0.034987) with: {'init': 'glorot_uniform', 'optimizer': 'adam', 'nb_epoch': 100, 'batch_size': 5} 0.699219 (0.009568) with: {'init': 'glorot_uniform', 'optimizer': 'rmsprop', 'nb_epoch': 150, 'batch_size': 5} 0.725260 (0.008027) with: {'init': 'glorot_uniform', 'optimizer': 'adam', 'nb_epoch': 150, 'batch_size': 5} 0.686198 (0.024774) with: {'init': 'normal', 'optimizer': 'rmsprop', 'nb_epoch': 50, 'batch_size': 5} 0.718750 (0.014616) with: {'init': 'normal', 'optimizer': 'adam', 'nb_epoch': 50, 'batch_size': 5} 0.725260 (0.028940) with: {'init': 'normal', 'optimizer': 'rmsprop', 'nb_epoch': 100, 'batch_size': 5} 0.727865 (0.028764) with: {'init': 'normal', 'optimizer': 'adam', 'nb_epoch': 100, 'batch_size': 5} 0.748698 (0.035849) with: {'init': 'normal', 'optimizer': 'rmsprop', 'nb_epoch': 150, 'batch_size': 5} 0.712240 (0.039623) with: {'init': 'normal', 'optimizer': 'adam', 'nb_epoch': 150, 'batch_size': 5} 0.699219 (0.024910) with: {'init': 'uniform', 'optimizer': 'rmsprop', 'nb_epoch': 50, 'batch_size': 5} 0.703125 (0.011500) with: {'init': 'uniform', 'optimizer': 'adam', 'nb_epoch': 50, 'batch_size': 5} 0.720052 (0.015073) with: {'init': 'uniform', 'optimizer': 'rmsprop', 'nb_epoch': 100, 'batch_size': 5} 0.712240 (0.034987) with: {'init': 'uniform', 'optimizer': 'adam', 'nb_epoch': 100, 'batch_size': 5} 0.751302 (0.031466) with: {'init': 'uniform', 'optimizer': 'rmsprop', 'nb_epoch': 150, 'batch_size': 5} 0.734375 (0.038273) with: {'init': 'uniform', 'optimizer': 'adam', 'nb_epoch': 150, 'batch_size': 5} ... 

Summary

In this post you discovered how you can wrap your Keras deep learning models and use them in the scikit-learn general machine learning library.

You can see that using scikit-learn for standard machine learning operations such as model evaluation and model hyper parameter optimization can save a lot of time over implementing these schemes yourself.

Wrapping your model allowed you to leverage powerful tools from scikit-learn to fit your deep learning models into your general machine learning process.

Do you have any questions about using Keras models in scikit-learn or about this post? Ask your question in the comments and I will do my best to answer.

Do You Want To Get Started With Deep Learning?

Use Keras Deep Learning Models with Scikit-Learn in Python

You can develop and evaluate deep learning models in just a few lines of Python code. You need:

Deep Learning With Python

Take the next step with 14 self-study tutorials and

7 end-to-end projects.

Covers multi-layer perceptrons, convolutional neural networks, objection recognition and more.

Ideal for machine learning practitioners already familiar with the Python ecosystem.

Bring Deep Learning To Your Machine Learning Projects

转载本站任何文章请注明:转载至神刀安全网,谢谢神刀安全网 » Use Keras Deep Learning Models with Scikit-Learn in Python

分享到:更多 ()

评论 抢沙发

  • 昵称 (必填)
  • 邮箱 (必填)
  • 网址