Autograd can automatically differentiate native Python and Numpy code. It can handle a large subset of Python’s features, including loops, ifs, recursion and closures, and it can even take derivatives of derivatives of derivatives. It uses reverse-mode differentiation (a.k.a. backpropagation), which means it can efficiently take gradients of scalar-valued functions with respect to array-valued arguments. The main intended application is gradient-based optimization. For more information, check out thetutorial and theexamples directory.
>>> import autograd.numpy as np # Thinly-wrapped numpy >>> from autograd import grad # The only autograd function you may ever need >>> >>> def tanh(x): # Define a function ... y = np.exp(-x) ... return (1.0 - y) / (1.0 + y) ... >>> grad_tanh = grad(tanh) # Obtain its gradient function >>> grad_tanh(1.0) # Evaluate the gradient at x = 1.0 0.39322386648296376 >>> (tanh(1.0001) - tanh(0.9999)) / 0.0002 # Compare to finite differences 0.39322386636453377
We can continue to differentiate as many times as we like:
>>> def elementwise_grad(fun): # A wrapper for broadcasting ... return grad(lambda x: np.sum(fun(x))) # (closures are no problem) ... >>> grad_tanh = elementwise_grad(tanh) >>> grad_tanh_2 = elementwise_grad(grad_tanh) # 2nd derivative >>> grad_tanh_3 = elementwise_grad(grad_tanh_2) # 3rd derivative >>> grad_tanh_4 = elementwise_grad(grad_tanh_3) # etc. >>> grad_tanh_5 = elementwise_grad(grad_tanh_4) >>> grad_tanh_6 = elementwise_grad(grad_tanh_5) >>> >>> import matplotlib.pyplot as plt >>> x = np.linspace(-7, 7, 200) >>> plt.plot(x, tanh(x), ... x, grad_tanh(x), ... x, grad_tanh_2(x), ... x, grad_tanh_3(x), ... x, grad_tanh_4(x), ... x, grad_tanh_5(x), ... x, grad_tanh_6(x)) >>> plt.show()
See thetanh example file for the code.
You can find a tutorialhere.
- Simple neural net
- Convolutional neural net
- Recurrent neural net
- Neural Turing Machine
- Backpropagating through a fluid simulation
- Variational inference in Bayesian neural network
- Gaussian process regression
- Sampyl, a pure Python MCMC package with HMC and NUTS
How to install
pip install autograd
Autograd was written by Dougal Maclaurin , David Duvenaud and Matt Johnson , and we’re actively developing it. Please feel free to submit any bugs or feature requests. We’d also love to hear about your experiences with autograd in general. Drop us an email!
We want to thank Jasper Snoek and the rest of the HIPS group (led by Prof. Ryan P. Adams) for helpful contributions and advice; Barak Pearlmutter for foundational work on automatic differentiation and for guidance on our implementation; and Analog Devices Inc. (Lyric Labs) and Samsung Advanced Institute of Technology for their generous support.
转载本站任何文章请注明：转载至神刀安全网，谢谢神刀安全网 » HIPS/autograd: Efficiently computes derivatives of numpy code