• Working out many functions and partial derivatives can be time-consuming and error-prone.
  • Function, even Julia functions, not just mathematical ones, are made up of elementary differentiable operations, such +, -, sin, cos, etc. So a computer program should be able to differentiate those functions.
  • Automatic Differentiation, AD, describes this process. The way it works is different from using analytical or numerical solutions.
  • JuliaDiff lists available differentiation packages in Julia.
  • Flux deep learning package uses Zygote.jl for AD.
using Zygote

f(x) = 3x^2 + 2x + 1

# Gradient of f at x=2
gradient(f, 2)


# gradient of vectors
x = [2, 1];
y = [2, 0];
f(x, y) = sum((x .- y).^2)

# gradient returns a tuple, with a gradient for each argument to the function.
grad = gradient((x,y) -> f(x, y), x, y)
([-2.0, 2.0], [2.0, -2.0])

# gradient of Julia functions
function pow(x, n)
 r = 1
 for i = 1:n
   r *= x
 end
 return r
end

gradient(x -> pow(x, 3), 5)
(75,)
Tags: julia-lang