The Backprop Typeclass
Most of the functions in this module require a
Backprop constraint on values
you wish to backpropagate. Even if you manage to get around it for the most
gradBP (the actual function to compute gradients) requires it on both
the inputs and outputs. Let's dig deeper into what it is, and how to define
The typeclass contains three methods:
class Backprop a where zero :: a -> a add :: a -> a -> a one :: a -> a
zero is "zero" in the verb sense -- it takes a value and "zeroes out" all
components. For a vector, this means returning a zero vector of the same
shape. For a list, this means replacing all of the items with zero and
returning a list of the same length.
one does the same thing but with one; the point of it is to be
one = gradBP
id --- the gradient of the identity function for your type.
add is used to add together contributions in gradients, and is usually a
Instances are provided for most common data types where it makes sense.
When defining your own custom types, if your custom type is has a single
constructor where all fields are instances of
Backprop, then GHC.Generics
can be used to write your instances automatically:
data MyType = MkMyType Double [Float] (R 10) (L 20 10) (V.Vector Double) deriving Generic
Nice type. Since it has a single constructor and all of its fields are already
Backprop instances, we can just write:
instance Backprop MyType
and now your type can be backpropagated!
For writing "primitive"
Backprop instances (types that aren't product types),
you can use the provided "helpers" from the Numeric.Backprop.Class module.
If your type is a
Num instance, you can use
instance Backprop Double where zero = zeroNum add = addNum one = oneNum
If your type is made using a
Functor instance, you can use
instance Backprop a => Backprop (V.Vector a) where zero = zeroFunctor add = undefined -- ?? one = oneFunctor
And if your type has an
IsList instance, you can use
instance Backprop a => Backprop (V.Vector a) where zero = zeroFunctor add = addIsList one = oneFunctor
Completely custom instances are also possible; you just need to implement
one as they make sense for your type. Just make sure that
you obey the laws for sane behavior!