Manipulating BVars

The most important aspect of the usability of this library is allowing you to seamlessly manipulate BVar s as as if they were just as, without requiring you as the user to be able to recognize or acknowledge the difference. Here are some techniques to that end.

Remember, a BVar s a is a BVar containing an a --- it's an a that, when used, keeps track of and propagates your gradient.

Typeclass Interface

BVars have Num, Fractional, Floating, Eq, and Ord instances. These instances are basically "lifted" to the BVar itself, so if you have a BVar s Double, you can use (*), sqrt, (>), etc. on it exactly as if it were just a Double.

Constant Values

If we don't care about a value's gradient, we can use auto:

auto :: a -> BVar s a

auto x basically gives you a BVar that contains just x alone. Useful for using with functions that expect BVars, but you just have a specific value you want to use.

Coercible

If a and b are Coercible, then so are BVar s a and BVar s b, using the coerceVar function. This is useful for "unwrapping" and "wrapping" BVars of newtypes:

newtype MyInt = MyInt Int

getMyInt :: BVar s MyInt -> BVar s Int
getMyInt = coerceVar

Accessing Contents

The following techniques can be used to access values inside BVars:

Traversable Containers

One that we saw earlier was sequenceVar, which we used to turn a BVar containing a list into a list of BVars:

sequenceVar :: (Backprop a, Reifies s W)
            => BVar s [a]
            -> [BVar s a]

If you have a BVar containing a list, you can get a list of BVars of all of that list's elements. (sequenceVar actually works on all Traversable instances, not just lists) This is very useful when combined with -XViewPatterns, as seen earlier.

Records and Fields

In practice, a lot of usage involves functions involving contents of records or data types containing fields. The previous example, involving a simple ANN, demonstrates this:

data Net = N { _nWeights1 :: L 20 100
             , _nBias1    :: R 20
             , _nWeights2 :: L  5  20
             , _nBias2    :: R  5
             }
  deriving (Show, Generic)

instance Backprop Net       -- can be automatically defined

To compute the result of this network (ran on an R 100, a 100-vector) and get the output R 5, we need do a matrix multiplication by the _nWeights1 field, add the result to the _nBias1 field...basically, the result is a function of linear algebra and related operations on the input and all of the contents of the Net data type. However, you can't directly use _nWeights, since it takes a Net, not BVar s Net. And you also can't directly pattern match on the N constructor.

There are two main options for this: the lens interface, and the higher-kinded data interface.

Lens Interface

The most straightforward way to do this is the lens-based interface, using viewVar or ^^..

If we make lenses for Net using the lens or microlens-th packages:

-- requires -XTemplateHaskell
makeLenses ''Net

or make them manually:

nBias1' :: Functor f => (R 20 -> f (R 20)) -> Net -> f Net
nBias1' f n = (\b -> n { _nBias1 = b }) <$> f (_nBias1 n)

then ^. from the lens or microlens packages lets you retrieve a field from a Net:

(^. nWeights1) :: Net -> L 20 100
(^. nBias1   ) :: Net -> R 20
(^. nWeights2) :: Net -> L  5  20
(^. nBias2   ) :: Net -> R  5

And, ^^. from backprop (also aliased as viewVar) lets you do the same thing from a BVar s Net (a BVar containing your Net):

(^^. nWeights1) :: BVar s Net -> BVar s (L 20 100)
(^^. nBias1   ) :: BVar s Net -> BVar s (R 20)
(^^. nWeights2) :: BVar s Net -> BVar s (L  5  20)
(^^. nBias2   ) :: BVar s Net -> BVar s (R  5)

With our lenses and ^^., we can write our network running function. This time, I'll include the type!

runNet :: Reifies s W
       => BVar s Net
       -> BVar s (R 100)
       -> BVar s (R 5)
runNet net x = z
  where
    -- run first layer
    y = logistic $ (net ^^. nWeights1) #> x + (net ^^. nBias1)
    -- run second layer
    z = logistic $ (net ^^. nWeights2) #> y + (net ^^. nBias2)

logistic :: Floating a => a -> a
logistic x = 1 / (1 + exp (-x))

Note that we are using versions of #> lifted for BVars, from the hmatrix-backprop library:

(#>) :: BVar s (L m n) -> BVar s (R n) -> BVar s (R m)

Higher-Kinded Data Interface

Using the lens based interface, you can't directly pattern match and construct fields. To allow for directly pattern matching, there's another interface option involving the "Higher-Kinded Data" techniques described in this article.

If we had a type-family (that can be re-used for all of your data types):

type family HKD f a where
    HKD Identity a = a
    HKD f        a = f a

We can define Net instead as:

data Met' f = M { _mWeights1 :: HKD f (L 20 100)
                , _mBias1    :: HKD f (R 20)
                , _mWeights2 :: HKD f (L  5  20)
                , _mBias2    :: HKD f (R  5)
                }
  deriving Generic

Then our original type is:

type Met = Met' Identity

deriving instance Show Met
instance Backprop Met

Met is the same as Net in every way -- it can be pattern matched on to get the L 20 100, etc. (the Identity disappears):

getMetBias1 :: Met -> R 20
getMetBias1 (M _ b _ _) = b

The benefit of this is that we can now directly pattern match on a BVar s Met to get the internal fields as BVars using splitBV as a view pattern (or the BV pattern synonym):

runMet :: Reifies s W
       => BVar s Met
       -> BVar s (R 100)
       -> BVar s (R 5)
runMet (splitBV -> M w1 b1 w2 b2) x = z
  where
    -- run first layer
    y = logistic $ w1 #> x + b1
    -- run second layer
    z = logistic $ w2 #> y + b2

runMetPS :: Reifies s W
       => BVar s Met
       -> BVar s (R 100)
       -> BVar s (R 5)
runMetPS (BV (M w1 b1 w2 b2)) x = z
  where
    -- run first layer
    y = logistic $ w1 #> x + b1
    -- run second layer
    z = logistic $ w2 #> y + b2

Now, the M w1 b1 w2 b2 pattern can be used to deconstruct both "normal" Mets, as well as a BVar s Met (with splitBV or BV).

Note that this HKD access method is potentially less performant than lens access (by about 10-20%).

Potential or Many Fields

Some values "may" or "may not" have values of a given field. An example would include the nth item in a list or vector, or the Just of a Maybe.

For these, the lens-based (prism-based/traversal-based) interface is the main way to access partial fields. You can use (^^?) or previewVar with any Traversal:

(^?)  ::        a -> Traversal' a b -> Maybe         b
(^^?) :: BVar s a -> Traversal' a b -> Maybe (BVar s b)

If the value in the BVar "has" that field, then you'll get a Just with the BVar of that field's contents. If it doesn't, you'll get a Nothing.

You can use this with any prism or traversal, like using _head to get the first item in a list if it exists.

If you have a type that might contain many values of a field (like a tree or list), you can use (^^..) or toListOfVar, which works on any Traversal:

(^..)  ::        a -> Traversal' a b -> [       b]
(^^..) :: BVar s a -> Traversal' a b -> [BVar s b]

This can be used to implement sequenceVar, actually:

sequenceVar :: BVar s [a] -> [BVar s a]
sequenceVar xs = xs ^^.. traverse

Tuples

The T2 pattern synonym is provided, which allow you to pattern match on a BVar s (a, b) to get a BVar s a and BVar s b. The T3 pattern is also provided, which does the same thing for three-tuples.

Note that T2 and T3 are bidirectional pattern synonyms, and can be used to construct as well as deconstruct.

Combining BVars

The following techniques can be used to "combine" BVars:

Foldable Containers

The "opposite" of sequenceVar is collectVar, which takes a foldable container of BVars and returns a BVar containing that foldable container of contents:

collectVar :: (Backprop a, Foldable t, Functor t, Reifies s W)
           => t (BVar s a)
           -> BVar s (t a)

Constructors

Sometimes you would like to combine a bunch of BVars into a BVar of specific container or data type.

isoVar

The simplest way to do this is using the isoVar, isoVar2, etc. family of functions:

isoVar2
    :: (Backprop a, Backprop b, Backprop c, Reifies s W)
    => (a -> b -> c)
    -> (c -> (a, b))
    -> BVar s a
    -> BVar s b
    -> BVar s c

So if we had a type like:

data DoubleInt = DI Double Int

We can combine a Double and Int into a DoubleInt using isoVar2:

isoVar2 DI (\(DI x y) -> (x,y))
    :: Reifies s W
    => BVar s Double
    -> BVar s Int
    -> BVar s DoubleInt

Higher-Kinded Data Interface

You can also use the "Higher Kinded Data" interface, as well. For our Met type above, you can use joinBV, or the BV pattern synonym:

makeMet :: Reifies s W
        => BVar s (L 20 100)
        -> BVar s (R 20)
        -> BVar s (L  5  20)
        -> BVar s (R  5)
        -> BVar s Met
makeMet w1 b1 w2 b2 = joinBV (M w1 b1 w2 b2)

Modifying fields

If you just want to "set" a specific field, you can use the lens-based interface with (.~~) or setVar. For example, if we wanted to set the _nWeights2 field of a Net to a new matrix, we can do:

myNet & nWeights2 .~~ newMatrix

or

setVar nWeights2
    :: Reifies s W
    => BVar s (L 20 5)
    -> BVar s Net
    -> BVar s Net

You can also use (%~~) or overVar to apply a function to a specific inside your value.

Prelude Modules

Finally, the Prelude.Backprop module has a lot of your normal Prelude functions "lifted" to work on BVars of values. For many situations, these aren't necessary, and normal Prelude functions will work just fine on BVars of values (like (.)). However, it does have some convenient functions, like minimum, foldl', fmap, toList, fromIntegral, realToFrac, etc. lifted to work on BVars. This module is meant to be imported qualified.

Moving On

Now that you know all about BVars, you really can just jump into the haddocks and start writing programs. The next section of this documentation is more details about the Backprop typeclass.

< Previous: A Detailed Look
Next: The Backprop Typeclass >

Site generated with

— © 2018