DerivativeID

We identify values produced for forward or reverse differentiation with a DerivativeID. We assign each derivative a unique DerivativeID with its own unique "sequence" number. Each instance of DerivativeID represents a separate set of perturbations, the product of any two of which is zero.

In nested derivatives, higher sequence numbers would always be used for the more inner derivative, and appear higher in the tree of Duals (or Reverses) representing a number. A sequence number of zero is used at the leaves where the value was produced without reference to any derivative operation; values for these are just wrappers around the raw data (either FloatScalar or FloatTensor).