BatchNormTraining

open class BatchNormTraining : BatchNormTrainingBase<BatchNormTraining>

A trainable Batch Normalization transform, as described in https://arxiv.org/abs/1502.03167 . When training is complete, use its @see inferenceMode property to get the computed affine transform. This version maintains an exponential moving average of the sum of the samples, sum of the squared samples, and sample count which are used to estimate the population mean and variance.

Epsilon is hardcoded to 1.e-5f.

Constructors

BatchNormTraining

fun BatchNormTraining(numFeatures: Int, momentum: Float = 0.1f)

Functions

cpu

open override fun cpu(): BatchNormTraining

equals

open operator override fun equals(other: Any?): Boolean

extractTangent

open override fun extractTangent(output: DTensor, extractor: (DTensor, DTensor) -> DTensor): TrainableComponent.Companion.Tangent

getSingleInput

open fun getSingleInput(inputs: Array<out DTensor>): DTensor

Helper to check that the layer was called with a single input. Returns that input if successful, else errors.

gpu

open override fun gpu(): BatchNormTraining

hashCode

open override fun hashCode(): Int

invoke

open operator override fun invoke(input: DTensor): DTensor

abstract operator fun invoke(vararg inputs: DTensor): DTensor

load

open override fun load(from: ByteBuffer): BatchNormTraining

store

open override fun store(into: ByteBuffer): ByteBuffer

open fun to(device: Device): OnDevice

trainingStep

open override fun trainingStep(optim: Optimizer<*>, tangent: Trainable.Tangent): BatchNormTraining

withTrainables

open override fun withTrainables(trainables: List<Trainable<*>>): BatchNormTraining

wrap

open override fun wrap(wrapper: Wrapper): BatchNormTraining

The wrap function should return the same static type it is declared on.

Properties

inferenceMode

open override val inferenceMode: AffineTransform

Freeze the batch norm transform that was used during training, returning an affine transform to be used for inference.

stats

open override val stats: Pair<DTensor, DTensor>

The computed running mean and variance.

trainables

open override val trainables: List<Trainable<*>>

Inheritors

BatchNorm2d