Activation Layers

Activation types and layers.

miopenActivationMode_t

enum miopenActivationMode_t

Activation layer modes

Values:

miopenActivationPASTHRU = 0

No activation, pass through the data

miopenActivationLOGISTIC = 1

Sigmoid function: \(1 / (1 + e^{-x})\)

miopenActivationTANH = 2

Tanh activation \( \beta * tanh( \alpha * x) \)

miopenActivationRELU = 3

Rectified Linear Unit \( max(0, x) \)

miopenActivationSOFTRELU = 4

\(log(1 + e^x)\)

miopenActivationABS = 5

Absolute value \(abs(x)\)

miopenActivationPOWER = 6

Scaled and shifted power \((\alpha + \beta * x)^{gamma}\)

miopenActivationCLIPPEDRELU = 7

Clipped Rectified Linear Unit \( min(\alpha, max(0,x)) \)

miopenActivationLEAKYRELU = 8

Leaky Rectified Linear Unit \( \alpha * x | x <= 0; x | x > 0 \)

miopenActivationELU = 9

Exponential Rectified Linear Unit \( \alpha * (e^{x} - 1) | x <= 0; x | x > 0 \)

miopenCreateActivationDescriptor

miopenStatus_t miopenCreateActivationDescriptor(miopenActivationDescriptor_t *activDesc)

Creates the Activation descriptor object.

Return
miopenStatus_t
Parameters
  • activDesc: Pointer to an activation tensor descriptor type

miopenSetActivationDescriptor

miopenStatus_t miopenSetActivationDescriptor(const miopenActivationDescriptor_t activDesc, miopenActivationMode_t mode, double activAlpha, double activBeta, double activGamma)

Sets the activation layer descriptor details.

Sets all of the descriptor details for the activation layer

Return
miopenStatus_t
Parameters
  • activDesc: Pointer to a activation layer descriptor (output)
  • mode: Activation mode enum (input)
  • activAlpha: Alpha value for some activation modes (input)
  • activBeta: Beta value for some activation modes (input)
  • activGamma: Gamma value for some activation modes (input)

miopenGetActivationDescriptor

miopenStatus_t miopenGetActivationDescriptor(const miopenActivationDescriptor_t activDesc, miopenActivationMode_t *mode, double *activAlpha, double *activBeta, double *activGamma)

Gets the activation layer descriptor details.

Retrieves all of the descriptor details for the activation layer.

Return
miopenStatus_t
Parameters
  • activDesc: Pointer to a activation layer descriptor (input)
  • mode: Activation mode enum (output)
  • activAlpha: Alpha value for some activation modes (output)
  • activBeta: Beta value for some activation modes (output)
  • activGamma: Gamma value for some activation modes (output)

miopenActivationForward

miopenStatus_t miopenActivationForward(miopenHandle_t handle, const miopenActivationDescriptor_t activDesc, const void *alpha, const miopenTensorDescriptor_t xDesc, const void *x, const void *beta, const miopenTensorDescriptor_t yDesc, void *y)

Execute an activation forward layer.

Return
miopenStatus_t
Parameters
  • handle: MIOpen handle (input)
  • activDesc: Descriptor for activation layer (input)
  • alpha: Floating point scaling factor, allocated on the host (input)
  • xDesc: Tensor descriptor for data input tensor x (input)
  • x: Data tensor x (input)
  • beta: Floating point shift factor, allocated on the host (input)
  • yDesc: Tensor descriptor for output data tensor y (input)
  • y: Data tensor y (output)

miopenActivationBackward

miopenStatus_t miopenActivationBackward(miopenHandle_t handle, const miopenActivationDescriptor_t activDesc, const void *alpha, const miopenTensorDescriptor_t yDesc, const void *y, const miopenTensorDescriptor_t dyDesc, const void *dy, const miopenTensorDescriptor_t xDesc, const void *x, const void *beta, const miopenTensorDescriptor_t dxDesc, void *dx)

Execute a activation backwards layer.

Return
miopenStatus_t
Parameters
  • handle: MIOpen handle (input)
  • activDesc: Descriptor for activation layer (input)
  • alpha: Floating point scaling factor, allocated on the host (input)
  • yDesc: Tensor descriptor for input data tensor y (input)
  • y: Data tensor y (input)
  • dyDesc: Tensor descriptor for input data tensor dy (input)
  • dy: Data delta tensor dy (input)
  • xDesc: Tensor descriptor for data input tensor x (input)
  • x: Data tensor x (input)
  • beta: Floating point shift factor, allocated on the host (input)
  • dxDesc: Tensor descriptor for data output tensor dx (input)
  • dx: Output data delta tensor dx (output)

miopenDestroyActivationDescriptor

miopenStatus_t miopenDestroyActivationDescriptor(miopenActivationDescriptor_t activDesc)

Destroys the activation descriptor object.

Return
miopenStatus_t
Parameters
  • activDesc: Activation tensor descriptor type (input)