athena.layers.commons

Utils for common layers.

Module Contents

Classes

PositionalEncoding	positional encoding can be used in transformer
ScaledPositionalEncoding	scaled positional encoding,
Collapse4D	collapse4d can be used in cnn-lstm for speech processing
Gelu	Gaussian Error Linear Unit.
TdnnLayer	An implementation of Tdnn Layer
GroupNormalization
InstanceNormalization	Instance normalization layer.
DownSampleBlock	conv2d downsample block for stargan, instance norm is used because batch size is 1
UpSampleBlock	conv2d upsample block for stargan, instance norm is used because batch size is 1
ConditionalInstanceNormalisation	CIN Block.
ResidualBlock	Residual Block with instance normalization.
Down2d_init
Down2d
Up2d	docstring for Up2d.
ZoneOutCell	Wrapper for LSTM cell to create ZoneOut Cell

Attributes

SUPPORTED_RNNS
ACTIVATIONS

class athena.layers.commons.PositionalEncoding(d_model, max_position=800, scale=False)

Bases: tensorflow.keras.layers.Layer

positional encoding can be used in transformer

call(x)

call function

class athena.layers.commons.ScaledPositionalEncoding(d_model, max_position=800)

Bases: PositionalEncoding

scaled positional encoding, reference: https://arxiv.org/pdf/1809.08895.pdf

build(_)

call(x)

call function

class athena.layers.commons.Collapse4D

Bases: tensorflow.keras.layers.Layer

collapse4d can be used in cnn-lstm for speech processing reshape from [N T D C] -> [N T D*C]

call(x)

class athena.layers.commons.Gelu

Bases: tensorflow.keras.layers.Layer

Gaussian Error Linear Unit.

This is a smoother version of the RELU. Original paper: https://arxiv.org/abs/1606.08415

Parameters

x – float Tensor to perform activation.

Returns

with the GELU activation applied.

Return type

x

call(x)

class athena.layers.commons.TdnnLayer(context, output_dim, use_bias=False, **kwargs)

Bases: tensorflow.keras.layers.Layer

An implementation of Tdnn Layer :param context: a int of left and right context, or a list of context indexes, e.g. (-2, 0, 2). :param output_dim: the dim of the linear transform

call(x, training=None, mask=None)

class athena.layers.commons.GroupNormalization(groups: int = 2, axis: int = -1, epsilon: float = 0.001, center: bool = True, scale: bool = True, beta_initializer='zeros', gamma_initializer='ones', beta_regularizer=None, gamma_regularizer=None, beta_constraint=None, gamma_constraint=None, **kwargs)

Bases: tensorflow.keras.layers.Layer

build(input_shape)

call(inputs)

get_config()

compute_output_shape(input_shape)

_reshape_into_groups(inputs, input_shape, tensor_input_shape)

_apply_normalization(reshaped_inputs, input_shape)

_get_reshaped_weights(input_shape)

_check_if_input_shape_is_none(input_shape)

_set_number_of_groups_for_instance_norm(input_shape)

_check_size_of_dimensions(input_shape)

_check_axis()

_create_input_spec(input_shape)

_add_gamma_weight(input_shape)

_add_beta_weight(input_shape)

_create_broadcast_shape(input_shape)

class athena.layers.commons.InstanceNormalization(**kwargs)

Bases: GroupNormalization

Instance normalization layer. References

• [Instance Normalization: The Missing Ingredient for Fast Stylization]

(https://arxiv.org/abs/1607.08022)

class athena.layers.commons.DownSampleBlock(filters, kernel_size, strides)

Bases: tensorflow.keras.layers.Layer

conv2d downsample block for stargan, instance norm is used because batch size is 1

call(x)

class athena.layers.commons.UpSampleBlock(filters, kernel_size, strides)

Bases: tensorflow.keras.layers.Layer

conv2d upsample block for stargan, instance norm is used because batch size is 1

call(x)

class athena.layers.commons.ConditionalInstanceNormalisation(in_channel)

Bases: tensorflow.keras.layers.Layer

CIN Block.

call(x, c)

class athena.layers.commons.ResidualBlock(out_channel)

Bases: tensorflow.keras.layers.Layer

Residual Block with instance normalization.

call(x, c)

class athena.layers.commons.Down2d_init(filters, kernel_size, stride)

Bases: tensorflow.keras.layers.Layer

call(x)

class athena.layers.commons.Down2d(filters, kernel_size, stride)

Bases: tensorflow.keras.layers.Layer

call(x)

class athena.layers.commons.Up2d(filters, kernel_size, stride)

Bases: tensorflow.keras.layers.Layer

docstring for Up2d.

call(x)

class athena.layers.commons.ZoneOutCell(zoneout_rate=0.0, **kwargs)

Bases: tensorflow.keras.layers.LSTMCell

Wrapper for LSTM cell to create ZoneOut Cell

inspired by: https://github.com/teganmaharaj/zoneout/blob/master/zoneout_tensorflow.py Published by one of ‘https://arxiv.org/pdf/1606.01305.pdf’ paper writers.

call(inputs, states, training=False)

Runs vanilla LSTM Cell and applies zoneout.

get_config()

athena.layers.commons.SUPPORTED_RNNS

athena.layers.commons.ACTIVATIONS