athena.layers.conv_module

Convolution module layer.

Module Contents

Classes

ConvModule

Convolution Module.

class athena.layers.conv_module.ConvModule(d_model, kernel_size=32, depth_multiplier=1, activation='gelu')

Bases: tensorflow.keras.layers.Layer

Convolution Module.

See section 2.2 of this paper for a description of this technique: Conformer: Convolution-augmented Transformer for Speech Recognition (https://arxiv.org/abs/2005.08100)

architecture:

pointwise_conv(.) # 2 * input_dim

glu(.) # input_dim depthwise_conv_1d(.) batch_norm(.) act(.)

pointwise_conv(.)

call(inputs, training=False)

Compute convolution module.

Parameters

inputs – Input tensor (#batch, time, channels).

Returns

Output tensor (#batch, time, channels).

Return type

tensor