athena.models.lm.transformer_lm

Transformer language model implementation

Module Contents

Classes

TransformerLM

Standard implementation of a RNNLM. Model mainly consists of embeding layer,

class athena.models.lm.transformer_lm.TransformerLM(data_descriptions, config=None)

Bases: athena.models.lm.nn_lm.NNLM

Standard implementation of a RNNLM. Model mainly consists of embeding layer, rnn layers(with dropout), and the full connection layer, which are all incuded in self.model_for_rnn

default_config

forward(inputs, input_lengths, training: bool = None)

do NN LM forward computation, for both train and decode.