pkg/

Package fofe provides an implementation of the Fixed-size Ordinally-Forgetting Encoding (FOFE) method.

Package lshattention provides an implementation of the LSH-Attention model, as describe in `Reformer: The Efficient Transformer` by N. Kitaev, Ł. Kaiser, A. Levskaya (https://arxiv.org/pdf/2001.04451.pdf).

Package syntheticattention provides an implementation of the Synthetic Attention described in: "SYNTHESIZER: Rethinking Self-Attention in Transformer Models" by Tay et al., 2020.

Package birnncrf provides an implementation of a Bidirectional Recurrent Neural Network (BiRNN) with a Conditional Random Fields (CRF) on tom.

Package bls provides an implementation of the Broad Learning System (BLS) described in "Broad Learning System: An Effective and Efficient Incremental Learning System Without the Need for Deep Architecture" by C. L. Philip Chen and Zhulin Liu, 2017.

Package slstm implements a Sentence-State LSTM graph neural network.

Package startransformer provides a variant implementation of the Star-Transformer model introduced by Qipeng Guo, Xipeng Qiu et al.

Package adanorm implements the Adaptive Normalization (AdaNorm) method.

Package fixnorm implements the fixnorm normalization method.

Package layernorm implements the Layer Normalization (LayerNorm) i method.

Package layernormsimple implements a simple version of LayerNorm (LayerNorm-simple).

Package rmsnorm implements the Root Mean Square Layer Normalization method.

Package rae provides an implementation of the recursive auto-encoder strategy described in "Towards Lossless Encoding of Sentences" by Prato et al., 2019.

Package rc contains built-in Residual Connections (RC).

Package horn provides an implementation of Higher Order Recurrent Neural Networks (HORN).

Package lstmsc provides an implementation of LSTM enriched with a PolicyGradient to enable Dynamic Skip Connections.

Package mist provides an implementation of the MIST (MIxed hiSTory) recurrent network as described in "Analyzing and Exploiting NARX Recurrent Neural Networks for Long-Term Dependencies" by Di Pietro et al., 2018 (https://arxiv.org/pdf/1702.07805.pdf).

Package nru provides an implementation of the NRU (Non-Saturating Recurrent Units) recurrent network as described in "Towards Non-Saturating Recurrent Units for Modelling Long-Term Dependencies" by Chandar et al., 2019.

Package rla provides an implementation of RLA (Recurrent Linear Attention).

Package srnn implements the SRNN (Shuffling Recurrent Neural Networks) by Rotman and Wolf, 2020.

Package charlm provides an implementation of a character-level language model that uses a recurrent neural network as its backbone.

Package contextualstringembeddings provides an implementation of the "Contextual String Embeddings" of words (Akbik et al., 2018).

Package evolvingembeddings provides a word embedding model that evolves by dynamically aggregating contextual embeddings over time during inference.

Package sequencelabeler provides an implementation of a sequence labeling architecture composed by Embeddings -> BiRNN -> Scorer -> CRF.

Package stackedembeddings provides convenient types to stack multiple word embedding representations by concatenating them.

Package tokenizers is an interim solution while developing `gotokenizers` (https://github.com/nlpodyssey/gotokenizers).

Package basetokenizer provides an implementation of a very simple tokenizer that splits by white-spaces (and alike) and punctuation symbols.

Package bart implements the transformer model introduced by Mike et al., 2019.

Package bert provides an implementation of BERT model (Bidirectional Encoder Representations from Transformers).