Transformer Architecture
Neural network architecture relying entirely on self-attention mechanisms to compute contextual representations in parallel.
Mechanism & Definition
A deep learning architecture introduced in 'Attention Is All You Need' (2017). Replaces sequential recurrent processing (RNNs) with parallelized self-attention layers, calculating pairwise relationship weights between all tokens in a sequence simultaneously using Key-Query-Value matrix multiplications.