← Back to Basic Science

Decomposable Attention

2016
Computer Science (theoretical)Machine Learning TheoryArchitecture Componentfoundational

Attention model that decomposes the problem of comparing two sentences into aligning and comparing sub-phrases using only feed-forward attention, with no recurrence (Parikh, Tackstrom, Das & Uszkoreit, "A Decomposable Attention Model," EMNLP 2016). One of the few pre-Transformer models to rely on attention without a recurrent network, cited by Vaswani et al. as a direct precedent for dispensing with recurrence entirely.

Originators

  • Parikh, A.
  • Tackstrom, O.
  • Das, D.
  • Uszkoreit, J.

Landmark Paper

W2413794162 ↗
Not retracted (OpenAlex)

Checked 2026-09-19 — interim signal only, see docs/BASIC_ROADMAP.md Phase 10

Connections

  • is precursor to Transformer
    basis: reasoned

    Vaswani et al. cite Parikh et al.'s Decomposable Attention (ref [27], their own footnote "in all but a few cases [27]") as the direct precedent showing attention alone -- without recurrence -- could carry a transduction task; the Transformer generalizes this to self-attention with multi-head projection and positional encoding.