STATE SPACE MODELS

Efficient Sequential Intelligence Beyond Self-Attention

The history of Artificial Intelligence has been characterised by successive architectural innovations, each seeking to overcome limitations encountered by previous computational paradigms. Early symbolic systems provided logical reasoning but struggled with uncertainty and scalability. Statistical learning introduced data-driven pattern recognition, while deep neural networks enabled increasingly abstract feature representation. More recently, Transformer architectures revolutionised language processing through attention mechanisms capable of modelling complex relationships across extensive bodies of information. These developments have produced remarkable advances in language generation, scientific reasoning, software engineering and multimodal learning.

Despite these achievements, contemporary Artificial Intelligence continues to encounter significant computational challenges. Transformer architectures require computational complexity that increases quadratically with sequence length because every element within an input sequence must compare itself with every other element through self-attention mechanisms. Whilst highly effective for many applications, this approach becomes increasingly expensive as sequences extend to hundreds of thousands or even millions of elements. Such limitations restrict efficiency within domains requiring continuous observation, real-time processing or exceptionally long contextual memory.

State Space Models provide an alternative computational philosophy. Rather than repeatedly comparing every element within a sequence, these architectures maintain continuously evolving internal representations that summarise relevant historical information. Incoming observations modify this internal state, allowing subsequent computation to depend upon accumulated contextual knowledge without requiring exhaustive pairwise comparisons. Consequently, State Space Models combine efficient sequential computation with powerful long-range memory, making them particularly attractive for applications involving continuous streams of information.

The resurgence of State Space Models demonstrates that progress within Artificial Intelligence increasingly depends upon integrating insights originating from numerous scientific disciplines. Concepts developed within control theory, dynamical systems, signal processing and applied mathematics are now influencing the design of next-generation neural architectures. This interdisciplinary synthesis represents one of the defining characteristics of modern Artificial Intelligence research.

From Control Engineering to Modern Artificial Intelligence

The origins of State Space Models extend considerably further than modern Artificial Intelligence. During the middle of the twentieth century, engineers, mathematicians and physicists developed state space methods to describe complex dynamical systems whose behaviour evolved continuously through time. Rather than representing systems through isolated equations, state space formulations provided unified mathematical descriptions capable of modelling interactions among numerous interconnected variables simultaneously.

Control engineering rapidly adopted these techniques because they enabled precise analysis of aircraft, industrial machinery, communication systems and electrical networks. By representing system behaviour through evolving internal states, engineers could predict future responses, design effective controllers and analyse system stability under changing environmental conditions. These mathematical principles subsequently became central to modern systems engineering.

Artificial Intelligence researchers also recognised the potential relevance of state space methods. Early sequential models, including recurrent neural networks, sought to preserve information through hidden internal states updated continually as new observations arrived. Although conceptually related to classical state space formulations, these early neural architectures encountered practical difficulties involving unstable optimisation, vanishing gradients and limited long-term memory.

The subsequent development of Long Short-Term Memory networks and Gated Recurrent Units introduced mechanisms capable of preserving information over longer temporal intervals, partially overcoming these limitations. Nevertheless, recurrent computation remained inherently sequential, restricting computational parallelism and limiting scalability compared with emerging Transformer architectures.

The remarkable success of Transformers temporarily shifted research attention away from recurrent computation. However, as researchers increasingly encountered computational limitations associated with quadratic attention complexity, interest returned to alternative sequential architectures. Modern State Space Models emerged from this renewed investigation, combining classical dynamical systems theory with advances in deep learning, efficient optimisation and specialised computational algorithms. Contemporary architectures therefore represent both a revival and a substantial extension of earlier state space concepts.

Dynamic State Transitions, Linear Algebra and Stability

The mathematical foundation of State Space Models derives from the representation of dynamic systems through continuously evolving internal variables known collectively as the system state. Rather than describing behaviour solely through observable inputs and outputs, state space formulations introduce hidden internal representations that preserve essential information regarding the history and current condition of the system.

Mathematically, the state evolves according to equations describing how previous internal states interact with new observations. The resulting internal representation provides sufficient information to predict future behaviour without requiring direct access to every historical observation individually. This property distinguishes State Space Models fundamentally from architectures relying upon explicit storage of complete input sequences.

Within Artificial Intelligence, these mathematical principles enable efficient modelling of long sequential data. Language, speech, biological signals, financial markets and sensor observations all consist of information evolving continuously through time. State Space Models process these sequences by updating internal representations progressively rather than repeatedly comparing every observation with every preceding element. Consequently, computational complexity increases approximately linearly with sequence length rather than quadratically, providing substantial efficiency improvements for extensive sequential data.

Linear algebra plays a central role throughout these formulations. Internal state transitions are represented through matrix operations describing how previous knowledge evolves in response to new information. Differential equations provide continuous formulations, while discrete approximations permit practical implementation within digital computation. Contemporary architectures additionally incorporate sophisticated parameterisation strategies ensuring stable optimisation whilst preserving expressive representational capability.

Stability Across Long Sequences

One particularly important mathematical characteristic concerns system stability. Classical control theory devoted extensive attention to ensuring that evolving state representations remained bounded despite continual environmental interaction. Modern Artificial Intelligence inherits these concerns because unstable state evolution rapidly degrades learning performance. Recent State Space Models therefore incorporate mathematically principled parameterisations guaranteeing stable dynamics throughout optimisation and inference.

Evolving Memory, Temporal Continuity and Hierarchical Representation

State Space Models differ fundamentally from attention-based architectures because memory emerges through continuously evolving internal representations rather than explicit comparison among input elements. Every new observation updates the internal state according to mathematically defined transition functions, allowing the model to preserve relevant historical information whilst discarding redundant details. Consequently, memory becomes an intrinsic property of system dynamics rather than an externally maintained database of previous observations.

Another defining architectural principle involves temporal continuity. Sequential information is treated as an evolving process rather than a collection of independent observations. This perspective closely resembles many natural systems in which present behaviour depends continuously upon accumulated historical experience. Human speech, financial activity, environmental monitoring and biological processes all exhibit such temporal continuity, making State Space Models particularly well suited to these domains.

Efficiency constitutes another central architectural principle. Since state updates require only local computation involving current observations and existing internal representations, computational requirements remain comparatively modest even for exceptionally long sequences. Modern implementations exploit specialised convolutional algorithms and efficient matrix operations, allowing extensive parallelism whilst preserving sequential reasoning capability.

Hierarchical representation further strengthens modern State Space Models. Multiple layers progressively transform evolving internal states into increasingly abstract conceptual representations, analogous to hierarchical processing observed within other deep neural architectures. Lower layers frequently capture local temporal relationships, while higher layers develop broader contextual understanding extending across substantially longer intervals. This hierarchical organisation contributes significantly to the expressive capability of contemporary State Space Models whilst maintaining computational efficiency.

Finally, State Space Models emphasise continuous adaptation. Internal representations evolve incrementally throughout processing, enabling models to respond dynamically to changing contextual information without repeatedly reconstructing complete attention patterns. This property supports efficient reasoning across extensive sequential information whilst providing a mathematically elegant framework for long-term contextual memory.

Bridging Continuous Dynamics and Discrete Computation

One of the defining characteristics of State Space Models is their ability to represent information through either continuous or discrete mathematical formulations. Classical systems theory traditionally employed continuous-time models to describe physical systems evolving without interruption, such as aircraft flight dynamics, electrical circuits or mechanical control systems. Artificial Intelligence, however, operates principally upon digital computers that process information in discrete computational steps. Modern State Space Models therefore establish mathematically rigorous relationships between continuous dynamical systems and their discrete computational implementations, allowing the theoretical advantages of continuous mathematics to be realised efficiently within digital architectures.

Continuous formulations provide important conceptual advantages because they describe information as evolving smoothly over time rather than through isolated computational events. Such representations closely resemble many natural processes, including speech, biological activity, environmental change and human cognition. When transformed appropriately into discrete computational procedures, these continuous models preserve their underlying dynamical properties whilst becoming compatible with efficient neural computation. Consequently, State Space Models combine the mathematical elegance of continuous systems with the practical requirements of modern Artificial Intelligence.

This relationship between continuous and discrete representations has become increasingly important as researchers seek architectures capable of processing exceptionally long sequences without losing contextual coherence. Unlike conventional recurrent models, whose memory frequently deteriorates as sequences lengthen, carefully designed State Space Models maintain stable representations over extended computational intervals. This stability derives directly from the underlying mathematical formulation rather than emerging solely through empirical optimisation, providing one of the principal theoretical strengths of the architecture.

The integration of continuous dynamical theory with discrete deep learning also illustrates a broader trend within contemporary Artificial Intelligence. Progress increasingly depends upon synthesising established scientific disciplines with modern computational techniques. State Space Models demonstrate how concepts originating within applied mathematics and engineering may be adapted successfully to address the challenges confronting large-scale neural computation.

Neural Parameterisation and Efficient Parallel Processing

The contemporary revival of State Space Models has been driven by the recognition that classical formulations, although mathematically elegant, required significant modification before becoming suitable for modern deep learning. Recent research has therefore focused upon redesigning state space architectures capable of exploiting graphical processing hardware efficiently whilst preserving the favourable mathematical properties associated with continuous dynamical systems.

Modern State Space Models differ substantially from their predecessors through the incorporation of learnable neural parameterisation, highly efficient convolutional computation and optimisation techniques specifically designed for large-scale Artificial Intelligence. Rather than relying upon manually specified system dynamics, contemporary models learn optimal state transitions directly from extensive collections of information through gradient-based optimisation. This combination of rigorous mathematical structure and flexible statistical learning represents one of the principal innovations responsible for the recent success of these architectures.

The development of specialised computational algorithms has also transformed practical implementation. Earlier sequential models processed information one observation at a time, limiting opportunities for computational parallelism. Contemporary State Space Models employ mathematical transformations permitting substantial parallel execution whilst preserving the sequential relationships fundamental to temporal reasoning. Consequently, they combine many of the computational advantages previously associated primarily with Transformer architectures whilst maintaining superior efficiency for exceptionally long sequences.

These developments have significantly broadened the potential applications of State Space Models. They are no longer viewed merely as specialised sequential algorithms but increasingly as general-purpose neural architectures capable of supporting language modelling, scientific computation, biological sequence analysis, robotics and numerous other forms of intelligent information processing.

Adaptive Selection Within Evolving Internal States

Among the most influential developments within contemporary research has been the emergence of Selective State Space Models. Traditional State Space Models process incoming information according to fixed dynamical rules, applying identical computational procedures regardless of the content or significance of individual observations. Although mathematically consistent, such uniform processing may prove inefficient because not every element within a sequence deserves equal attention or long-term retention.

Selective State Space Models address this limitation by introducing adaptive mechanisms capable of determining dynamically which information should influence the evolving internal state. Rather than updating memory identically for every observation, the model learns to preserve information of continuing importance whilst allowing less relevant details to diminish naturally over time. This capability introduces an important element of computational selectivity analogous to attentional processes observed within biological cognition.

The emergence of selective processing has significantly strengthened the competitiveness of State Space Models relative to attention-based architectures. Instead of explicitly comparing every element with every preceding observation, selective state transitions determine internally which information should remain influential throughout future computation. This selective memory enables exceptionally long contextual reasoning whilst preserving computational efficiency.

Selective Retention and Intelligent Memory

Such developments also reflect broader principles governing intelligent behaviour. Effective cognition depends not upon remembering everything equally but upon retaining information that remains relevant whilst allowing transient details to fade appropriately. Selective State Space Models therefore provide an increasingly sophisticated computational approximation of adaptive memory processes central to intelligent reasoning.

Compressed Memory Across Extended Sequences

Memory constitutes one of the defining challenges confronting Artificial Intelligence. Many intellectual tasks require information presented thousands or even millions of computational steps earlier to remain available during subsequent reasoning. Conventional sequential neural architectures frequently struggle with such long-range dependencies because information gradually degrades as processing continues. Transformer architectures addressed this challenge through attention mechanisms enabling direct comparison across complete input sequences, although at considerable computational expense.

State Space Models approach memory from an entirely different perspective. Rather than storing previous observations explicitly, they maintain continuously evolving internal representations summarising accumulated contextual knowledge. Incoming information modifies these representations according to carefully designed state transition functions, allowing relevant historical information to remain accessible over extended intervals without preserving every observation individually.

This approach offers several important advantages. Computational complexity remains approximately proportional to sequence length, permitting efficient processing of exceptionally long information streams. Memory also becomes inherently dynamic because internal representations evolve continuously in response to changing contextual requirements. Rather than functioning as static repositories of previous observations, State Space Models maintain living representations capable of adapting throughout ongoing computation.

Long-range dependency modelling consequently becomes an emergent property of the evolving system state. Linguistic context, scientific relationships, causal structures and temporal patterns may all remain influential across extensive computational intervals without requiring exhaustive comparison among individual observations. Such capability is particularly valuable for document analysis, scientific literature, genomic research, financial forecasting and numerous other domains characterised by extensive sequential information.

Recent empirical investigations suggest that carefully designed State Space Models may preserve meaningful contextual information across substantially longer sequences than many conventional architectures whilst requiring significantly reduced computational resources. These observations have stimulated considerable interest in their potential role within future generations of large-scale Artificial Intelligence.

Linear Complexity and Scalable Context Processing

Computational efficiency has become one of the defining considerations influencing contemporary Artificial Intelligence research. The extraordinary success of increasingly large neural architectures has been accompanied by rapidly escalating demands for computational infrastructure, energy consumption and financial investment. Consequently, architectural innovation now focuses not solely upon improving predictive performance but also upon enhancing computational efficiency and scalability.

State Space Models contribute significantly to these objectives through their fundamentally different computational organisation. Because internal states evolve incrementally rather than requiring exhaustive pairwise comparison among sequence elements, computational complexity increases approximately linearly with sequence length. This contrasts sharply with the quadratic complexity associated with conventional attention mechanisms, particularly when processing extensive contextual information.

The implications extend beyond theoretical efficiency. Lower computational complexity permits larger contextual windows, reduced memory requirements and more practical deployment across diverse computational environments. Scientific organisations processing extensive observational data, enterprises analysing continuous operational information and autonomous systems operating within dynamic environments all benefit from architectures capable of sustaining efficient long-term reasoning without prohibitive computational expense.

Scalability similarly improves because expanding contextual length no longer requires proportional expansion in computational workload. Researchers may therefore investigate increasingly sophisticated applications involving extensive scientific datasets, legal archives, engineering documentation and continuously generated sensor information without encountering the severe computational limitations associated with many earlier neural architectures.

These characteristics suggest that State Space Models occupy an increasingly important position within the evolving landscape of Artificial Intelligence. Rather than replacing existing architectures entirely, they provide an alternative computational paradigm particularly well suited to applications requiring efficient long-range memory, continuous reasoning and scalable sequential computation.

Complementary Strengths of State Space and Attention Models

The emergence of State Space Models has inevitably invited comparison with Transformer architectures, which have dominated Artificial Intelligence research since the introduction of the attention mechanism. Although both architectural paradigms seek to model complex sequential information, they differ fundamentally in how contextual knowledge is represented, maintained and utilised throughout computation. Understanding these differences is essential for appreciating the strategic significance of State Space Models within the future development of Artificial Intelligence.

Transformer architectures represent contextual relationships explicitly through self-attention, allowing every element within an input sequence to compare itself directly with every other element. This mechanism provides exceptional flexibility because relevant contextual information may be retrieved regardless of its position within the sequence. Consequently, Transformers have demonstrated remarkable success across language understanding, reasoning, translation, software engineering and multimodal learning.

This flexibility, however, carries significant computational cost. Since every input element interacts with every other element, computational and memory requirements increase quadratically as sequence length expands. While this characteristic remains manageable for comparatively short contexts, processing documents extending across hundreds of thousands or millions of tokens rapidly becomes computationally expensive. Increasing context windows therefore requires substantial computational infrastructure and correspondingly greater energy consumption.

State Space Models approach the same challenge from an entirely different perspective. Rather than constructing explicit attention relationships among all observations, they preserve contextual understanding through continuously evolving internal representations. Historical information is incorporated into the system state as computation progresses, allowing relevant knowledge to influence future reasoning without repeated pairwise comparison. Computational complexity therefore increases approximately linearly with sequence length, making State Space Models particularly attractive for applications involving extensive contextual information.

The distinction should not be interpreted as a competition in which one architecture necessarily replaces the other. Instead, each possesses complementary strengths. Transformer architectures remain exceptionally effective where flexible contextual interaction and parallel computation dominate performance requirements. State Space Models provide substantial advantages where efficient long-range memory, continuous sequential reasoning and computational scalability become primary objectives. Increasingly, researchers are exploring hybrid architectures capable of combining these complementary computational philosophies within unified intelligent systems.

Efficient Long-Context Architectures for Language Modelling

Large Language Models have transformed contemporary Artificial Intelligence through their ability to perform language understanding, reasoning, summarisation, software development and numerous other intellectual tasks within unified computational frameworks. Until recently, these systems have depended predominantly upon Transformer architectures. The emergence of State Space Models has introduced the possibility that future language models may employ fundamentally different computational organisations whilst maintaining or even extending current capabilities.

Language processing presents particular challenges because meaning frequently depends upon relationships extending across very long textual contexts. Scientific publications, legal documents, technical manuals and historical archives often require reasoning over information distributed throughout hundreds of pages. Transformer architectures can address these problems by enlarging context windows, although computational expense increases correspondingly.

State Space Models offer an alternative solution by maintaining evolving contextual representations capable of preserving relevant historical information throughout extensive documents. Rather than remembering every preceding word individually, the model develops compressed internal representations capturing the conceptual relationships most relevant for subsequent reasoning. This capability enables efficient processing of exceptionally long textual sequences whilst reducing computational demands.

Such characteristics suggest considerable strategic importance for future generations of Large Language Models. As demand increases for systems capable of analysing complete books, scientific literature, legislative archives and enterprise knowledge repositories, architectures combining extensive contextual memory with computational efficiency will become increasingly valuable. State Space Models therefore represent an important candidate for supporting the next generation of large-scale language processing.

Moreover, their mathematical foundations permit continual contextual evolution rather than repeated reconstruction of complete attention patterns. Consequently, language models based upon State Space Models may prove particularly well suited to ongoing conversations, continuous document analysis and applications requiring persistent contextual understanding extending across prolonged interactions.

Sequential Intelligence Across Science and Enterprise

The practical significance of State Space Models extends well beyond language modelling. Numerous scientific and enterprise domains require efficient processing of extensive sequential information whilst maintaining coherent contextual understanding over prolonged periods. These requirements align closely with the strengths offered by State Space Models.

Scientific research provides numerous examples. Genomic sequencing generates extremely long biological sequences whose interpretation depends upon complex interactions extending across extensive regions of genetic material. Climate science similarly requires analysis of continuous environmental observations collected over prolonged periods, while astronomy processes enormous streams of observational data originating from distributed telescopic systems. State Space Models provide computational frameworks capable of preserving long-term contextual relationships throughout these extensive datasets without incurring prohibitive computational costs.

Engineering applications similarly benefit from efficient sequential modelling. Industrial monitoring systems continuously generate operational information describing equipment performance, environmental conditions and maintenance requirements. Intelligent analysis of these data streams requires architectures capable of identifying evolving patterns whilst retaining historical operational context. State Space Models naturally support such requirements through continuously updated internal representations reflecting accumulated system behaviour.

Financial institutions increasingly analyse extensive streams of market activity, economic indicators and transactional information. The ability to preserve meaningful temporal relationships over prolonged intervals whilst adapting continuously to changing market conditions represents a significant strategic advantage. State Space Models provide efficient mechanisms for modelling such evolving environments without requiring exhaustive comparison among all historical observations.

Healthcare also represents an important application domain. Patient histories frequently extend across many years and include laboratory investigations, imaging studies, clinical consultations, pharmaceutical treatments and physiological monitoring. Intelligent clinical decision support depends upon integrating these diverse temporal observations into coherent representations supporting diagnosis, prognosis and treatment planning. State Space Models offer promising computational frameworks for managing such complex longitudinal information whilst maintaining computational practicality.

These examples demonstrate that the significance of State Space Models extends beyond any individual application domain. Their architectural principles address fundamental challenges associated with sequential reasoning, contextual memory and computational scalability encountered throughout numerous scientific, industrial and organisational environments.

Hybrid, Multimodal and Hardware-Optimised Research Frontiers

Research into State Space Models is progressing rapidly as investigators seek to extend their theoretical foundations, computational efficiency and practical applicability. One of the most active areas concerns increasingly sophisticated selective state mechanisms capable of determining dynamically which information should be retained, strengthened or discarded throughout ongoing computation. Improvements in adaptive memory management are expected to enhance reasoning capability whilst preserving computational efficiency.

Hybrid State Space and Attention Architectures

Another important direction involves hybrid neural architectures combining State Space Models with attention mechanisms. Rather than viewing these paradigms as mutually exclusive, researchers increasingly investigate systems in which attention provides flexible local reasoning while State Space Models maintain efficient long-range contextual memory. Such hybrid architectures may ultimately provide the most effective balance between representational flexibility and computational scalability.

Multimodal Artificial Intelligence constitutes another expanding research frontier. Future intelligent systems must integrate language, visual perception, sound, sensor information and structured numerical data within unified computational frameworks. State Space Models possess significant potential for representing temporal relationships across these diverse information modalities, supporting richer contextual understanding than isolated modality-specific architectures.

Research also increasingly investigates hardware optimisation. As State Space Models become more widely adopted, specialised computational implementations are being developed to exploit their favourable mathematical properties more efficiently. Such optimisation is expected to strengthen both training performance and operational deployment across increasingly diverse computational environments.

Longer-term investigation focuses upon increasingly general sequential intelligence. Contemporary State Space Models already demonstrate impressive capabilities for maintaining contextual continuity across extensive sequences. Future developments may extend these principles towards continually learning systems capable of adapting their internal dynamical representations throughout prolonged operational lifetimes. Such capability would represent an important step towards Artificial Intelligence exhibiting persistent memory, continual learning and adaptive reasoning across changing environments.

State Space Models as Foundations for Scalable Sequential Intelligence

State Space Models represent one of the most important architectural innovations within contemporary Artificial Intelligence because they reintroduce mathematically principled dynamical systems into the centre of modern neural computation. By replacing explicit pairwise attention with continuously evolving internal representations, these architectures provide an elegant solution to one of the defining challenges confronting large-scale Artificial Intelligence: the efficient modelling of extensive sequential information without prohibitive computational complexity.

This white paper has explored the historical origins of State Space Models, demonstrating how concepts originating within systems engineering, applied mathematics and control theory have been transformed into powerful computational architectures supporting modern deep learning. Their contemporary resurgence illustrates that progress within Artificial Intelligence increasingly depends upon integrating diverse scientific traditions rather than relying upon isolated technological innovation.

The analysis has shown that State Space Models differ fundamentally from Transformer architectures in their treatment of contextual memory, sequential reasoning and computational organisation. While attention mechanisms remain exceptionally powerful for many forms of contextual interaction, State Space Models provide substantial advantages where long-range dependency modelling, continuous memory and computational scalability become dominant considerations. Their approximately linear computational complexity offers particularly significant advantages as Artificial Intelligence increasingly addresses extensive scientific, industrial and organisational information.

The growing relevance of State Space Models extends across Large Language Models, enterprise computing, scientific research, healthcare, engineering and numerous other application domains requiring efficient sequential intelligence. Their capacity to preserve evolving contextual representations without exhaustive pairwise comparison provides a compelling foundation for future intelligent systems capable of sustained reasoning across exceptionally large information environments.

Looking ahead, continued advances in selective state mechanisms, hybrid neural architectures, multimodal integration and specialised computational optimisation suggest that State Space Models will occupy an increasingly important position within the future landscape of Artificial Intelligence. Rather than representing a temporary alternative to existing neural architectures, they embody a broader computational philosophy in which intelligent behaviour emerges through stable, adaptive and continuously evolving internal representations. As Artificial Intelligence continues progressing towards increasingly capable, efficient and contextually aware systems, State Space Models are likely to become one of the principal architectural foundations supporting the next generation of intelligent computational technologies.

Bibliography

• Dao, T. and Gu, A. (2024) ‘Transformers are State Space Models: Generalised Models and Efficient Algorithms’, Proceedings of the International Conference on Machine Learning.
• Gu, A., Goel, K. and Ré, C. (2022) ‘Efficiently Modelling Long Sequences with Structured State Spaces’, International Conference on Learning Representations.
• Gu, A. and Dao, T. (2024) ‘Mamba: Linear-Time Sequence Modelling with Selective State Spaces’, International Conference on Learning Representations.
• Kailath, T. (1980) Linear Systems. Englewood Cliffs, New Jersey: Prentice Hall.
• Kalman, R.E. (1960) ‘A New Approach to Linear Filtering and Prediction Problems’, Journal of Basic Engineering, 82(1), pp. 35-45.
• Ljung, L. (1999) System Identification: Theory for the User. 2nd edn. Upper Saddle River, New Jersey: Prentice Hall.
• Oppenheim, A.V. and Schafer, R.W. (2010) Discrete-Time Signal Processing. 3rd edn. Upper Saddle River, New Jersey: Prentice Hall.
• Särkkä, S. (2013) Bayesian Filtering and Smoothing. Cambridge: Cambridge University Press.
• Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł. and Polosukhin, I. (2017) ‘Attention Is All You Need’, Advances in Neural Information Processing Systems, 30, pp. 5998-6008.