Loading [MathJax]/jax/output/CommonHTML/jax.js
Skip to yearly menu bar Skip to main content


In-Person Poster presentation / top 25% paper

Associative Memory Augmented Asynchronous Spatiotemporal Representation Learning for Event-based Perception

Uday Kamal · Saurabh Dash · Saibal Mukhopadhyay

MH1-2-3-4 #43

Keywords: [ Associative Memory ] [ event-based camera ] [ set processing ] [ memory augmented neural network ] [ spatiotemporal representation ] [ event-based perception ] [ attention ] [ object recognition ] [ Applications ]


Abstract: We propose EventFormer, a computationally efficient event-based representation learning framework for asynchronously processing event camera data. EventFormer treats sparse input events as a spatially unordered set and models their spatial interactions using self-attention mechanism. An associative memory-augmented recurrent module is used to correlate with the stored representation computed from past events. A memory addressing mechanism is proposed to store and retrieve the latent states only where these events occur and update them only when they occur. The representation learning shift from input space to the latent memory space resulting in reduced computation cost for processing each event. We show that EventFormer achieves 0.5% and 9% better accuracy with 30000× and 200× less computation compared to the state-of-the-art dense and event-based method, respectively, on event-based object recognition datasets.

Chat is not available.