Multimodal modelling has seen great interest in recent years with fantastic results and applicability over a wide range of tasks. A particular feature of such applicability has been the development of conditional generation, and the chaining of such conditional models to generate cross-modally. This however has meant that the question of representations, and what being cross-modal entails, has been eschewed in favour of high generative quality---something that leaves things as black-boxes from the perspective of human inspection and interpretability. In this talk, I will touch upon some recent and ongoing work in our lab towards learning unsupervised models that capture structured representations, which can be constrained across modalities to address questions of interpretability through multimodal grounding.