Training self-driving systems to be robust to the long-tail of driving scenarios is a critical problem.Model-based approaches leverage simulation to emulate a wide range of scenarios without putting users at risk in the real world.One promising path to faithful simulation is to train a forward model of the world to predict the future states of both the environment and the ego-vehicle given past states and a sequence of actions.In this paper, we argue that it is beneficial to model the state of the ego-vehicle, which often has simple, predictable and deterministic behavior, separately from the rest of the environment, which is much more complex and highly multimodal.We propose to model the ego-vehicle using a simple and differentiable kinematic model, while training a stochastic convolutional forward model on raster representations of the state to predict the behavior of the rest of the environment.We explore several configurations of such decoupled models, and evaluate their performance both with Model Predictive Control (MPC) and direct policy learning.We test our methods on the task of highway driving and demonstrate lower crash rates and better stability.