In many statistical problems, incorporating priors can significantly improve performance. However, using prior knowledge in differentially private query release has remained underexplored, despite such priors commonly being available in the form of public data, such as previous US Census releases. With the goal of releasing statistics about a private dataset, we present PMW^Pub, which---unlike existing baselines---leverages public data drawn from a related distribution as prior information. We provide a theoretical analysis and an empirical evaluation on the American Community Survey, showing that PMW^Pub outperforms state-of-the-art methods. Furthermore, our method scales well to high-dimensional data domains, where running many existing methods would be computationally infeasible.