In various fields of natural science, there exists a high demand for accurate simulations of physical systems. For example, the weather forecasting requires a large-scale simulation of physical systems described by partial differential equations (PDEs). To reduce the computational cost, recent studies have attempted to build a coarse-grained model of the systems by using deep learning. Many training strategies for deep learning have developed for images or natural languages, but they are not necessarily suited for physical systems. A physical system demonstrates similar phenomena in most points (e.g., sunny days) but exhibits a drastic behavior occasionally (e.g., typhoons). Roughly speaking, a physical system dataset suffers from the class imbalance, whereas previous studies have rarely focused on this aspect. In this paper, we propose an imbalance-aware loss for learning physical systems, which resolves the class imbalance in a physical system dataset by focusing on the hard-to-learn parts. We evaluated the proposed loss using physical systems described by PDEs, namely the Cahn-Hilliard equation and the Korteweg-de Vries (KdV) equation. The experimental results demonstrate that models trained using the proposed loss outperform baseline models with a large margin.