Combining physics with machine learning is a rapidly growing field of research. Thus far, most of the work in this area focuses on leveraging recent advances in classical machine learning to solve problems that arise in the physical sciences. In this workshop, we wish to focus on a slightly less established topic, which is the converse: exploiting structures (or symmetries) of physical systems as well as insights developed in physics to construct novel machine learning methods and gain a better understanding of such methods. A particular focus will be on the synergy between the scientific problems and machine learning and incorporating structure of these problems into the machine learning methods which are used in that context. However, the scope of application of those models is not limited to problems in the physical sciences and can be applied even more broadly to standard machine learning problems, e.g. in computer vision, natural language processing or speech recognition.