Learning from on-device data has enabled intelligent mobile applications ranging from smart keyboards to apps that predict abnormal heartbeats. However, due to the sensitive nature of this data, expert annotation is seldom available. Consequently, existing federated learning techniques that learn from on-device data are unable to capture expert knowledge via data annotations, mostly relying on unsupervised approaches. In this work, we explore an alternative way to codify expert knowledge: using programmatic weak supervision, a principled framework that leverages labeling functions in order to label vast quantities of data without direct access to the data itself. We introduce Weak Supervision Heuristics for Federated Learning (WSHFL), a method to interactively mine and leverage labeling functions that annotate on-device data in cross-device federated settings. Experiments on two sentiment classification tasks show that WSHFL is both efficient and effective at these tasks.