Discovering high-entropy alloys (HEAs) with high yield strength (YS) is an important yet challenging task in materials science. However, YS can only be accurately measured by very expensive and time-consuming real-world experiments, hence cannot be acquired at scale. Learning-based methods could facilitate the discovery process, but the lack of a comprehensive dataset on HEA yield strength has created barriers. We present X-Yield, a materials science benchmark with 240 experimentally measured ("high-quality") and over 100K simulated (imperfect or "low-quality") HEA yield strength data. Due to the scarcity of experimental data and the quality gap with imperfectly simulated data, existing transfer learning methods cannot generalize well on our dataset. We address this cross-quality few-shot transfer problem by leveraging model sparsification "twice" --- as a noise-robust feature learning regularizer at the pre-training stage, and as a data-efficient learning regularizer at the few-shot transfer stage. We then propose a bi-level optimization framework termed Bi-RPT that jointly learns optimal masks and automatically allocates sparsity levels for both stages. The effectiveness of Bi-RPT is validated through extensive experiments on our new challenging X-Yield dataset, alongside other synthesized testbeds. Specifically, we achieve an 8.9-19.8% reduction in terms of the test MSE and 0.98-1.53% in terms of test accuracy, merely using 5-10% of the hard-to-generate experimental data.