Test-time adaption (TTA) has emerged as a new paradigm for reconciling distribution shifts between domains without accessing source data. However, existing TTA methods mainly concentrate on uni-modal tasks, overlooking the complexity in multi-modal scenarios.In this paper, we delve into the multi-modal test-time adaption and reveal a new challenge named reliability bias. Different from the definition of traditional distribution shifts, reliability bias refers to the information discrepancies across different modalities derived from intra-modal distribution shifts. To solve the challenge, we propose a novel method, dubbed reliable fusion and robust adaption (RFRA). On the one hand, unlike the existing TTA paradigm that mainly repurposes the normalization layers, RFRA employs a new paradigm that modulates the attention between modalities in a self-adaptive way, supporting reliable fusion against reliability bias. On the other hand, RFRA adopts a novel objective function for robust multi-modal adaption, where the contributions of confident predictions could be amplified and the negative impacts of noisy predictions could be mitigated. Moreover, we introduce two new benchmarks to facilitate comprehensive evaluations of multi-modal TTA under reliability bias. Extensive experiments on the benchmarks not only verify the effectiveness of our method but also give some new observations to the community. The code and benchmarks will be released.