The test has low power (efficiency) for moderate to large sample sizes. The Wilcoxon–Mann–Whitney U two-sample test or its generalisation for more samples, the Kruskal–Wallis test, can often be considered instead. The relevant aspect of the median test is that it only considers the position of each observation relative to the overall median, whereas the Wilcoxon–Mann–Whitney test takes the ranks of each observation into account. Thus the other mentioned tests are usually more powerful than the median test. Moreover, the median test can only be used for quantitative data.2
However, the null hypothesis verified by the Wilcoxon–Mann–Whitney U (and so the Kruskal–Wallis test) is not about medians. The test is sensitive also to differences in scale parameters and symmetry. As a consequence, if the Wilcoxon–Mann–Whitney U test rejects the null hypothesis, one cannot say that the rejection was caused only by the shift in medians. It is easy to prove by simulations, where samples with equal medians, yet different scales and shapes, lead the Wilcoxon–Mann–Whitney U test to fail completely.3
However, although the alternative Kruskal-Wallis test does not assume normal distributions, it does assume that the variance is approximately equal across samples. Hence, in situations where that assumption does not hold, the median test is an appropriate test. Moreover, Siegel & Castellan (1988, p. 124) suggest that there is no alternative to the median test when one or more observations are "off the scale."
Brown, GW; Mood, AM (June 1948). "Homogeneity of several samples". American Statistician. 2 (3): 22–23. JSTOR 2682086. https://www.jstor.org/stable/2682087?seq=1 ↩
http://psych.unl.edu/psycrs/handcomp/hcmedian.PDF [bare URL PDF] http://psych.unl.edu/psycrs/handcomp/hcmedian.PDF ↩
Divine, George W.; Norton, H. James; Barón, Anna E.; Juarez-Colunga, Elizabeth (2018-07-03). "The Wilcoxon–Mann–Whitney Procedure Fails as a Test of Medians". The American Statistician. 72 (3): 278–286. doi:10.1080/00031305.2017.1305291. ISSN 0003-1305. https://doi.org/10.1080%2F00031305.2017.1305291 ↩