Every opponent of Mohammad Ali has gone to the wall in the contest with him so far.
Here, the underlined idiom means-