It just depends on which ranking system. US News? All they did was ask people about their opinions, I believe. I don't think they used any objective indices. I'd give it about 5% of the "meaningful" outcome variance of what matters, if we're putting statistics to it. If that.
I'd be more interested in things like number of publications, impact factors of the publications typically pursued by the department, collaborative relationships among other institutions/departments, how active professors are in influential organizations (e.g. presidents of certain groups), grants, certification/dual degree opportunities, etc. These are more objective and, in my opinion, far more relevant than US News rankings.
For the record: I am interviewing at UNC Chapel Hill, which is ranked #2 by US News. My "less rational" side of me is clearly very influenced by that, but objectively I know it shouldn't really matter since they earned that #2 spot based on opinions.