I have to say that I agree with you, now that my head is somewhat clear after receiving my first acceptance. Going purely off of the rankings, I saw that the "#1" program in my field has a professor with a Ph.D from the "#5" school. Perhaps the distance between #5 and #1 isn't very far or the rankings don't mean everything. If you're concerned about the ranking of your school, just check around to see what kinds of opportunities graduates from that program have. If you like what you see, then the ranking is just a meaningless number, right?