I agree with everything you said, however, the difference in rankings is completely negligible.  Stanford, Cal, and MIT's rankings always swap through 1-3 depending on which ranking criteria and website you look at.   
 
As other people mentioned, fit/adviser/research is much more important.  
 
I would also like to point out, future careers.  If you are hoping to get a job in California/Semi-Valley, I am willing to bet that having a degree from Cal or Stanford is much more beneficial than having a degree from MIT.  The networking possibilities or internship opportunities will be far easier to come by and greater than if you were in MA.