I completely get where you're coming from, however Berkeley is just as well regarded as MIT inside the field of physics. I think the best way to go about this is to ask yourself about what the goal of your PhD is. Are you envisioning it as a tool to get a job in finance? Or do you want to pursue industry/academia? I think the answer will guide you to determining the path that's most conducive to reaching your goal. That's not to say that you'll never change your mind in the process. Also you mention you'll be happier at MIT because of the people, but will the research be good enough as well, because that will be your primary focus for 5-7 years. I've known a handful of grad students at my current school who came in wanting to work in high energy theory and they ended up having to leave 3-4 years in because they couldn't find a faculty advisor (I should add that our physics department is one of the best in the world too) so even at good schools, making sure that you'll have someone to work with and look out for you is important.
Just my two cents. You have great options and ultimately I don't think there's a wrong choice. GL!