Remember, we are not going to graduate school to enjoy the campus, the community, or the local culture. Of course, it's a big bonus to feel that those things excite you too, but we can't have it all, and ultimately you should commit to an institution because you believe you can love and explore your field the way you want to most at that school. I believe that the way you said you felt when you visited your future undergraduate campus, which you described as an epiphany moment; is how you should feel about your field (not necessarily the school). That's how you should feel about the professors you plan to work with, and about your own abilities and future if you enter that particular program. I don't know if I'm trying to convince you or me, so take this for what it's worth. With that said, If it truly feels wrong to you, then in all likelihood, it is wrong for you.