This was me before my second visit, tbh. But because of my graduate school naïveté, and coming from a small and poorly funded state school, I decided it was best for me to take into account what I wanted to get out of a graduate program (and the surrounding city, as well). Ultimately, at the end of my second visit (South Carolina), I immediately knew which program would be best for me to develop as a scholar. It had nothing to do with the faculty or students there, in fact they were all great, but it's just a matter of fit and where I'd be happiest (despite SD's high cost of living). I think taking that visit, with an open mind, will give you reassurance in the future about your decision!