Totally having the same issue, so *bump*!
Even though small college towns often offer a number cultural activities, my concern - and maybe the original poster's - concern is that bigger cities just necessarily have more diversity and more diverse activities, and that goes a long way if that's an important part of your lifestyle.