I grew up in Boston (actually a city right next to Tufts) and I also was a Tufts undergrad. If you're in Boston, then you're almost in the among the echelons of Harvard and MIT. I've had a lot of people from Boston mistake Tufts for an Ivy League school. Despite being a college town though, it is really hard to find Tufts t-shirts and gear outside of the campus (unlike Harvard). Tufts is famous for its medical school and has a medical center (and subway stop) named after it near downtown which would explain its reputation in the city. Otherwise, it might be recognized for its engineering program or biomedical research.
Outside of Boston, Tufts is usually recognized in the northeastern states. Outside of those states, you might only get recognition among educated folks and employers in specialized fields. Tufts people are everywhere, hidden all over the place though. I love getting random shout-outs while wearing Tufts gear.