I'm not insinuating much about New Haven, but when I went there just to visit for a couple of days, I definitely did not get the very safe cozy feeling that the residents seem to claim (though I might have been just unlucky). Cutting through the crap, when I arrived, I had a very short distance to walk to the hotel (very close to campus), and I just about got mugged. I say "just about" because he only got out to give him my money and credit cards before a cop trolled down the street and he scrammed. Oh, and then when I toured the physics building, I talked to some professor about New Haven, and he joked about how a lot of people living off the west side of campus had been "dying of lead poisoning", if you get his meaning.
Yes, I know, I was walking alone that night, and it's one thing to joke about crime near by and another to have to worry about it. Still though, it really detracted from the attractiveness of the locale as opposed to a place with good restaurants and night life sans the thrill of danger. I could do without that thrill.
That being said, I'm from South Chicago, and if you like it in places like that and get by well enough, then you'll do plenty fine in New Haven. It's no better, no worse.
Good lord though, the campus itself is beautiful, and of course, you feel quite safe when there.