Hi there - If you aren't excited about living in DC, aren't comfortable living/working in a gritty neighborhood and/or feel out of place on a campus that's 99% African American (I'm assuming you know that Howard is a prominent HBC - historically black college), you will be pretty unhappy. I've lived in DC for a decade, love the ease of public transportation, the ethnic restaurants (there's a delicious "soul veg" spot across from Howard that's run by black, vegan, Ethiopian Jews - no joke!), the crowds and the mix of cultures. The Howard/Shaw neighborhood is cheaper than most and improving...but it is one of the city's more rough-and-tumble neighborhoods, the area around campus looks really run down, and the Howard area is not somewhere I'd walk alone or at night. DC's housing is also incredibly expensive (1 bedrooms from $1400 to $3200), so the cost of living is a lot more than you'd see elsewhere.
The city is an unbelievable perk for people who love it...but would make for a miserable 5 years if you're not happy here. As a fellow small-town PA girl, I'd recommend visiting the school if you haven't already, walking around the neighborhood, taking the bus and metro and seeing if you'd feel comfortable. It sounds to me like you'll be happier somewhere smaller and less urban, so don't be afraid to wait for the right fit. If you got into one PhD program, you'll get into another!