Yeah, I totally get that. With scholarship, Columbia ended up being cheaper for me (including cost of living) that the other schools I applied to for a masters even though they were in smaller cities. And, I wasn't considering a PhD seriously at the time, so I wasn't considering funded programs really.
I think it is great to apply to CUNY! I keep telling myself not to make any decisions without the most information possible, and I just won't have the information about how much a school will really cost until I get in and see their funding package. That being said, I did not like living in NYC as a student. I don't live a fancy lifestyle, but I had to live in a neighborhood that was not designed for young professionals or English speakers to afford it and be close to school, going out for a drink didn't fit into my budget (because drinks are 10-18 dollars even at lunch), and sit down dinner definitely didn't either. So I lived in New York, but only experienced the noise and dirt and none of the food, museums, or culture. I was used to DC where museums are free and you can find cheap happy hour drinks at a nice place for $5. Or pregame on cheap traders joes wine then go to a bar and buy nothing. Trader Joe's in NYC (except 1) don't sell wine!!!! Also I didn't live near central park, and generally found the greenspace disappointing and crowded.