Columbia offers graduate student (subsidized) housing to its PhD students. The rent is much lower than market rates, and is generally within about 1/2 mile of campus. If you'd rather not live on campus, you don't have to--but once you're out of the housing system, it's very hard to get back it. My advice is to apply for housing when they offer it--you can always turn it down later if you decide against. I'd recommend against the dorms, since you'll live mostly with undergrads--unless this appeals to you. You'll probably have a roommate, generally another grad student, who could be great or terrible depending. But if it doesn't work out, you always have the option to move--either within the CU housing system or to somewhere "off campus." The leases aren't binding--you can break the lease any time if you want to move.
The housing office tries to accommodate lifestyles (e.g. being kosher), although they can't guarantee it. Nonetheless, there's really no harm in applying.