I second all of the above..esp ones about department atmosphere, TA/RA workload and type, and town. Other stuff that might not occur to you at first glance:
--Related to atmosphere: What kind of relationship exists between cohorts? Between classes? Will you get support from advanced students? Chances to work closely with other departments (if this is important to your research)? Is faculty a consistent, supportive presence at student-run lectures and events or is that relationship more one-way?
--Follow up on class sizes for TA-ships or teaching assignments, as well as chances for non-teaching related funding. It will greatly reduce your workload to have small classes or instructorships alternating with fellowships, etc.
--the student health plan and quality of local medical care.
--Accessibility of airport / getting around town generally: Will you be able to get home conveniently and at a reasonable price however often you wish? Can you get away from campus for a weekend? (is there anywhere you would want to go?). Make sure there is somewhere around to avoid burnout, including places that will keep you away from your students (who may not yet have grasped the borders between your professional life and your personal life--I've been stopped while walking with my partner, in Starbucks, etc...). This would include opportunities for any social or personal stuff you like to do, whether or not you expect to have time for it.
--the political and religious atmosphere of both town and campus, if either is important to you.
One other thing to keep in mind for the sake of making things run smoothly--as a current student, I'd remind anyone taking this step that any student you're emailing has an absurd workload and doesn't know you from Adam's off ox. In general, avoid asking questions you could find the answer to on the department website (size of stipends, etc), open-ended "What should I do in case of x?) sorts of things related specifically to your application or decisions, or barraging one student with 100 questions. This is not to say we're not happy to answer questions, but be respectful of our time, introduce yourself a bit to give us some context (esp. since we may be seeing you in the fall!), and thank us when you're done please.