Oh, I'm really excited to hear everyone's answers to this.
I would ask...
1) How are advisors chosen? (if applicable, many of the schools I applied to, the DGS assigns the first advisor and then you have the opportunity to ask someone else for the dissertation phase)
2) Which courses do the TAs typically teach?
3) How much input, if any, do they have in regards to the courses they are assigned?
4) Does the department offer any funding for travel/conferences?
5) What kind of summer support, if any, does the program offer?
Obviously these will change if these questions are addressed online but the amount of information available on programs varies vastly in my experience.