I think this decision should made by thinking about how "ready" you are to dive into the research. Since you mentioned that you have to change fields, I would think you need to prepare by taking courses, learning about the field, etc.
I would suggest taking the fellowship + TA. It's okay that you do not have an advisor assigned, you will eventually once you get to the campus. Generally, its easier for a professor to take you into their group if you are already funded by other means. This way, they can test you out before giving you an RA (less risky) and you can configure out if they are the right fit for you.
Take the offer from University A if you don't feel confident in just jumping into research that you don't know much about, and if you want to be a little more relaxed the first year or so. Take the offer from University B if you feel confident to jump into research and if you really dislike teaching.