Will you be a teaching assistant or have your own courses that first year? If you haven't been an adjunct or had teaching experience yet (usually you may have some experience while doing an MA), it can be intimidating. I'm guessing there should be some sort of training somewhere. If you are expected to have your own ENGL 1 and ENGL 2 courses from the first semester, that seems a bit crazy to me.
I think a big factor is how you foresee yourself adjusting to grad school. Learning to write 15-25 page papers for each class and read a book per class per week can be a huge jump from undergrad experience. To add teaching service on top of that when students are just transitioning is a lot. A fellowship for the first year seems like the better deal to me and perhaps you can eventually seek opportunities at that school to at least TA for literature-heavy courses.
But ultimately, if the teaching experience is what you value more, and you think you are willing to push through the transition phase, then it's your decision.