If by better fit you mean that the program will lead to more happiness in the long-term - (you'll be ready to study exactly what you want and the other programs will not provide the same jumping-off point), then yes, I'd say you are crazy to turn it down because of 50,000 dollars (or $20,000 if masters).
If by better fit you mean that both positions will lead to the same future point and that the lower- paying university would just be nicer to work at for the next 5 years, then you aren't crazy. The difference in short-term happiness may not be worth 50,000.
Furthermore, depending on the TA-ship responsibilities (or other contigencies of the the funding), the issue of stress and time you spend not doing research can widen or narrow the gap. How much would you pay to be able to focus on your research and finish w/in 5 years with published papers?
Your favorite school is paying 20,000. Some other school is paying 30,000 a year. Is that other school your second choice, or is your second choice paying something more along the lines of 25,000? That would be a difference of $25,000 dollars, which seems very bearable.
Other things to consider. You'll always have the opportunity to apply for fellowships and make up the difference in money. Are there differences in income tax between the states? If the top choice state doesn't tax and the other state does, the difference in funding will be smaller than it seems. What about student fees? I'm at a university where fees cost nearly 2000/ year, and we have to pay them out of pocket.
