Thank you @juilletmercredi, I have really taken your comments to heart. And everyone else's. I will be visiting soon and I am hoping that it will give me enough of an insight, or at least an gut feeling about whether or not I would be happy at the programs. I see that you are working in industry - what did the rest of your cohort do? Did you feel generally supported in your program?
Thank you as well @hats. Maybe Temple was not a perfect example. The university ranks very highly for engineering and agriculture, but not high for my field. It is still a great fit for my research and I would receive a stipend and tuition waiver for five years. I was not thinking about prestige when I made my applications, I was thinking about research fit. The Ivy was a fluke: both a great fit and top 10 or so for my field. The others are ranked lower but well-regarded for my area of research. I would be pleased to attend any of them. I suppose all that remains is to visit and speak to current students.
What I really wanted from this thread was to hear that I would not be considered crazy for turning down an Ivy, especially for a lower ranked school. What I got is that it is fine as long as it is for the right reasons and not just out of fear of failure.