Actually the graduation rate of ICES is not that high (http://www.ices.utexas.edu/about/facts-figures/). Also the information I have is that each year they admit on average 15-16 PhD students and on average 4-4.5 finish it successfully. Of course it does not mean that a large number of students fail on the exams. There are millions of reasons not to complete a program. However, it is a bit concerning considering how competitive is the program and how good are the students they admit, and still they are not able to finish it. On the other hand, if you manage to graduate you are one of a few (the "market" is not flooded with ICES PhDs) which is very good.
I also think that ICES is a better choice compared to GaTech especially considering they give you more money. You must be very good that is why they offer you such good funding. Probably you are also an US citizen. Quite naturally, since it is the US, there are considerably more and nicer funding options for US citizens. I am an international candidate.
You are totally right that the question is what I want more since one of my choices is CS and the other is more like Applied Math (depending on the track you take). This introduces a good amount of differences.
So if I decide to go to ICES and you don't change your mind, we will meet there .