I'll be applying for PhD programs in the fall as I finish my masters thesis. If all the stars align, I'll be starting my PhD at age 42, so I guess that puts me in the geriatric ward. To answer the original question, I'd bet money that there are far more of us old codgers pursuing PhD's than you might think. Not a thing wrong with it, and I personally couldn't do it any other way.
After getting my BA, I wanted nothing to do with graduate school. I landed a help desk job that developed into a software developer career. I make pretty good money doing it and the job is about as stable as anyone can expect given the current economic climate. The problem? I burned out about ten years ago -- and I'm still doing it.
I took a few English classes a few years back at a local university because I was listless and I missed academia. At the time I didn't realize how much. I eventually decided I had to give the tenured professor schtick a shot or I'd regret it.
I say all this because I'm aware of the odds, and to say that they are daunting is like standing in a mosh pit and saying, "Gee, this is loud." I figure that knowing exactly what I want, including a very specific research focus, helps mitigate the risks somewhat, but that just makes what I want to do slightly less ridiculous. That said, I'm not quite huffing the magic pixie dust either, since I know I can convert the PhD into a technical writing/editing gig with my previous software experience or (worst case) go back to software development. There are options, and I sorely need a change. I'm sure not happy doing what I'm doing now. So to answer Swagato, this is why I'm forging ahead with this insanity.
I think it's important to go for broke and do everything in your power to pursue what you love. But plan B's, as cliche as they are, help take some of the pressure from plan A's. And anything that makes a plan A more viable is a good thing.