I don't think there's any way for me (or anyone here, really) to give you a definitive answer, or even a good one. Nonetheless: If you are comfortable with the language of "calling" (and I suspect you are), do you feel "called" to pursue the vocation of an academic? If so, go for it.
One thing I will note, as a current PhD student and friend of many Christian professors: The balance you would like to strike as a professor and preacher will be significantly difficult to attain. The two vocations you feel drawn to, teaching and ministering, are both full-time gigs. Pastoral ministry (as you know) is much more than just preaching on weekends, just as teaching is much more than showing up to lecture on Tuesdays and Thursdays. What I'm trying to say is: be careful. That is a huge load you are looking to take on.