Bayesian vs Classical Statistics

[Cavalerius]

It seems that certain institutions (e.g., Duke and UT-Austin) are heavily Bayesian. I suppose that if one were to attend one of these institutions, then one's research would be Bayesian in nature. Has anyone with little experience in Bayesian statistics and uncertainly as to whether they wished to adopt a Bayesian perspective attended any such institution? If so, were you ultimately satisfied with the research and methods?

Edited January 31, 2019 by Cavalerius

[Stat Assistant Professor]

I did my PhD research on Bayesian statistics and I am still continuing to work on that in my current postdoc. I think there are a lot of exciting developments and directions for research in this field, both theoretical and methodological/applied. In particular, a major direction of research right now is making Bayesian methods more computationally tractable and scalable to massive datasets (something that has traditionally been a challenge for implementations based on MCMC). This is pursued through various alternatives to traditional MCMC like variational inference, sequential Monte Carlo, etc.

A lot of it is a matter of taste, I suppose. Some people like the Bayesian perspective, since you get a posterior distribution of the parameters and not just a point estimate. So you can get automatic uncertainty quantification from the posterior density *and* point estimates (posterior mean, median, or mode) under a unified framework. Whereas in "classical" statistics, the uncertainty quantification is based on long-term coverage rather than on actual probabilities (e.g. "if we repeated this experiment a large number of times and constructed these confidence intervals, 95% of the confidence intervals would contain the true parameter").

[Gauss2017]

I think a lot of ML and AI is frequentist based but the wave of the future is to use causal inference in ML and AI

[Cavalerius]

Thank you, Stat PhD Now Postdoc Gauss2017, for your opinions and insights. It seems that at many top institutions there are now a fair number of Bayesians, and so, assuming that one is able to work with a faculty member doing research of this type, there is the potential to do Bayesian research at any number of schools. Nevertheless, at a school like Duke, there really seems to be no choice but to do Bayesian research.

Because I am still deciding what type of research I would like to, I would not necessarily want to be limited in my exposure to both classical and Bayesian statistics, but I also do not want to miss the chance to do promising research at a school that focuses primarily on just one or the other, especially if in the future the Bayesian perspective becomes the dominant one.

[Stat Assistant Professor]

20 minutes ago, Cavalerius said:

Thank you, Stat PhD Now Postdoc Gauss2017, for your opinions and insights. It seems that at many top institutions there are now a fair number of Bayesians, and so, assuming that one is able to work with a faculty member doing research of this type, there is the potential to do Bayesian research at any number of schools. Nevertheless, at a school like Duke, there really seems to be no choice but to do Bayesian research.

Because I am still deciding what type of research I would like to, I would not necessarily want to be limited in my exposure to both classical and Bayesian statistics, but I also do not want to miss the chance to do promising research at a school that focuses primarily on just one or the other, especially if in the future the Bayesian perspective becomes the dominant one.

A couple things:

1) Even if a department leans more heavily in one direction (Bayesian vs. frequentist), your coursework will most likely expose you to "classical" statistics. Actually, at my PhD program (which leans more heavily towards Bayesian), most of the core PhD statistics classes (linear models, GLMs, inference, mathematical statistics, design of experiments) were taught almost exclusively from within the realm of "classical"/frequentist statistics, and the Bayesian courses were only offered as electives. I had to teach myself the Bayesian stuff for my research. 

2) In your research, you will most likely need to do a ton of reading and surveying literature/books when you first start out, and then, it will be necessary to expose yourself to both frequentist and Bayesian strategies. Even though my research was on Bayesian statistics for high-dimensional data, I still had to learn all about LASSO, ridge regression, SCAD, MCP, etc. (all of which are frequentist methods for dealing with high-dimensional problems). If you want to learn something like Bayesian deep learning or Bayesian dimension reduction, you will almost certainly need to be somewhat familiar with neural networks, PCA, etc. from the frequentist perspective. Even if your thesis research is on Bayesian, you will no doubt have to learn the frequentist analogues and strategies for solving the same problems, and it is not that hard to pick up on it.

3) You can always change your research area and teach yourself a new area later on if you find that you prefer one approach over another. A PhD is just something to certify that you were able to conduct academic research at a bare minimum publishable level, but it does not prevent you from moving on to something totally different later. My PhD advisor's thesis research was on frequentist nonparametrics and he continued to do stuff like that for awhile, but in the late 1980s/early 1990s, he became a Bayesian. One of his former PhD students also no longer works on Bayesian statistics in her postdoc, but instead, on recommender systems and process data from a frequentist point of view.

Edited January 31, 2019 by Stat PhD Now Postdoc

[Cavalerius]

Wow, Stat PhD Now Postdoc, thank you for the detailed analysis. I was not sure when I asked the question what to expect, but your comments have been extremely helpful in giving me a better perspective on this matter. Thanks so much!

[bayessays]

Just a few caveats to StatPhdNowPostdoc, who gave great advice as always and is very knowledgeable.  I believe he went to Florida, which while more Bayesian-leaning than most, is not nearly as much so as Duke or UT Austin. At both of these schools, even the coursework is mostly oriented to Bayesian statistics from the beginning. You'll obviously get some of the basics of more classical statistics too, but the curricula are vastly different. UT has at least one professor who is not a Bayesian, though I don't think the same is true of Duke.  These places will also likely be populated by students who are mostly interested in Bayesian statistics - whether for philosophical reasons, because it's the cool thing to do, or whatever reason.  There is a history of very strong opinions in the Bayesian vs frequentist debate.  If you go to one of these two schools, I would recommend that it's because either 1) you really exclusively want to be a Bayesian or 2) you are so philosophically ambivalent that you realize all of statistics is just tools made up to solve problems, and you are ok with spending a few years focusing on just one set of those tools and not learning the others. 

There are very few young die-hard frequentists, but if you might become one, I would suggest not going to one of these programs. A Bayesian PhD is also incredibly likely to consist of a lot of very computational work, so I would be ready for that.

[Cavalerius]

Thanks, bayessays, that is good to know! I do not want to shy away from new challenges or turn down excellent programs (and as you intimate, the Bayesians seem to be the "cool kids"). My background is more mathematical than computational, so while I am not opposed to computation, I have always treated it as a means to an end and focused more on theory. I would probably classify myself as someone in the second category you mention. I would like to develop the best methods to solve the problems in which I am most interested, be those methods frequentist or Bayesian. For reference, my main interest is in stochastic processes.

[Gauss2017]

Those were two really great answers from Statnow and Bayssays.  I believe Stanford is considered mostly a frequentist school.  So you may want to look at how they approach things for comparison sake