Stat Assistant Professor

Even if you were to do well in your Masters program, I would consider UC Berkeley, Columbia, and Yale to be completely unrealistic. You have too many B's, and the competition for these schools is very stiff. Some of these schools only accept very few domestic applicants to begin with, and I'm afraid you won't be able to compete against applicants from Ivy schools, Stanford, UChicago, MIT, etc. with higher GPAs and possibly some solid research experience. 
 UC-Davis, UCLA, UC Irvine, and UT-Austin are reaches as well, IMO. It seems as though the UC schools are all very competitive, regardless of their rank (except for maybe UC-Riverside), because of their desirable locations. But I'm not sure how open UC-Davis would be to accepting their own Masters students as long as you perform well in their program, though -- that might be something to look into.
I would say that in order to be competitive for PhD programs, you have to get all A's in your Masters program, especially since you got a B in a graduate Statistics course. Definitely also take a full year of analysis and get A's to make up for your B in undergrad and possibly one or two other advanced math class (e.g. proof-based linear algebra) to show that you can succeed in math-heavy courses. The second year of a Statistics PhD program is pretty theoretical for the most part. If you do well in your Masters, you might be able to get into a program like TAMU or Iowa State. However, the most realistic schools would probably be those in the range of 37-80 of the USNWR rankings (i.e. those ranked below Yale). For your profile, I would consider TAMU and ISU to be the upper end of the schools you should be applying to for Statistics PhDs. 

Even if you were to do well in your Masters program, I would consider UC Berkeley, Columbia, and Yale to be completely unrealistic. You have too many B's, and the competition for these schools is very stiff. Some of these schools only accept very few domestic applicants to begin with, and I'm afraid you won't be able to compete against applicants from Ivy schools, Stanford, UChicago, MIT, etc. with higher GPAs and possibly some solid research experience. 
 UC-Davis, UCLA, UC Irvine, and UT-Austin are reaches as well, IMO. It seems as though the UC schools are all very competitive, regardless of their rank (except for maybe UC-Riverside), because of their desirable locations. But I'm not sure how open UC-Davis would be to accepting their own Masters students as long as you perform well in their program, though -- that might be something to look into.
I would say that in order to be competitive for PhD programs, you have to get all A's in your Masters program, especially since you got a B in a graduate Statistics course. Definitely also take a full year of analysis and get A's to make up for your B in undergrad and possibly one or two other advanced math class (e.g. proof-based linear algebra) to show that you can succeed in math-heavy courses. The second year of a Statistics PhD program is pretty theoretical for the most part. If you do well in your Masters, you might be able to get into a program like TAMU or Iowa State. However, the most realistic schools would probably be those in the range of 37-80 of the USNWR rankings (i.e. those ranked below Yale). For your profile, I would consider TAMU and ISU to be the upper end of the schools you should be applying to for Statistics PhDs. 

Even if you were to do well in your Masters program, I would consider UC Berkeley, Columbia, and Yale to be completely unrealistic. You have too many B's, and the competition for these schools is very stiff. Some of these schools only accept very few domestic applicants to begin with, and I'm afraid you won't be able to compete against applicants from Ivy schools, Stanford, UChicago, MIT, etc. with higher GPAs and possibly some solid research experience. 
 UC-Davis, UCLA, UC Irvine, and UT-Austin are reaches as well, IMO. It seems as though the UC schools are all very competitive, regardless of their rank (except for maybe UC-Riverside), because of their desirable locations. But I'm not sure how open UC-Davis would be to accepting their own Masters students as long as you perform well in their program, though -- that might be something to look into.
I would say that in order to be competitive for PhD programs, you have to get all A's in your Masters program, especially since you got a B in a graduate Statistics course. Definitely also take a full year of analysis and get A's to make up for your B in undergrad and possibly one or two other advanced math class (e.g. proof-based linear algebra) to show that you can succeed in math-heavy courses. The second year of a Statistics PhD program is pretty theoretical for the most part. If you do well in your Masters, you might be able to get into a program like TAMU or Iowa State. However, the most realistic schools would probably be those in the range of 37-80 of the USNWR rankings (i.e. those ranked below Yale). For your profile, I would consider TAMU and ISU to be the upper end of the schools you should be applying to for Statistics PhDs. 

Gradient boosting is essentially an additive model tailored to decision trees, and the concept of additive models was first developed by Friedman and Stuetzle at Stanford. It is possible that somebody else suggested the idea of boosting for tree-based models, but the gradient boosting machine (GBM) paper that gets cited the most often was written by Friedman when he was at Stanford. The generalized additive model (GAM), an important development in nonparametric regression, was also developed by two Stanford statistics faculty, Hastie and Tibshirani (albeit this was before they joined the Stanford faculty).
Nobody is claiming that there are *no* people who have revolutionized the field at other programs. Obviously, at UC Berkeley, you have Michael Jordan and Martin Wainwright, and Lucien Le Cam also spent his career at UCB. But there is certainly a higher concentration of such "field-changing" folks at Stanford than at any other school. Bootstrapping, compressed sensing, lasso/L1 regularization methods, additive models -- all very "revolutionary" developments in Stats -- came from people who are at Stanford.

Yeah, in terms of academic placements, Yale has done exceptionally well. This leads me to believe that Yale S&DS must be viewed favorably among many departments. Of course, it's really the responsibility of the applicants themselves to make themselves 'stand out' (you can't just rely on the brand name of your alma mater). But the fact that Yale has produced so many of these outstanding job market candidates who got tenure-track jobs at University of Chicago Statistics, Columbia Statistics, UPenn Wharton Statistics, and Princeton OFRE speaks to the department's strengths. I am not sure about Yale's industry placements, but I can't imagine it being *much* different than the industry placements for UChicago. 

I don't think prestige should be a huge factor because I think the ratings of these programs is closer than they appear and I've seen a lot of Yale grads doing very well lately.  You're going to be extraordinarily successful at both places, so I'd go based off program vibes and which department/city you think will help you be most productive and happy the next few years.  To me it sort of sounds like you want to go to Yale.

The distinction is very vague, to be sure. I would look at which journals they are publishing in. If a lot of their work is in places like Annals of Statistics, Annals of Probability, or Bernoulli, it is probably mainly theoretical. If in JASA-Theory and Methods, Biometrika, or JRSS-B, it is probably a mix of the theory and methodology. If in JASA-Applications and Case Studies, Annals of Applied Statistics, Biometrics, Journal of Computational and Graphical Statistics, or JRSS-C, it is mostly methodological/applied (these may have one theorem but usually not more than two).
Some other journals like Statistica Sinica, Journal of Multivariate Analysis, Journal of Machine Learning Research, Bayesian Analysis contain both heavily applied and heavily theoretical articles (and everything in between), so to gauge this, you will need to read the abstract of the article and scroll through it to see how many theorems there are.

I am not inclined to answer this on a public forum. But to make my own personal assessment, I look at student outcomes (i.e. where the PhD graduates have placed), the caliber of the faculty (i.e. current big names and "rising stars"), and the research output (i.e. how actively are the faculty publishing? And which venues are they publishing in?). Note that to assess how "strong" the research output is, it may be a bit different for Statistics vs. Biostatistics. For example, Biostat departments might possibly put heavier weight on publications in journals like Biostatistics, Biometrics Practice, and JRSS: Series A, as well as publications in top specialized journals (like BMC Bioinformatics, Nature Genetics, and American Journal of Human Genetics if the faculty member works in statistical genetics). These venues enjoy wide prestige in the Biostatistics community, but you might not see as many Statistics faculty members publishing in them. 

Based on the criteria I mentioned, I think that Yale S&DS is quite under-ranked, given their PhD placements, caliber of faculty, and research output.  

Congrats on your outstanding acceptances! IMO, Yale is a very good program with some excellent professor (e.g. John Lafferty and Harrison Zhou) who are very renowned and well-known in the field of statistics. Yale's academic placements are also fairly impressive -- they've placed PhD alumni at UC Berkeley, University of Chicago, UPenn Wharton, and Princeton ORFE to name a few. This is an excellent placement record, which leads me to believe that many top schools view the Yale S&DS program favorably. If you are leaning towards industry, then I think that either Chicago or Yale would be fine. It seems like you have especially good vibes from Yale though, so I would definitely not think it's insane to pick Yale over a "higher ranked" school.
I am not really sure how their USNWR rank was determined but I truly believe that Yale is under-ranked relative to the department's strength (in terms of its faculty and student outcomes). Rankings are based on popular perceptions (surveys sent out to academic statisticians), which might be slow to change. I think the rankings are still fairly good for the most part, but there are a few that I think are over-ranked or under-ranked relative to their current strength -- Yale is one of those (I believe it's under-ranked).

Congrats on your outstanding acceptances! IMO, Yale is a very good program with some excellent professor (e.g. John Lafferty and Harrison Zhou) who are very renowned and well-known in the field of statistics. Yale's academic placements are also fairly impressive -- they've placed PhD alumni at UC Berkeley, University of Chicago, UPenn Wharton, and Princeton ORFE to name a few. This is an excellent placement record, which leads me to believe that many top schools view the Yale S&DS program favorably. If you are leaning towards industry, then I think that either Chicago or Yale would be fine. It seems like you have especially good vibes from Yale though, so I would definitely not think it's insane to pick Yale over a "higher ranked" school.
I am not really sure how their USNWR rank was determined but I truly believe that Yale is under-ranked relative to the department's strength (in terms of its faculty and student outcomes). Rankings are based on popular perceptions (surveys sent out to academic statisticians), which might be slow to change. I think the rankings are still fairly good for the most part, but there are a few that I think are over-ranked or under-ranked relative to their current strength -- Yale is one of those (I believe it's under-ranked).

Well, if your chances of continuing on in a PhD program at UPenn are well above average, then I would strongly consider UPenn AMCS. I know that several of the professors in the Wharton Statistics Department supervise PhD students in AMCS. I also know that some folks in the DBEI Department supervise students in other departments as well (e.g. some CS students who work on NLP are co-advised by faculty in DBEI). Some DBEI PhD students have also been advised by Wharton Statistics faculty (e.g. Edward Kennedy who is now a rising star professor at CMU), so it seems like they're pretty flexible about whom you can work with.  
Is the UPenn Masters funded (either fully or partially through TA or RA)?? That's also something to consider. 

Johns Hopkins AM&S is a pretty good choice, I think. The school is certainly prestigious, and the department doesn't have bad academic placements. I saw that they recently placed PhD graduates in TT positions at University of Pittsburgh and Indiana University Statistics Departments.
UPenn is obviously very prestigious too, but is the AMCS Masters funded? And is there a clear "Masters-to-PhD" route? i.e. do (m)any of the Masters students get to continue on to the PhD program at Penn as well? I would inquire about this possibility (but phrase it like, "If I perform well in the program and if I were to do research with a professor during my Masters, would it be possible to transfer to the PhD program? How many AMCS MS students can continue on to the PhD program at UPenn?") UPenn has some great people for ML/Statistics, including those working in current "hot" fields like differential privacy and deep learning.

Before starting the PhD, I did review Calculus, probability and statistics (at the Casella/Berger level), real analysis, and linear algebra (including proofs). I think that this was helpful for the classes that I had to take in my first year, since it was fresh in my mind right before starting classes.  
I didn't study any measure theory, stochastic processes, functional analysis, etc. before starting the PhD. I don't think this is really necessary, but if you are very interested in it, it could potentially be useful... though I should note that by the time you start research, chances are you will forget most of this stuff. At that point, you can just relearn what you need for your research and fill in any gaps as you go.  But I do recommend reviewing some of the topics in my first paragraph because you'll be able to use that stuff right away when you start taking classes (rather than possibly needing it one or two years later when you start your dissertation research).

Before starting the PhD, I did review Calculus, probability and statistics (at the Casella/Berger level), real analysis, and linear algebra (including proofs). I think that this was helpful for the classes that I had to take in my first year, since it was fresh in my mind right before starting classes.  
I didn't study any measure theory, stochastic processes, functional analysis, etc. before starting the PhD. I don't think this is really necessary, but if you are very interested in it, it could potentially be useful... though I should note that by the time you start research, chances are you will forget most of this stuff. At that point, you can just relearn what you need for your research and fill in any gaps as you go.  But I do recommend reviewing some of the topics in my first paragraph because you'll be able to use that stuff right away when you start taking classes (rather than possibly needing it one or two years later when you start your dissertation research).

It's not a measure-theoretic treatment (such as a class in stochastic processes you might take after a first semester course out of a book like Billingsley), but Resnick's Adventures In Stochastic Processes is one of the most entertaining math books I have ever read - highly recommended.  @Stat Assistant Professor is much more of an expert, so defer to his opinion, but I don't think a huge knowledge of martingales will be needed; the only time I see martingales come up is in some people's theoretical work on Bayesian asymptotics.  Keep in mind that you won't really have to learn much of this stuff that you don't want to -- for instance, if you don't like martingales or functional analysis, most people at Duke will be doing research that is much more applied.  My personal advice is always to just focus on mastering the materials in your classes and if you are starting research, to learn that material.  Unless you really love doing the extra math, the effort:reward ratio is probably not worth it.

So I do mostly research in the area of Bayesian statistics (though not exclusively), and I have done both applied and theoretical research in this area. 
I would say for theory: it is pretty important to know analysis and linear algebra well and to be comfortable with probability theory and stochastic processes. Unless you are doing very hardcore theoretical research (and there are some people who do that), you don't need to know measure theory that well, but you should be comfortable with it. Plus, measure theory/probability theory can be pretty useful for Bayesian nonparametrics. In Bayesian nonparametrics, you frequently replace finite-dimensional prior distributions with stochastic processes (e.g. Dirichlet process, Gaussian process, etc.), and it can be useful to know a little bit of measure theory and probability theory. 
For methodology/applications: obviously be familiar with the Bayesian paradigm, as well as MCMC (Gibbs sampling and Metropolis-Hastings) and maybe variational inference. Most of the time, the posteriors are intractable, so you do need to do approximate inference. It would be useful to be familiar with some of the "classical" models for linear regression, classification, and semiparametric/nonparametric methods for regression/classification. I think once you specialize in a particular research area (e.g. spatial statistics, functional analysis, topological data analysis, etc.), you can learn that stuff on your own. There's no need to study it prior to starting your research, unless you are very interested in it.  
For programming: Be proficient with programming in R and comfortable with using C/C++. Since R can be a bit slow and have a lot of overhead (compared to C/C++), you may prefer to code in C/C++ and integrate this code with R. R is great for creating plots and visualizations, etc., but if you are going to run MCMC (for example), you may prefer to use C/C++ and then integrate this with R, because your code will run a lot faster.

I think working with an Assistant Prof is probably fine. I have seen some TT faculty who had Assistant Professors as their PhD supervisors and who still landed many campus interviews for tenure-track positions. The most important things to consider when working with an Assistant Professor are:
whether their research is a good "fit" and whether they can help you to be competitive in the job market for academia or industry (either because they can help you publish in the top tier journals/conferences or because they have solid industry connections), and whether they are productive enough (by your department's standards) to earn tenure. If both criteria apply, then I say go for it. Besides, getting a TT position is the sum of many different parts, not just one thing. If your research is in a "hot" area that a hiring department currently lacks expertise in or if their job ad expresses special interest in recruiting applicants from your subfield, then I would think that you would enjoy certain advantages, regardless of who your advisor is. I also think that adcoms consider the strength of the recommendation letters too, not just whom they're written by.
It is a good idea to try to get your work noticed, though, so you can hopefully get a letter of recommendation from somebody who is influential in the field. One of my letter writers when I was on the market was from a pretty prominent name in the field, and this person was neither my PhD or postdoc supervisors... but I interacted with this person fairly regularly and they were familiar with my work, so they were able to write a very good letter for me. I believe that helped a lot.

I've seen it happen (PhD->industry->academia), but usually, the scenario was like this. The person got their PhD and decided to go work in industry. They didn't like it and discovered that they preferred academia, so they went to do a postdoc instead after 1 year out in industry. Then after the postdoc, they got a faculty position.
It's much less common for people to return to academia after a number of years. I've seen it happen but this almost always entailed taking a (non-tenure track) lecturer position somewhere, or sometimes a job at a teaching-oriented college. The reason it's harder to return after many years (for research universities anyway) is because academic hiring is based so heavily on your research output (i.e. your publication record) and the department's needs. If you've been working in industry for longer than a year, it is very difficult to keep writing papers and publishing (whereas if you've been out of your PhD for only a year, you could still have fairly recent papers from your dissertation that have been published or that you're working to get published somewhere).  It's also hard to keep up with current trends in academic research if you've been out for a very long time, and those tend to change rapidly these days. If your research area isn't at least a moderately sized area of interest to the statistics/biostatistics/machine learning research communities, then it will be hard to get hired. 
However, I suppose you could theoretically be competitive for academic positions after years spent in industry if you were in a role where you could continue publishing. Or you could also be competitive if you were to go do a postdoc after years spent in industry and used the postdoc as a springboard for getting back in into academia. I think I may have seen a couple of people do this (do a postdoc after years spent in industry). It should be noted that these people usually did Biostatistics postdocs and then went into Biostat departments. 

Agreed, I would weigh a fellowship offer very heavily. I was on fellowship where I only had to teach/TA for one year. The other three years were totally free to spend on courses and research. This ultimately shaved an entire year off of the (typical) PhD completion time -- finished in 4 years instead of 5, because I only had to focus on my research after the second year, and I had no other responsibilities. 
It should be noted that if you are interested in academia, then finishing "faster" is *NOT* always advised. If you're waiting for an Annals of Statistics or JASA paper to be accepted, then it's better to wait another year so you can have that on your CV. This way, you will be more competitive on the job market (and you can also get other papers done/accepted/published too in that time). However, being able to finish faster is great if you want to go into industry or if you have a good postdoc lined up that can make your profile more competitive than it would be if you stayed another year in the PhD program. 

In my opinion, you should be basically asking yourself 3 questions about a fellowship - forget anything about prestige.
1. How much money will it give you over the 5 years?
2. How much time will it save you from doing stuff you don't want over 5 years?
3. Will it give you flexibility in finding an advisor? (For instance, since I have a fellowship, professors don't have to find me themselves so I had more freedom to work with who I wanted)

I've seen it happen (PhD->industry->academia), but usually, the scenario was like this. The person got their PhD and decided to go work in industry. They didn't like it and discovered that they preferred academia, so they went to do a postdoc instead after 1 year out in industry. Then after the postdoc, they got a faculty position.
It's much less common for people to return to academia after a number of years. I've seen it happen but this almost always entailed taking a (non-tenure track) lecturer position somewhere, or sometimes a job at a teaching-oriented college. The reason it's harder to return after many years (for research universities anyway) is because academic hiring is based so heavily on your research output (i.e. your publication record) and the department's needs. If you've been working in industry for longer than a year, it is very difficult to keep writing papers and publishing (whereas if you've been out of your PhD for only a year, you could still have fairly recent papers from your dissertation that have been published or that you're working to get published somewhere).  It's also hard to keep up with current trends in academic research if you've been out for a very long time, and those tend to change rapidly these days. If your research area isn't at least a moderately sized area of interest to the statistics/biostatistics/machine learning research communities, then it will be hard to get hired. 
However, I suppose you could theoretically be competitive for academic positions after years spent in industry if you were in a role where you could continue publishing. Or you could also be competitive if you were to go do a postdoc after years spent in industry and used the postdoc as a springboard for getting back in into academia. I think I may have seen a couple of people do this (do a postdoc after years spent in industry). It should be noted that these people usually did Biostatistics postdocs and then went into Biostat departments. 

There are a few Statistics departments that are in business schools. For example, at University of Pennsylvania, the Statistics department is in the Wharton School of Business. In spite of this, it does not seem as though most of the statistics faculty at UPenn work on any particular business/economic applications, and most of them are pretty theoretical. I think at Temple University, the Statistics department is also in the business school, and not all of their faculty work on business applications of statistics.
As to how reputable the NYU program is, I really do not know much about this program. It seems very, very small (< 10 students across all years). I guess a good starting point would be to look at the publication records of Statistics faculty in the IMOS department at NYU and see where they are publishing. If you're interested in industry, it might not be so important that faculty are publishing consistently in top-tier journals -- but you do want to work with a faculty member who is reasonably productive so you can graduate!

I personally wouldn't weigh a 1-year fellowship very highly (it would be different if it were for all 5 years and included summer funding without work duties).  I've been a TA at 3 different programs; 2 semesters I had about 10 hours/week of work, 1 semester I had about 5 hours a week, and 2 semesters I had less than 1 hour per week.  Most people don't do research their first year anyways, and I almost can't imagine TA duties being so heavy that they interfere with taking courses even if you have the full 20 hours/week.  A lot of people I know found their TA assignment to be a nice break from their routine (although probably moreso not during COVID when you actually personally interact with students).  I agree with statsguy's general point that you should consider all these aspects, but I think it would be short-sighted to tip your decision based on a very temporary inconvenience at worst.

I don't think they're necessarily directly comparable, since Annals of Statistics pertains mainly to mathematical statistical theory (and is indeed the most prestigious stats journal for statistics theory). I would say among methodologists/theoreticians, Annals of Statistics is considered more prestigious.
 However, Annals of Applied Statistics is considered a top-tier journal and has had some very influential papers appear in it. For example, the original Bayesian additive regression trees (BART) paper (BART is one of the top-performing ML methods for prediction) and the pathwise coordinate optimization paper by Friedman et al. appeared in Annals of Applied Statistics. 

University of Wisconsin seems to do better in terms of academic placements. They've had some very solid grads, like Ming Yuan at Columbia University. I also think that Madison is a better location than State College personally.
If I were you, I would look carefully through faculty websites and see what journals and/or conferences the faculty are publishing in. If you see a lot of JASA, Annals (any of the stats or probability journals), JRSS, Biometrika, ICML, NeurIPS, COLT, etc., then I would consider that program to be very strong and promising for a future academic career.
I would also consider things like coursework, qualifying exams, etc. At some schools, they have a lot more course requirements, and some have two written exams rather than one. That could add to the length of study. I know of some people who graduated from Wisconsin Statistics with their PhD in three or four years, so I'm not sure if they have fewer exams/course requirements.