Pursuing a PhD in Statistics & Data Science for professional reasons - overcoming feeling of inadequacy due to "passion"

[Bacaw]

Quick background info: I hold a master's degree in statistics, which I earned immediately after undergrad.  I have worked as a quantitative analyst and data scientist for the last three years in the finance industry.

Quick summary of my dilemma: I enjoy my professional career, but my work often feels stale or mundane due to a relative lack of complexity.  I am considering a PhD to advance my professional career.  I would consider myself a data science enthusiast, but I would stop short of calling it my passion.  I am trying to shake feelings of inadequacy in a pool full of individuals more passionate about research and academia than myself.

---

For the past three years, I have pondered the option of leaving the private sector and returning to graduate school for my PhD.  My master's degree in statistics provided me with a fantastic foundation in statistical modeling and machine learning.  However, at the same time, my graduate degree also revealed to me the much more vast realm of knowledge well beyond my understanding - deep learning, NLP, etc.  I feel there is very much a ceiling to the data science roles in industry I am well-qualified for currently.  Three years into my career, I am growing tired of logistic regressions and random forests.  Simply put, I am bored with my current work.  I want my work to be more mentally stimulating.  I want the fulfillment that comes from tackling more difficult analytic problems.

I believe a PhD would provide me the guidance to expand both the breadth and depth of my knowledge.  I very much miss the rate at which I was learning new things in school compared to in my professional career (I am still learning in my career, albeit at a slower rate and without much emphasis on techniques to solve problems beyond what I can already solve).  I realize a vast amount of information is available on the internet to sharpen my skills, but I believe the leap I want to make requires more structure and guidance through a formal education.

With that said, I am very much focused on a professional career.  My struggle comes with knowing that many other PhD students set out with aspirations in academia - which quiet honestly are not for me.  I believe that most folks in academia would consider their subject matter to be their "passion", however I would be more inclined to say that data science a strong interest of mine, but more of a means to an end.  I see myself as a well-rounded individual (or at least I try), and I value a balance in life that allows me to pursue many interests, life goals, and recreational outlets.  I am having trouble shaking the feeling that I would not be devoted enough to pure, scholarly study of my subject to successfully complete a PhD.

Is this a common feeling for professionally-focused students?  Or would it be more wise for me to continue my career as a masters-level data scientist?

[bayessays]

Given that you don't want to pursue an academic career and want to still be a data scientist afterwards, in my opinion you would just be taking a low paid 5 year (probably stressful) sabbatical from your job.

If you already have a master's degree in statistics, the additional classes you would be taking would only be either highly theoretical math classes that you'll have no use for or electives.  For the electives, you can just take free courses online in topics like NLP and deep learning that will be as good as any at a PhD program, and you'll have full control over your learning.

After classes are done, you'll spend 3 years working alone on some very specific problem for a quarter of your current salary, and then you'll come out probably making less money than if you stayed at your current job 5 more years. 

I think getting the PhD only makes sense if you want to pursue academia, switch fields into something like biostatistics, or if your really have some deep internal need to have the experience of getting the PhD.  You seem to be like being a data scientist and want to continue doing it, so I would take the money you're making now, attend some meetups or take some online classes, but i don't see it being worth the opportunity cost for you, which is probably something like $250-500k over 5 years.

PhDs get the same boring data science job as people with master's degrees.

[DanielWarlock]

Hello. I had a similar dilemma about 2 and half years ago when I also worked in finance as a "quant analyst". Look. It sounds good to be a "financial quant". But the truth is that the work is pretty routined and most of your co-workers are disillusioned: they either do it really half-assed so that they can spend more time with their family, or they become "workaholic" in order to be promoted. After a while, you feel that you want to get away from this.

I'm not saying that "financial quant" is not good. On the contrary, it has several upsides as mentioned by Bayessaid. But the decision solely hinges on whether you like creative work. This is important. If you are a guy who just wants to do routine stuff and stay relaxed, then academia is not for you: you will need to not only work hard but think hard and stay active. But if you enjoy the freedom of exploring things, then you should do it. This is what I thought 2.5 years ago. I basically thought about it and quitted my job. I then contacted a prof and started doing research. Simple as that! You just need to make a decision if you want to change. You can start your research career today if you want.

*Also a correction on opportunity cost: a "financial quant" typically makes 100K-500K and a PhD makes 30K. Over five years, it is 0.35M-2.35M lost. So you can lose as much as a 2 million! 

Thanks.

 

 

Edited February 21, 2020 by DanielWarlock

[Stat Assistant Professor]

OP: Since you are already working in the field of data science and you do not seem to have much interest in pursuing a research career, I am not sure that a PhD would really give you much (other than the intellectual stimulation, I suppose, if you are independently wealthy and can afford to take a 4-6 year "break" from industry). Certainly, there are many Statistics PhD graduates who work as data scientists or in non-academic careers, but for the most part, they were not already working in data science prior to starting the PhD -- and a lot of them were international students for whom getting a PhD in STEM actually makes sense if they want to get a U.S. work visa.

I would carefully evaluate your reasons for wanting a PhD and whether it is worth the opportunity cost. I wouldn't do it unless you really have a strong drive to create new knowledge and do original research. Even then, this motivation may not be sufficient once you weigh it against your ultimate career goals, opportunity costs, and the sheer difficulty of the whole ordeal. You seem intellectually curious, so I would do as the posters above suggested and take free online classes or study topics in your own free time if you want to learn. But unless you have a drive to create new knowledge (however incremental), it may not be worth your time.

[MathStat]

On 2/20/2020 at 9:31 PM, bayessays said:

Given that you don't want to pursue an academic career and want to still be a data scientist afterwards, in my opinion you would just be taking a low paid 5 year (probably stressful) sabbatical from your job.

If you already have a master's degree in statistics, the additional classes you would be taking would only be either highly theoretical math classes that you'll have no use for or electives.  For the electives, you can just take free courses online in topics like NLP and deep learning that will be as good as any at a PhD program, and you'll have full control over your learning.

After classes are done, you'll spend 3 years working alone on some very specific problem for a quarter of your current salary, and then you'll come out probably making less money than if you stayed at your current job 5 more years. 

I think getting the PhD only makes sense if you want to pursue academia, switch fields into something like biostatistics, or if your really have some deep internal need to have the experience of getting the PhD.  You seem to be like being a data scientist and want to continue doing it, so I would take the money you're making now, attend some meetups or take some online classes, but i don't see it being worth the opportunity cost for you, which is probably something like $250-500k over 5 years.

PhDs get the same boring data science job as people with master's degrees.

Thank you, @bayessays, for your insightful and honest comment, as always.

While I barely started my Stat PhD, it is sad and discouraging to read this (despite hearing this before and sort of being aware of the issue) :  "PhDs get the same boring data science job as people with master's degrees." 

Of course most of us working towards the Phd love stimulating and intellectually challenging work and we will try our best for a good / decent academic placement. However, as it has been discussed here before, with fantastic information and tips from @Stat PhD Now Postdoc, academic jobs are simply put, a big lottery  for the most part. Even if we assume you played your cards right and secured a great academic placement, you have to deal with the "tenure chase" stress for several years. One world-class famous professor from my undergrad told me that he doesn't know whether he would choose this same path again. 

At the other extreme, many people around me (including the original poster, @Bacaw) have described their data science roles (ranging from small to fancy, big-name companies) as repetitive, menial, boring work,  where they use the same methods/tools almost every day. I admit I am not very knowledgeable about industry jobs and perhaps you can still find a fulfilling and stimulating data science position if you look hard enough. These descriptions are also very subjective and depend a lot on the individuals' preferences and visions. 

The question is: is there any middle ground option that is perhaps not as extreme as the two above-mentioned paths? Would top-notch research positions in industry such as Microsoft research, Facebook AI research, Google research etc. be the answer? I heard the hiring process for Microsoft research is nearly as competitive and rigorous as that for a top academic job. If anyone has any experience on this and is willing to share, I would love to hear tips about preparing for such industry research roles. And of course, if people think there are other answers to the question of which kinds of jobs (if any) would have both the perks of industry and academia, I would really appreciate the advice. 

Thanks everyone in advance and sorry for hijacking this thread with a somewhat personal worry. However I think it is reasonably relevant to the topic discussed here. 

[bayessays]

@MathStat First off, my statement about the jobs being boring was just my personal opinion and related to the original poster's comment that a PhD might get them a "less boring" data science job.  From my experience, the PhD serves as, perhaps, an easier way to get interviews at places like Facebook/Google/etc, but the jobs don't require PhDs and you won't do anything different if you have one.  Because the OP felt this way, I felt that sentence was an appropriate way to get my point across, but there are a million jobs that I would find boring that other people wouldn't, so I don't mean to discourage people from pursuing data science careers.

People around here mention Microsoft/Facebook/Google research arms, but honestly, I don't think these are realistic options for most people.  Most people working on research are computer scientists and work on machine learning algorithms.  They might have a couple statisticians if you choose the right research area, but I don't think these can be considered a backup.  These companies have large sections of the faculty at Stanford's department on their payroll, so it is not a backup for an academic job.  Many of the people who get regular data scientist jobs at these companies could compete for tenure-track positions if they chose to.

There are a few majors differences between academia and the data science world off the top of my head, that could cause people to be unhappy moving into industry:

1) In academia, a lot of people go in to pursue theoretical statistics.  If you hate analyzing data, a data science career is not going to be an acceptable alternative.

2) Most companies don't know how to use data scientists.  They expect magic, or won't let you have a say in data collection/study design, or don't listen to you, or want you to fudge the numbers to get the results they want.  Turnover among data scientists is incredibly high even at some top companies because of frustration.

3) At least half your job will be writing SQL.  (which can be fine - SQL is cool)

4) Technological concerns trump statistical concerns.  Creating simple data visualizations, etc that fit into existing software flows will have the biggest impact, so working on anything innovative is hard because nobody will use it.

5) Obviously a completely different lifestyle in terms of time management than academia, although the top companies are basically university campuses with free food and massages.

If you went into statistics because you are a self-described "data nerd" who loves digging through data, trying different ways of analyzing it, and have a certain mindset, I think a career in data science could be rewarding.  And you will also become very rich, so it's not all bad.

[Robbentheking]

Full disclose: I have not worked in industry as a data scientist and my opinion is just from word of mouth.

I think people that say that PhD's get the same data science jobs as MS students are not totally correct. Certainly this can be the case, but if you are motivated to work hard in your PhD and prove you have more research ability than being a package-calling coding monkey, my impression is that there definitely are roles out there for PhDs that are more 'interesting', and not just at the hyper-competitive facebook, google, etc. For example, I know from a friend that Grubhub has research teams that do different work than he does as an MS graduate. 

More importantly, I think it's certainly possible to enjoy a job like your current one further if you have a deeper understanding of the methods you're using, and still more importantly, I wouldn't care about losing out on 5 years of earning potential if you're fundamentally unhappy at the moment. 

Anyone who goes into a PhD thinking 'this is a waste of time if I don't get an academic job' is really putting themselves in a bad position. Academia is way too competitive for that. 

[bayessays]

I would be very interested to see an example of a job that requires a PhD in statistics (not machine learning), outside of the pharmaceutical world, that is very different from what you could get at a same or similar company with a master's degree. I have never seen such a job.

It may be true that OP could enjoy their job more if they understood what they're doing better - but a PhD doesn't accomplish this goal.  They have already taken all the relevant coursework. They are equipped to read Wikipedia.  OP specifically mentioned things like NLP/deep learning which are not even topics covered in statistics departments.

[Geococcyx]

Well, NLP and deep learning are often topics that come up in CS departments -- perhaps OP should be looking at CS Master's or PhD's instead (insofar as they're looking at any graduate programs at all)?  Granted, I don't know that those open any more doors than a Stat PhD (which we've established to be limited anyways), but maybe OP's unfulfilledness is partially because they're interested in working on different, more CS-oriented problems?

(To avoid my only contribution being armchair psychology: @Bacaw, is there a reason you'd prefer statistics over maybe CS or math or engineering if you went back to grad school?)

[Robbentheking]

25 minutes ago, bayessays said:

It may be true that OP could enjoy their job more if they understood what they're doing better - but a PhD doesn't accomplish this goal.  They have already taken all the relevant coursework. They are equipped to read Wikipedia.  OP specifically mentioned things like NLP/deep learning which are not even topics covered in statistics departments.

Lol, how does it not accomplish the goal? It's not a necessary condition, but it's certainly a sufficient one. In a PhD program, you can get paid a living wage to read Wiki all day, every day.

As far as Statistics specifically, obviously it's up to OP to pick a department that suits his/her interests.

[bayessays]

1. This is a post about getting a PhD in statistics in a statistics forum from an OP who has a MS in statistics.  He did not ask whether he should go back to school for a decade to get CS pre-reqs and then apply to CS PhD programs. 

2. He can take 5 minutes out of his day to study the logistic regression Wikipedia at either his current job or his PhD, so I have no idea what point you're trying to make here.

[Robbentheking]

4 minutes ago, bayessays said:

 He did not ask whether he should go back to school for a decade to get CS pre-reqs and then apply to CS PhD programs. 

A ridiculous exaggeration. I have an MS in stats, and got acceptances from CS PhD programs despite only taking algorithms,  complexity theory,  and deep learning during my MS, and not taking any CS courses in undergrad. If you're studying ML, which is presumably what OP wants to do, they wouldn't care if you don't know how change the working directory in your terminal if you have a decent math background.

13 minutes ago, bayessays said:

This is a post about getting a PhD in statistics in a statistics forum from an OP who has a MS in statistics. 

'PhD in Statistics & Data Science'. He is clearly interested in ML, which is something that takes place both Statistics and CS departments. The idea that anything I said is off topic is also ridiculous. 

10 minutes ago, bayessays said:

He can take 5 minutes out of his day to study the logistic regression Wikipedia at either his current job or his PhD, so I have no idea what point you're trying to make here.

If you think someone truly interested in anything is going to be satisfied with 5 minutes of additional knowledge from Wikipedia, I have nothing to say.  Try holding a full time job and keeping up with any current literature in a meaningful way.

[bayessays]

We can agree to disagree.  Spending 4 years and giving up a quarter million dollars writing a dissertation on a narrow topic you'll never use in industry with the idea that you'll have a few more minutes to read blog posts during your day is a bad idea and I don't have anything more to say on this.  If you are just going to keep hijacking this thread to mock my opinions instead of offering your own, you can find another forum.

[BL250604]

1 hour ago, Robbentheking said:

Lol, how does it not accomplish the goal? It's not a necessary condition, but it's certainly a sufficient one. In a PhD program, you can get paid a living wage to read Wiki all day, every day.

I wouldn't exactly say this is the case. You get paid a living wage to work very hard and grapple with complex ideas. Reading wiki, while sometimes is a supplement, is certainly not the main activity of a Ph.D. candidate in Statistics. 

[Robbentheking]

1 hour ago, BL250604 said:

I wouldn't exactly say this is the case. You get paid a living wage to work very hard and grapple with complex ideas. Reading wiki, while sometimes is a supplement, is certainly not the main activity of a Ph.D. candidate in Statistics. 

For sure. I meant this figuratively, in the sense that you get paid to understand things that (hopefully) interest you, whereas at an industry job, learning what interests you is not your main work. 

[DanielWarlock]

1 hour ago, bayessays said:

We can agree to disagree.  Spending 4 years and giving up a quarter million dollars writing a dissertation on a narrow topic you'll never use in industry with the idea that you'll have a few more minutes to read blog posts during your day is a bad idea and I don't have anything more to say on this.  If you are just going to keep hijacking this thread to mock my opinions instead of offering your own, you can find another forum.

I feel that there has been too much aggression and negativity. Technically, you are getting paid to learn stuff, which seems to me a pretty good deal. I used to work. At that time, I needed to do research and read paper after 7pm--I just wanted to sleep and I felt terrible. For my masters, I essentially pay tens of thousands to take same classes with PhD and do research with less support. What are we even complaining about here? I'd still do PhD if I were to go back to my old job after that.

Edited February 24, 2020 by DanielWarlock

[Stat Assistant Professor]

I do know several Statistics PhD graduates who now work as Research Scientists at Facebook, Google Brain, and Amazon (some after spending a summer being a Research Scientist-Machine Learning intern). I don't really know what they do as far as "research" goes and/or if that is any different from "regular" data scientists, though. If you want to do *really* basic research like computing theory or mathematical foundations of statistical learning outside of academia, the opportunities will indeed be very limited (e.g. Microsoft Research, maybe some national labs).

Anyway, I would figure out what your priorities in your work/non-work life are and go from there. Even I decided that I was not going to be a postdoc for more than three years and that I would go back to industry if I couldn't find an academic job within 3 years of finishing my PhD (fortunately I found one during the second year of my postdoc). If I had to leave academia, though, I would have definitely missed academic research but I think I would have been fine too -- if I didn't find work sufficiently satisfying/stimulating, I would have redirected that energy into making my non-work life satisfying (maybe even by taking some MOOC's to keep my brain active, likes bayessays suggested).

[MathStat]

First of all, congratulations @Stat PhD Now Postdoc for finding an academic job! That is amazing news and I am very glad to hear that. 

Thank you everyone for all your insights. 

I think my approach right now would be to enjoy the opportunity given to me to focus for 5 years on work and research I find interesting and see what happens later. One of my classmates who is a masters student and does not plan to pursue a PhD also tells me that you *can* find interesting work in industry without a phd and perhaps without even a masters, as was his experience - the issue is that the first one or two years may be more boring work, until you convince your colleagues that you have original ideas that are worth pursuing. However, if you have valuable ideas that you can show lead to promising results which could benefit the business, you will be allowed to branch out and conduct your own work (more of less), and thus gain more independence. This is in the finance industry. I don't yet know that much about tech, but hopefully there can be similar stories in tech too.  

[Bacaw]

Thank you everyone for offering your perspectives.  I realize this can be a contentious topic, so I especially appreciate you all for putting less popular opinions out there.

Perhaps I could elaborate more on my current situation and future goals.  @bayessays mentioned that PhDs get the same boring jobs as M.S. graduates, and I am certainly aware that is true to a certain extent.  I work with several of them.  My current role certainly does not require a PhD.  There are many great things to acknowledge about my current role: the work-life balance is good, and I have learned some skills (my ability to code is immeasurably better than it ever was in school).  However, my firm will outright admit it is not very analytically mature, and it will take a very longtime to move forward.  I realize many companies likely share the sentiment, but I also know many others (mostly in tech) are on or near the cutting edge of analytics.

I want to pursue a role that will challenge me more, but I know I will need to sharpen my skills to land such a role.  Although many employers include my current one emphasize personal development time, in reality no employer will really offer all that much time for development opportunities.  One option would be to spend more time outside of work learning new techniques and technologies, but there is certainly a limit to the amount of time on top of work I could happily allocate to this goal.  As I mentioned before, I strive to be a balanced individual, and I have many hobbies, interests, and goals beyond data science.  I find a PhD appealing for this reason: learning and research would be my job.  I am fortunate to know many PhD students that work hard on their research but have a variety of outside interests that make them more well-rounded individuals.

Maybe I should further describe what attracts me to PhD study as well as what deters me from staying in academia beyond completion of a PhD.  Things I find appealing: learning and applying cutting-edge techniques as well as researching new techniques, (some) freedom to pursue specific interest areas.  Things I do not find appealing: grant proposals, competing for postdocs, a less secure future.  I want to tackle difficult problems, but in an industry setting.  I understand staying current is difficult to do long-term, but I am under the impression I could be far more valuable by spending a few years broadening my skills beyond elementary statistical modeling and machine learning.

 

The opportunity cost is very valid, but I would not be considering a PhD program as an option if I only cared about money.  I have comfort in the fact that 1) I have already saved and invested a substantial amount in my few years of professional work and 2) I could be well-compensated once again following my studies.  I think there could be a great uptick in my quality of life and sense of fulfillment by being back in a university setting, unless I am grossly misunderstanding what PhD studies look like (which of course is what motivated me to make this post).

As one last remark, the program I am looking at is considered to be an interdisciplinary program in Statistics and Data Science, meaning it overlaps with Computer Science among other fields.  I will continue to research the department, but my understanding is that this program would be less like a traditional statistics PhD program.

[bayessays]

First, I apologize for contributing to the distractions above.  As @Robbentheking says, sometimes things are taken literally that are meant figuratively.  In this case, I may have bluntly exaggerated some of the negatives of the choice to go back, because what I would personally want in this situation is for possible flaws in the plan to be pointed out. It is hard to distill your experience into posts on an internet forum sometimes, but I know we all all trying our best to help others who are going through similar choices.

@Bacaw A lot of what you are saying resonates with me.  I have been in a very similar position to you - got an MS, became a data scientist at one of the big companies, and am going back for the PhD for many of the exact same reasons that are appealing to you, although I do plan to pursue an academic career.  I'll just leave a few more thoughts of things to consider that would weigh in my decision in your shoes. 

1) Is there any opportunity for promotion at your current job?  If you could get into a position where you have one or two junior data scientists working for you in the next couple years, I would consider staying.  Having a lot of experience with the data science job market and seeing how people get to high positions, the common theme seems to be that a lot of people start low, work their way up to some sort of team lead/managing position within their first company, and then the entire world is open to them.  Experienced data science team leads are much harder to find than people without managing experience, so from a career development perspective, it might make more sense to prioritize gaining that experience rather than more education.

2)  Especially if you want to live somewhere that isn't Bay Area/Seattle/NYC, the data science job market is pretty tight.   I know people who had trouble getting a first interview coming out of top PhD programs.  Academic experience is often not valued highly by companies, so staying at your job and having a few years of career things you could mark on your resume may help you more than re-starting as someone who has been out of the workforce for a few years.  Nobody can really predict how the job market will be in 5 years, and the market is being flooded with new talent, so I'd take the current extremely high salaries in case they come back down to earth.

3) Regarding having more time to do stuff, I just doubt you'll have much more actual free time in a PhD to do this extra learning.  Between TAing, classes, homework, that's already a lot of time per week that's almost equivalent to what you're doing at your job probably.  Most data science jobs are at a computer all day.  Most people I know could spend quite a bit of time in their job just browsing the internet.  Can you read some data science blogs, do tutorials, go to meetups?   This seems like a more balanced approach for someone who doesn't necessarily view this as a passion and wants to live a balanced life.  Going all-in for a PhD seems like the opposite of balance.  In addition to incorporating some of this learning in your day-to-day,  if you want to, you could switch jobs a couple times in the next five years and take a month or two off in between.  There are short 1 week immersion boot camps in things like deep learning or NLP.  You could use the additional money you save over the next five years to retire and do whatever you'd like afterwards, etc.

Anyways, this is obviously a very personal decision and only you can weigh the cost vs benefits.