Seeking for advice on rather sensitive reality of ML/stat/Data related research

[Statmaniac]

Hi all,

I want to ask for advice from more experienced researchers in this forum on some of the ongoing thoughts I have towards ML/stat/algorithmic/computational research. As a PhD student researching machine learning/statistics, I have been overwhelmed by a chain of negative aspects I saw in the actual research phase. Below are some features I find very uncomfortable staying in these fields.

1. A significant portion of research papers are not reproducible. I understand writing a paper that perfectly explains all the tiny-gritty details of the methodology is very difficult, but at the same time, I realized that there are just way too many papers not explaining details enough for readers actually to use methods they introduced in the papers. What is worse is that a significant amount of papers are error-prone, and I have seen reviewers simply disregarding such errors treating them "trivial". An upper-year student who graduated last year had found at least three major errors in his advisor's previous paper, but his advisor ignored them. He fixed all the theories by himself, submitted the correct version of the proof, and got published with his advisor's name on it. Moreover, this was published in one of the top 4 venues, which statistics people would highly consider. His advice to me was simply to accept the reality and to graduate with PhD without making a fuss. It seems that the more experienced people take a somewhat flexible attitude that "learn what you need to and ignore something doesn't feel right/make sense," a.k.a. look at the bigger picture, not the details or "You do not need to do the correct research, as research, by nature, is prone to error".  I agree that understanding the bigger picture is important, but it seems the way research is done dismisses details often too much. I try to follow/learn this attitude, but it just seems very hard and somewhat arbitrary.

2. Due to publication pressure, I feel that there are so many meaningless papers. In fact, I also submitted two papers this year, and I am not proud of any of my work. I hardly find these can be used by practitioners. Methodologies these days are much more complicated than in the past, but I felt that it became too complicated to be actually useful in practice. My advisor seems to be satisfied with my work, but it seems that he can't understand why I am not happy. I guess I am also another one who is just merely trying to survive in this crazy "competition" instead of doing "real" and "meaningful" research.

These thoughts have negatively influenced my research work to the degree that I started to question whether this is indeed the way I want to spend the rest of my career. I was fascinated and excited by creative/beautiful ideas of bridging theories with actual data analysis or solving some real-world problem using my quantitative skills, but in reality, it seems that, by the nature of the discipline, there's a lot of darkrooms which I didn't see before. I am sure some other people in the forum have once in their life had similar feelings. I wonder how they dealt with such feelings and moved on. 

Edited March 6, 2021 by Statmaniac

[Stat Phd]

10 hours ago, Statmaniac said:

Hi all,

I want to ask for advice from more experienced researchers in this forum on some of the ongoing thoughts I have towards ML/stat/algorithmic/computational research. As a PhD student researching machine learning/statistics, I have been overwhelmed by a chain of negative aspects I saw in the actual research phase. Below are some features I find very uncomfortable staying in these fields.

1. A significant portion of research papers are not reproducible. I understand writing a paper that perfectly explains all the tiny-gritty details of the methodology is very difficult, but at the same time, I realized that there are just way too many papers not explaining details enough for readers actually to use methods they introduced in the papers. What is worse is that a significant amount of papers are error-prone, and I have seen reviewers simply disregarding such errors treating them "trivial". An upper-year student who graduated last year had found at least three major errors in his advisor's previous paper, but his advisor ignored them. He fixed all the theories by himself, submitted the correct version of the proof, and got published with his advisor's name on it. Moreover, this was published in one of the top 4 venues, which statistics people would highly consider. His advice to me was simply to accept the reality and to graduate with PhD without making a fuss. It seems that the more experienced people take a somewhat flexible attitude that "learn what you need to and ignore something doesn't feel right/make sense," a.k.a. look at the bigger picture, not the details or "You do not need to do the correct research, as research, by nature, is prone to error".  I agree that understanding the bigger picture is important, but it seems the way research is done dismisses details often too much. I try to follow/learn this attitude, but it just seems very hard and somewhat arbitrary.

2. Due to publication pressure, I feel that there are so many meaningless papers. In fact, I also submitted two papers this year, and I am not proud of any of my work. I hardly find these can be used by practitioners. Methodologies these days are much more complicated than in the past, but I felt that it became too complicated to be actually useful in practice. My advisor seems to be satisfied with my work, but it seems that he can't understand why I am not happy. I guess I am also another one who is just merely trying to survive in this crazy "competition" instead of doing "real" and "meaningful" research.

These thoughts have negatively influenced my research work to the degree that I started to question whether this is indeed the way I want to spend the rest of my career. I was fascinated and excited by creative/beautiful ideas of bridging theories with actual data analysis or solving some real-world problem using my quantitative skills, but in reality, it seems that, by the nature of the discipline, there's a lot of darkrooms which I didn't see before. I am sure some other people in the forum have once in their life had similar feelings. I wonder how they dealt with such feelings and moved on. 

That’s a great point I have seen that with most conference papers. Are you publishing in conferences?

 

the truth is that conferences are unreliable due to the sheer volume of papers.

many of of my friends (newbie grads like you) are “reviewing” for conferences. They have no clue how to really assess the quality of paper (as they tell me personally!)

also most of these friend just want to have conference papers to put on their CV and get tech job later. And their advisors just want to grow their cv, the advisors rarely read the papers, they just write the intro and conclusion, and give a cursosry look 

so it’s really depressing, I agree with you...

[Stat Phd]

But I think you have the right mindset, you should try do your best but still be pragmatic. Are you looking for an industry job or academic job?

[bayessays]

.

[Euler17]

In relation to you not being proud of your recent papers, remember that you probably know the shortcomings/holes in your work better than anyone else. I have struggled with being completely unsatisfied with 2 of the papers I wrote this year because I felt like there were so many ways that they could be improved, but I trusted my professors that they were ideas worth writing about. And maybe I don't get around to improving upon them in later papers the way I envision, but once they are out there others have the opportunity to build on them if they'd like. I def agree with most of the issues you bring up, just wanted to give my two cents on that point.