Evaluating program reputation

[songofgallifrey]

I'm hoping you all can provide some guidance on how to evaluate the reputation of a graduate program and how reputation should fit into my decision-making process.

I've applied to 5 PhD programs for the fall 2017 cycle: two are joint programs in social policy and sociology, two are in linguistics (and specifically sociolinguistics), and one is in second language education. I'm interested in linguistic identity and language policy, which is why I've applied to different kinds of programs. I've been admitted to the second language education program already, and am waiting to hear back from the others; however, I'm already thinking about how I'm going to decide among my options, assuming that I get in to one or more of the other programs I applied to.

The most nebulous factor in my decision-making process is reputation, both of the program and the faculty I hope to work with. I've also had mixed results finding information on the amount of  funding available for students in each program/department (both in terms of tuition/stipend and research/conferences), admission statistics, and post-graduation placement for PhD students. I've taken a look at sources like U.S. News rankings, but ultimately those rankings aren't specific to the programs I'm interested in and only reflect what I already know about how prestigious Ivy League schools are. I've also tried figuring out how often faculty members are cited, but I haven't found that to be a super fruitful exercise. Otherwise, the only information I have about reputation and everything that entails (resources, job prospects, etc.) is from word of mouth, and I'm not super comfortable making a life-altering decision based on one person saying that x school is good for x topic.

Do you have resources that you've found helpful for thinking about a program's reputation? Is anyone else applying to similar programs and have thoughts on this? Should I really care about reputation, or do you think there's something else that I should pay more attention to? There are some other threads on program reputation that make me think that reputation is something I should be paying attention to, but there were some mixed opinions on that. I appreciate your feedback.

Edited December 22, 2016 by songofgallifrey

[TakeruK]

I'm in a different field than yours but this is a factor that transcends fields!

Does your field have school visits? A lot of the information you are looking for, especially about specific resources to students, is best learned during a school visit. These things aren't usually well publicized on websites. If you can't visit, try to set up some Skype calls with profs and other grad students. You can definitely ask about things like whether or not students feel like they can go to the conferences they want to go (or if they always have to make hard decisions due to limited funds), whether or not there are department-wide or university-wide sources for funding if your advisor doesn't have money for you, whether the stipend is enough to live on, what is the quality of life etc.

For determining reputation, yes, reputation within the field is important, but overall reputation could also be important, depending on your goals. When I was choosing, I was deciding between two top schools in my field. However, one was a private school that was well known beyond my field, and another was a public state flagship school that had little academic clout outside of a few fields. In this case, because the "in-field" reputation was equal (I'd rank these programs as #1 and #2 in the country), I put a lot of consideration on the overall school ranking. The reasons to consider the overall rankings (e.g. US News):

- I might want to consider non-academic jobs, and someone outside of my field would be more impressed with a PhD from the private school instead of public state school.
- The private school draws a lot more money from alumni and philanthropists than the public school, often translating into more resources for research and sometimes extra perks for students too. I wasn't sure of this at first, but after I got here, I was amazed by the difference in availability of resources at this private school compared to my past experiences. 
- Usually, but not always, the higher ranked schools are in areas with other academic institutions, so it's easier for you as a student to access collaborators and colleagues in other universities. It's also more likely that your university will be hosting big events for your field, so it increases your exposure.

Finally, you wrote about determining these reputations. I want to say that this only really matters that the "big picture" level. I'm not quite sure, but from the description of what you're doing, it sounds like you might be trying to discern differences too small to matter. When I considered "reputation", I mentally divided the schools into three categories of rankings. The specific numbers may vary due to the size of your field, but for me, there were top tier programs that would probably be ranked 1-10 nationally. Then there's another tier that might rank 10-30. And then there's everything else. I don't think reputation really matters when differentiating between rank #3 and rank #7. It's all the same. I would consider reputation as a factor only when you have to decide between these two different "tiers".

Here's how I would determine the school's reputation (and the reputation of its members). Some of these things aren't probing reputation directly, however, what we really care about is the advantages that you get from going to a school with a strong reputation, so we can look for these benefits/advantages instead

- Word of mouth is actually not that bad. But make sure you get the right opinions. If you have good relationships with a wide variety of professors, talk to them about your grad school options. They might share lots of thoughts with you. Yes, it is just word of mouth and people's opinions, but that's exactly what "reputation" is anyways. It's not an objective score, it's other people's subjective opinions. Again, you won't be able to get precise information like "School X is the #3 school" from asking a bunch of people. But if you talk to people about "good graduate programs in X", and you hear certain school names over and over again, then that tells you something.

- I look at the papers of the students in the program. See what kind of work they're doing, what kind of resources they can use.

- I go to the major conference websites for my field (we have a big national one every January) and search the abstract database. It allows me to sort by institution, so you can see what the students are doing at these meetings. In my field, oral presentations are more prestigious than poster presentations, so you can see how often people from School X give talks vs. give posters. You can also get a sense of how often the school sends students to these meetings. Oh also, some conferences have student presenter prizes---find out where they're from.

- I go to the society webpages for the major society in my field and look at the past awards. My society awards 3-5 prizes each year, for different things like "Career Achievement Prize", "Young scientist" (i.e. a prof with a PhD in the last 7 years) etc. This is a good way to determining an individual person's reputation, because if they win one of these, it means they are recognized by the field as a superstar. These awards require nomination from others in the field and are very competitive. If you notice that some schools often have winners then that tells you something!

- In my field, there is a journal called "Annual Reviews of Astronomy & Astrophysics" (similar Annual Reviews exist for a lot of other fields). Scan the table of contents and see who is writing the review articles. These people are generally going to be considered the foremost experts in their topic. These authors are specifically invited to write these reviews. See who they are and where they are working. For the topics specific to your area of research, read the review article. They usually have a very extensive literature review as they will often summarize all the knowledge and work on the topic to date. See who gets cited. Which works are considered field-defining?

- Go to the department's website and look at the list of faculty. Usually they will say the title of the professor too, like the "Jackie A. Smith Professor of Biology" or the "Rachel E. Miller Chaired Professor of Physics". These special professor titles are usually because the money came from a donor that endowed the position. If you see a lot of these types of titles on a department page, you can infer that they are getting a sizable chunk of income from donors.

- If your field has national graduate fellowships, for example like the NSF GRFP, you can often search the database of winners. Find out if students in the department you're interested will often win them.

- If your field has national prize postdoctoral positions, find out where these winners got their PhDs.

- Finally, one of the advantages of going to a high ranked school is that you're more likely to have distinguished speakers come visit for the seminar and colloquium series (and you're more likely to have many of these talk series). Go to the department website and look up who the past invited speakers have been. Are they getting the top researchers in your field to visit? Would the variety and quantity of talks and visitors interest you?

[rising_star]

Everything @TakeruK has said is true. I just wanted to add that in many fields, word of mouth is the only way to rank programs/departments. In my field, rankings are basically garbage because there are so many differences in areas and a school that's strong in Apple Studies may not be strong in Basketweaving, even though both are key parts of the discipline. Consequently, word of mouth is the way to go. Though I'd point out that you want to pay attention to word of mouth about the advisor and the department. A department's reputation and strength in a subfield can be dramatically affected by the reputation of just one or two people. If those people are well-known in the subfield and you're considering academia, it may not be a problem if the department as a whole is ranked 10-30 because you're working with the person to work with in that area. Does that make sense? 

[songofgallifrey]

@TakeruK thank you so much for your detailed answer! This is all really helpful information for me. To your question about school visits, I know that at least one of the schools I applied to has an open house event, and I've been put in touch with a graduate student in the program I've been admitted to already, so I hope to ask her these questions about funding and job prospects.

and thank you @rising_star for your comment. the distinction between departmental and individual professor reputation - and the impact of that distinction on academic vs. non-academic job prospects - makes a lot of sense. The school I've been admitted to so far is in the 10-30 bucket, but one of my LOR writers said that this program should be among my top 3 choices because of the professor I would work with (whose individual reputation elevates that of the department, as you described).

 

Edited December 22, 2016 by songofgallifrey

[Concordia]

I'm also thinking about this problem-- which will be a real first-world problem if I get into enough places to worry about it.

I just finished a master's with a very good grade in the thesis, with feet potentially in the door at two world-famous places for doctoral work.  (One gave me the degree and will give me a bye on some of the admission filters b/c of the grades they have already given me.  My adviser is at the other.)  Each place, in addition to having a fearsome reputation, would be an excellent place to (a) get to know people not just in my area, but in a lot of other disciplines, and (b) study comfortably without getting too distracted.  There are a number of prospective supervisors in each place who are somewhat interested in my specific area (of History).  And the world at large will know exactly where I went when I'm done.

To weigh against these opportunities, unsolicited comments from a few people have suggested that the guy who is neck-deep not just in my general area, but in my decade and the specific problem I want to look at for my dissertation is not at these two places, but in a good but not drop-dead-famous university in a large metro area.   He is young and inexperienced as a supervisor, but well-liked and willing to be helpful to me.  The university he's in was best-known for a completely unrelated field, but has been working hard to make their history department well-staffed.

So here's the question: consensus on this board has it that the adviser means more than anything else.  That, however, seems to be most true in the sciences, where you really do need specific research grant pipelines, laboratories with particular hardware, etc.  In the humanities, where scholars do a certain amount of intellectual skipping around as their interest changes over time, is it necessarily better to find someone who could co-author your next paper (but might not be in a preferred learning environment), or be happy with excellent teaching in a well-funded, stimulating place that will put you more or less on track to do what you want to do today, while giving you the flexibility to migrate away from your interest in 2017?  Giving one a fishing license, and not just a map of today's fishing hole, in other words?

Put another way, I've seen a few comments here on the order of "yeah-- Princeton's OK and they have a post-Marxist feminist gender scholar who's working on the late 1670s, but if you really want a post-Marxist feminist gender scholar analyzing the early 1670s who's left-handed, you really need to look at East Armpit State."  I exaggerate, of course, but I am wondering how people would decide when they're cutting it too fine on any of these issues.

[If it matters, age and a few other factors may make me a little too unconventional for many of the tenure-track gigs that people are justifiably worried about getting.  I think at this point that I am better off not stressing out about joining that queue, but instead looking for a flexible Plan B right off the bat.]

Edited December 26, 2016 by Concordia

[TakeruK]

2 hours ago, Concordia said:

So here's the question: consensus on this board has it that the adviser means more than anything else.  That, however, seems to be most true in the sciences, where you really do need specific research grant pipelines, laboratories with particular hardware, etc.  In the humanities, where scholars do a certain amount of intellectual skipping around as their interest changes over time, is it necessarily better to find someone who could co-author your next paper (but might not be in a preferred learning environment), or be happy with excellent teaching in a well-funded, stimulating place that will put you more or less on track to do what you want to do today, while giving you the flexibility to migrate away from your interest in 2017?  Giving one a fishing license, and not just a map of today's fishing hole, in other words?

It may be "more" true in the sciences, but not all of the sciences are what you describe! Some of the theoretical researchers in my department have no labs, no hardware, few grants and a lot of intellectual skipping around. I'm kind of in between---I don't require any hardware (other than a computer) for my own work, but we do write proposal for access to shared nation-wide resources (telescopes). However, the time spent collecting the data is like 1-2 weeks per year, and the rest of the time is working at my desk. 

In these cases, I do think the advisor still plays a large role. I think this might suggest that this applies outside of the sciences too, but I'm not in your field so I can't know for sure. Having an advisor with a good reputation matters more than just grants/hardware/money. Letters from your advisor might have a little bit more weight (although I think most people are careful enough not to let knowing the letter writer affect their evaluation of you too much). However, your advisor should be your #1 champion when it comes to promoting your work and helping you join the network of professionals in your field. Someone with connections and resources can help you do this better than someone who doesn't.

That said, I don't really agree that advisor reputation is strictly more important than department reputation. I think they are both important factors you need to consider together. Look for a place that can give you enough in both, even if it means there were other places that had better fits in only one of these aspects. That's my opinion anyways. From your description here, it sounds like the original two places you were considering will offer something in both of these aspects, but the latest option presented to you only has the good advisor fit factor.
 

2 hours ago, Concordia said:

Put another way, I've seen a few comments here on the order of "yeah-- Princeton's OK and they have a post-Marxist feminist gender scholar who's working on the late 1670s, but if you really want a post-Marxist feminist gender scholar analyzing the early 1670s who's left-handed, you really need to look at East Armpit State."  I exaggerate, of course, but I am wondering how people would decide when they're cutting it too fine on any of these issues.

[If it matters, age and a few other factors may make me a little too unconventional for many of the tenure-track gigs that people are justifiably worried about getting.  I think at this point that I am better off not stressing out about joining that queue, but instead looking for a flexible Plan B right off the bat.]

If Plan B isn't some sort of academia-related work, then I don't think you have very much to gain from working with the best post-Marxist feminist gender scholar compared to working with a "good" post-Marxist feminist gender scholar. To me, the only reason to even consider this super advisor is if you want to be the next great post-Marxist feminist gender scholar. Otherwise, if you are considering a wide variety of opportunities after your degree, you're much better off at the "Princeton" school instead of the other one. That is, people who aren't also post-Marxist feminist gender scholar won't care if you worked with their best colleague, or not.

Finally, I don't know if this applies to your field, but you can still collaborate with this great prof at the other school without being their student. If you have a good idea, write to them and see what happens (after discussing with your own advisor of course).

[Concordia]

In a pinch, there can be such things as co-supervisors from outside the walls, but that discussion depends on a ton of other things going right.

Thanks for the thoughts.  I am still rather short of facts that will take this problem beyond the hypothetical, but those may become more evident as winter wears on.

 

[TakeruK]

4 hours ago, Concordia said:

In a pinch, there can be such things as co-supervisors from outside the walls, but that discussion depends on a ton of other things going right.

Thanks for the thoughts.  I am still rather short of facts that will take this problem beyond the hypothetical, but those may become more evident as winter wears on.

To clarify, I don't mean co-supervision with the other prof. I meant working with this other prof as a collaborator! Co-supervision is a lot tougher, especially if you want it formalized due to regulations and also due to the commitment of the other prof being on your committee etc. etc. But if you have a good project idea and want to collaborate, it gives them a much lower level of commitment. They basically just need to put in as much as they want to get out of this relationship with you.

[DBear]

@TakeruK mentioned the school's overall ranking in the decision-making process. I just wanted to add on to this because, all else being (somewhat) equal, the overall quality of the school may impact your quality of education and the experience you have more than you think, especially if you plan to do anything interdisciplinary and/or are required/encouraged to have faculty from another department on your dissertation committee. If the program you're applying to is excellent in that field, but only that field, then that could limit the resources available to you in terms of research etc. 

 

[Concordia]

I was sort of thinking in those lines, especially since there's a fair bit of self-education going on.

 

[post from many months ago just "restored" for some reason]

Edited March 24, 2017 by Concordia

[Concordia]

Anyway, the problem just became a hair less hypothetical.   No obviously bad choices, especially since my career isn't going to be going in any pre-determined fashion that I can lay out today.  But-- my master's supervisor, whom I like very much, says I'm almost certainly getting into Oxford (i.e., he's seen my name on a list on his desk).  

The other pending applications are Cambridge (with a young rock star who is almost certainly an excellent teacher, although I might get turned away if he decides that he's not looking at my area just now, or is too popular with the jeunesse d'oree to spare me the time) and one of the less-famous London colleges, albeit one with a rapidly rising history department and a young prof who would love to have me be his/her first PhD advisee.  We had a great interview, and I got some great guidance on refining my proposal.  It helped that there aren't a lot of students who give a crap about this person's specialty.   

On the plus side for Option 3, the advisor was recommended to me by one of the first two candidates, and just did a book on the half-decade I'm proposing to look at.  There are some potentially interesting colleagues in the history department as well.

On the down side, I have no idea of who else goes to this place and if they're around to chat with after sunset.  I'm thinking not just about other historians or even social science/humanities people--- just intelligent people who are willing to speak to non-scientists in English.  I haven't spent any time lurking on their campus talking to anyone who can tell me about what life is like there.   

SO apart from all other factors, I'm not sure if I would do better in a smaller department or a smaller city.  The choice may be between (a) having a supervisor dying to have me make her/him look good and score points vs. esteemed rivals, or (b) locking myself away from the huge city that I love and having more easily-defined living arrangements, transport, study space, social groups, etc.  That's not a totally trivial consideration-- my bad time management and tendencies to depression make me wonder if I don't have ADHD.   And, like a lot of people in academia, I am an introvert.  

Sorry to ramble.  At the moment, even the good part of the story is unofficial, so I shouldn't jinx it.

Edited March 24, 2017 by Concordia