Apply for specific professor or department?

[coolgod]

Hi, I am interested in researching a very niche part of quantum computing which unfortunately not many professors do. I've seen many guides telling me to apply for schools where there are 2-3 Profs I want to work with in mind, but this is very difficult for me since most schools often only have 1 professor working on something close to my interest. These professors are even sometimes in odd departments where they are the only one working on quantum computing and while the majority researchers are in Physics or CS dept. I'm applying for a Ph.D. in the US for 2017 and my question is should I apply to schools and specific departments where there is only 1 person working on what I like, or should I apply to larger departments or schools who research quantum computing but their interests are less similar to mine? I feel like I could survive a 5 yr Ph.D. in other areas of quantum computing but for some reason, I feel strongly attracted to this niche area. Has anyone had a similar experience like me and ultimately went into their non-favourite research area for Ph.D.? If so how do you feel about it now? Is it possible that I get accepted by a professor who's current research interests aren't exact what I'm looking for but lets me work on a project that matches my interest?
Thank you.

Edited November 10, 2016 by coolgod
added tags and made it more specific

[rising_star]

If I were you, I'd apply to a mix of both, in consultation with those who will be writing your recommendation letters. The question you have to consider is what you would do if that one professor left the school and they were the only person you wanted to work with. At that point, would you be able to stay where you are or would you want to leave for another institution?