Hi, first congrats on getting all the interviews.
Well, for me personally, I don't think it's a bad idea not attending an interview at a university where you know for sure you will not go. And it really depends on whether you have an offer on your hand or not. In my case, after I secured an offer that I know I will accept at the end, I basically just treat other interviews as free travels. On the other hand, it also depends on your schedule as well. Interviews sometimes can be very disruptive to your current work. So if I am sitting on an offer and very busy with my current work, I will definitely not attend an interview that I'm not interested in.
However, it is another story if you have not got an offer yet.
Best of Luck!