Do either of them offer the substitute of phone interviews, and if so, say how they will be weighted compared to in person ones? Some schools, especially ones that do not fund airfare, know it's unrealistic for students to visit everywhere, and make it clear they weight both equally, some offer them but it seems to put you at a disadvantage. You could contact the programs and say it may not be feasible (no need to say why) and ask if there are other options. If you don't go, you can always visit later if you get an offer.
Otherwise, I'd go with the factors I'm using to make my decision in general, mainly how much does the advisor and the program fit me? Location is a distant third concern for me (though I also didn't include schools I absolutely wouldn't go to because of that), but it's also something that can be important to see in person. So can all of it really, so I'm glad for the most part, my interviews don't conflict (had some close calls though!), but I have also found that I could learn a lot through phone interviews (some places do those first anyway).