As a current grad student who has hosted people for interview weekends before, I wouldn't hold this against anyone. We know how difficult the application and interview process is and that sometimes shit like this happens. I think as long as you make it clear that you're obviously still interested in the place that you're physically at and are just keeping your options open, you should be fine.