I went to CalArts for undergrad for two years for Graphic Design before I took a few years off and then transferred to ArtCenter for Illustration/Painting –– so while I can't speak about the Photography department specifically, I do have some experience with the school in general and living in the area.
CalArts was an amazing magical place. Because they have the performing arts schools like Dance and Music, and hold the gallery openings every week, there was always cool stuff happening on campus. The community aspect on campus was very much a real thing, and there was a lot of interesting cross-disciplinary collaboration happening quite organically. There was way more social interaction between metiers, and between undergraduate and graduate students, than at ArtCenter. (Do not regret transferring in the slightest, but way different vibe between the two schools.) I made a lot of friends at CalArts that I'm still close to, many of them from other programs. We would still go back to CalArts for performances, gallery shows, etc. even years after they graduated. CalArts and its alumni, faculty, students are a genuinely relevant presence in the art and creative scene in LA too.
Valencia kind of sucks, but if you don't mind driving, or make friends who like driving, it's really not that isolated from most of LA. We would drive to Hollywood just to eat dinner sometimes (Hollywood is about a 25 minute drive no traffic, Downtown is like 45, stopping myself now before I turn into that SNL sketch and start listing freeway routes.) Santa Clarita area is also relatively affordable for Southern California. It's easy and convenient to live there in terms of normal life stuff like going to the grocery store, etc., which is nice if you're spending most of your time on campus in the studio.
If you think the CalArts program would allow you to make better work, and the LA scene in general seems viable, I would not necessarily let the location dissuade you.