I also went to a claremont school (Harvey Mudd). I really enjoyed living in claremont. There certainly are some areas in claremont that have expensive housing, but you can find fairly reasonable apartments as well. While there are about 5000 undergrads in all the 5c's you won't really need to compete with them for housing as something like 98% choose to live on campus. There are some decent apartments just north of Harvey Mudd at xing of Foothill and Mills. I know some people who lived there and they really liked it. It is true that there are some areas around claremont that aren't very nice, but if you have a car there are definitely nice places to live that aren't right next to campus. Because it is not right in the middle of LA you can live in other areas and not have too much trouble with traffic. You probably don't want to live any farther west than Covina otherwise traffic on the 210 will be a pain but if you search around you can find something both nice and cheap.
As far as living in claremont goes, the on campus live at the claremont colleges tends to cater towards the undergrads that make up the majority of the population and other than visiting the village area there is very little to do in actual claremont. However, you are only 20-30 minutes away from Pasadena, which is a very nice town, and maybe 45 from downtown LA/Orange county. I know a few CGU students who suffer the traffic and actually live out in Pasadena, but it is expensive so that might not be the best choice financially.
I will try my best to answer any specific questions you have, I graduated recenly so I should be able to give you up-to-date info.