Your hard stats matter a lot, way more than they ever did for undergraduate applications.
Having a publication, research experience, good LORs, and okayish GRE scores will mean that you get accepted into plenty of pretty good programs, but when it comes to the absolute best, your GPA is just very low. These programs would only rarely accept a candidate below ~3.8, and you're not even borderline for that. Even your 3.55 at Berkeley is low when you're applying to do physics. Given that you have experience and personal connections at Berkeley, that was going to be the good program you'd get into if you were going to get into one.
So yeah, it looks like your GPA has killed your apps. I hope you applied to some lower programs, and if you haven't, make sure you do that next year if you apply again. And look into grade replacement schemes too.