I spoke with a representative at my school who handles student matters and she gave me a few options. I have the right to change my advisor, I can speak to the chair, the dean, and I can file a formal grievance at the university. The issue is those mechanisms, although in place, don't translate onto the reality - reality which is filled with politics, ego, and professors who will go out of their way to protect their reputation.. I'm afraid if I take this out in the open, I will only shoot myself in the foot, because my advisor is very influential and respected. The representative also told me that there are different dynamics among advisors and students and sometimes there may be a lack of personality fit which causes lack of meetings and responses (I also said that the advisor doesn't have the knowledge in my area of research, which was clearly shown as they were wrong on most things). The fact that I still went with what the advisor told me, was still my decision so it is only on me. With regards to yelling at me, I should simply tell the advisor not to do it.. The thing is, if I am dealing with someone so proud and with such a personality, then calling them out (even if I'm right), would only make things worse..
At this point, I'm not even sure how this relationship should look like and what I have the right to expect. To me, it seems like being a PhD advisor is the easiest job in the world, because no matter what you do, what you say, or how you say it, it is still the student's fault, and there is really no accountability to them (and even though it is their 'service', the advisor does get paid for students being registered for credits with them).
Right now, I'm just trying to suffer through it.