We had a student in my undergrad that did this. There was always a constant "yes" or "mmmhmmm" coming from him. It was especially annoying when the professor would ask something along the lines of, "everyone understand?" (He would immediately say yes, and the professor would move on, anyone hanging on had to reel the professor back in).
No one ever did anything, it kind of screwed him over a few times because he was only following about half the time. Some professors call you out by asking you a question. He started doing it a bit less because of that.
I'm not sure why it reminds you of black congregations. That comment seems more racist than bringing it up to him. I can see that you're annoyed, and I honestly understand where you're coming from, but this has really nothing to do with race, and I don't think he would think that if you brought it up to him. I've found that at least our classmate just wanted sort of suck up to the professor. Make the prof think more highly of him.
I don't really have any advice of bringing it up. We never brought it up to the student, because it is a little hard to word it in a good way that doesn't sound like you're awkwardly asking them to shut up. Personally I always sat away from him so that at least if he mumbled, I couldn't hear it.