I applied for PhD programs after working for some time in industry. I would say the hardest part was getting recommendation letters - not that people were unwilling to provide them, but it felt slightly awkward emailing professors from a few years ago, re-introducing myself, and asking for recommendations. Other than that, I don't think it was much different than applying as a traditional BS graduate - although there were several times when I wish I had an academic advisor who could help me with my personal statements, give advice on which programs were a good fit, etc. On my visit weekends to campuses, there were several people who were currently working...all of whom had varying degrees of comfort in telling their employers they were applying to graduate school. If you're wary about telling your employer, then obviously there's that challenge.
In summary - yes, it was intimidating coming back to academia after being in industry, but very well worth it. I'm happy with my application cycle, and I think I have a MUCH better idea of what I want to do then when I was a BS graduate.