Well, the job of the chaplain is to provide comfort.

Can a Catholic chaplain provide comfort to a Buddhist? To a polytheist? To an atheist?

So why can't an atheist do the same? There are no gods, that's all fantasy, but that doesn't prevent me from having empathy with a religious person. It doesn't prevent me from knowing about their beliefs so that I can offer them suggestions tuned to those beliefs.