The priest is probably a Tillichian-style believer. It's mainly borrowed Heideggerian existentialism at work. It's possible to believe in God, yet not think he exists. It's a problem of using the word "exists"-- if you believed that God is what supports all of existence and laid the groundwork of reality, you wouldn't think he exists either. "I am who am" and all that. So.
"Words are deeds." - Wittgenstein