I think God gives his authority to the humble. The authority the church or any society takes is self evidently a "taken" thing not a "given" thing. "Those that were first will be last" springs to mind when I think of this.
That is certainly true and could be used to define what is really of God and what is not.
God gives to those that follow him and others follow those that humble themselves before God. Yet when that humble attitude disappears eventually so does OGds authority on that person disappear.
The formal church on the other hand, along with the other human institutions, take for themselves an authority as if it is theirs by right. They have forgotten that as God gives, so can he also take away.