I was part of that following for a couple months too, years ago.
It is the members of their following who are just shills, that I dislike.
Those who know he's full of it and his claims are full of it, yet they like to join up in stringing everyone along and stirring up arguments and proselytizing believers for the cause or just for the heck of it.
My dear, everyone one has been duped in the past, mainly because we want things to be true and they simply aren't and there are people who will prey upon our hopes for whatever purpose they have. You came to realize something didn't add up and you broke away from them. The people I have little sympathy for are the ones who now see the truth about him and yet cling to him like like a frightened child clings to it's mother. They have seen the truth and know he's a confirmed multiple liar, hoaxer and all round nut case, but they can't let him go. Reminds me of a cult following really......and if you can believe what's being said, Dyer is doing a lot of expeditions where people pay to have the privilege to go on a wild goose chase with him showing them the way.
I can only hope that they come around much as you did at some point before they themselves become as jaded as Dyer.