Yep, it took me all of three seconds to reverse this setting (admittedly, when I had been told where to look).

I connect to gchat using a third party program, and my contact list in this program is still full of unwanted contacts that I have to manually delete.

I said do it before you join to save people in a similar situation.