Because the loudest anti-immigration messaging paints immigrants as criminals, invaders, or parasites, so the whole movement gets tagged with that stink.
Plenty of people oppose illegal immigration on legal/economic grounds without ever calling anyone a parasite. Lumping all opposition in with the loudest voices is just a lazy dodge.
