Part of the reason for a weird phrase like "illegal immigrant" catching on is that it looks like you're being weirdly picky when you try to argue against it on the basis that entering a country illegally is just a misdemeanour, not a crime. Such arguments presuppose a respect for the law which in this case I suspect it doesn't deserve.