An odd prime number p is called anti-elite if only finitely many Fermat numbers are quadratic non-residues to p. This concept is the exact opposite to that of elite prime numbers. We study some fundamental properties of anti-elites and show that there are infinitely many of them. A computational search among all the numbers up to 100 billion yielded 84 anti-elite primes.