During Harry Potter's life, he was one of the wizards who entered the Wizarding World without understanding that there was a distinct separation between those who used magic and those who didn't. But those who had no magical inclination were known as muggles. Though it sounded like a derogatory term, it was actually used in kindness. That said, there were still many who used magic who saw muggles as enemies and lesser beings because of their magical limitations.
Source:https://www.cbr.com/muggles-triumphs-struggles-wizarding-world/