Remember when Rumple said that the Hat can be used to travel to magical worlds? I think he really meant that it can access worlds in their Universe. Recently, it's known that Victorian England can be accessed through the Hat, and Victorian England isn't a magical world. The presence of Magic is there because the White Rabbit and Jafar used it but other than that, it ain't Magical. Like the Land Whithout Magic, the presence of Magic is there, like what the Dragon said, as inhabitants from the Magical Universe uses Magic, somehow, here. But, the Land Without Magic cannot be accessed by the Hat.
Any suggestions?