Requimortis
No one was practicing fel magic at the time. The Highborne brought the Burning Legion to Azeroth through their brazen, blatant abuse of magic lead by their queen, Azshara. The Pandaren sealed themselves away to avoid destruction. As it turns out, Liu Lang was not content with that decision and set out for the world upon the back of the turtle now know as the "Wandering Isle". Every new Pandaren created starts there and it is also the birthplace of Chen Stormstout. Much like the Orcs who were at one point greatly corrupted by the demons, there is no reason the Pandaren can not use the devastating demonic magics to their benefit. Do you have any other arguments I might poke holes in? Or are you content with letting this one sink alone?
The abuse of magic caused the Burning Legion to notice Azeroth,
The Burning Legion practised Fel magic (is where it is from after all) what corrupted Xavius, whom then Corrupted Azshara (and the Highborne) whom went about and cause the Great Sundering.
One would assume the Pandaren Empire forever being best of friends with the Highborne would of had some sort of Connections within the Kaldorei Empire to tell them that it was not just them overdoing it with the magic.