The entrance to Ain’asel is not as obvious as it seems. The realm was laying on another plane, so a human for long time could enter it only through enchanted places, if they slept in or stayed there long enough for the gates to open. That’s how the legends of fey appeared in Avras.
Now, the portals are open, the gates shattered, and the humans aknowledged the fairy existence hard way. But still, spells enveloping Shadowlands (which border with Avras) are deadly for all who would just want to come – it’s hard path, before one enters Lesser Realm, not to mention Dal’coler.
And there, it were not the fae who shattered gates. This, will be revealed later.