Tower of Babel Theory
In short, there are many symbols all over the stone tower that indicate the architects were heathens. They desecrated the symbol of the Triforce with a statue depicting it under a hunched-over man licking his privates. Also, the entrance is a giant humanoid head with a flaming finger pointing to the sky; contempt for the goddesses. There are four structures around the giant's head which look like phalluses pointing to the heavens. Could imply rape and dominance of the four male giants over the female goddesses.
A strange "portal" is above the temple, both on the inside and the outside, so it is not a mistake. This is likely a portal to the heavens, where the builders would attempt to take the Sacred Realm for themselves.
Their weapon is the Giant's Mask, fashioned in the likeness of their gods. It would take powerful magic to create such a mask.
The Garo, who seem to have control over the Stone Tower, likely use Shadow Magic like their Sheikah "alternates" in Hyrule.
The tower was flipped when the Goddesses shined their light on it, and the portal to heaven became a portal to hell; a twisted, sandy wasteland. The statues in there suggest they found Majora's Mask in the wasteland.
They must have been the "ancient tribe" the Happy Mask Salesman mentioned. They are the only tribe found in the game, after all, and it fits well.
They used Majora's Mask as a part of their Shadow Magic, and attempted to use it to assault the Sacred Realm, likely fused with the power of the Giant's Mask.
The "tribe of legend" disappeared.
So, what do we know about the Twili and the Fused Shadows? The Twili use Shadow Magic, and the Fused Shadows are a combination of all the great dark powers they used. They were a tribe that was sealed away for their assault of the Sacred Realm.
Note that Zant is a high ranking person among the Twili, and Garo Masters are high-ranking among the Garo. They both have similarly-shaped robes, use a unique twin-sword style, and have obelisk-like helmets.
Does it seem a little hard to believe that the Garo became the Twili?
On the back of the Fused Shadows, there is the Garo "eye" symbol, which is exactly like the Sheikah eye. It also appears on the top of Midna's throne.
Note, the only way to see the eye symbol is to rip the model straight from the game and view it in a 3D modeling program. It's under the curly parts in the back.
100
