Now this theory is based mostly off of speculation, so there are quite a few holes, but I put a fair bit of thought into this.
To start, we know that the Snow Elves were driven into exile by the Ancient Nords who forced them to seek refuge with the underground Dwemer. It's revealed in the Dawnguard DLC that the Dwemer demanded that the Snow Elves must sacrfice their sense of sight in order to stay with them, and that this was supposedly accomplished by using a toxin. We also know that the Dwemer seemed to completely vanish all at once, with no warning or signs of any struggle.
Now we know that Snow Elves were a proud people, and that their relationship with the Dwemer was sketchy at best, and that there were several elves who strongly opposed this idea. My theory starts with the idea that at least one, or possibly a group of Snow Elves, decided to do something about this by resorting to consorting with Daedra, Clavicus Vile in particular. Clavicus Vile is well known for making unfair bargans, and manipulating mortals for his amusement, so when the Snow Elves came to him asking him to make their race dominant over the Dwemer, or maybe seeking revenge on them for taking their sight. The Dwemer weren't stupid, however, and apparently didn't trust the Snow Elves to begin with, so it's possible that there was a failsafe to their toxin that would cause them to become the twisted Falmer if for some reason the Dwemer were no longer around to continue adminstering the toxin.
Another thing that somewhat supports this theory it that Falion in Morthal claims to have communicated with the Dwemer. If Clavicus Vile were somehow involved in their dissapearnce, it is likely that they weren't killed off or destroyed, but instead displaced to some realm of Oblivion, making it entirely possible that a man like Falion to have somehow made contact with them.
Now like I said, this is all speculation, so I'd like to see what you all might have to say about it, whether it's disproving or adding to the theory.