There is one located in an urn inside of the Shrine of Mehrunes Dagon. (This may be a leveled occurrence, however it has not been confirmed. The Shrine of Mehrunes Dagon is known to have different loot based on level)
There is one near the Shrine of Mehrunes Dagon. Take the side path to the left and follow it up the mountain, then look for a small altar on the right.
North of Riften – A settlement of Shor's Stone – there is a chance one can be found in Sylgja's house or her possession.
During the Thieves Guild quest, "The Pursuit", one may be found in a bowl on Mercer's desk in the tunnels underneath Riftweald Manor, along with many other gems. One can also be found on his person in The Raggon Flagon - Cistern.
Upon completion of "Bow to the Master," 5 random gems are given to the Dragonborn as a reward. One of which can be a Flawless Amethyst.
Can be found as easy random loot in certain containers with the Prowler's Profit perk.
One can be found in Korvunjund during the Civil War questline. When ordered by Legate Rikke to search for a lever to open a portcullis, the Dragonborn will find a dagger on a pedestal. Grab the dagger and remove it. The pedestal will activate and open a passage to hiddden treasure. Next to the chest is a Flawless Amethyst.