To the south of Swindler's Den, two flawless amethysts can be found at the Shrine of Stendarr. One is in plain view and the other is in a small chest, other gems may spawn instead of the flawless amethyst.
During the Thieves Guild quest, "The Pursuit", one may be found in a bowl on Mercer's desk in the tunnels underneath Riftweald Manor, along with many other gems. One can also be found on his person in The Raggon Flagon- Cistern.
Upon completion of "Bow to the Master", 5 random gems are given to the player as a reward. One of which can be a flawless amethyst.
Can be found as easy random loot in certain containers with the Prowler's Profit perk.
One can be found in Korvunjund during the Civil War questline. When ordered by Legate Rikke to search for a lever to open a portcullis, you will find a dagger on a pedestal. Grab the dagger and remove it. The pedestal will activate and open a passage to hiddden treasure. Next to the chest is a flawless amethyst.
There is a chance one can be found in Mistveil Keep to the chest on the right of the throne (if you are looking at the throne with the main doors behind you). However, this may just be when Maven Black-Briar is Jarl of Riften.