The Mammoth Tusk for Ysolda can be found inside Halted Stream Camp, which has several of them. Alternatively, they can be found on mammoths located at giant camps throughout Skyrim. There is also an easy steal if the Dragonborn heads to Jorrvaskr, in a separate bedroom to the hall, belonging to Vignar; or from on top of a cupboard behind Hulda at The Bannered Mare.
The Black-Briar Mead can be found at the Black-Briar Meadery. Alternatively this item can be bought at most Taverns in Skyrim. The closest seller is The Dead Man's Drink in Falkreath. Another bottle of Black-Briar Mead can be found in the underground area under the castle in Helgen when you escape with Hadvar (it can be found next to a skeleton and a metallic helmet, which both are right by an empty, wooden wagon).
Song of the Alchemists cannot be obtained without stealing*. After accepting the quest, a copy appears inside Bards College. It is not marked as stolen, but if caught, it'll still count as stealing Unless the player is a member of the college. It can also be stolen from Anise's Cabin. If this quest is taken when the player's Alchemy level is higher than adept level then the player will not receive the reward (skill increase).
Only one Rare Gifts quest can be active at a given time, quest dialogue will not show up with the other characters if another Rare Gifts quest is active.