The Mammoth Tusk for Ysolda can be found inside Halted Stream Camp, which has several of them. Alternatively, they can be found on mammoths located at giant camps throughout Skyrim. There is also an easy steal if the Dragonborn heads to Jorrvaskr, in a separate bedroom to the hall, belonging to Vignar; or from on top of a cupboard behind Hulda at The Bannered Mare, however if you sell some Firewood to Hulda it can be taken freely.
Song of the Alchemists cannot be obtained without stealing*. After accepting the quest, a copy appears inside Bards College. It is not marked as stolen, but if caught, it'll still count as stealing. It can also be stolen from Anise's Cabin. (*There is an exception to this: should the Dragonborn decide to join- or is already a member- of the Bards College, then taking the book will not be counted as stealing) If this quest is taken when the player's Alchemy level is higher than adept level then the player will not receive the reward (skill increase).
Only one Rare Gifts quest can be active at a given time, quest dialogue will not show up with the other characters if another Rare Gifts quest is active.