The Mammoth Tusk for Ysolda can be found inside Halted Stream Camp, which has several of them. Alternatively, they can be found on mammoths located at giant camps throughout Skyrim. There is also an easy steal if the Dragonborn heads to Jorrvaskr, in a separate bedroom to the hall, belonging to Vignar; or from on top of a cupboard behind Hulda at The Bannered Mare.
Song of the Alchemists cannot be obtained without stealing*. After accepting the quest, a copy appears inside Bards College. It is not marked as stolen, but if caught, it'll still count as stealing. It can also be stolen from Anise's Cabin. (*There is an exception to this: should the Dragonborn decide to join- or is already a member- of the Bards College, then taking the book will not be counted as stealing) If this quest is taken when the player's Alchemy level is higher than adept level then the player will not receive the reward (skill increase).