Inside the cave there is a shrine to Sheogorath. Also note one secret passage to a chamber with a chest. The rocks will crumble down revealing the passage when one touches the wall. On the map it is the short tunnel just north of the entrance. The cave is inhabited by scamps.
People in Leyawiin report strange noises and smells coming from Rosentia Gallenus's House. She has accidentally acquired the Staff of the Everscamp, which produces a steady population of four scamps to follow the staff-bearer.