I haven't completed the quest yet so he stays with me. He's annoying, but he sure makes for a great distraction when encountering bosses and many enemies. Dragons... Trolls... Giants... Everyone ignores me because Cujo jumps right in and gnaws on 'em. I went to one location with him and Serana (bears and a wolf in a cage) and those two got into a pretty massive fight with each other. It was pretty entertaining to see two invinceable followers go at it with each other. 4th of July on my TV - and I just sat back and grabbed the popcorn. The issue was remedied as soon as I traveled into a cave, door, etc. Then they were friends again.
What a g'boy.