Inside, there is only one route that is open. It leads to a room containing many cells and three ghosts. At the back of this room, on a table, will be a key. This key will unlock any of the locked cells (expert-level locks). When facing the table with the key, there will be a cell on the left that has a hole in its wall, leading over to the next block of cells, (via another cell). Through this cell and out of its conjoined cell is another ghost up the stairs at the opposite end of the room from its entrance. At the end of the hall, enter the room on the right, turn right to a chest. The book Song of Hrormir (increases two-handed skill), will be leaning against this chest.
The chest that has the Song of Horomir leaning against it also has a master-level lock, however, the key found earlier opens it.
There is a Guard's Orders note in the same room with the book The Legend of the Red Eagle. When entering, turn to the left, pick up the note that reads "We must get the prisoners and execute them by killing or by drowning, and escape this prison before the storm", suggesting a storm was why the prison was abandoned.
In the first room of the prison (with the ghosts), there will be a skeleton on the ground holding the Prisoner's Plan, which suggests that there was a plan for the inmates to overpower the guards and escape.
There is a trap door in the last cell on the right in the first cell-block, (opposite the cell with the hole in the wall), that leads outside.
PC 360 PS3 It is possible that followers will be unable to find a path leading to the entrance of this location and so will not follow inside, however on exiting and crossing the river they will try to join back up. Fast traveling will reunite the Dragonborn and follower(s).