You can get an unlimited amount of arrows by doing the following trick:
- Find a Guard training.
- Pickpocket his current arrows out then place one arrow of choice.
- Collect the arrows shot.
It may be a good idea to get your arrow back if you plan on doing extremely unlawful activities, such as killing chickens or slapping them stupid kids.