Norfolk County Council do a booklet about walks around Aylsham: Most of them start at the Bure Valley Railway Station or the market place (free parking if you look around)
Aylsham has a lovely church to check out as well.
The long walk of the whole of the Bure Valley goes from Aylsham to Hoveton if you're feeling more ambitious - More info
http://www.norfolk.gov.uk/view/NCC129965
Not too far away from Aylsham is the Blickling Hall Estate Walk
