Thanks to whomever suggested Godstone Farm, we tried it out last weekend - my wee lad had a great time as did the rest of the family.

I cobbled together this list from the various posts, bit of a TL:DR I guess..

- Petts Wood
- Keston Ponds
- Eltham Palace
- Chartwell House, Rochester Castle and the superb National Fruit collection at Brogdale which also has a miniature railway.
- Godstone Farm
- The Pinetum at Bedgebury - fantastic walking or hire a bike with the kids
- Bluebell Railway
- Knowle Park in Sevenoaks
- Ightham Mote nearby
- The Bluebell and Kent & East Sussex Railways
- Sheffield Park - handily close to the Bluebell Railway
- Bedgebury
- Camber Sands, Winchelsea, Rye and Hastings
- Box Hill
- RHS gardens at Wisley

Links
http://chislehurst-caves.co.uk/
http://www.bexleyheritagetrust.org.uk/hallplace/
http://www.highrocks.co.uk/
http://www.shoreham-aircraft-museum.co.uk/