I can recommend Ightham Mote but more for the lovely country walks around the perimeter as much as the gardens which is free. Close by is the village of Stone Street with some great pubs including the Snail- great for Sunday lunch!