Try the Arches in Greenwich. Not that easy to get to from HOP unless you have a car but I put up with the journey without one 'cos it's worth it. Nice pool - always at least 3 lanes free for adults at most times of the day plus they have a separate leisure pool for families. It's not sleek or modern but is kept reasonably clean and provides a much pleasanter swimming experience than E Dulwich in my opinion.
At weekends it's nice to follow a swim with a walk around Greenwich Park which is just a couple of minutes away.