Firstly, I'd use the link I gave above to complain to TFL individually about the service and see what response you get.
Armed with that, it's then probably worth contacting local councillors and/or MP to see if they can get a better response from TFL or escalate it any way. I'm not sure if it's something that the Forest Hill Society might be able to add some weight against too. If enough people start to make a noise about it and the number of complaints increase, then something might get done.
I have been more aware of watching out for the P4 while on the bike since this thread came out - although I generally only share the route with them between Dulwich Village and the Horniman. I must say even in the last week, I've been surprised by how many times they come along in pairs or with a less than 5 minute gap which suggests that there are much longer gaps elsewhere in the service. On Tuesday and Wednesday night, the South Circular was entirely snarled up though and I'm sure that on Weds, I passed three of them!