There's more to do in Crystal Palace, and some better architecture, but you have to balance that with being that bit further out with poorer transport links. It's also a bit rougher (albeit no downtown Mogadishu) and suffers from being split between three boroughs, meaning no-one really takes responsibility for the area as a whole. I concede that Forest Hill doesn't appear to fare much better under Generalissimo Bullock.

They're not wildly dissimilar areas though and it's easy to get between the two - 5 mins on the train, a little longer on the buses. As an observation, although the converse might not be true, if you lived in Crystal Palace you would have little reason to go to Forest Hill.