Much as I love the Horniman, my favourite museum in Britain is the National Museum of Scotland in Chamber St, Edinburgh.
I have many fond memories of days there as a child. And the extension that opened a few years back showed that even display cases and spaces can be stunning in their own right. The Andy Goldsworthy walls and the fabulous 'people' sculptures that hold some early jewellery are amazing.
Large parts of it is closed at the moment since the older half is being refurbished but when it re-opens next year, it's well worth a trip to Edinburgh (particularly given how many other things Edinburgh as to do too!)