I would say it is more akin to saying that mathematics is something that is discovered, rather than invented. The one subject in which you can know things.
I'd agree that maths is discovered rather than invented, but the same goes for Uranus, or any other part of someone's anatomy.
What's different about maths is that it promises, and often delivers, knowledge without always needing to be validated by external observation. As such, maths has a dangerous allure for those who are hostile to modern science. I think such people also see maths as essential to science, then pick up that pure maths isn't actually science, and conclude that there is something flawed about the whole discourse of modern science.
BTW, is this actually what is talked about on these inter faith walks?