Did you invite them onto your property? No, then call the police!! I doubt they were going to sweep your front yard or plant roses.

If it looks dodgy and they are on your property then call the police. I think the police would rather turn up to a non incident than another burglary or as you were in, something far worst.