You cannot claim something over something you don't even have from the beginning. I'm outside the town which means I'm outside the jurisdiction. I'm not even on any plot that the town owns so why in the flying fuck is it that the town already has rights over something outside their town? If you want to expend go build around my house.
Seriously, this isn't even mentioned in the book we are given and in the rules. This is just plain out bullying. It is as if I have to join a town just so I have security over my house which is nuts. I built it near the town so I can get to the center faster. And wouldn't me being closer mean you have a customer near by?