Thankfully the borders between duchies, as well as the Central Lands, were not protected by border patrols. At least, not by the actual armies of the kingdom. Considering that the Central Lands are the jewel of the kingdom, the Valorian family is strict on criminal activities in their own territories. Some small-time gangs definitely exist in this place too, but nothing even remotely on the scale of the Vesper Consortium or the Phantom League.
The Central Lands are like Pyongyang, made to look good while the rest of the country is left to fend for themselves.
That's not to say slavery and such don't exist here, it does very much so. However, only the legal side of it, meaning debt slaves, criminal slaves, etc. Kidnapping people to turn them into illegal slaves is significantly less common in the Central Lands.