Spatial and territorial organization is an important factor in the configuration of control and surveillance strategies at work in our society. Architecture in general and certain buildings in particular have been key devices in exercising said control and surveillance. In these pages we look at the building structures in facilities specially designed for the control and custody of their occupants. We begin with the Casas de Misericordia, which appeared in the 16th century and were designed to house marginalized people mostly from urban environments, and go over the morphologies of prisons, hospitals, and quarantine stations. We analyze the transfer of building structures from one kind of establishment to another, and discuss how their specific functions progressively fixed their morphologies. Finally, we focus on the discourse built in Spain, from its origins in the 16th century until its realizations in the first half of the 19th century, when a marked institutional specialization took place and building structures became more stable, dealing with theoretical proposals as well as what was done in the practice.