The old castle was transformed into a country residence in 1451 by Michelozzo on commission to Cosimo the Elder. The new and sumptuous villa, surrounded by gardens and a vast park for hunting, it became the favourite summer residence of the Medicis and their guests. The nineteenth-century transformations, ordered by prince Marcantonio Borghese, largely cancelled the original aspect.

The villa is also connected to the world of technology. Indeed, a manufactory that made the famous majolica was built next to the castle. There were two kilns, one used to make an average quality production, the other for a high quality production. Founded in the late 15th century by potter brothers Pietro and Stefano di Filippo, the majolica manufactory at Cafaggiolo produced authentic masterpieces throughout the 16th century, destined to the grand-ducal family (vases, jugs and cups), as well as to city institutions, such as the Santa Maria Nuova Hospital of Florence.