Marquis Pietro Torrigiani inherited a small garden towards the end of the 18th century, which he successively enlarged with the purchase of several bordering lands. Between 1813 and 1814, he entrusted Luigi de Cambray Digny with remodelling the park. The result was a large English garden with a romantic flavour and a rich variety of plants, sculptures and buildings, which once finished became the subject of a guide to facilitate visits by guests. Built in 1821 by architect Gaetano Baccani, the tower hosted a library on the first floor, while an octagonal room on the second floor housed a collection of astronomical instruments, including several telescopes. The terrace on the top was intended for contemplating the sky and astrological speculation. The tower indeed represented the final stage of an initiatory itinerary that started at the beginning of the park with the statue of Osiris, Egyptian god of death and resurrection.

The garden is particularly important especially for the history of botany. In the early 18th century, Pier Antonio Micheli, along with other lovers of natural science, founded the first European botanical society and conducted experimental cultivations of many plant species. In 1839, A. Pucci drafted an inventory of the species in the garden, recording 5500 plants in pots and about 13,000 plants in soil. In 1841, the garden hosted the meetings of the Botanical Section of the Third Congress of Italian Scientists under the presidency of Antonio Targioni Tozzetti.

Though without a part of the works of art that once adorned it, Torrigiani Garden today is one of the best conserved English gardens inside the walls of Florence.