The castle's original building is thought to date back to the 12th century, a time when the San Martino family was gaining prominence in the Canavesana area. The first transformation of the original manor house was carried out in 1646 on the wishes of Count Filippo San Martino, whose design is attributed to Amedeo di Castellamonte. After the death of Filippo di Agliè, the castle had a largely symmetrical configuration, with an inner and outer courtyard, which opened towards the village, and the east façade characterised by two pavilion towers overlooking a garden with flower beds. In 1763, the Savoy family bought the castle as a dowry for Carlo Emanuele III's second son, Benedetto Maria Maurizio, Duke of Chiablese, at which point a new, magnificent project was undertaken to redevelop the complex, the work of architect Ignazio Birago of Borgaro. The village was also involved in the extensive renovation plan, with the construction of the current parish church, which was connected to the castle by a two-storey covered gallery, which is still in existence. Birago called many artists linked to the Court of Turin to Agliè, such as the brothers Filippo and Ignazio Collino for the sculpture of the fountains, and Giuseppe Bolina for the decorative elements of the large entrance hall. The gardens and park were also designed, with a stretch of water along the longitudinal axis, which has been preserved to this day in the part of the roof garden near the castle. During Napoleonic rule, the castle was turned into a hospice, while the park was subdivided and given to private owners, after the Restoration, in 1823 the castle returned to royal ownership. Two years later, the last adaptation of the flats by King Carlo Felice began. The castle was redecorated by court artists, including the sculptor Giacomo Spalla, who fitted out the Sala Tuscolana, with artefacts found in the Villa Rufinella in Frascati, owned by Carlo Felice and Queen Maria Cristina. In the second half of the 19th century, the Green Gallery was refurbished. Between 1830 and 1840, the park was extended to its current size of 34 hectares and radically altered by Xavier Kurten in the "English" landscape style, discarding green symmetries for a Romantic-style layout, which can still be seen today. In 1939, the castle was sold to the state and turned into a museum, retaining its structure and furnishings.