Built around 1054 by the Normans for defence purposes, the Aragonese Castle became the home of the Acquaviva Counts and was later transformed into a pleasant stately residence through several alterations. The ancient megalithic walls, dating from the 6th-4th centuries BC, are still visible at the base of the Torre Maestra, the only surviving example of a Norman tower.
The trapezoidal plan is distinguished by four towers in the corners: the Maestra or Norman tower, about 25 metres high, composed of regular limestone ashlars and started in the 11th century by the first Count of Conversano Goffredo Altavilla the Norman, another tower similar to the previous one, smaller in size, with elegant Gothic windows and a belvedere embellished with arches, the cylindrical tower, which became the symbol of Conversano, replacing one of the four quadrangular towers of the castle, the polygonal tower built by Andrea Matteo Acquaviva, son of Giulio Antonio. The latter has a peculiar shoe shape, with slightly sloping walls to cushion artillery shots, a crenelated top suitable for cannon mouths and narrow slits along the sides for the placement of ammunition. Opposite the polygonal tower, the original drawbridge is found.
Outside, the façade is adorned with windows and balconies dating back to the 19th century. On the inside, the castle houses the Municipal Art Gallery, with 17th-century paintings from Finoglio's "Jerusalem Delivered" cycle, the Francesco Netti Art Gallery, an exhibition of historical clothing entitled "Nuptials at the Castle", a space for temporary exhibitions and a conference room. Disabled access is from Piazza Conciliazione.