The Castle Hill is dominated by two castles. The Old Castle (Castello Vecchio) was mentioned the first time in a written document in the year of 1150 as „Castrum Groxi“, but archaeological results indicate the construction already for the end of the 10th century. The dominating church spire from a later period was partly restructured at the end of the 19th century. The first construction-phase of the corresponding small church is dated by archaeological results already into the 7th or 8th century. Most probably built to cover the two medieval tombs cut into the rock. The New Castle (Castello Nuovo) was built between 1350 and 1375 by the famous family Visconti, the feudal lords of Milano. Erected as a stronghold it was perfectly adapted to the strategic needs of the 14th century. It had a double defensive wall, a moat and a solid keep (donjon) in the inner courtyard.

The oldest and still visible building on the castle hill of Grosio, the Castello Vecchio at the southern end of the hill was most probably built at the end of the 10th or the beginning of the 11th century and was mentioned for the first time in a document in the year of 1150. Called as a “Castrum Groxi” it became known even as “Castello di San Faustino”, named after the two Roman martyrs Saints Faustinus and Jovita, in whose honor the castle chapel was dedicated to. The preserved walls still show the original extension of the structure. From an architectural point of view it is interesting that the western wing of the castle was not only built on the rock but was partially cut out from the rock to create more space. The spire was probably erected in a later phase when the decline of the Castello Vecchio had already started and for sure was partially restored at the end of the 19th century. The castle chapel, from which remained only in its foundation walls, once covered a double-grave that was cut into the rock. Furthermore archaeological excavations proved the existence of a previous building, most probably a hall church from the 7th or 8th century that was already built to cover the graves. Archaeological remains of the burials itself could not be found. According to written sources the Castello Vecchio was inhabited until the 16th century.

The Castello Nuovo was built as a stronghold between 1350 and 1375 due to the military-strategic needs of the 14th century. The castle is surrounded by a double defense wall and had next to e solid keep even a fortified internal tower. Apart from a military expedition by Giovanni Cane in 1376 to Bormio, which had its starting point here, the Castello Nuovo was never involved in military confrontation or siege. Today the castle is only preserved as a castle ruin because after 1526 the new political power in Valtellina ordered the destruction of all fortifications in the valley. That way the dominion of the Three Leagues wanted to avoid any potential danger that could emanate from their own castles. During the period of the Bündner Wirren (1620-1639), the Thirty Years War in Europe, the structure was used by the French under Duke Rohan as a military base and got partially restored. The Castello Nuovo is nowadays the best-preserved example of all the historic defensive structures in the Valtellina.
Between 1992 and 1997 in the area of the Castello Nuovo archaeological excavations could document the remains of a bronze and iron age settlement, dating from the middle of the 2nd to the end of the 1st millennium BC. Some of the small finds are now exposed in the excavation museum. The studies of those excavation results could prove for that time period alternating cultural connections and relations into the Upper Rhine Valley (Graubünden/CH) in northwestern direction as well as to the southeastern region of the Alps, into Trentino and South Tyrol.