It used to be the centre of an aristocratic estate, which was separated from the royal and free town of Varaždin in terms of ownership and law. The construction of the present-day fortress started in the 14th and it finished in the 19th century. Its oldest part is a central tower. Its benches and canopies, which can be found on the ground floor, are the most beautiful examples of Gothic secular plastic in northern Croatia. During the wars against the Turks in the 16th century, it was rebuilt as a Renaissance Wasserburg- a castle surrounded by high earthen embankments with bastions and a double water belt. The rebuilding was carried out by Italian builders from the Lake Como area. The construction manager was Domenico del Lallio, the main royal fortress builder on the Slavonian Military Border in the second half of the 16th century. In the past it was owned by a number of important aristocratic families, such as the Counts of Celje, Ivan Ungnad, Juraj Brandenburg and Croatian Ban Toma Erdӧdy and his descendants. It became the property of the town of Varaždin in 1925.
The Varaždin City Museum was opened in the Old Town in 1925 in honour of the millennium anniversary of the Kingdom of Croatia. The Culture and History Department of the Museum houses numerous collections which are nowadays exhibited in stylish rooms. The most valuable collection, which can be seen by visitors in 40 museum rooms, consists of guild items, furniture, historical portraits, weapons, paintings, porcelain and glass as well as other everyday items which were used by the aristocracy and wealthy citizens from Varaždin. Particularly valuable are collections of items which belonged to two famous people from Varaždin, a politician and a writer Ivan Kukuljević Sakcinski and a linguist Vatroslav Jagić.
