On ground steeped in history, between the mountain ranges of the Ester- and Ammergebirge, where it is proved that the Roman road was built around 200 after Christ during the reign of Septimus Severus and led towards the North, castles were set up later, which dominated the landscape and which also included the castle of the counts of Eschenlohe. This castle was located on a hill – the Vestibühl – at the Southern end of the town.
The possessions of the counts of Eschenlohe extended to the south via Partenkirchen and Mittenwald far into the Ulten Valley in South Tirol, to the north until Murnau and to Lake Staffelsee. Further north, their possessions were spread between those of the counts of Andechs. It is assumed that they were related to each other, which is concluded from the fact that the coats of arms of both dynasties of counts are identical.
Udalschalk, presumably a brother of Count Berthold I of Eschenlohe, was ordained Bishop of Augsburg in 1184. Berthold III, mentioned in documents for the first time in 1246, was without male descendants and sold his complete possessions in 1294. Part of it, including Eschenlohe, became the property of the territory of the Bishop of Augsburg; this territory in turn was sold to Emperor Ludwig in 1332, who handed it over to the Ettal monastery. The possessions Partenkirchen and Mittenwald were sold by Berthold III at a price of 1000 marks of lot silver and an annual contribution of 5 cartloads of wine from Bozen to the Bishop of Freising – for the rest of his life. There is no evidence anymore of the name of the dynasty of the counts of Eschenlohe as from the 14th century. After the dilapidation of the castle, settlers built their houses from the stones of the castle. A presentable settlement was set up.