The newly married artist couple Carl and Olga Milles acquired a property 1907 on the cliff of Herserud high above lake Värtan on the island of Lidingö. Their intention was to build a home incorporating space for their art studios.

The house was designed by architect Carl M. Bengtsson and was built 1908. During the following half-century Millesgården was expanded and developed in collaboration with Carl's half-brother, architect Evert Milles. Between 1911-13 the first addition was built, an open-air studio in the form of a loggia wing, intended to improve Carl's work environment. The sculptor had contracted a serious case of silicosis from inhalation of the dust from his stone carving.

As Milles income increased from the many new sculpture commissions during the 1920's, adjoining properties were acquired along the south slopes. With the construction of the middle terrace and the Small Studio both the area of the sculpture garden increased as well as providing further work space for making sculpture.

During the Milles couple's absence 1931-50, while Carl was professor at Cranbrook Academy of Art in Michigan, USA, building activities at Millesgården came to a halt. But Evert Milles continued to draw up blueprints, in preparation for future building projects.

In 1936 Millesgården was constituted into a foundation, which was donated to the Swedish people. In anticipation of Carl and Olga's return to Millesgården, 1950 saw the construction of the spacious lower terrace with monumental replicas of free-standing and fountain sculpture from Sweden and the USA.

The lower terrace with the couple's new home, Anne's house designed by Evert Milles and furnitured by founder of furniture copmany Svenskt Tenn, Estrid Ericsson, was nearing completion at the time of Carl Milles' death, September 19, 1955.