The garden in its present form was created by Hornel with the aid of his family between 1901-1933 and was probably continued by his sister until 1950. No layout plans are known to exist. The Trust has a collection of early photographs which show the garden in c.1930.
Broughton House was originally the town house of the Murrays of Broughton in Wigtownshire and Cally. It was purchased by Edward Atkinson Hornel in 1901. Born in Victoria, Australia, in 1864, he came to Britain with his parents 18 months later and settled in their native Kirkcudbright. He spent most of his life there prior to 1880 when he went to Art College in Edinburgh. He was a contemporary of W.M. Frazer RSA, Sir D.Y. Cameron, W.S. McGeorge and Wishart, the latter two of whom he spent time with at the Verlat Academy in Antwerp, Belgium. In 1883, his painting 'A Glimpse of Kirkcudbright' was accepted by the Royal Scottish Academy. He declined the offer of an Associateship of the Academy in 1901. In the same year, he purchased Broughton House and subsequently added the gallery and also remodelled the garden at the back of the house. Hornel was one of the founder members of the Glasgow School of Impressionist painters. He travelled extensively with fellow member George Henry, in particular to Japan, Burma and Ceylon, which not only provided inspiration for his painting but also material for the garden which he had established at Broughton House. Charles Oppenheimer, the landscape painter, was his friend and also his tenant of the neighbouring house on High Street.
Hornel compiled a fine library over many years and contributions were received from many of his contemporaries, among them, Thomas Frazer of Maxwellknowe, Dalbeattie. It now contains some 15,000 books.
Throughout his life, Hornel was looked after by his sister, Elizabeth, who shared his interests in art and gardening. On his death in 1933, he left the life-rent of the house to her. On her death in 1950 the Trust, which now manages his estate, took over his Bequest. Hornel wished that Broughton House 'be preserved as a public art gallery for the benefit of the people of the Stewartry and visitors thereto'. The Friends of the Hornel Art Gallery & Library help to raise funds to augment the Trust.