In the 12th and early 13th centuries, the manor belonged to the de Morvilles, one of whom was Lord High Constable of Scotland. By around 1260, the greater part of the lands belonged to Sir William de Abernethy and it stayed in that family's ownership until 1643 when the 9th Laird sold it to Sir Andrew Fletcher, Lord Innerpeffer. Lord Innerpeffer supported the Royalist cause and was fined L5,000 by Cromwell in 1648. He died in 1650 and Andrew, the 3rd Fletcher laird, born in 1653, was later known as 'Fletcher of Saltoun' and as 'The Patriot' for his opposition to the Union of Parliaments. He became an exile, travelling to Spain and Hungary before returning to Scotland with William of Orange. He died in 1717 in London. His brother, Henry, looked after the house and lands at Saltoun and succeeded him as 4th Laird; it was his wife, Margaret Carnegie of Pitarrow, who introduced the barley mill to Scotland. She also introduced weaving mills, for the weaving of 'Hollands' in c.1710. The next laird was another notable historic personality, Lord Milton, the distinguished judge, and the formal gardens shown in the 18th century plans were designed by him and featured several fountains. He was succeeded by his first son, Andrew, who approached Robert Burn to build the library wing in 1775. He was succeeded by his brother John, General Fletcher, who had spent most of his army career in America and Canada, before returning to manage the estate at Saltoun following contraction of a fever. He died in 1803 and the house was leased over a period of about ten years, during the next laird's minority.
In 1818 John Hay drew up plans for a new kitchen garden with conservatories and an extensive shrubbery to the south of the garden on the opposite side of the river. The kitchen garden and hothouses are shown to have been carried out on the 1st edition OS map of c.1853. In 1819, William Burn was commissioned to carry out extensive additions to the house, which totally encompassed the earlier work of his father. Thomas Dick Lauder, writing in 1846, refers to the bowling green and luxuriant wilderness of evergreen trees and shrubs, but bemoans the fact that the laird had been advised to remove the formal gardens. By 1853, only informal planting is shown around the house and along the river. There is no sign even of the bowling green with its 'yew hedge of immense height and thickness'. By 1885, John Fletcher who succeeded as 9th Laird in 1879 held nearly 4,000 acres in the shire. He married Bertha Mansel Talbot of Margam Park, Glamorgan to which their son Andrew succeeded in 1918. He lived mainly at Margam until his death in 1950 when he was succeeded by his son, John. No major changes took place from then until 1970 when the Fletcher family converted the Stable Court for their own use as a dwelling. The house was sold with about three acres of land in 1970 and was converted into flats. A further area of the policies is let on a 99 year lease to the flat owners. The 12th Laird, Andrew, succeeded his uncle in 1972.