The estate of Sound covered the westerly part of Shapinsay, passing from the Tulloch family to the Buchanans in the 17th century and to the Feas of Clestrain, in Stronsay in the 1720s. The House of Sound was burnt by Hanoverian troops in 1746 but was replaced. Laings of Papdale acquired the estate in 1770 and then in 1782 it was sold to Major Thomas Balfour.
Balfour began a series of agricultural improvements on Shapinsay and c 1785 founded Balfour village which was settled with 'joiners, carpenters, weavers, tailors…' (Sinclair, 1795-8). His considerable improvements involved:
'an extensive garden laid out; the lands are substantially inclosed and judiciously cultivated with the English plough; many barren fields are, by cultivation, made fertile…. In short Cliffdale, which is the name of this gentleman's seat, taken in conjunction with its appendages, exhibits to the eyes of a stranger coming from the sea, or from Kirkwall, rather the appearance of a neat little villa in the vicinity of some opulent city, than of a gentleman's house recently raised in a remote sequestered part of the kingdom.'
Built by Balfour, Cliffdale was an L-shaped house, the focus of a designed landscape (Matheson, 1819). The walled gardens from the 17th century House of Sound, the woodland plantation, the Doocot, the principal east-west access routes and Balfour village all comprise an overall scheme.
By 1844 the whole island had been acquired by the Balfour Estate when Captain William Balfour bought out Samuel Laing of Papdale. The estate was transferred to the Balfours of Trenabie in Westray.
In 1843-4, Colonel David Balfour commissioned the architect David Bryce to design Balfour Castle. Thus Cliffdale House was incorporated as service accommodation with, to the south, a new range consisting of a Scots Baronial mansion. It took more than three years to build, largely from stone quarried on the island, and was completed in 1847.
The Castle grounds were remodelled by David Bryce in association with the garden designer Craigie Inglis Halkett of Cramond (Edinburgh), whose best known work is the garden of St. Martin's Abbey near Perth, undertaken in 1855. Halkett's precise role is unknown although a letter from Bryce to David Balfour in 1847, clearly suggests that Bryce designed the architectural elements of the terraced gardens, leaving the layout of flower beds to be determined separately. The Halkett family was acquainted with a branch of the Balfour family based in Perth, and this connection may have led to Craigie Inglis' commission.
During the 1850s there was an extensive landscape scheme aimed at improving the setting of the Castle and its approach from the village. Part of the village was demolished to allow a diagonal approach from the north east, through the woodland. Bryce designed a gateway with corbelled parapet, mock portcullis and flanking lodges forming a new entrance from the harbour, c 1850. The Dishan Tower, a circular doocot dating from the 17th century, was adapted as a 'douche house' and a 'two holer' lavatory, flushed by the tides, was constructed on the pier.
The old Walled Gardens were replaced by a new Walled Garden to the north east of the Castle sheltered by the woodland. Earthwork terraces and rubble built ha-has were also constructed to the east and the south of the Castle respectively. To the west of the Castle, terraced and sunken gardens were laid out to link with a series of garden enclosures, formerly associated with the House of Sound, including a gateway which was adapted for use as a sheltered seat. The woodland was extended westwards to provide shelter and enclosure for the new Castle and its gardens.
The scheme at Balfour Castle was one aspect of Colonel Balfour's comprehensive programme of agricultural improvements throughout Shapinsay. This involved land improvement through drainage, deep cultivation and enclosure within a distinctive gridiron pattern of drystone dykes. Crofters were resettled to maximise the productivity of the land and numerous agricultural innovations were introduced to improve the quality of livestock and cereal production in Shapinsay and throughout Orkney. A model farm was developed to the north of Balfour village and a mill constructed in 1883. In order to improve transport links with the Mainland and other islands, the harbour was improved, its pier having been extended in 1861.
During the Second World War, the Castle was opened to the public at stated times during the summer months. Its many objets d'art and the extensive gardens attracted many visitors and the proceeds, given in war time to war charities, were later disbursed among the various social organisations represented on the island. The Balfour family remained at Balfour Castle until 1962 when the estate was sold to the Zawadski family.
The Castle retains its original fixtures and furniture and now operates as a Castle Hotel. The Walled Gardens still function as a productive garden and together with the grounds are maintained by the Zawadski family and gardeners. The terraced flower gardens have been replanted with conifers to provide additional shelter. Garden restoration and conservation is ongoing.