The Caves of Dysart lie in the south-eastern corner of Dysart House's garden and adjacent to the ruin of St Serf's Kirk, Shore Road. St Serf's Cave, the largest, is traditionally said to have been the retreat or desertum of St Serf, who founded the church of Culross 697-706, although he is also credited with rearing St Kentigern (520s). The cave was a place of pilgrimage in the 16th century. It is claimed that this 'desertum' is the origin of the name Dysart (Millar 1895 p.94)
James II started the construction of Ravenscraig Castle as a base on the Forth, in 1460. It was completed by his widow, Queen Mary. Subsequently it passed into the ownership of William Sinclair, Earl of Orkney when, in 1476, James III compelled him to exchange the Earldom of Orkney for the estates of Dysart and 'Ravensheugh' with the Castle. The Sinclairs occupied Ravenscraig Castle until the mid-17th century. As early as the 16th century, they appear to have had a house on the modern-day site of Dysart House, called 'The Hermitage' (Millar 1895). A date-stone, 1588 or 1688, in the walled garden may be a re-used fragment from this house.
History relates a succession of disputes and representations between the Sinclairs and Burgh Council, centering on free passage to, and usage of Dysart harbour. The harbour was foremost in trade with the Low Countries, Dysart itself being called 'Little Holland', as well as being a long-established and crucial port in the coal, salt and fish trade (Gifford 1992, p.288).
John Sinclair, master of Sinclair (1683-1750) owned the property in the early 18th century. Following a duel which resulted in a declaration of 'wilful murder', he lived abroad until 1712, when he was pardoned and returned to live at Dysart. A Jacobite, he commanded the Fifeshire and Aberdeenshire horse in 1715 and defeated Argyll at Dunblane. Once more, following a sentence of attainder, Sinclair lived abroad. During this time, on the death of his father in 1723, his estates were settled on his brother James. In 1722 the house was destroyed by fire and many family records were lost. On Sinclair's return in 1726 he rebuilt the house, renaming it Dysart House. In 1748 he secured an excambion with the Burgh Council, whereby he obtained the 'East Croft' at the west end of Dysart 'to build a house for his own dwelling' (Donaldson 1983).
General James St Clair (d. 1762), who served in Flanders and France (1745-6), and then as ambassador to Vienna and Turin in the 1750s, succeeded his brother. He purchased lands in Roslin, Ravenscraig and Dysart and, in 1755–6, altered the house. In 1754 he enclosed Lethemwell Parks, Aislerhead parks and the Aisler Rock. The latter areas, lying on the coast to the south of the harbour, are said to be so-called from the French 'aisler', ashlar, and refer to the stone sources exploited in the construction of Ravenscraig Castle and St Serf's Tower (Swan and MacNeill 1997, p.17). After the General's death the estate passed to a nephew, Colonel James Paterson (d.1789), who assumed the name of Sinclair.
In the 1770s there appear to have been a succession of disputes with Colonel Sinclair regarding public access to coastal amenities. One related to the Letham Wells, which the townsfolk had a right to access by two foot roads, one leading from a 'lower part of the Town along the Beach, and by another road from the Upper part of the Town to the North-west corner of the Lethemwells park ..' (Donaldson 1983 S17, 1777). This right of access was upheld in 1777, as was their right to quarry and remove stones from the foreshore, although their claim to pasture their horses on the Drying Green, an area within Letham Park, was refused. Colonel Sinclair retained Aislerhead or the Fort as his exclusive property. The 'Fort' is a high rock commanding the harbour, which terminates the Sailors Walk. It is said to have been fortified by Oliver Cromwell, although there are no remains of any work (OSA 1794, p.522).
The estate passed to Sir James St. Clair Erskine Bt.of Alva, 2nd Earl of Rosslyn (1762-1837) in 1789, who made major architectural and landscape changes to the property. He sought a land exchange with the Burgh in 1790, to obtain the West Croft, which John Sinclair had given to the Burgh Council in 1740. It lay at:
'the Harbour of Dysart on the West side thereof, bound by the Harbour of Dysart and that wynd or High Way called HOTT POTT Wynd on the East, the Sea flood on the South, the arable land called Normand's Aikers and the Ashlerhead and that little spot of Ground called the Fort on the west, and the highway leading from the Burgh of Dysart to the Town of Dunniker on the North' (Donaldson 1983 S17, 1777).
This clearly relates to the area of land on Ashlar Head, bounded by the Sailors Walk to the east. The Council refused the exchange. Erskine may have sought to regain this land to ensure his privacy within Dysart House, for the Burgh Council proposed that:
'the Upper part which overlooks the ground at St James' house the Town should at its expense enclose this ground and make a footway for communication to the Harbour under the surface of the Bank, so low as to prevent seeing across St James' wall and to plant the Brae under the paling of the Ashlurhead Rock with Broom Bush. Further that all the Brae or Bank under the Ashlur Paling should be enclosed and planted by the Town, and a walk for the inhabitants be formed under it' (Donaldson 1983, 10 November 1790 Vol.3).
Sir James accepted the Council's recommendation and offered to carry out the proposals and bear the costs.
The house was enlarged between 1808–14. Sometime during the late 18th or early 19th century a complex of garden walks were laid out. The Dysart Caves were ornamented in a picturesque manner, with the addition of steps and gothic arched tunnel vaulting linking them as a 'feature' into the garden walks. In order to obtain the West Croft ground, Sinclair offered the Town, in 1807, the gift of ground called the 'Fort', to be used as a walk. This may indicate the formal adoption of the Sailors Walk as a public sea-walk. In 1835 estate survey shows a bridge or 'viaduct' connecting the top garden terrace at Dysart House with the 'pleasure grounds' on the west side of Hot Pot Wynd. A walled garden and stables are shown along with a detached flower garden, in the middle of the pleasure grounds.
A new inner harbour at Dysart was constructed from a former quarry, to the north of the existing harbour. In 1835 a new building at the harbour was used for processing whale oil. Lord Rosslyn, affected by the smell, took action within the House of Lords to arrest this process, and won. Although the harbour belonged to the Burgh Council, the main user was the Earl of Rosslyn's Collieries, the main family wealth resting on coalmining royalties.
In about 1883, Dysart House was described as 'a plain but commodious mansion with beautiful gardens, commanding a splendid view across the Forth' (Groome 1883). By 1894, the detached flower garden had been removed and a new carriage drive led through the park and pleasure grounds to the north front, crossing Hot Pot Wynd by a bridge set to the north of the earlier bridge (1st edition OS 25", 1894). These changes may have been carried out by Francis Robert Sinclair, 4th Earl of Rosslyn (1833-90), who succeeded in 1866. By his time the estate comprised 3,000 acres in Fife, worth £9,000 in rents and coalmining royalties, with a further 100 acres in Midlothian including Rosslyn Castle.
Due to the 5th Earl's bankruptcy, the house, grounds and castle were acquired, in 1896, by Sir Michael Nairn (d.1915), of the famous Kirkcaldy linoleum manufacturers. He extended the existing park wall around the seaward side of the estate for privacy, filling in the hand-holds and footholds cut in the rock and cutting off free access along the coastal path, which allowed Dysart folk a customary right of access to dig for bait on Pathead beach, by Ravenscraig Castle. This resulted in a court action in 1898.
His son, Sir Michael Nairn conveyed the former pleasure grounds and the area of Dysart Park know as Three Trees Park to the parish of Kirkcaldy and Dysart, for a public park, in 1929. In the same year, Dysart House and garden were sold to the Carmelite Order and remain in their possession today.
Public park use has inevitably changed the nature of the former policies. A new lodge, Mid Lodge, was built shortly before or after transference of the land to the District Council. Tennis courts and playing fields now occupy former parkland west of the walled garden.