The Isle of Raasay was long-held by Macgille-Chaluim, the MacLeods of Raasay. By the late 17th century they had abandoned their ancient seat at Brochel Castle situated in the north of the island and built a three storey tower, Kilmoluag Castle, reputedly at or near the site of Raasay House. A new house was built c 1720 – the shell of which is encased in the present house. In 1745 Malcolm MacLeod, 10th Chief, joined the Jacobite cause and this new house fell victim to government reprisals following the defeat at Culloden in 1746. Little is known of that building, apart from the fact that Cumberland's troops 'brunt Rasay's good house to ashes, as also the whole houses upon the island excepting two small villages that escaped their sight… They likewise found all Rasay's furniture and silver-plate hid in a cave…'
Nevertheless, the stone walls remained so in c 1747 and, in c 1762, John MacLeod commenced remodelling it with a new south-facing front of five bays, flanked by lower wings. The dramatic landscape comprising the main view from the house was amplified, by means of its being deliberately framed. This was formed by flanking plantations, splayed slightly outwards to accentuate the perspective, extending to the sea on either side of the front green and framed by Glamaig and Ben Lee on the west and Garbh-bheinn and Bla Bheinn to the east. In 1773 John MacLeod entertained Dr Samuel Johnson (1709-84) and his future biographer, James Boswell (1740-95), at Raasay House. Johnson described the house as 'a neat, modern fabric of eleven rooms' while Boswell commented on its picturesque qualities, situated with 'a fine verdure about it, - with a considerable number of trees; - and beyond it hills, and mountains in gradations of wildness.' He described the 'good garden well stocked with kitchen stuff, gooseberries, raspberries, currants, strawberries, apple-trees. There is a tolerable southern wall on which fruit trees have been tried, but have been neglected. ' A drawing of 1815 (Daniell, 1815) shows the house from the north with this walled garden in the foreground and Skye in the background.
James MacLeod, the 11th chief who succeeded in 1787, established further plantations, which formed the nucleus of the 19th century park. A two-storey seven-bay classical southern frontage, taller and longer than the pre-existing house, was added c 1790-1805. A landscape painting of c 1823 depicts the facade against a landscape backdrop of towering hills (Roberts, 1980, p.230), although the artist rotated the house to show its southern façade against its southern setting. It was referred to in 1841 as a 'very splendid modern house' built between 1787 and 1815 (New Statistical Account, 1845). The promontory jutting centrally southwards into Churchton Bay was set out with a Battery during the Napoleonic Wars. As at Islay House (q.v. Inventory Supplementary Volume, Strathclyde, forthcoming) this was also treated as an ornamental landscape feature.
The MacLeod family sold the estate for £37,000 c 1843 to the Rainys of Edinburgh, the first of several mainly absentee landlords who owned the house until the late 1970s. From 1846, the first Rainy laird cleared 14 townships, created sheep farms and extended Raasay House in 1848, probably commissioning Charles Wilson (1810-63) as architect. Thereafter Rainy's son, who was resident at Raasay, undertook some more imaginative agricultural reform.
In 1875 the Wood family bought Raasay estate and developed the island as a sporting estate. This may explain the need to add a further wing, designed by Alexander Ross, to the house in 1876-7. Herbert Wood built a deer fence to protect the parks and policies of Raasay House, including the Home Farm. It enclosed an area from the coast south of Oskaig up to Creachan Lodge, south-eastwards along the hill to Glen Lodge and then south to a point near the pier. Ornamental trees and shrubs were planted along walks, chiefly from Raasay House to Dun Borodale, the old Manse and the Free Church and along to Loch a' Mhuilinn, reached by way of the ruins of St Mao-luag's Chapel.
By the early 20th century the estate was in the ownership of Wallace Thorneycroft who collaborated with William Baird & Co, ironmasters, in locating and analysing the potential of Raasay ironstone. Bairds purchased the island estate in May 1911, setting up a mine, railway, kilns, new pier and workers housing at Suisnish. Production ceased in 1918. Raasay House then became a hotel, temporarily, and was eventually bought by the Highland and Islands Development Board in 1978. By this time the house had suffered from a lack of investment and maintenance and a new use was needed.
Highlands and Islands Enterprise now own Raasay House and parts of the estate (2001). Other areas of the estate remain in private ownership. Raasay Outdoor Centre, in the East Wing of the house, offers accommodation and courses in water-sports and rock climbing, while the West Wing is used as a Community Hall.