According to Burke's History of the Landed Gentry of Great Britain, the earliest known ascendant of the Holdens of Nuthall Temple was Henry Holden of Wilne of Derbyshire, born in the first half of the 16th century. The succession was then as follows: Henry's eldest son Robert of Aston (1594-1659), Robert's second son Samuel of Aston (1636-1692), Samuel's eldest son Rev. Thomas Holden, Rector of Aston (d.1726), Thomas's son Millington (1707-1755), and Millington's son Robert (d. 1780).
Through various marriages the Holdens acquired lands in Nottinghamshire, Yorkshire and Lincolnshire. Felley Priory, Nottinghamshire, was inherited by the Holdens through Thomas's marriage to Elizabeth, daughter and co-heir of Gilbert Millington of Felley Priory. The marriage in 1725 of Alexander Holden (1680-1769), son of Samuel Holden, to Mary Atkinson, daughter of Robert Atkinson of Newark (d. 1699) was of particular significance in bringing the Atkinson and Standish estates to the family. In 1717 Alexander Holden purchased the manor of Hawton, Nottinghamshire from the Newdigate family. Alexander and Mary's son Atkinson inherited the Holden estates and the Nottinghamshire and Yorkshire estates from his mother which on his death passed to his brother Robert (1722-1808) who also purchased estates at Darley, Derbyshire.
Robert Holden died without issue and his estates were inherited by his cousin Robert (1769-1844), the son of Robert of Newark mentioned above. In 1819 Robert Holden purchased Nuthall Temple and the manor of Nuthall, Nottinghamshire, from the Sedley family, after applying for an Act of Parliament to allow him to sell his settled estates in Derbyshire and Yorkshire. In 1800 Robert had married Mary Anne Drury Lowe (1783-1840), the daughter of William Drury (Lowe) (1783-1827) of Locko Park, Derbyshire. On the death of his father in law, Robert Holden managed the Drury Lowe estates although he was outlived by his mother in law. When she died in 1849, the Drury Lowe estates passed to William, the eldest son of Robert and Mary Anne Holden, who changed his name from Holden to Drury Lowe. The Holden estates passed to the second son, Robert Holden (1805-1872), known as 'The Colonel'.
Robert was succeeded by his brother Atkinson Alexander Holden (1808-1882), Vicar of Spondon and Rector of Hawton. The estates then passed to his son John (1848-1913) and subsequently to John's younger brother Robert (1853-1926), Rector of Nuthall.
Robert was succeeded by his son Robert Millington Holden (1885-1950) who sold Nuthall Temple and bought Chetwode priory in Buckinghamshire where he lived until 1930 when he purchased Sibdon Castle in Shropshire.