During the summer of 1864, when Scott first became involved with Tewkesbury, he made a real attempt to be more sensitive towards his family’s feelings. He conceded to Caroline’s dislike of The Grove by starting to look for a new house and after ‘many disappointments & difficulties we found a suitable residence at Ham’. In the autumn of 1864, they were able to move in before ‘the commencement of our Hampstead winters’.

This was the Manor House which Scott rented from the Earl of Dysart, who owned nearby Ham House, one mile south of Richmond in Surrey. The Scott’s new home could hardly be in a more different situation than The Grove. It stands on flat land within a bend of the river Thames and has no distant views. Scott described it, in 1872, as ‘an old Georgian house founded on one of much earlier date It has extensive & beautiful gardens & My dearest wife much enjoyed it’. It has a much grander appearance than The Grove, as would have befitted Scott’s improved status, although about the same size. The previous occupants were a family of four with six living-in servants and a gardener’s family of six in a separate cottage. There was obviously ample room for the seven Scotts and their five servants and Scott converted the gardener’s cottage into a drawing office so that he could work closer to Caroline.

It is a classic Georgian house in brown and red brick with symmetrical three-storied, five bay front from which the three centre bays project forward under a pediment. A central porch is supported by Doric columns and was connected to a gate on the outside road by a covered way. Scott may have provided this to enable Caroline to have a sheltered route to awaiting carriages but this and the central gate, have now gone and have been replaced by a pair of carriage gates with a drive up to the front door. In the twentieth century, the rear of the house was transformed into a two-storied building with new wings projecting from either side of the old house.

Scott’s own architectural ideas may have been at odds with the philosophy of Georgian architecture but when it came to choosing his own accommodation, he clearly appreciated the simple elegance of the style and its appropriateness for his own purposes. He seems to have tried to ensure that the house would be suitable for his physically impaired wife and certainly Caroline liked the house and its garden. Here she would not have to negotiate the steps and slopes of the Hampstead heights, and here she was to find more congenial companionship. But the quest for a healthier environment for the children was soon to have tragic consequences with the death of his son Albert. The following Christmas vacation Albert:

availed himself of its facilities for boating and nearly every day went on the river with Alwyne for a row in a boat he had hired for the vacation. Alas! how little did we think that this harmless recreation would be the cause of so much grief!

It was on Sunday 22 January 1865, that Albert first complained about feeling stiff but Scott, as usual, seems to have been busy, and knew nothing of his son’s condition until the following Wednesday. The next day, ‘alas that it should have so happened!!’ He did not recover and died in the afternoon of Monday 30 January 1865. The house at Ham was never the same for the family and by 1869 were on the move again, albeit temporarily.

The three-year lease on Rooks Nest expired in October 1872 and the sadly depleted Scott family moved back to Ham. He wrote:

I had on my dear wife’s decease at one time thought that My sister in law Helen Oldrid might keep house for me – but My sons thought it would be dull for them & other arrangements were made.

Helen Oldrid was the spinster member of the Oldrid cousins and lived in the family home at South Place Boston, which she had latterly shared with her brother John Oldrid and his wife Euphemia. When John became the vicar of Alford in 1863, she sold the old house at Boston and moved into a cottage opposite the vicarage at Alford. But she must have felt somewhat isolated when John remarried so soon after Euphemia’s death and it may have been a kindly gesture by Scott to suggest that Helen should move in with his family at Ham.

Dukinfield had obtained a place at Christ Church Oxford and with Alwyne still an undergraduate there the boys would only be at Ham during the vacations. The ‘other arrangements’ which Scott implemented as a result of the boys complaints appear to have been that their brother John, with his wife and rapidly growing family, would move in instead, a much more lively arrangement!