Sometime in around 1863-4, Sir George Gilbert Scott was commissioned by Frederick Benjamin Twistleton, 16th Lord Saye and Sele, whom he knew through his work on Hereford Cathedral where Twistleton was an archdeacon, to restore his castle. George Gilbert junior took over the project in 1865, so Scott did not carry out much work here although the geometric windows at the west end of the north and south walls may be his, while the Jacobean style doors and doorcases in the rooms at the west end of the first floor were probably designed by his son.

http://www.broughtoncastle.com/archive_slade.htm#scott