The original Pinebanks house is believed to have been built between 1880 and 1882 by Norwich solicitor John Odin Howard Taylor.
He had premises in Upper King Street, was a keen chess player, and he is believed to have been very influential behind the scenes in city life, as he was the secretary of the company which built Prince of Wales Road in the 1860s.
He lived at Pinebanks until he died on May 15, 1890, after a stroke.
Other notable people to live there include T C Jarrold, who was known as Herbert Jarrold, of the department store family.
The folly tower, which was built in the garden at the same time as the house, was listed as a grade-II structure in May 2006. Built of random knapped flint, with red brick and limestone dressings, the tower features oriel and mullioned windows
The tower, also known as Taylor's Folly or Taylor's Tower, also has a shield with the incised inscription 'HM Queen Kapiolani ascended this tower 6 June 1887'. Kapiolani, the Dowager Queen of Hawaii, was touring England on the occasion of the golden jubilee of Queen Victoria.
Its listed building entry, quoting Towering Follies by the Journal of Norfolk Gardens Trust, said: 'A prominent landmark at the top of the scarp slope above the River Wensum. Comparatively late for a folly tower, this is a good example of an eclectic High-Victorian garden building.'
While it is only the tower folly which is listed, Broadland's conservation officers felt the original house also made a positive contribution to the appearance and character of the area.
They also felt the loss of the house would sever the architectural relationship between the house and tower, leaving the tower isolated and harming its historical significance.
After Pinebanks was closed down, investigative surveys discovered a 'Zero' radio bunker at the site, dating from the Second World War.
The zero stations are often mistaken for air-raid shelters but would actually have been used as a hideout for the Special Duties Organisation, which were a branch of the Auxiliary Units.
Comments: Our rules
We want our comments to be a lively and valuable part of our community - a place where readers can debate and engage with the most important local issues. The ability to comment on our stories is a privilege, not a right, however, and that privilege may be withdrawn if it is abused or misused.
Please report any comments that break our rules.
Read the rules here