The Grade II listed manor, which is thought to be the oldest house in Portsmouth and was mentioned in the 1086 Doomsday Book, is said to house a choir of nuns.
The spirits of whispering children and Sir Roderick of Porchester, who was murdered outside the manor in the middle ages, are said to be among the ghosts haunting the property.
And if their aim was to prevent new people moving in, it would seem that they have succeeded - for the time being at least.
After failing to sell at auction with a reserve price of £375,000, Wymering Manor in Portsmouth, Hants, has now been donated to the Wymering Manor Trust, along with a £30,000 grant for repair work.
The group will attempt to restore the house to its former glory before opening it as a tourist attraction - a project that is expected to cost some £500,000.
Portsmouth City Council bought the property in the 1960's and leased it to the Youth Hostel Association until 2006. According to the Daily Mail newspaper, it was then sold to a private organisation after the cost of the upkeep became too much for the council.
However, the development never took place and it was returned to the council, which has now been forced to give it away due to a lack of interest from serious buyers.
Jeremy Lamb, a chartered surveyor who tried to sell the property in 2010, told the Daily Telegraph: "It's certainly a unique house.
"Although they patrol it on a 24-hour basis because it attracts lots of people who are intrigued by its levels of paranormal activity, they refuse to work alone there at night."
Other properties that have failed to sell due to rumours of haunting include a former town house in Derby, which is said to house a total of 14 ghosts.