The team and how we did it all
Foreclosed is one of the largest data projects thus far by the King's Investigative Workshop. A unique database was built from diverse sources and analyzed for key trends.
This project is the result of months of interviewing, data gathering and data analysis by students in the investigative workshop and MJ program at the school of journalism at the University of King’s College.
Information about foreclosures in Nova Scotia is not gathered in one place, so the students spent more than a week building a database of foreclosure auctions advertised in Nova Scotia newspapers in 2017. 2017 was chosen because it can take up to two years for a foreclosure to reach the last step in the process, resale by a lender of a property purchased at auction.
The students searched archives of daily and weekly newspapers owned by SaltWire Network and Advocate Media throughout the province, including microfilmed editions of the Chronicle Herald kept at Halifax Public Libraries, and captured the images or text of the ads. For microfilmed papers, ads had to be individually imaged.The same was true for online archives that included ads as images within a page. In some other cases, text could be extracted directly from the ads.
caption
Foreclosure auctions are advertised in printed newspapers. The King’s team extracted key information from the ads into a database.For images, including images from microfilm, optical character recognition was used to convert the images to readable text.
The ads all use a similar format, with variations. A script written in the Python programming language was used to extract data from the text extracted from the ads, and compile it into a csv file for import into spreadsheet and database software. As OCR does not produce perfectly accurate results, especially with lower quality images such as those obtained from microfilm readers, the resulting data was then checked by hand by the student team to correct errors.
The ads yielded key details such as the location of the property under foreclosure, the plaintiff and defendant in the court action, the names of lawyers and the scheduled date of the auction.
Additional information was gathered for each property. Foreclosure orders and deeds were obtained from provincial land registry records. These provided such details as the date of the order, the total amount owed as assessed by the court at the time of an order and the purchaser and the highest bid at the auction, which for third party purchasers is the amount paid for the property.
From publicly available information, including an open data set of property sales and the Viewpoint.ca real estate site, the students added information on the price obtained at the first arms-length resale of the property after the auction sale.
The completed data was then reviewed for errors, with each key data point being checked at least twice by human eyes, and three times for critical data.
The dataset was analyzed using Microsoft Excel and MySQL database software to reveal patterns and trends reported in the stories.
Altogether, the final dataset contained 50 separate columns of information, including that gathered, and additional columns derived through calculations. It represents the only known such database in Nova Scotia.
The team used the database to find key patterns, such as auctions that may have produced surpluses that could go back to the borrower, and lender resales that produced a possible profit. Students then viewed court files for the most likely candidates.
Information about auctions occurring during early 2019 was gathered from ads in the Chronicle Herald and postings at the Halifax courthouse. Students attended about foreclosure 20 auctions, all in Halifax. The team was able to confirm patterns evident in the data, such as maximum bids of lawyers for the lenders through interviews and by attending auctions. About a third of auctions examined in 2017 were scheduled for the Halifax courthouse.
Conventional reporting using documents and human sources rounded out the effort.
Limitations
There are limitations on the data analysis. As the starting point was ads for foreclosure auctions, the analysis leaves out foreclosure actions that did not lead to an order for foreclosure and sale, or for which the auction was not advertised prior to its being cancelled. It also does not include cases where a property owner was sent a demand letter by a lawyer but no action was started. For all these reasons, the data doesn’t reflect the full breadth of mortgage default in Nova Scotia but instead reflects cases for which an order was issued, a sale scheduled and the sale advertised in a newspaper. This may skew the data toward cases more difficult to resolve through payment of amounts owed.
It is also possible that some foreclosure auction ads were missed, either because they appeared in a small weekly newspaper not included in those papers searched, or because the searchers overlooked them. This risk is largely mitigated by redundancy built into the search. The ads have to be published twice, and most often ads will appear in a local paper and the provincial edition of the Chronicle Herald, or will appear twice in the Chronicle Herald, for which both provincial and metro editions were searched. The number of ads found also closely matches summary data on the number of foreclosure orders provided by the Nova Scotia Department of Justice.
The information is only as accurate as the original source material.
Any analysis that includes only one year captures a snapshot of conditions at that time. Based on interviews for this project, there is no reason to believe patterns found in another year would have been substantially different.
Team members
The reporting and writing team was made up of MJ student Bethanee Diamond, and workshop students Ava Coulter, Ian Gibb, Piper MacDougall, Isabel Ruitenbeek, Julia-Simone Rutgers, Jane Sangster and Stacey Seward. The faculty advisor and project editor is Fred Vallance-Jones.
