The housing market in Ontario is regulated by Tarion, a not-for-profit consumer protection organisation established by the Government of Ontario in 1976. When a homeowner buys a new-build house in Ontario, the warranty is registered with Tarion.
If the buyer finds a problem in the construction, such as damp, electrical, or structural, usually submits a form to Tarion on an online portal using free text. Depending on the issues at hand, the homeowner’s submission may resolve itself, or could escalate to costly and time consuming interventions such as a physical inspection by a surveyor or appropriate professional.
Since Tarion has been in operation since 1976 and homeowners' submissions and builders' responses are mostly in the form of plain text, the management of Tarion were aware that their text data would be a goldmine for AI initiatives such as predictive analytics, or even predictive natural language processing models, which is why they opted for AI strategy consulting with us. If a homeowner’s complaints are not addressed by a builder, the issue is likely to escalate and require a costly and stressful inspection by a surveyor or other professional.
Tarion contracted Fast Data Science as part of an AI strategy consulting engagement. The engagement consisted of three phases:
Data discovery: we explored the data already present in Tarion’s Salesforce system and built some small sandbox predictive models. We found that there was plenty of predictive value in both the plain text of a homeowner’s submission, and the accompanying metadata, and it would be possible to predict the likelihood of a submission escalating to physical inspection, major structural defect, or even litigation.
Opportunity identification: we held a number of brainstorming sessions with VPs and other stakeholders throughout the business. These conversations were led by the VPs who were able to discuss what they would like to see AI to achieve for their division and for the organisation as a whole.
Opportunity prioritisation: we collated the candidate data science and NLP initiatives and asked stakeholders in Tarion to rate them on a three-point scale in terms of their impact on homeowners, builders, staff, and Tarion’s finances. For example, the initiative which was most mentioned by stakeholders across departments was, “when a homeowner submits a form, triage it according to the likelihood of escalation”. Some other exciting initiatives were of high interest to one particular department, but not necessarily in other areas, such as “when a homeowner begins typing a submission, provide assistance as to whether their complaints are under warranty”. We also ranked these items according to technical difficulty and located them on an interactive scatter plot built in Plotly Dash, allowing stakeholders to explore the ranked initiatives and segment them by various criteria:
AI opportunity prioritisation exercise. Projects can be categorised on two axes, difficulty and impact. Attractive initiatives would be towards the bottom right of the graph, that is, having a high impact for low technical difficulty.
The final deliverable for the engagement with Tarion was a presentation with the CEO, and a 42-page report detailing our investigation and recommendations for the next steps.
If your organisation has business processes where customer issues may escalate, and you suspect that your unstructured text data could hold some predictive value, please contact us to arrange a free consultation! One way to think about it is, if I had budget to hire a smart graduate to assess every single incoming document, form, phone call, or email, and triage them, what would I want them to do?. If you think that this smart graduate would make a large impact on your business, the chances are that AI could too. You’d be surprised what NLP models can achieve, especially after the explosion in power of LLMs.
A lot of the discussion I’ve seen online about NLP is centred on the way that an NLP model could triage submissions according to a straightforward semantic set of categories, such as electrical, damp, construction, in the case of a housing regulator. It would be very easy to send GPT-3 a simple prompt and ask it to make this kind of categorisation. But what’s really interesting and far more valuable to a business is to triage issues according to future expectations, such as expected litigation costs or probability of escalation to mediation: this is where AI comes into its own. A lot of business implementations of AI which I’ve seen have missed this point, and stop short of making future predictions.
At Fast Data Science, we don’t just focus on technology. Most of our time on an AI strategy engagement is spent talking to stakeholders, running surveys reminiscent of social sciences or market research, and communicating findings. When business opportunities have been identified, the technical issues can resolve themselves. To focus first on technology (“we want to do XYZ with GPT-3”) is to put the cart before the horse.
Our key recommendation for Tarion was to begin laying the foundations for natural language processing and AI projects. In this case, this would be a data lake, data warehouse, and ETL (extract,transform, and load) pipeline, using tools such as Snowflake, Google BigQuery, or Apache Spark. This would pave the way for some high-impact predictive analytics projects to run very smoothly, such as predicting homeowners' escalations, or predicting home builders entering into insolvency. These key projects had been identified as being of extremely high impact to nearly all stakeholders, as well as only being moderately difficult to implement.
Review of Fast Data Science after the Tarion project on our Google profile
What we can do for you
