Can AI and NLP score sales leads for Customer Relationship Management?

Natural language processing can distinguish customers from salespeople

Is it possible to use natural language processing (NLP) to distinguish between unwanted sales approaches and promising leads for a business’s customer relationship management? If so, this would be a great application of AI in business.

I receive a number of unsolicited business approaches each week, and it is often difficult for me to identify who is approaching me for business or with a genuine problem in machine learning that I can help with, and who is looking to sell me something. Often the salespeople use language such as ‘partnership’ rather than directly stating that they want to sell me a product or service, and so it is not always straightforward to work out who is who.

Sometimes I have been caught by surprise - for example, I have assumed that an email was a sales approach, and then the person has phoned me later and turned out to be a genuine business prospect, and on other occasions I have spent half an hour speaking with somebody before I realised that they were trying to sell me a product. So the distinction is very difficult even for a human to make.

I developed and trained a machine learning model to differentiate between potential business prospects and likely sales approaches, and I set it up to text me when a genuine business prospect is detected.

Lead management with CRM software

I have tried out using customer relationship management software (CRM) such as Salesforce, Monday.com and HubSpot, which are very good at managing leads, but do not distinguish between good and bad prospects. The software allows you to create leads, but won’t assess the chance of that lead converting to a sale (at least I have not found that feature).

AI to detect emails from potential future clients

After experiencing the frustration of the CRMs, I had the idea of building a machine learning model to use NLP classify my incoming emails.

I selected all the new contact requests in my Gmail and then used Google Takeout to save the messages to my computer.

I saved 149 unsolicited messages. I had replied to 31 of them, which means that 31 of them were interesting for me, and the remaining 118 were mostly people trying to sell me products.

A small dataset of 149 documents is not sufficient for a neural network or deep learning approach. So I used a Naive Bayes classifier, which simply uses the presence or absence of words in documents to make a decision. This model ignores word order and syntax, so it is sometimes called a ‘bag of words’ model because each document is reduced to a bag of unordered words without context.

I found that the top themes in the true leads (those people who appeared to be potential customers) are often phrases such as “we are looking for” or “we have a project” - so people who are interested in buying a service from me will describe very directly and succinctly what they need and how quickly they need it.

Interestingly, these people are also more likely to use my name or mention Fast Data Science by name. For example, Dear Thomas, we have an urgent natural language processing project where we need X - would you be available to talk this week?

Word cloud of the top words which indicate that a message is a promising lead for follow-up

On the other hand, the non-leads, who are mostly salespeople and jobhunters, often use phrases like “my services” and “our SEO business”, and include plenty of links. For example, Dear Sir/Madam, I am an expert in SEO and I want to write an article for your website.

These people tend to use more generic greetings rather than explicitly using my name or my company’s name, and talk more about themselves and the services they offer.

A person’s name is to that person, the sweetest, most important sound in any language.

Dale Carnegie, ‘How to Win Friends and Influence People’ (a bestselling self-help guide first published in 1939 and widely used for training salespeople)

The word cloud below summarises the top terms in the non-lead approaches:

Word cloud of top terms indicating that a message is a sales approach rather than a lead. You can see that first person pronouns and links are more prevalent in sales approaches than in genuine leads.

When I trained my classifier on 80% of the data and tested on the remaining 20%, it achieved 61% accuracy in identifying promising leads - the ROC curve is shown below.

The problem of unbalanced data

I found that when I used only 149 emails without any adjustment, I ended up with a text classifier that classified most leads as useless. Because most of the examples that the classifier sees during training are non-leads, the classifier ends up being a little conservative and tends to classify nearly everything as “uninteresting”.

To combat this, I added some extra copies of the “interesting” emails to the training data, resulting in a more balanced dataset. This technique is called upsampling and it’s one of the many ways that we can combat class imbalances in machine learning.

When I upsampled the data, the performance of my model went down slightly, but I found that the model was more practical to use as a classifier as it didn’t tend to miss so many promising sales leads.

Taking it a step further and predicting revenue with natural language processing

The obvious question that arose after this was, is it possible to use a text classifier to predict the final outcome of a business relationship, i.e. the total amount invoiced?

I tried using my CRM to connect my leads to my invoice data. I was able to train a text classifier, however by this stage the data became very sparse (as the majority of dialogues do not lead to a sale), and the classifier did not perform very well. (You might want to read about training text classifiers on sparse data) However a larger business would definitely be able to build a lead-to-revenue predicting model.

Using other non-text features of the message

Besides linguistic features, there are a number of other features which can indicate the potential of a sales prospect. Among these

• the number of links in the message - it is well known that spam filters often block messages with hyperlinks, and my classifier clearly came to the same conclusion,
• the location of the person,
• the email address of the person (are they writing from a corporate email address or a Gmail or Hotmail account?),
• the time of day and day of week,
• the IP address of the sender if available,
• the size of the prospect’s company. Hubspot identifies a contact’s organisation from their email domain, and auto-fills key data about the company such as revenue, headquarters, industry and number of employees. It would be possible to hook into this extra third-party data and use it to further refine the model.

Using the text classifier

I tried using Azure Functions to get my text classifier model running live on the web instead of my laptop (deploy it). Azure Functions is a nice lightweight solution, as I only pay a few pennies every time the model is used - I don’t need to pay for the time that it is out of use.

I set up the tool Zapier to monitor the incoming emails for sales leads, and send me an SMS when a message has a greater than 20% probability of being a potential business lead. Let’s see where this gets to.

Zapier is a website which allows you to set up actions such as sending an SMS on certain conditions, using a no-code interface. Here is my flow for sending myself an alert when a promising lead comes in.

I began receiving text alerts for promising sales leads, with probabilities included. This is a screenshot of some of my SMS alerts for the potential leads.

Conclusion

It took me about an hour to build a text classifier to distinguish between unwanted sales approaches and promising leads, and another hour or so to get it connected so that I am alerted by SMS to promising sales leads.

A large number of the text-based features which emerged from training this model could generalise well, and are not specific to Fast Data Science, or the consulting and technology industries.

Correspondents who talk about themselves or their company are likely to be selling something, whereas those who talk about me, my work, or my company are more likely to want to buy from me. I would imagine that business people in other industries may have made similar observations.

Correspondents who mention ‘partnerships’ are particularly tricky to classify, as this phrasing is used by some genuine clients but also seems to be used by salespeople to obfuscate a sales approach.

As customer relationship management software becomes more sophisticated, we may see a move towards AI being used throughout the end-to-end sales pipeline and available as an off-the-shelf part of the CRM software.

It would be wonderful if a CRM could flag incoming leads as high or low probability of converting based on linguistic features, allowing sales teams to focus their efforts on prospects with a higher chance of converting, or a higher expected revenue, or to maximise revenue per hour worked.

I am not aware of a CRM package offering this level of analysis out of the box, but I would be interested to hear more if you aware of anybody taking this approach. Please contact me and I would be keen to hear from you.