If you’ve ever bought something on Amazon or other large online retailers, you’ll have noticed the ‘similar products’ that the site recommends to you after you’ve made your purchase. Sometimes they’re not the best suggestion, but in my experience most of the time they hit the mark.
This is an area of machine learning called recommender systems.
How do recommender systems work? In the case of online retailers, the standard approach is to fill out huge matrices and work out the relationships between different products. You can then see which products normally go together in the same basket, and make recommendations accordingly. This is called collaborative filtering and it works mainly because most products have been purchased thousands or millions of times, allowing us to spot the patterns.
Fast Data Science - London
A table of products and how often they are bought together. This is the basis of collaborative filtering, which is one recommender systems algorithm.
Now imagine you run a dating website. Let’s simplify and say your site only caters for male-female pairings. How do you recommend a female to a male user who’s just registered?
This is when things get tricky. There are many users, new users are registering all the time, and most users have made few contact requests.
In this case we can work with what we do have:
Recommender systems on text
One approach which I like to use is a deep learning approach called vector embeddings, which goes like this:
A dating recommender system might assign all users to a location in vector space, and match users who appear close together.
Of course the tricky bit is how to go from a profile text and image, to a vector. This is something that Convolutional Neural Networks (CNNs) are very good at.
Vector embeddings can be useful for making recommendations in other industries too:
A house sale website recommender system might need to make use of text and image data
There are off the shelf recommender systems that you can use for online retail or movie recommendations. But for text or image based recommendations really you need a custom solution, and this is extremely complex to build.
I have set up Fast Data Science Ltd to provide consulting services in this area after 10 years' experience working with machine learning on natural language data. If you have lots of text or image data and you’d like to build a custom recommender system I’d love to hear from you. Please contact me here or write a comment below.
Organisations of all scales and nearly across all sectors are now becoming increasingly data-driven, especially as larger data storage systems and faster computers continue to push the performance envelope.
Because of the extensive use of technology, and the division of labour, the work of the average gig economy worker has lost all individual character, and, consequently, all charm for the employee.
The impact of AI on human resources The world of work is rapidly changing, both due to traditional data science practices being adopted by more organisations, and due to the increasing popularity of generative AI tools such as ChatGPT, and Google’s BARD among non-technical workers.