AI in business and industry

· Thomas Wood
AI in business and industry

Your NLP Career Awaits!

Ready to take the next step in your NLP journey? Connect with top employers seeking talent in natural language processing. Discover your dream job!

Find Your Dream Job

AI in business and industry

Artificial intelligence (AI) is a hot topic in business, but many companies are unsure how to leverage it effectively. I will discuss some practical applications of AI, focusing on areas where it can deliver real impact.

There are lots of ways that we see AI being used in industry. Generative AI may capture headlines and media attention, but, in my experience, the commonest uses of AI that we’re seeing in industry today are still traditional machine learning and statistics. I would contend that where you are most likely to see a return on investment for an AI project is in the development of very simple models, perhaps those that can be worked out with pen and paper, when they are combined with business data and in-house business knowledge, although there is a huge potential for hybrid models where deep learning, NLP, and generative AI are combined with data from business processes such as project cost and success data.

Is AI even needed?

Sometimes, the answer is no. Complex AI models might not be necessary when you have good, clean data. For example, if you lack access to clinical trial cost data, gathering it is the bigger hurdle, not building a fancy model.

AI technologies used in business and industry

Fast Data Science - London

Need a business solution?

NLP, ML and data science leader since 2016 - get in touch for an NLP consulting session.

Traditional machine learning and statistics

If you work for an educational institution, and you know five numbers about a particular student, can you predict the likelihood that the student will fail their next exam?

This is a prediction problem that we address with simple machine learning models such as linear regression, or even simple averages, as well as more sophisticated models.

Before exploring more sophisticated machine learning, it’s worth trying some simple solutions such as averages, or a basic linear regression. These tools are simple, quick, cheap to deploy, and easy to adapt to predict numbers like cost or risk. For example, you can make an explainable formula that expresses a student’s risk of failing, or an employee’s risk of resigning, or a clinical trial’s risk of ending uninformatively, on a scale from 0 to 100 which people can calculate using pen and paper.

This approach has a number of advantages, namely that it’s

  • Simple
  • Quick
  • Cheap to deploy
  • Easy to predict numbers like cost or risk

On the downside, simple models are limited in what they can achieve, and they need you to have clean structured input (not images or text) to start with, which is not always available.

Deep learning

Another kind of AI which is becoming widespread in companies and in industry today is deep learning. An example of that would be applications like computer vision, where you might want to classify a street image as a fire hydrant or as a manhole cover. Deep learning models, or artificial neural networks, are very loosely inspired by the structure and function of the human brain. Neural networks can process massive amounts of data, allowing them to tackle intricate tasks that were once beyond the reach of machines. Deep learning models are excellent for tasks like automated visual inspection. In this capacity we can see that they are generally a cost saver, allowing a business to cut some operating costs, but not always achieving the degree of transformation that can be achieved by a smart very business-savvy prediction model.

Examples of what deep learning can do

1. Automated visual inspection: Deep learning excels at image and video recognition. In manufacturing, for instance, deep learning algorithms can be trained to automatically inspect products for defects on a production line. This significantly improves quality control by identifying issues early on, saving time and resources. Imagine a car factory where deep learning can detect even the smallest paint imperfections or misaligned parts on a moving assembly line.

2. Predictive maintenance: Deep learning can analyse sensor data from machinery to predict potential failures before they occur. This allows for proactive maintenance, preventing costly downtime and production delays. For example, wind turbine farms can leverage deep learning to analyse vibrations and other sensor data to predict component failure, enabling technicians to schedule maintenance before a breakdown occurs.

3. Advanced robotics: Deep learning empowers robots with greater autonomy and adaptability. Robots can be trained to handle complex tasks in unstructured environments, such as warehouses or logistics centers. Deep learning algorithms can help robots navigate obstacles, identify and pick objects of various shapes and sizes, and even perform delicate tasks that previously required human intervention.

4. Personalised customer experiences: Deep learning can personalize the customer experience across various industries. In retail, deep learning algorithms can analyze customer purchase history and browsing behavior to recommend relevant products or offers. Similarly, in e-learning platforms, deep learning can personalize learning paths for students based on their individual strengths and weaknesses.

5. Drug discovery and medical diagnosis: Deep learning is making significant strides in the healthcare sector. Deep learning algorithms can analyse medical images like X-rays or MRIs to detect diseases with high accuracy, aiding in early diagnosis. Additionally, deep learning can be used to examine vast datasets of medical research to accelerate drug discovery and development.

Generative AI

Generative AI has taken the world by storm with its ability to generate human-quality text. These AI models, like GPT-3, Gemini, Copilot, LLaMA, and others, are trained on massive datasets of text and code, and can produce near-human text or images, and even output code in programming languages like Python!

In the business world, generative AI offers a range of possibilities which are being explored in organisations:

  • Content creation: Generate marketing copy, product descriptions, or social media posts, freeing up human writers to focus on more strategic tasks.
  • Personalised communication: Craft personalised emails or chat messages that adapt to individual customer needs or preferences.
  • Code completion: Tools like GPT can suggest lines of code or complete functions, accelerating software development.

However, it’s important to remember that generative AI is still under development. While the outputs can be impressive, they may require human editing to ensure accuracy and factuality.

Hallucinations are an issue, and you need to be careful if the process is safety related or business-critical, and also understand your legal obligations where it comes to data protection, sensitive data, proprietary and copyrighted data, and personally identifiable information in general.

Many organisations are starting to draft generative AI policies to ensure that employees stay on the right side of the law when using these tools. If you would like assistance in drafting an AI policy, an AI strategy, or AI and data governance in general, please contact Fast Data Science.

Retrieval augmented generation (RAG)

On top of generative AI, we also have RAG (Retrieval Augmented Generation) and Prompt Engineering, where we send the user’s query to GPT together with extra information, like legal information or industry-specific information. We send this to GPT and then get a result which takes advantage of everything that GPT can do, as well as all the information we’ve got in our database.

Our Insolvency Bot, which we have developed in collaboration with an interdisciplinary team at Royal Holloway University, Gunnercooke and the University of Surrey, showcases how a generative AI model’s performance can be augmented using retrieval augmented generation (RAG) with a database of domain specific knowledge, in this case around English and Welsh insolvency law. You can read about how we developed the Insolvency Bot in our accompanying blog post.

We found that RAG (prompt engineering) significantly reduced the incidence of AI hallucinations and wrong answers when we evaluated our RAG model on a test set.

RAG significantly improved the performance of the generative AI model in the insolvency domain

RAG significantly improved the performance of the generative AI model in the insolvency domain

Traditional natural language processing (NLP)

A form of AI which is sometimes overlooked is traditional natural language processing. Fast Data Science focuses on NLP, although we also operate in the other areas of machine learning and AI.

Example applications of NLP are often this kind of scenario: you have a document, you want to find all the companies or the drug names or the product names and product codes mentioned in the text, or you want to find all the dates and times or you want to connect them semantically or find out if there is a positive or negative sentiment about a particular company.

With traditional NLP we can analyse vast amounts of text data to extract meaning and insights, such as clinical trial protocols.

Typically, a company would have unstructured data that has accumulated over the years. This is very common in certain industries, such as legal, financial, medical, pharmaceuticals, and insurance. Often there is an adverse event, or costly incident that it’s desirable to avoid.

For example, in shipping, a ship enters a port and is detained for 5 days by port authorities for violating the MOU rules, costing $5 million. When vessels are inspected, an inspection report is written up as a PDF.

An NLP project which would have low business value would be: read the inspection report and classify violations into “lifeboats”, “fire safety” and “crew conditions”

A high business value initiative might be: read the inspection report and predict that vessel is 60% likely to be impounded on next stop in port of Singapore.

To understand the power of NLP, imagine you had a smart graduate reading the documents who never sleeps, and they have access to the business data. What would they do on seeing a new document?

Traditional NLP, while less flashy than generative AI, is a vital tool for businesses that deal with large volumes of text data. Its ability to extract meaning and structure from language unlocks valuable insights that can inform better decision-making.

How to get business value out of AI

Some of the most common and the most beneficial initiatives that you can do in AI would be things like predicting the costs or the risk or the duration of a project.

So for example, imagine you have data on 100 clinical trials that have been run in the past within your organisation. For each clinical trial trial, you know a few things about them, and you know for the past clinical trials, how much it cost. What you want to do is for a future upcoming clinical trial where you know these data points, predict the cost.

And that’s something that’s really valuable in all kinds of industries, predicting costs or predicting risk of failure, predicting if a building project is going to fail and result in litigation, or predicting if a ship entering into a harbour is likely to be impounded by the port authorities. These are all things that really quite powerful in terms of the impact on the bottom line or on the risk exposure of an organisation.

Actually, we spend a lot of time talking about generative AI when really we should be looking at problems and models from a business value perspective.

  • Focus on the problem, not the technology: Start by identifying a clear business problem, such as predicting project costs or customer churn. Then, explore if AI is the best solution. Often, simpler methods like averaging historical data can be highly effective.
  • The power of data: The success of AI hinges on data quality and availability. Cleaning and integrating data from disparate sources is often the most significant hurdle in AI projects.
  • Simple is powerful: Don’t overcomplicate things. Linear regression models or simple averaging can deliver impressive results for tasks like cost prediction or customer segmentation.

It’s also a good idea to plan your data science project with a kickoff meeting and prepare for it to take a while to get access to data.

Hybrid AI: combining approaches

Quite often we can get a huge amount of value by combining generative AI or traditional NLP with business-oriented models. For example, in our Clinical Trial Risk Tool, we are using NLP to extract key features from trial protocols, and then a simpler linear model to calculate cost and risk.

This is called hybrid AI and can be very powerful and is often the option that delivers the biggest monetary impact for a business.

Unstructured text data

Many businesses have a treasure trove of untapped potential: unstructured data like emails, reports, and contracts. NLP can unlock this data’s value. Imagine an “always-on” analyst who reads reports, classifies violations, and even predicts future problems – that’s the power of NLP in a hybrid model.

Fast Data Science: AI and NLP experts at your service

At Fast Data Science, we specialise in NLP solutions. We can help you transform your unstructured data into actionable insights, improving efficiency, reducing risk, and giving you a competitive edge.

Ready to get started?

Focus on the practical applications of NLP and capture the hidden monetary and efficiency value in your data. Contact Fast Data Science today and see how NLP and AI can take your business to the next level.

We have also provided some resources for planning and running data science projects in a handy checklist on the Resources section of our website, together with an in-browser Gantt chart generator for NLP projects, a data science roadmap planner, a project cost planner spreadsheet, and a project risk tool.

References

Ribary, M., et al. Insolvency Bot: A GPT-based Legal Advice Tool for Small Businesses in Distress [Long Paper Version]. Zenodo, 12 Sept. 2023, doi:10.5281/zenodo.10029735.

Wood TA and McNair D. Clinical Trial Risk Tool: software application using natural language processing to identify the risk of trial uninformativeness. Gates Open Res 2023, 7:56 doi: 10.12688/gatesopenres.14416.1.

T. Mikolov et al.. Efficient Estimation of Word Representations in Vector SpacearXiv:1301.3781 (2013)

Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. ImageNet classification with deep convolutional neural networks. Communications of the ACM 60.6 (2017): 84-90.

Unlock Your Future in NLP!

Dive into the world of Natural Language Processing! Explore cutting-edge NLP roles that match your skills and passions.

Explore NLP Jobs

Fast Data Science webinar on AI and NLP in pharmaceuticals
Data science

Fast Data Science webinar on AI and NLP in pharmaceuticals

On 29 May, Thomas Wood presented a webinar on how AI and Natural Language Processing (NLP) can transform clinical trials in the pharmaceutical industry.

Clinical trial cost modelling with NLP and AI
Data scienceDeep learning

Clinical trial cost modelling with NLP and AI

Modelling risk and cost in clinical trials with NLP Fast Data Science’s Clinical Trial Risk Tool Clinical trials are a vital part of bringing new drugs to market, but planning and running them can be a complex and expensive process.

Semantic similarity with sentence embeddings
Data scienceNatural language processing

Semantic similarity with sentence embeddings

In natural language processing, we have the concept of word vector embeddings and sentence embeddings. This is a vector, typically hundreds of numbers, which represents the meaning of a word or sentence.

What we can do for you

Transform Unstructured Data into Actionable Insights

Contact us