Retrieval-augmented generation (RAG) and natural language processing

· Thomas Wood
Retrieval-augmented generation (RAG) and natural language processing

Natural language processing (NLP) is revolutionising how businesses interact with information. But large language models, or LLMs (also known as generative models or GenAI) can sometimes struggle with factual accuracy and keeping up with real-time information.

If ChatGPT was trained on data until a certain year, how can it answer questions about events that happened after the cutoff point?

Retrieval-augmented generation (RAG) allows LLMs such as ChatGPT to stay up to date in their responses.

Natural language processing

Want to learn more?

Liked what you’ve just read? Get in touch for an NLP consulting session.

What is retrieval-augmented generation (RAG)?

Remember the old mobile phones which completed a sentence by taking into account the previous words? That’s all the LLMs are doing.

An LLM is a super-powered autocomplete. It excels at understanding language patterns but can lack domain-specific knowledge. LLMs are notorious for hallucinating when they don’t know the answer.

We can mitigate the problem of hallucinations and inaccuracies by taking the user prompt, and leveraging an external knowledge base and prepending or appending some useful information which we think the LLM should know, before we pass the prompt to the LLM. For example, if the user has a query about English insolvency law, we can send the user’s original question, together with some relevant information retrieved from a database.

Modifying the prompt sent to an LLM is also called prompt engineering.

With RAG, we augment the request by retrieving relevant documents from the knowledge base and feeding them to the LLM along with the original prompt. This empowers the LLM to generate more accurate and up-to-date responses.

A demonstration of the Insolvency Bot, a use case of RAG (retrieval augmented generation) in the legal domain.

How retrieval augmented generation can saving money and time

Here’s how RAG and prompt engineering can benefit businesses:

  • Reduced Costs: Training and maintaining LLMs can be expensive. RAG allows businesses to leverage existing LLM models and enhance their capabilities with targeted knowledge bases. This eliminates the need for costly retraining on ever-growing datasets.
  • Improved Accuracy: Although RAG doesn’t eliminate LLM hallucinations and inaccuracies, it can be shown to improve the performance by grounding responses in reliable sources. An LLM can be prompted to give relevant citations. This builds trust with customers and reduces the risk of disseminating misinformation. We found that RAG gave a significant improvement in accuracy of GPT-3.5 and GPT-4 in the legal domain (Ribary, M., Krause, P., Orban, M., Vaccari, E., Wood, T.A., Prompt Engineering and Provision of Context in Domain Specific Use of GPT, Frontiers in Artificial Intelligence and Applications 379: Legal Knowledge and Information Systems, 2023. https://doi.org/10.3233/FAIA230979, see our writeup here).
  • Faster Information Retrieval: RAG allows businesses to tap into the power of external knowledge bases, making information retrieval faster and more efficient. This frees up employees’ time for more strategic tasks.

Real-world applications of retrieval augmented generation

  • Domain-specific question answering Try our Insolvency Bot, which uses RAG to retrieve legal precedents and English statute law in order to answer questions about English insolvency procedures. It is not legal advice but it’s a demo which we developed in conjunction with a team at Royal Holloway and presented at the JURIX legal tech conference in 2023.
  • Customer Service: RAG-powered chatbots can answer customer queries with higher accuracy, reducing the need for human intervention and saving on support costs. For example, Fast Data Science are using RAG for our chatbot on fastdatascience.com. Try it out!
  • Market Research: Analyze vast amounts of customer reviews and social media data using RAG to gain deeper insights into customer preferences.
  • Internal Knowledge Management: RAG can streamline access to internal documents and resources, improving employee productivity.

The Future of NLP

RAG represents a significant step forward in NLP. By combining the power of LLMs with external knowledge, businesses can unlock new levels of efficiency, accuracy, and cost-effectiveness in information retrieval. As technology evolves, RAG is poised to play a central role in the future of human-computer interaction.

Your NLP Career Awaits!

Ready to take the next step in your NLP journey? Connect with top employers seeking talent in natural language processing. Discover your dream job!

Find Your Dream Job

Natural Language Processing | What is NLP and how can it help my business? (video)
Natural language processingAi for business

Natural Language Processing | What is NLP and how can it help my business? (video)

This new video explains natural language processing: what it is, how it works, and what can it do for your organisation. Natural Language Processing (NLP) is a branch of Artificial Intelligence (AI) that focuses on giving computers the ability to understand human language, combining disciplines like linguistics, computer science, and engineering.

A/B test calculator (Bayesian)
Data science consultingAi for business

A/B test calculator (Bayesian)

This free A/B test calculator will help you compare two variants of your website, A and B, and tell you the probability that B is better. You can read more about A/B testing in our earlier blog post on the subject. You may also be interested in our Chi-Squared sample size calculator which will help you calculate the minimum sample size needed to run a Chi-Squared test, given an expected standardised effect size.

A/B testing
Data science consultingAi for business

A/B testing

See also: Fast Data Science A/B test Calculator (Bayesian) A/B testing is a way you can test two things, Thing A, and Thing B, to see which is better. You most commonly hear about A/B testing in the context of commercial websites, but A/B testing can be done in a number of different contexts, including offline marketing, and testing prices.

What we can do for you

Transform Unstructured Data into Actionable Insights

Contact us