Natural Language Processing and Text Analysis

Our main area of focus is natural language processing (NLP). The manager, Thomas Wood, studied a Masters in 2008 at Cambridge University in Computer Speech, Text and Internet Technology and since then he has been working exclusively in machine learning and mostly in NLP. In 2018 he founded Fast Data Science to deliver data science consultancy, focusing on NLP.

We have built NLP pipelines from scratch, and worked on natural language dialogue systems, document classifiers and text based recommender systems. For these tasks we have used both traditional machine learning techniques as well as the state of the art such as neural networks. We normally use Python for our NLP work.

NLP examples

Examples of applications of natural language processing include:

  • Natural language understanding
  • Natural language dialogue systems
  • Text analysis
  • Topic analysis – clustering
  • Document classification
  • Document-based recommender systems
  • Unstructured data analysis
  • Document anonymisation

The power of NLP (interactive word2vec graph)

Below you can see a representation of some technical terms used in a dataset of clinical trial documents in 3D space.

Words with similar meanings and usages are close together. Words are colour-coded into clusters which correspond to groups such as diseases (cluster 3), verbs (clusters 1, 6 and 8), etc. If you move the mouse over a word, you can see that word’s cluster number, and the word’s nearest neighbours. A word’s nearest neighbours tend to be words with similar meaning or function, such as synonyms.

This is a demonstration of how natural language processing can be used to find synonyms and common topics in a completely new set of text documents, in totally unsupervised fashion.

The word vectors were calculated in 128 dimensions using the word2vec algorithm on Google Cloud Platform and reduced to three dimensions using t-SNE. The words were assigned to 15 clusters using the k-Means clustering algorithm.

NLP and unstructured data

Today many companies, in particular in certain industries such as healthcare, pharmaceuticals, legal, and insurance, have large amounts of unstructured data. This is typically data in text format, which may even be unscanned documents, PDFs, HTML, or any other file type.

Unstructured data is very difficult to deal with but can contain a goldmine of information. Fast Data Science specialises in extracting value from organisations’ unstructured datasets.

Natural Language Processing applications in healthcare

Natural Language Processing applications in healthcare Natural Language Processing applications in healthcare

AI and natural language processing are being increasingly adopted across the healthcare sector. This technology is sometimes called healthtech or MedTech. NLP is being used to compare and detect changes in clinical reports, extract clinical concepts such as MeSH terms from electronic medical records, and develop human-to-machine natural language dialogue systems to improve the healthcare experience.

We have worked on a number of projects in healthcare, including:

Natural Language Processing technologies at Fast Data Science

We do a lot of natural language processing with Python. We have worked on a variety of NLP models, including:

  • Bag of words, tf*idf, cosine similarity
  • NLP pipelines, lemmatisation, parsers, chunkers
  • Deep neural networks
  • Clustering: Latent Dirichlet Allocation
    • This is useful for extracting topics from a set of unstructured documents, for example legal documents, survey responses, factory error reports, etc.
  • Search engines and search term recommenders
  • Google Natural Language, AWS, Microsoft Azure
Natural Language Processing word cloud
Topic detection is an NLP technique that allows you to discover common themes in a set of unstructured documents.

Natural Language Processing in Python and R

We work with the following programming languages and frameworks:

  • TensorFlow
  • Keras
  • Python NLTK
  • R

Examples of past Natural Language Processing projects

NLP projects we have worked on for major household names include

  • a spoken dialogue system to control a smart home
  • an unsupervised text analysis program to analyse text descriptions of manufacturing defects (Boehringer Ingelheim)
  • a model to classify jobseekers’ CVs into industries and salary bands (CV-Library).
  • analysis of survey responses (White Ribbon Alliance)

Contact us to hire an NLP data scientist today!