Above: video of Thomas Wood presenting Harmony at the Pydata on 27 March 2024
Update: you can download the slides from the presentation here
Link to the meet up: Meetup.com.
I will present our work on Harmony, harmonydata.ac.uk, which is a free online AI research tool that uses generative AI and LLMs to help psychologists analyse datasets. It uses Python, Pandas and HuggingFace Sentence Transformers to find similarities between questionnaires.
Psychologists and social scientists often have to match items in different questionnaires, such as “I often feel anxious” and “Feeling nervous, anxious or afraid”.
This is called harmonisation.
Harmonisation is a time consuming and subjective process. Going through long PDFs of questionnaires and putting the questions into Excel is no fun.
We’ve been working on an open source Python library and free web tool called Harmony which uses natural language processing and generative AI models to help researchers harmonise questionnaire items, even in different languages.
Fast Data Science is a leading data science consultancy firm providing bespoke machine learning solutions for businesses of all sizes across the globe. With a focus on innovation and collaboration, Fast Data Science empowers businesses to leverage the transformative power of data.
Unleash the potential of your NLP projects with the right talent. Post your job with us and attract candidates who are as passionate about natural language processing.
Hire NLP Experts
You are probably familiar with traditional databases. For example, a teacher at a school will need to enter students’ grades into a system where they get stored, and at the end of the year the grades would need to be retrieved to create the report card for each student. Or an employee database might store employees’ home addresses, pay grades, start dates, and other crucial information. Traditionally, organisations use a structure called a relational database, where different types of data are stored in different tables, with links between them, and they can be queried using a special language called SQL.
A problem we’ve come across repeatedly is how AI can be used to estimate how much a project will cost, based on information known before the project begins, or soon after it starts. By “project” I mean a large project in any industry, including construction, pharmaceuticals, healthcare, IT, or transport, but this could equally apply to something like a kitchen renovation.
Senior lawyers should stop using generative AI to prepare their legal arguments! Or should they? A High Court judge in the UK has told senior lawyers off for their use of ChatGPT, because it invents citations to cases and laws that don’t exist!
What we can do for you
