Exploring use cases of GPT models in healthcare

Project owned by AI Sweden and part of GPT Summer internship program English
342d ago update
Exploration of how generative (GPT) AI can be used in healthcare. We will explore 4 use-cases:

  • Making a chat bot for medical professionals that can find and give information about patients.
  • Making a chat bot for patients that can keep track of medical history, appointments etc.
  • Making an AI assistant that simplifies information gathering from the intranet.
  • Explore how to connect GPT-model to gather information from internet to answer medically related questions (1177).

Note: Documents are of imaginary patients since in practice these are very sensitive and for legal reasons cannot "leave" the hospital. They were however made to be similar in structure to real documents, with guidance from data scientists working in the field.

Partner: Sahlgrenska University Hospital (Västra Götalandsregionen)

Link to code repository: https://github.com/Felix-Nilsson/gpt-internship

pages

Resources

2023-09-07 17:40 Post
The internship is now over with the final event successfully held on the 29th of August. 

This summer the team developed a chatbot application covering four use cases for LLM's in healthcare, specified by AI Competence Center at Sahlgrenska University Hospital. They were the following:

  • An assitant for a patient to answer questions about their medical records.
  • A chatbot that can answer general medical questions using selected sources on the internet.
  • An assitant for doctors that can help them to quickly find information about patients.
  • A chatbot that can help medical staff with administrative questions, e.g looking up phone numbers, email, policies etc.

Our final product was a  website where the use cases were implemented in a chat interface. 

Key concepts were Retrieval Augmented Generation (RAG), word embeddings, agents (OpenAI function calling) and vector databases.

Even though this application was made as a proof of concept in mind,  we are very satisfied with the result. We are happy to hear that VGR will use our project as a stepping stone for future projects.

Finally we want to thank AI Sweden and VGR for the opportunity to work with and explore the possibilities of LLM's during the summer!

2023-08-25 08:18 Post
This was our final week working with our project and it was mostly spent adding small features and redesigning the UI. The main feature added was the ability to save chats, enabling multiple conversations at once.

We have also presented our application to groups at VGR where we hope that we gave good directions and ideas to what is possible, both with this application but also with LLMs in general.

Next week we will have our final event and we hope that as many partners as possible will attend!

We want to thank AI Sweden and VGR for the opportunity to explore LLMs in healthcare during the summer and we are excited to see what will happen in the future!

2023-08-18 07:46 Post
Since the internship is coming to an end, we have been focusing our energy to illustrate our project and its findings. We have created a presentation and a poster for the final event that is being held August 29th .

We also got the opportunity to present our project to a group of people at VGR, where we had an interesting discussion that gave us good insights to what the next steps would be. Another couple of presentations are scheduled to make sure that we can share our findings with as many people as possible.

Finally, our codebase is now public and you are all welcome to take a look. Hopefully you can find something that can help you in your work! Link can be found on this project page.

As we are approaching the end of this project and our knowledge sharing event, we have created a preliminary version of our presentation.

We have finalized our front page for the code repository (README) that includes instructions on how to run our application as well as some general information and how we test.

As for the application itself, we have worked on integrating the different chatbots to make it easier to move between them. (For example, so a patient can ask a question about their medical record and then follow up that question by a question to the internet-bot (1177.se, FASS.se, internetmedicin.se) in order to get a clarification about some detail.)

We have explored the possibility of running a local model and have come to the conclusion that it would not be reasonable with the time left and the resources at our disposal. But, we still consider it to be an interesting possibility that would be valuable to explore further.

Next week, we will meet with people from AI Competence Center at Sahlgrenska/VGR and demo our application.

We will also continue working on refining our presentation and adding any final touches to the application.

2023-08-04 07:18 Post
This week  we worked on integrating the testing via promptfoo as a github-action. This means that every time the prompts are edited in some way, we will run a test script to ensure that the quality was not diminished. It will enable us to more systematically measure improvements in the actual prompts.

For this to work as intended we needed to refactor our codebase to store prompts in a single directory, which we also started working on.

Further, we also worked on some minor UX improvements. For example we now have more nicely formatted links provided by our LangChain bot, as well as further additions to some settings menus for the chatbots.

Overall we are satisfied with our progress, even if there still are some tasks to finish. Next week we will start to work on our final presentation, and keep fixing various issues. We may also explore the possibilities of using a local model like LLaMa-2 rather than an API like with GPT-3/4, which may be preferable in a healthcare context.

2023-07-31 07:35 Post
These two weeks are summarized in this post since members of the group went in and out of vacation. From now on, all members are back.

Week 5 was spent adding features and fixing several smaller miscellaneous problems. The most prominent feature added was a settings menu where you could choose tools and what type of language to use. 

Week 6 was focused on testing and documentation. We wrote a draft for a set of guidelines when working with LLMs in healthcare with topics that we believe are important to consider. The new test suite was created using promptfoo that enables us to test prompts faster. 

For week 7 we want all currently planned features to be implemented and refine our test suite.

2023-07-14 12:07 Post
During week 4 we managed to get a fully functioning application with our new frontend/backend-setup. What we also did was to transition our data storage from pkl-files in to a chroma database, speeding up document retrieval and answer generation. 

In parallel with this we also added the following functionalities to our application:

  • Possibility to reset chat
  • Switch between different clients
  • Option to view sources of answer

To finish the week of we had a meeting with a doctor that gave us valuable input on the state of the application, as well as what could be improved. 

We want to continue to refine the product and  improve user experience.  Some time during the coming two weeks we will go on vacation so the progress might slow down a little.

2023-07-07 11:23 Post
At the beginning of the week we started and finished the development of working versions of our intranet- and internet-bot. The intranet-bot uses similarity search among documents to find relevant information, while the internet-bot uses LangChain agents to access 1177.se, internetmedicin.se and FASS.se. We do not have access to their respective API's, instead we used the DuckDuckGo search engine which might hamper our results slightly.

The rest of the week was dedicated to building a new frontend in Sveltekit as we believe that this would give us more flexibility and is a better long term solution. Paired with this we built a server backend in Flask/Python in order to integrate our models well. 

If we look back at our goals for this week we can say that we achieved what we planned. Looking ahead to next week, we hope to have a fully functioning application in our new setting.

Please note that the title is just a placeholder and may be subject to change.

Attributes

Health
Generative AI