LLM4SAMPERS4

PoC/Research created by Trisha Chandrasekar English
22d ago update
Summary Objectives Solution Outcome
SAMPERS is the national transport model system, owned by Trafikverket. It is an essential tool used to assess transport infrastructure projects across Sweden. The new version, SAMPERS 4, was launched on April 2, 2024. 

As with anyone learning a novel software, it can be challenging for new and even experienced users to navigate efficiently. This complexity often leads to prolonged learning curves impacting efficiency and decision-making processes in transport modeling.

To address these challenges, I have worked on LLM4SAMPERS4 as a part of my Master's Thesis. This is a Retrieval Augmented Generation (RAG) solution which involves the development of a specialized Q&A chatbot designed to streamline the interaction between SAMPERS 4 users to help understand the SAMPERS 4 system better.

Objectives of LLM4SAMPERS4:

  • Efficient Information Retrieval: Enable users to quickly and accurately access information from the SAMPERS 4 knowledge base.
  • Enhanced User Understanding: Provide clear, understandable explanations of the modules, files, and functions within SAMPERS 4 to assist users in navigating its complexities.
  • Context-Aware Assistance: Offer tailored responses to user inquiries, focusing on the specific context of their projects and the nuances of transport modeling in SAMPERS 4.
The solution is a RAG pipeline. (Note that this is not ChatGPT but a solution I have worked on).

It is as seen in the image.

The system was tested by actual users of SAMPERS 4. 

Results

- The results were far more detailed than responses given by ChatGPT and more specific to SAMPERS 4.

- It was able to give the text files which is what had to be used.

- It can capture information from images and tables as well.

- It could answer questions with respect to the transportation project scenario specified

Painpoints

#1 It struggled to retrieve information from tables sometimes. Possible reason could be due to conflicts of information about a topic with the same name. This can be rectified by improving method of retrieval.

#2 User has to be very specific about how the question is being asked.

Future Improvements

- Improving method of information retrieval (For example, using a MultiQuery technique where the question will be rephrased in different ways to retrieve all relevant information from different perspectives)

- Using a Prompt Template to aid users to be more specific and concise in their questions

- Leverage years of experience of solving a project by using documentation to teach the model how it can think when solving a problem

Attributes

Transportation & Logistics
IT & Software
More Efficient
Practioner
Optimization
Generative AI