This post marks the end of our summer internship at AI Sweden and Zenseact. We are proud over what we have accomplished during our time here.  Zensei is now deployed and used by Zenseact employees.  Zensei boasts the following capabilities:

–  Basic chat function.
    – –  Chat with Zensei either in a channel or through DMs and receive an answer complete with sources and links to the various relevant internal documentation used for the answer.

- Ingrained search function.
   – – Instead of asking a question to Zensei, you can instead enter different keywords and Zensei will fetch the most relevant sources from the internal documentation.

- Feedback integration.
    – – Feedback can be given dynamically and it will be used to further help improve the selection of sources for each answer.

–  File analysis. Attach your own files in your message and Zensei will attempt to analyse and provide feedback.

    – – Code analysis: Attach your code files and Zensei will help with debugging or provide feedback and comments.

   – – Data analysis: Attach data files, such as .csv files, and Zensei can analyze and summarize trends and provide comments and insight.

  – – Text analysis. Attach text files and Zensei will read and analyze. You can for example get Zensei to summarize meeting minutes or explain documentation.

We are very satisfied with Zensei and our time spent this summer. We would like to thank AI Sweden and Zenseact for the opportunity and for welcoming us at the office.

The last two sprints have been focused on wrapping up the project.

Development of Zensei will continue after our internship ends, we are currently working on handing over the project to an existing team at Zenseact. To make the handover easier we are documenting the repository and how everything functions so that nothing is lost when we leave.

We have also been working on preparing two presentations, one that was held this week at Zenseact and also the AI Sweden Knowledge Sharing event next week.

This week Zensei was also deployed in Slack for all employees of Zenseact to try out and chat with.

This post summarizes the last three weeks of work since it is the first week the whole team is back together because of vacations.
Sprint 5:
When the user rates the answer in Slack, that rating is now added as a rating to the sources that Zensei used for creating the answer. This should with time lead to bad sources being used less.

Added a separate fitness measure to the sources that is based on keywords, date and user rating. This is to be used as a second step after selecting the top sources based on distance to pick out the most viable sources for the question. Might need some tweaking to function just right.

We also investigated possibility of using Llama 2 instead of chatGPT to be able to keep the solution in-house. However due to the complexity of setting it up and the hardware requirements it was not further pursued.
Sprint 6:

We started the process of cleaning, documenting and commenting our code. This is not only good practice for us but good for the eventual hand-over of the project.
We have examined and rewritten old code to better fit into the project as whole. We have written a docstring for every function in the project as well as comments explaining the code in most functions.

We also spent a considerate amount of time this sprint debugging and stress testing the code. Some unexpected bugs reared their heads which we had to fix.

For a new feature we added the possibility for tailored prompts for different use cases. For example a different specific prompt when using Zensei to analyze data or asking about code than the default chatbot prompt.

Sprint 7:

This sprint we added the ability to give detailed written feedback to Zensei directly in Slack which gets saved locally by Zensei for later use.

To continue the documentation of the code, we have written a read-me file for users as well as for new developers on how Zensei works and how to work on it.

A new feature was added that allows the user to attach files to their message which Zensei can now read and analyze, as shown in the attached image. This can be useful if you want to discuss code you've written or this months KPI numbers for example.
 

We have started to design and work on a front-end web page for Zensei that users that don't use Slack will be able to use. We also plan to add more features to the web page that we are unable to add to the slack version due to slacks limitations.

Some time this sprint was also spent on planning and performing a demo presentation for Zenseact.

This sprint we made several smaller updates to Zensei.

We implemented the ability to reenter an existing thread in Slack at any time, allowing the user to continue any old conversation with Zensei long after the initial question. This was accomplished by fetching the conversation data from the thread and adding this to the chatbots history as well as reembedding the original question in the thread to retrieve the correct sources from the original thread. Due to the fact that we always retrieve our sources at a temperature of 0 the sources will be identical no matter how many times we embed the same question allowing the sources to be identical when reviving an old thread.

A lot more data was also added this week. This should improve answers to questions that did not generate correct answers earlier. We also simplified the process of fetching, cleaning and embedding the data to be able to update the data when necessary.

We deployed the Slack bot on a virtual machine to be able to run it continuously and it is now available for some employees to chat with. As of now it is mostly used to test and evaluate the chatbot.

The next steps are to evaluate the performance with the added data and to adjust the prompt and ranking of the sources based on that.

This sprint we got the green light for using real data. But before we could use it we cleaned it from personal information (email adresses, phone numbers and names) to comply with GDPR. 

This week we also held a short beta test for Zensei in Slack with another team to collect some user feedback. This feedback was collected through reactions in slack (thumbs up or thumbs down). We also added the ability to show all the sources that Zensei used if you click on the eyes emoji.

The next step is to make use of the user feedback and to add more data.

For this sprint we have continued development on both the Slack bot and the standalone app and added several features. The bot now also has a name, Zensei.

We gave the bot the ability to handle follow-up questions and to keep the context of the previous messages. The sources that were available to the bot when formulating the answer is now displayed in the answer as hyperlinks. 

The updates to the Slack bot is that it now can handle multiple requests at once through multi-threading, and that it allows for follow-up questions by staying in the thread for 3 minutes after the latest message while retaining the context of the conversation.

We updated the front-end interface  which now is equipped with some debugging capabilities and provides the user with metadata accompanied by the response from the bot as well as a way to easily read the relevant documents provided as context to the bot.

We generated a large database of dummy data using gpt-3.5 that Zensei currently uses for information. The next step is to get green light for using real data and to add some feedback from the user to be able to rate the reliability of the sources.

This week was the first planned sprint of the project. The main goal for this sprint was to create a demo of the system flow using generated dummy data following the flowchart above.

We have created a solution for downloading, structuring, and embedding data in a database. We then read from the data base to extract context relevant to the query and then send this context along with the query to the gpt-3.5-turbo model to form an appropriate answer. The current ways of interacting with the bot is through a slack bot or through a rudimentary standalone application.