Cutting onboarding time by 40% with iGPT, our internal AI assistant for platform engineering
Infraspec’s internal engineering team
A lot of the infrastructure work we do builds on things the team has already figured out before.
We had built up a lot of knowledge around Terraform, Kubernetes, Ansible and the way we approached platform engineering across different projects. New engineers at Infraspec had a lot of similar questions when they started working. Some of the answers were documented, and a lot more were in the heads of senior engineers who had solved the same problem on an earlier project.
That works when the team is small. As the team grew and more projects were running at the same time, it started slowing everyone down.
That’s the whole reason why iGPT was built.
2×
junior engineer productivity
30%
improvement in project efficiency
Why not just use a general LLM?
Our new engineers had a lot of context to pick up before they could get comfortable with the kind of infrastructure work we do. We already had all the knowledge to provide, but finding the right bit of it was still taking time.
A general LLM could definitely explain Terraform or Kubernetes. It just wouldn't have been able to tell you how we’ve set things up on our own projects or why we picked one approach over another. We wanted engineers to ask iGPT what they’d ask a senior engineer, and get an answer based on how Infraspec does things.
How did we get our own knowledge into it?
We started with the documentation that we had, along with FAQs and other internal knowledge we wanted iGPT to use. We cleaned up all that material before indexing it.
We then created embeddings from the content and stored them in a vector database.
When someone asks iGPT something, it pulls the relevant material from our docs and uses them while answering. This keeps the response tied to the way we’ve actually worked on these problems.
What did we use to build it?
We used OpenWebUI for the chat interface and connected it to the knowledge we already had internally. We used GPT-3.5 as well as open-source models like Gemma and Llama to generate responses.
We added a feedback mechanism as well, so engineers could tell us when an answer needed some work. Now the team had something concrete to use while improving the responses over time.
How did we test it?
We first rolled iGPT out to a smaller group and let them use iGPT before opening it up to the whole team. Our domain experts worked with the development team during that period to check if the right information was being retrieved and whether the answers were accurate enough.
We kept making changes based on the feedback from those users. Once the team was comfortable with how it was performing, we rolled it out more widely and continued monitoring it.
What changed for the team after rolling this out?
New engineers could get to the context they needed much faster, without having to rely as much on someone who had already worked through the problem. That made onboarding easier for them and also reduced some of the back-and-forth for experienced engineers who were otherwise being pulled in to answer those questions.
The change showed up in the numbers as well. Onboarding time came down by 40%. Also, junior engineer productivity doubled, and overall project efficiency improved by 30%.
Want to make your infrastructure work harder for your team?
Tell us what your engineers are wrestling with.