Summarizing Tickets and Searching Internal Docs
Customer Support / Internal EngineeringFine-tuning a model to summarize support tickets and building a RAG chatbot over years of internal documentation, for two separate enterprise teams.
Overview
A regional customer-support team was drowning in ticket volume and needed fast, accurate summarization, and a separate internal engineering organization had no fast way to search years of documentation and issue history. Canvas Labs Partners fine-tuned an open model for the first team's ticket summarization and built a retrieval-augmented chatbot grounded in internal knowledge for the second.
52% ROUGE score
summarization quality on the fine-tuned ticket model
Self-serve Q&A
replacing manual search through years of documentation
Challenge
One team was drowning in ticket volume, needing fast, accurate summarization just to keep up with what was coming in. Another had years of internal documentation and issue history with no fast way to search it, so an answer that already existed somewhere in the archive took real effort to find.
Solution
For the regional customer-support team, Canvas Labs Partners fine-tuned an open model using PEFT and LoRA, tuned specifically to the team's own ticket format and tone rather than a generic summarization style. The model reads each incoming ticket and produces a summary an agent can act on immediately.
For the internal engineering organization, Canvas Labs Partners built a LangChain-based retrieval-augmented chatbot, grounded in internal documentation and issue-tracker history through a vector database. An engineer can ask a question in plain language and get an answer sourced from years of accumulated knowledge, in seconds rather than digging through the archive.
Both systems apply the same two techniques to different knowledge problems: fine-tuning to teach a model one team's specific format and tone, retrieval-augmented generation to ground a model in a specific archive rather than its general training data. One turns a ticket into a summary in the team's own voice; the other turns a question into an answer sourced from internal history.
Results & benefits
The fine-tuned summarization model scores 52 percent on ROUGE, the standard measure of summary quality, so the regional customer-support team gets a usable ticket summary in the team's own tone rather than a generic one to edit by hand. The internal engineering organization's retrieval-augmented chatbot turned years of documentation and issue-tracker history into a self-serve resource: an engineer asks a question and gets an answer grounded in the archive, instead of searching for it themselves. Neither result required replacing the underlying knowledge, only building a faster way to reach it.
Customer
Support team & engineering org
Services
- Agentic AI consulting & development
- LLM fine-tuning & model adaptation
- RAG & knowledge retrieval systems
Tech stack
About us
Canvas Labs Partners is a software consulting team that designs and ships production agentic AI systems: multi-agent pipelines, tool-calling architectures, and human-governed automation, together with the trading infrastructure, blockchain, mobile, and product engineering those agents plug into. We publish our work as a case ledger rather than a portfolio, so every claim on this site links to the engagement behind it. Engagements follow the same four-step discipline, Map, Bound, Instrument and Govern, whatever the stack, and every entry in the ledger carries a result we are willing to be measured against.
Delivered individually, in a prior role, by one of Canvas Labs Partners’ founding team members. Not by the firm itself.
Have a system like this one?
We built this one end to end. If you are facing something similar in customer support, book a free discovery call and we will tell you what it would take.