- Twig Dev Notes
- Posts
- AI Dev Notes: Jan 12, 2026
AI Dev Notes: Jan 12, 2026
Explore AI, LLM, RAG, Agent, MCP Techniques with the Twig dev team
Chandan Maruthi
January 13, 2026 • Estimated Reading Time: 4 minutes
Highlights21 RAG Strategies - over 400 DownloadsI am excited to share the 21 RAG Strategies Ebook was downloaded over 300 times over this weekend. It great to see how important RAG is in todays enterprise AI. Remember to share what you liked or disliked about it - Chandan |  n |
AI will be the most transformative technology humanity has ever created.
Engineering Notes
Curated Docs vs User Generated Data?
Why User Generated Data hurts RAG
UGC like Slack, Tickets etc is short length and very specific to a topic. This created sharp embeddings. Embeddings on curated data on the other hand like documents, confluence, help docs etc are lengthy and often cover multiple topics. Making the embeddings broad. In vector search, we found that UGC embeddings beat curated data. This problem was not solved even when we added reranking algorithms.
How we solved it
Separate pipelines, data sources are marked as UGC vs Curated. We ran separate retrieval pipelines for each path. In vector search we picked the top N results from each pipeline. Then we ran a rerank on the top xN results. This allowed the best results to showup in vector search.
Future Research
I wonder if there is a unit size of data. What happens if all data is reduced to that unit size so embeding are always based on the same size of text. Would this help the quality of embeddings?
About Twig
Learn how you can ship RAG projects faster with Twig
Twig is an AI engineering platform built for teams shipping Retrieval-Augmented Generation (RAG) and agentic systems to production. It automates the hardest parts of enterprise AI—data ingestion, chunking, vectorization, and evaluation—so developers can focus on building intelligent copilots and support automation. Trusted by fast-growing startups and enterprise teams alike, Twig helps you go from prototype to production up to 80% faster.
MCP: Use AI inside of your custom apps with Twigs MCP interface
Auto-ingestion: Instantly connects and cleans data from sources.
Smart chunking: Optimizes context size for precise retrieval.
Auto-indexing: Embeds and stores data at scale.
Self-evaluation: Monitors and improves answer quality.
Fast deploy: Ships RAG pipelines to production 80% faster.
Agent Builder
Build AI Agents in Minutes, Not Months
Twig’s Agent Builder makes creating intelligent, task-specific AI agents as simple as assembling building blocks. Instead of stitching together APIs, prompts, and retrieval systems manually, teams can define an agent’s role, connect it to knowledge sources, and deploy it instantly—all from one unified interface. The Agent Builder handles context management, memory, and reasoning flow automatically, so developers can focus on outcomes, not infrastructure.
From Prototype to Production Effortlessly
Each agent built on Twig is powered by enterprise-grade RAG and evaluation pipelines, ensuring reliable, context-aware responses across real business data. Teams can customize behaviors, track performance, and iterate without writing extra code. Whether it’s a support bot, knowledge assistant, or internal co-pilot, Twig turns AI agent creation into a fast, repeatable process—making enterprise-ready automation accessible to every developer.