Introduction
The landscape of building LLM applications is shifting dramatically. The intricate scaffolding layers that once seemed essential—indexing frameworks, query engines, retrieval pipelines, and handcrafted agent loops—are fading away. In a recent interview, Jerry Liu, CEO and co-founder of LlamaIndex, describes this collapse not as a crisis but as an evolution. Developers now face a simpler yet more strategic challenge: instead of orchestrating deterministic workflows, they must focus on context as the true differentiator. This guide walks you through the steps to adapt your approach, reduce reliance on heavy frameworks, and position your projects for the next wave of AI-native development.
What You Need
- Understanding of current AI agent patterns – familiarization with managed agent diagrams, Model Context Protocol (MCP), and Claude Agent Skills plugins.
- Access to modern LLMs – models like GPT-4, Claude 3, or Codex that can reason over large unstructured data and self-correct.
- A lightweight tooling stack – for example, LlamaIndex, MCP connectors, and a code editor that supports natural language prompts (e.g., Claude Code).
- Document parsing capabilities – opt for agentic document processing with OCR to extract data from locked file formats (PDFs, images, etc.).
- Modular integration mindset – willingness to swap tools (OpenAI Codex vs Claude Code) without rewriting everything.
Step-by-Step Guide
Step 1: Acknowledge the Shift from Frameworks to Context
The first step is recognizing that the scaffolding layer—those carefully composed indexing layers, query engines, and retrieval pipelines—is no longer the bottleneck. Models now reason over massive amounts of unstructured data, often better than humans. As Liu notes, “The new programming language is essentially English.” Start by accepting that heavy orchestration frameworks may become obsolete. Your focus should shift from how to connect components to what data to feed the model.
Step 2: Embrace Natural Language as Your Primary Interface
With coding agents like Claude Code or OpenAI Codex generating most of your implementation code—Liu says about 95% of LlamaIndex’s code is AI-written—reduce your dependency on hand-coded libraries. Write prompts that instruct the AI to build retrieval pipelines or query structures. Instead of diving into documentation, simply point the agent at your data sources. This collapses the barrier between programmers and non-programmers. Practice: Describe your desired data flow in natural language and let the agent create the scaffolding dynamically.
Step 3: Consolidate Agent Patterns into a Managed Diagram
Agent patterns have converged toward what Liu calls a “managed agent diagram.” Rather than building custom orchestration for every workflow, use a harness layer that combines tools, MCP connectors, and skills plugins. Identify the core actions your agents need (e.g., search, data extraction, API calls) and plug them into a unified interface. This reduces complexity and makes your stack portable across different models. Action item: Review your current agent architecture—remove any manually coded loops that can be handled by an MCP or plugin.
Step 4: Invest in High-Quality Context Extraction
When scaffolding collapses, context becomes your moat. Agents must decipher file formats (PDFs, Office docs, images) to extract the right information. Liu’s team at LlamaIndex has focused on agentic document processing via OCR because “there’s a core set of data locked up in file format containers.” Deploy parsing solutions that provide higher accuracy and cheaper extraction. Test your agent’s ability to extract structured knowledge from various file types; the quality of this context directly impacts your application’s performance.
Step 5: Keep Your Stack Modular and Model-Agnostic
There is growing concern about builders being locked into specific platforms. Liu emphasizes that “whether you use OpenAI Codex or Claude Code doesn’t really matter. The thing that they all need is context.” Design your system so that the context layer remains separate from the model interface. Use standard protocols (e.g., MCP) to ensure you can swap the underlying LLM without rewiring every integration. Checklist: Can your document parser work with any agent? Are your tool connectors independent of the model provider?
Step 6: Iterate with Simpler Primitives
Liu highlights that it’s “just way easier for people to build even relatively advanced retrieval with extremely simple primitives.” Start by testing your application with a minimal set of retrieval components—perhaps just a vector store and a basic query engine. Let the LLM handle the reasoning, self-correction, and multi-step planning. Avoid over-engineering pipelines; the model’s ability to operate on raw or lightly processed data is improving rapidly. Measure performance and add complexity only when necessary.
Tips for Success
- Don’t fight the collapse – Embrace the reduction of manual scaffolding. Your time is better spent curating high-quality context data.
- Monitor emerging standards – MCP and other protocols evolve quickly; stay updated to avoid lock-in.
- Use agent skills plugins – Pre-built skills for Claude or Codex can save weeks of integration work.
- Test parsing accuracy – Regularly benchmark how well your context extraction pipeline handles diverse file formats.
- Let AI write the code – Encourage your team to dictate requirements in natural language and review AI-generated implementations.
- Plan for portability – Even if you currently use one model, design your data layer to interface with others.
Remember: the scaffolding may be falling, but that clears the way for context to become your strongest competitive advantage. Build around that, and you’ll thrive in the new AI landscape.