On this article, you’ll find out how LangChain, LlamaIndex, and uncooked API calls every resolve a distinct layer of the LLM software stack, and the way to decide on amongst them primarily based on what your challenge truly requires.
Subjects we’ll cowl embody:
- What every possibility is designed to do, said plainly with out advertising and marketing spin.
- How the three approaches examine on efficiency, token overhead, debugging readability, and code quantity.
- A sensible determination framework for selecting the correct stage of abstraction earlier than you construct — and earlier than that selection turns into costly to undo.
Let’s not waste any extra time.
Introduction
You’ve gotten a working immediate. The mannequin is giving good solutions. Then the following requirement lands. Possibly it’s reminiscence; the mannequin wants to recollect what was stated three messages in the past. Possibly it’s retrieval — the mannequin must reply questions on paperwork it was not skilled on. Possibly it’s device use; the mannequin must verify a database, run a calculation, or name an exterior API earlier than it may reply. All of a sudden, a single shopper.chat.completions.create() name just isn’t sufficient, and you’re standing on the first actual architectural determination in your LLM challenge.
Three paths exist from that second: attain for LangChain, attain for LlamaIndex, or construct a skinny layer on prime of the uncooked SDK your self. Getting this selection fallacious doesn’t break the prototype. It breaks the manufacturing system six months later, if you end up debugging stack traces 40 frames deep, paying 2.7x what you ought to be on token prices, or spending a dash migrating away from breaking API adjustments.
LLM API spend doubled from $3.5 billion to $8.4 billion between late 2024 and mid-2025. These are actual manufacturing budgets. The framework layer — the code that sits between your software and the mannequin — immediately determines how a lot of that spend is doing helpful work versus paying for abstraction you didn’t want.
This text offers you an sincere comparability: what every possibility truly is, the place it genuinely wins, the place it prices you, and a call framework you should utilize tomorrow.
The Panorama in Plain English
Earlier than evaluating trade-offs, it helps to know what every possibility truly is — not what its advertising and marketing says, however what drawback it was constructed to unravel.
- LangChain began in October 2022 as a general-purpose framework for chaining LLM operations collectively. Its core concept was that constructing actual purposes required composing a number of steps — immediate templates, mannequin calls, output parsers, reminiscence, instruments — and there ought to be a normal method to try this. It has grown into the most important LLM framework by adoption: 119K GitHub stars, 500+ integrations, and a sprawling ecosystem. The LangChain crew now builds LangGraph, a separate bundle for stateful, graph-based agent workflows, because the really useful technique to construct manufacturing brokers inside the ecosystem.
- LlamaIndex (launched as GPT Index in November 2022) was constructed to unravel a distinct drawback: getting LLMs to motive over your individual knowledge. Its design is organized round knowledge ingestion, chunking, embedding, indexing, and retrieval. The place LangChain is about orchestrating what occurs between steps, LlamaIndex is about making the retrieval step itself as correct and environment friendly as doable. It sits at 44K GitHub stars with 300+ knowledge connectors via LlamaHub, overlaying sources like Notion, Google Drive, Slack, PDFs, and databases.
- Uncooked API calls means utilizing the OpenAI Python SDK, the Anthropic SDK, or any mannequin supplier’s shopper immediately — no orchestration layer, no abstractions past what the supplier ships. You write the immediate, name the mannequin, and deal with the response your self. This isn’t the primitive fallback it’s typically introduced as; it’s the method manufacturing groups are more and more migrating again to for workloads the place the framework’s complexity stopped paying for itself.
The vital factor to know earlier than studying any comparability is that these three choices usually are not competing on the identical dimension. LangChain is an orchestration toolkit. LlamaIndex is a retrieval toolkit. Uncooked API calls are a stance on how a lot abstraction you want. Many manufacturing techniques use two of them collectively. The query is at all times: given what I’m truly constructing, which layer of abstraction earns its price?
LangChain: The Orchestration Layer
LangChain’s power is assembling complexity. In case your software entails a number of steps, a number of instruments, conditional routing, reminiscence throughout turns, or brokers that motive earlier than performing, LangChain offers the constructing blocks for all of it, with connectors to 500+ providers and a neighborhood giant sufficient that somebody has already solved many of the edge instances you’ll encounter.
LangGraph, constructed by the identical crew and steady at v1.0 since October 2025, is the place the intense agent work lives now. It fashions agent workflows as directed graphs, the place nodes are Python features, edges are state transitions, and a central typed state object flows via the complete execution. It has built-in persistence by way of checkpointers to SQLite, PostgreSQL, or Redis, which implies brokers can pause mid-workflow, persist their state, and resume hours later. That’s genuinely exhausting to construct your self and is considered one of LangChain’s clearest justifications in a manufacturing context.
The sincere trade-offs are price naming immediately. LangChain provides ~10ms framework overhead per step, and LangGraph provides ~14ms. For many human-facing purposes that make LLM calls taking 1–3 seconds every, that is irrelevant. For prime-throughput pipelines processing hundreds of requests per minute, it compounds. Stack traces from LangChain manufacturing errors routinely span 15 to 40 frames of inside framework code; discovering the precise supply of a bug is slower than in a system you wrote your self. And for easy use instances, one documented comparability discovered LangChain incurring 2.7x larger prices than a local implementation for a primary RAG pipeline — the abstraction overhead consumed tokens that didn’t should be consumed.
LangChain v1.0 (October 2025) dedicated to API stability after a turbulent v0.1 via v0.3 interval that pressured a number of breaking migrations. That historical past is price figuring out. For brand new tasks, the soundness concern is essentially resolved. For groups operating v0.x code in manufacturing, the migration price to v1.0 is actual.
Here’s a working LangChain LCEL chain — the fashionable technique to compose LangChain operations.
Conditions:
|
pip set up langchain langchain–openai python–dotenv |
Tips on how to run: Save as langchain_chain.py, add OPENAI_API_KEY to your .env, run python langchain_chain.py
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 |
# langchain_chain.py # A LangChain LCEL chain: immediate template → mannequin → output parser # Conditions: pip set up langchain langchain-openai python-dotenv # Tips on how to run: python langchain_chain.py
import os from dotenv import load_dotenv from langchain_core.prompts import ChatPromptTemplate from langchain_core.output_parsers import StrOutputParser from langchain_openai import ChatOpenAI
load_dotenv()
# ── MODEL ───────────────────────────────────────────────────────────────────── # ChatOpenAI wraps OpenAI’s chat fashions. Swap the mannequin string to change # to gpt-4o-mini (cheaper) or claude-3-5-sonnet (by way of langchain-anthropic) — # the chain code beneath stays equivalent both method. This mannequin portability # is considered one of LangChain’s real benefits over uncooked API calls. llm = ChatOpenAI( mannequin=“gpt-4o”, temperature=0.2, api_key=os.getenv(“OPENAI_API_KEY”) )
# ── PROMPT TEMPLATE ─────────────────────────────────────────────────────────── # ChatPromptTemplate defines the message construction with named variables. # {matter} will get stuffed in at runtime — templates are reusable and versionable. immediate = ChatPromptTemplate.from_messages([ (“system”, “You are a concise technical explainer. Keep answers under 100 words.”), (“human”, “Explain {topic} in simple terms.”) ])
# ── OUTPUT PARSER ───────────────────────────────────────────────────────────── # StrOutputParser extracts the textual content content material from the mannequin’s AIMessage response. # With out it you get again an AIMessage object relatively than a plain string. parser = StrOutputParser()
# ── CHAIN (LCEL) ────────────────────────────────────────────────────────────── # The pipe operator (|) builds a sequential chain: immediate → llm → parser. # LCEL (LangChain Expression Language) makes the composition readable and # helps streaming, batching, and async execution with the identical interface. chain = immediate | llm | parser
if __name__ == “__main__”: # invoke() runs the complete chain synchronously outcome = chain.invoke({“matter”: “vector embeddings”}) print(outcome)
# stream() yields tokens as they arrive — no code adjustments wanted for streaming print(“n— Streaming response —“) for chunk in chain.stream({“matter”: “RAG pipelines”}): print(chunk, finish=“”, flush=True) print() |
What this does: Three objects — immediate, llm, parser — are related with the | operator. LangChain’s LCEL executes them so as: the template fills in {matter}, passes a formatted message to the mannequin, and the parser extracts a plain string from the response. The identical chain helps .invoke(), .stream(), .batch(), and .ainvoke() with none adjustments to the chain definition itself. That interface consistency is the clearest argument for LangChain on tasks that want a number of execution patterns.
Right here is similar basis prolonged to a tool-using agent with LangGraph.
Conditions:
|
pip set up langchain langchain–openai langgraph langchain–neighborhood python–dotenv |
Tips on how to run: Save as langchain_agent.py and run python langchain_agent.py
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 |
# langchain_agent.py # A LangGraph ReAct agent with two instruments: net search and a calculator # Conditions: pip set up langchain langchain-openai langgraph langchain-community python-dotenv # Tips on how to run: python langchain_agent.py
import os from dotenv import load_dotenv from langchain_openai import ChatOpenAI from langchain.instruments import device from langchain_community.instruments import DuckDuckGoSearchRun from langchain_core.messages import HumanMessage from langgraph.prebuilt import create_react_agent
load_dotenv()
llm = ChatOpenAI(mannequin=“gpt-4o”, temperature=0, api_key=os.getenv(“OPENAI_API_KEY”))
# Net search — no API key required search = DuckDuckGoSearchRun()
@device def calculate(expression: str) -> str: “”“ Consider a protected mathematical expression. Use for arithmetic or proportion calculations. Enter: a Python math expression string (e.g., ‘1500 * 0.08’). ““” strive: outcome = eval(expression, {“__builtins__”: {}}, {}) return f“Consequence: {outcome}” besides Exception as e: return f“Error: {str(e)}”
instruments = [search, calculate]
# create_react_agent wires collectively the LLM, instruments, and a built-in ReAct loop. # The agent thinks, calls a device, reads the outcome, and continues till performed. agent = create_react_agent(llm, instruments)
if __name__ == “__main__”: outcome = agent.invoke({ “messages”: [HumanMessage(content=“What is 15% of 2400?”)] }) print(outcome[“messages”][–1].content material) |
What this does: create_react_agent abstracts the complete reasoning loop. The mannequin decides whether or not to make use of a device, LangGraph executes the chosen device, feeds the outcome again into the message historical past, and repeats till the mannequin has a closing reply. What would take 50+ strains in a uncooked implementation is 4 strains right here. That abstraction is suitable whenever you want it. The query the following part addresses is: when do you not?
LlamaIndex: The Retrieval Layer
LlamaIndex was designed from the bottom up for one job: serving to LLMs motive over exterior knowledge. That focus is each its largest power and the clearest sign for when to make use of it. In case your software’s central problem is “how do I get the mannequin to reply precisely from my paperwork,” LlamaIndex is the proper place to begin.
The efficiency numbers mirror that specialization. LlamaIndex indexes paperwork 2.5x sooner than LangChain and hits sub-200ms question latency for 10,000 paperwork. Its framework overhead of ~6ms compares favorably to LangChain’s ~10ms and LangGraph’s ~14ms. On the token stage, LlamaIndex makes use of ~1.6K tokens per question versus LangChain’s ~2.4K — a 33% distinction that provides up rapidly at scale.
The architectural motive for these variations is that LlamaIndex treats retrieval as a first-class primitive, not a composable part. Its 5 core abstractions — knowledge connectors, node parsers, indices, question engines, and workflows — are designed to work collectively out of the field. Hierarchical chunking preserves parent-child relationships between doc sections. Auto-merging retrieval recombines associated chunks at question time. Sub-question decomposition breaks complicated queries into easier ones and merges the outcomes. You get all of this with much less code: LangChain requires 30–40% extra code than LlamaIndex for equal RAG pipelines.
The place LlamaIndex is weaker is on the agent facet. Its Workflows system handles async, event-driven pipelines effectively, however stateful multi-turn brokers with built-in persistence require extra guide implementation than LangGraph. LangGraph’s checkpointing — the place an agent pauses, persists its full state, and resumes later — is one thing LlamaIndex Workflows can obtain however doesn’t present out of the field. For doc Q&A and data retrieval, this not often issues. For long-running agentic workflows with human-in-the-loop necessities, it issues an important deal.
Here’s a full LlamaIndex RAG pipeline, from doc ingestion to question.
Conditions:
|
pip set up llama–index llama–index–llms–openai llama–index–embeddings–openai python–dotenv |
Tips on how to run: Save as llamaindex_rag.py and run python llamaindex_rag.py
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 |
# llamaindex_rag.py # Full LlamaIndex RAG pipeline: ingest paperwork → index → question # Conditions: pip set up llama-index llama-index-llms-openai # llama-index-embeddings-openai python-dotenv # Tips on how to run: python llamaindex_rag.py
import os from dotenv import load_dotenv from llama_index.core import VectorStoreIndex, Doc, Settings from llama_index.llms.openai import OpenAI as LlamaOpenAI from llama_index.embeddings.openai import OpenAIEmbedding
load_dotenv()
# ── GLOBAL SETTINGS ─────────────────────────────────────────────────────────── # LlamaIndex v0.10+ makes use of a world Settings object as a substitute of ServiceContext. # Configure your LLM and embedding mannequin as soon as right here — all pipeline elements # choose them up routinely. Swap fashions right here to alter the entire pipeline. Settings.llm = LlamaOpenAI( mannequin=“gpt-4o”, temperature=0, api_key=os.getenv(“OPENAI_API_KEY”) ) Settings.embed_model = OpenAIEmbedding( mannequin=“text-embedding-3-small”, # Quick and cost-effective for many RAG duties api_key=os.getenv(“OPENAI_API_KEY”) )
# ── DOCUMENTS ───────────────────────────────────────────────────────────────── # In manufacturing, change with: SimpleDirectoryReader(“./docs”).load_data() # LlamaHub offers 300+ connectors for Notion, Google Drive, PDFs, databases. # Paperwork created inline right here to maintain the instance totally self-contained. paperwork = [ Document( text=( “LlamaIndex is a data framework for LLM applications. “ “It specializes in document ingestion, chunking, embedding, and retrieval. “ “Core abstractions: data connectors, node parsers, indices, query engines, “ “and workflows. LlamaHub provides 300+ pre-built data connectors.” ), metadata={“source”: “llamaindex_overview”} ), Document( text=( “LangChain is a general-purpose LLM orchestration framework. “ “It excels at chaining operations, multi-step agents, tool use, and memory. “ “LangGraph — the recommended way to build stateful agents in the LangChain “ “ecosystem — stabilized at v1.0 in October 2025.” ), metadata={“source”: “langchain_overview”} ), Document( text=( “Raw API calls use the OpenAI or Anthropic SDK directly with no framework. “ “This approach has the lowest latency and highest transparency. “ “Best for simple, one-off tasks where framework abstraction adds no value. “ “As complexity grows, a thin internal wrapper is usually preferable to “ “adopting a full orchestration framework.” ), metadata={“source”: “raw_api_overview”} ), ]
# ── INDEX ───────────────────────────────────────────────────────────────────── # from_documents() handles the complete pipeline: chunk → embed → retailer. # By default, vectors are saved in reminiscence. For manufacturing, cross a vector retailer: # index = VectorStoreIndex.from_documents(docs, storage_context=storage_context) # the place storage_context factors to Pinecone, Weaviate, Chroma, and many others. index = VectorStoreIndex.from_documents(paperwork)
# ── QUERY ENGINE ────────────────────────────────────────────────────────────── # as_query_engine() creates a retrieval + technology pipeline in a single name. # similarity_top_k=2 retrieves the two most related chunks per question. # response_mode=”compact” merges retrieved chunks earlier than passing to the LLM — # reduces token utilization in comparison with “default” mode, which sends every chunk individually. query_engine = index.as_query_engine( similarity_top_k=2, response_mode=“compact” )
if __name__ == “__main__”: questions = [ “What is LlamaIndex best suited for?”, “How does LangChain differ from LlamaIndex?”, “When should I use raw API calls instead of a framework?”, ]
for q in questions: print(f“Q: {q}”) response = query_engine.question(q) print(f“A: {response}n”) |
What this does: Settings.llm and Settings.embed_model configure the complete pipeline as soon as. VectorStoreIndex.from_documents() handles chunking, embedding, and indexing in a single name — a course of that takes 30–40% extra code in LangChain. as_query_engine() then creates a retrieval + technology pipeline with two strains. The similarity_top_k and response_mode parameters provide you with management over the retrieval conduct with out requiring you to assemble the retrieval elements your self. That’s the LlamaIndex worth proposition in concrete kind: much less meeting, extra retrieval high quality.
A two-column structure diagram evaluating LlamaIndex and LangChain RAG pipelines facet by facet (click on to enlarge)
Uncooked API Calls: The Minimal Path
The default assumption in most LLM developer communities is that you simply begin with uncooked API calls and graduate to a framework as your challenge grows. The sample price inspecting in 2026 is the reverse: groups that began with LangChain and are quietly rewriting to uncooked SDKs.
The OpenAI Brokers SDK, launched in March 2025 with 26,900 GitHub stars and 10.3 million month-to-month downloads, offers device use, multi-agent handoffs, built-in tracing, and guardrails in a minimal bundle. Its overhead per device name is 2–5ms versus LangChain’s 10–30ms. Groups migrating from LangChain to uncooked SDKs sometimes see a 40–60% discount in code quantity and a 70–90% discount in month-to-month framework upkeep burden.
The argument for the uncooked path just isn’t that frameworks are dangerous. It’s that the worth of an abstraction layer relies upon totally on whether or not it’s hiding complexity you truly face. In 2022, constructing immediate chains and dealing with device calls reliably required framework help as a result of vendor APIs have been inconsistent. By 2026, OpenAI and Anthropic have absorbed device calling, streaming, operate schemas, and multi-turn reminiscence into their native SDKs. The framework’s abstractions now not conceal significant variations. They conceal readability.
Uncooked API is constantly the quickest possibility, with no framework overhead and no additional LLM requires orchestration. Frameworks add 100–500ms of Python overhead per agent step. For latency-sensitive workloads — real-time buyer help, voice brokers, and high-throughput pipelines — that overhead is actual and price avoiding.
Here’s a full tool-using agent constructed on the uncooked OpenAI SDK in below 80 strains.
Conditions:
|
pip set up openai python–dotenv |
Tips on how to run: Save as raw_api_agent.py and run python raw_api_agent.py
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 |
# raw_api_agent.py # A whole tool-using agent on the uncooked OpenAI SDK — no framework. # That is ~75 strains together with feedback. Examine it to the LangChain equal. # Conditions: pip set up openai python-dotenv # Tips on how to run: python raw_api_agent.py
import os import json from dotenv import load_dotenv from openai import OpenAI
load_dotenv() shopper = OpenAI(api_key=os.getenv(“OPENAI_API_KEY”))
# ── TOOL DEFINITIONS ────────────────────────────────────────────────────────── # The mannequin reads these descriptions to resolve when and the way to name every device. # Clear, particular descriptions are extra necessary right here than in any framework — # there is no such thing as a wrapper to fill in gaps. TOOLS = [ { “type”: “function”, “function”: { “name”: “calculate”, “description”: ( “Evaluate a mathematical expression. Use for arithmetic, “ “percentages, or numerical computation. “ “Input: a Python math expression as a string.” ), “parameters”: { “type”: “object”, “properties”: { “expression”: { “type”: “string”, “description”: “A Python math expression, e.g. ‘1500 * 0.08′” } }, “required”: [“expression”] } } }, { “kind”: “operate”, “operate”: { “identify”: “get_word_count”, “description”: “Depend the variety of phrases in a given string of textual content.”, “parameters”: { “kind”: “object”, “properties”: { “textual content”: {“kind”: “string”, “description”: “The textual content to depend.”} }, “required”: [“text”] } } } ]
# ── TOOL IMPLEMENTATIONS ────────────────────────────────────────────────────── def calculate(expression: str) -> str: strive: outcome = eval(expression, {“__builtins__”: {}}, {}) return str(outcome) besides Exception as e: return f“Error: {e}”
def get_word_count(textual content: str) -> str: return str(len(textual content.break up()))
# Maps device identify → Python operate for dynamic dispatch within the loop beneath TOOL_DISPATCH = {“calculate”: calculate, “get_word_count”: get_word_count}
# ── AGENT LOOP ──────────────────────────────────────────────────────────────── def run_agent(user_message: str) -> str: “”“ A whole ReAct-style agent loop utilizing uncooked OpenAI device calls. The mannequin decides whether or not to name a device or return a closing reply. The loop continues till the mannequin stops requesting device calls. Each step is seen — no framework wrapping, no hidden logic. ““” messages = [ {“role”: “system”, “content”: “You are a helpful assistant.”}, {“role”: “user”, “content”: user_message}, ]
whereas True: response = shopper.chat.completions.create( mannequin=“gpt-4o”, messages=messages, instruments=TOOLS, tool_choice=“auto”, # Mannequin decides: name a device or reply immediately temperature=0, )
message = response.selections[0].message messages.append(message) # At all times add the assistant message to historical past
# No device calls = the mannequin has its closing reply if not message.tool_calls: return message.content material
# Execute every device name the mannequin requested for tool_call in message.tool_calls: identify = tool_call.operate.identify args = json.hundreds(tool_call.operate.arguments) fn = TOOL_DISPATCH.get(identify) outcome = fn(**args) if fn else f“Unknown device: {identify}”
# Device outcome goes again into the message historical past. # The mannequin reads this on the following iteration to resolve what to do subsequent. messages.append({ “position”: “device”, “tool_call_id”: tool_call.id, “content material”: outcome, }) # Loop — the mannequin now processes the device outcomes
if __name__ == “__main__”: queries = [ “What is 18% of 3500?”, “How many words are in: The quick brown fox jumps over the lazy dog?”, “Split 240 items into groups of 16. How many groups?”, ] for q in queries: print(f“Q: {q}nA: {run_agent(q)}n”) |
What this does: The agent loop is totally clear. There is no such thing as a framework between you and the mannequin’s response. The whereas True loop runs till message.tool_calls is empty, which occurs when the mannequin decides it has sufficient info to reply immediately. Each message — system, person, assistant, and gear outcome — is in a plain Python record you possibly can examine, log, or modify at any level. That transparency is the uncooked path’s core benefit: when one thing breaks, you recognize precisely the place to look.
Head-to-Head Comparability
The identical job was evaluated throughout three dimensions. All measurements mirror present benchmarks from impartial evaluation cited all through this text.
Framework Overhead and Efficiency
| Metric | Uncooked API | LlamaIndex | LangChain (LCEL) | LangGraph |
|---|---|---|---|---|
| Framework overhead | ~0ms | ~6ms | ~10ms | ~14ms |
| Token overhead (per question) | 0 | ~1.6K | ~2.4K | ~2.0K |
| Per device name latency | 2–5ms | N/A | 10–30ms | 10–30ms |
| Stack hint depth on error | 2–5 frames | 5–10 frames | 15–40 frames | 15–40 frames |
| Debug transparency | Excessive | Medium | Low | Low |
Code Quantity: Similar RAG Activity, Three Methods
That is probably the most concrete technique to really feel the trade-off. All three implementations beneath reply the identical query from the identical context doc:
| Implementation | Traces of code | Framework set up measurement | Debugging readability |
|---|---|---|---|
| Uncooked OpenAI SDK | ~20 strains | openai solely | Full visibility |
| LlamaIndex | ~15 strains | llama-index + plugins | Medium |
| LangChain LCEL | ~18 strains | langchain + langchain-openai | Low–medium |
For a primary one-document Q&A, the distinction is marginal. The place LlamaIndex’s code benefit compounds is whenever you add chunking methods, a number of paperwork, re-ranking, metadata filtering, and hybrid search — every of which requires extra meeting in LangChain than in LlamaIndex.
When Every One Breaks
Understanding when every method fails is as helpful as figuring out when it succeeds.
| Failure mode | Uncooked API | LlamaIndex | LangChain |
|---|---|---|---|
| Retrieval accuracy degrades | You constructed it, you repair it | Tune chunking/index technique | Tune every pipeline part individually |
| Agent loops indefinitely | Add max_iterations manually | Workflow timeout | max_iterations parameter |
| Immediate adjustments break output | Quick, apparent | Quick, apparent | Might propagate via chain silently |
| Mannequin API adjustments | Replace SDK | Replace llama-index bundle | Replace langchain-openai + retest |
| Debugging a manufacturing error | Direct, small stack | Average | Deep stack traces, exhausting to isolate |
| Scaling to excessive throughput | Optimum | Good | Framework overhead compounds |
Full Working Instance
The identical doc Q&A job applied 3 ways. Similar enter doc, similar query, totally different path via the stack. Learn these facet by facet and the trade-offs turn into concrete.
Conditions:
|
pip set up openai langchain langchain–openai llama–index llama–index–llms–openai llama–index–embeddings–openai python–dotenv |
Tips on how to run: Save as three_ways.py and run python three_ways.py
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 |
# three_ways.py # The identical doc Q&A job applied 3 ways: # Uncooked OpenAI SDK, LlamaIndex, and LangChain LCEL. # Similar enter. Similar output. Totally different path via the stack. # Conditions: pip set up openai langchain langchain-openai llama-index # llama-index-llms-openai llama-index-embeddings-openai python-dotenv # Tips on how to run: python three_ways.py
import os import time from dotenv import load_dotenv
load_dotenv()
QUESTION = “What’s retrieval-augmented technology and why does it matter?”
CONTEXT_DOC = “”“ Retrieval-Augmented Technology (RAG) is a method that improves LLM responses by fetching related context from an exterior data base earlier than producing a solution. As a substitute of relying solely on coaching knowledge, RAG retrieves probably the most related doc chunks and contains them within the immediate. This reduces hallucinations, retains solutions grounded in your precise knowledge, and permits the mannequin to reply questions on info it was by no means skilled on. ““”
# ───────────────────────────────────────────────────────────────────────────── # APPROACH 1: RAW OPENAI SDK # When to make use of: easy, one-off calls the place full visibility issues most # ───────────────────────────────────────────────────────────────────────────── def raw_api_answer(query: str, context: str) -> str: “”“Reply a query utilizing context, by way of uncooked OpenAI SDK — no framework.”“” from openai import OpenAI shopper = OpenAI(api_key=os.getenv(“OPENAI_API_KEY”))
# The whole lot is specific: the system immediate, the context injection, # the message construction. Nothing is hidden in a framework abstraction. response = shopper.chat.completions.create( mannequin=“gpt-4o”, temperature=0, messages=[ { “role”: “system”, “content”: ( “Answer questions using only the provided context. “ “If the answer is not in the context, say so clearly.” ) }, { “role”: “user”, “content”: f“Context:n{context}nnQuestion: {question}” } ] ) return response.selections[0].message.content material
# ───────────────────────────────────────────────────────────────────────────── # APPROACH 2: LLAMAINDEX # When to make use of: document-heavy retrieval the place you need optimized RAG out of the field # ───────────────────────────────────────────────────────────────────────────── def llamaindex_answer(query: str, context: str) -> str: “”“Reply a query utilizing LlamaIndex — purpose-built retrieval pipeline.”“” from llama_index.core import VectorStoreIndex, Doc, Settings from llama_index.llms.openai import OpenAI as LlamaOpenAI from llama_index.embeddings.openai import OpenAIEmbedding
# Configure as soon as — all pipeline elements choose it up Settings.llm = LlamaOpenAI( mannequin=“gpt-4o”, temperature=0, api_key=os.getenv(“OPENAI_API_KEY”) ) Settings.embed_model = OpenAIEmbedding( mannequin=“text-embedding-3-small”, api_key=os.getenv(“OPENAI_API_KEY”) )
# from_documents() = chunk + embed + index in a single name # For a number of paperwork, cross a listing: from_documents([doc1, doc2, doc3]) index = VectorStoreIndex.from_documents([Document(text=context)])
# as_query_engine() = retriever + generator, wired collectively routinely query_engine = index.as_query_engine(similarity_top_k=1) return str(query_engine.question(query))
# ───────────────────────────────────────────────────────────────────────────── # APPROACH 3: LANGCHAIN LCEL # When to make use of: workflows that can develop to incorporate brokers, reminiscence, or routing # ───────────────────────────────────────────────────────────────────────────── def langchain_answer(query: str, context: str) -> str: “”“Reply a query utilizing a LangChain LCEL chain.”“” from langchain_core.prompts import ChatPromptTemplate from langchain_core.output_parsers import StrOutputParser from langchain_openai import ChatOpenAI
llm = ChatOpenAI( mannequin=“gpt-4o”, temperature=0, api_key=os.getenv(“OPENAI_API_KEY”) )
immediate = ChatPromptTemplate.from_messages([ (“system”, “Answer using only the provided context. “ “If the answer is not in the context, say so.nnContext:n{context}”), (“human”, “{question}”) ])
# The identical chain helps .stream(), .batch(), .ainvoke() — no code adjustments wanted chain = immediate | llm | StrOutputParser() return chain.invoke({“context”: context, “query”: query})
# ───────────────────────────────────────────────────────────────────────────── # RUN ALL THREE AND COMPARE # ───────────────────────────────────────────────────────────────────────────── if __name__ == “__main__”: approaches = [ (“Raw OpenAI SDK”, raw_api_answer), (“LlamaIndex”, llamaindex_answer), (“LangChain LCEL”, langchain_answer), ]
for identify, fn in approaches: print(f“n{‘=’*60}”) print(f“Method: {identify}”) print(f“{‘=’*60}”) begin = time.perf_counter() reply = fn(QUESTION, CONTEXT_DOC) elapsed = time.perf_counter() – begin print(f“Reply: {reply}”) print(f“Time (excluding LLM): seen in wall clock”) |
What this does: All three features obtain the identical QUESTION and CONTEXT_DOC and return a string reply. The uncooked API model manually constructs the message record and extracts the response. The LlamaIndex model makes use of from_documents() and as_query_engine() to deal with the pipeline. The LangChain model assembles a immediate, mannequin, and parser with the | operator. At this scale — one doc, one query — the variations are minimal. Feed this operate 500 paperwork and a fancy question, and the hole between LlamaIndex’s purpose-built retrieval and the opposite two approaches opens up considerably.
Wrapping Up
The framework determination just isn’t about which possibility has probably the most GitHub stars or probably the most options. It’s about matching the abstraction stage of your device to the precise complexity of your drawback.
For easy, one-shot duties, uncooked API calls are sooner to put in writing, sooner to run, and simpler to debug than any framework. For doc retrieval at any significant scale, LlamaIndex earns its dependency via higher chunking, sooner indexing, and fewer code. For stateful brokers with reminiscence, instruments, and multi-step reasoning, LangGraph’s persistence and graph-based management stream are genuinely exhausting to copy cleanly with a hand-rolled loop.
The sample that most manufacturing groups converge on by mid-2026 just isn’t a single framework however a layered stack: uncooked SDK for the easy calls, LlamaIndex for the retrieval layer, LangGraph for the agent loop, and LangSmith for tracing throughout all the pieces. None of these selections locks you out of the others. They compose.
The sensible rule is that this: begin with the minimal possibility that handles your present necessities, and add a framework whenever you hit an issue the framework was constructed to unravel — not earlier than. A retrieval drawback you encounter is a motive so as to add LlamaIndex. A state administration drawback you encounter is a motive so as to add LangGraph. Including both earlier than you are feeling the ache they handle means including upkeep overhead for a future drawback that will not arrive within the form you anticipated.
