Summary
AI frameworks package repeatable LLM application work: prompts, tool calls, retrieval, state, routing, retries, and observability. They exist because the first demo is often simple, but the second and third feature quickly create the same engineering problems again. A practical default for beginners is: start with the smallest thing that solves the job, then adopt a framework when the project needs structure that is no longer easy to maintain manually.Why It Matters
A direct API call is enough when the task is a single prompt, a small script, or a prototype where you can see every step. A framework starts to make sense when the project needs:- memory across turns or sessions
- retrieval from documents, databases, or private knowledge bases
- tool use such as search, database queries, calendar actions, or code
- stateful control flow with branches, loops, retries, or checkpoints
- collaboration patterns where several roles or agents divide the work
- observability so failures can be traced and improved
Which part of my current project is becoming repetitive, fragile, or hard to reason about?
Mental Model
Think of framework choice like choosing between writing a script, using an app starter kit, or using a workflow engine:Comparison Table
Scenario Walkthroughs
Scenario 1: Simple chatbot
- No framework: store the message list, call the model, and return the answer. This is usually enough for a prototype.
- LangChain: becomes useful when the chatbot needs tool calls, structured outputs, streaming, tracing, or reusable prompt/model wiring.
- Verdict: start without a framework. Move to LangChain when the chatbot becomes an application rather than a single chat loop.
Scenario 2: Document QA over internal PDFs
- No framework: parse files, chunk text, create embeddings, store vectors, retrieve relevant chunks, assemble context, and handle citations yourself.
- LlamaIndex: provides a focused path for loading documents, building an index, and querying over private data.
- LangChain: can also build retrieval flows, especially when retrieval is one part of a broader tool-using agent.
- Verdict: choose LlamaIndex when document retrieval is the core product; choose LangChain when retrieval is one tool inside a larger agent app.
Scenario 3: Multi-step research workflow
- LangGraph: model the process as explicit nodes such as plan, search, synthesize, review, and revise. Use checkpoints when the workflow may need to pause, resume, or recover.
- CrewAI: model the process as roles such as planner, researcher, writer, and editor, each with a task.
- Verdict: choose CrewAI for a fast role-based prototype; choose LangGraph when execution control and state are the main risks.
Useful Defaults
- Use no framework for the first working version unless retrieval, tools, or state already dominate the problem.
- Use LlamaIndex when the question is “how do I make this data searchable and answerable by an LLM?”
- Use LangChain when the question is “how do I build an LLM application that connects models, prompts, tools, and integrations?”
- Use LangGraph when the question is “how do I control and recover a multi-step process?”
- Use CrewAI when the question is “how do I divide this work across named agent roles?”
- If a task is deterministic, use a normal function or workflow. Add an agent only when it provides useful coordination, tool use, or decision-making.
Citations
- LangChain documentation
- LlamaIndex documentation
- LangGraph documentation
- CrewAI documentation
- Current official readings are listed in
external_readings.
Reading Extensions
- Agent Frameworks: intermediate-level comparison of conversation-first, graph-first, skill-first, and engineering-first frameworks.
- Agent Runtime Building Blocks: the runtime primitives that frameworks wrap.
- Ecosystem Overview: the full ecosystem lane.
Update Log
- 2026-06-02: Revised into a beginner decision guide with current official links, clearer mental models, and less API-specific guidance.
- 2026-05-04: Initial draft for beginner-oriented framework comparison.