How Does AI Think? Part 5 — AI in the Wild: Applications, Building Blocks & the Developer's Toolkit

📖 This is Part 5 of the "How Does AI Think?" series. Part 1 · Part 2 · Part 3 · ← Part 4: Under the Hood

Introduction — From Theory to Practice

Parts 1–4 traced the ideas, architectures, and engineering behind AI. But understanding how transformers work is different from knowing how to build something useful with them.

This final part bridges that gap. We'll explore where AI is making a real impact today, learn the building blocks every developer needs (RAG, vector databases, embeddings, agents), tour the open-source ecosystem, and build an intuition for choosing the right model and deployment strategy for your project.

Less theory, more practice. Let's go.

1. Real-World AI Applications (2024–2026)

Beyond the hype, AI is transforming specific domains in measurable ways. Here are the areas where the impact is already undeniable:

1.1 The Impact Numbers

2. The RAG Pattern

LLMs have a fundamental limit: they only know what was in their training data (up to a cutoff date). They can't access your company's documents, today's news, or your personal notes — unless you give them that information at inference time.

RAG (Retrieval-Augmented Generation) solves this by combining a retriever (finds relevant documents) with a generator (the LLM that uses those documents to answer):

2.1 Why RAG Matters

• No hallucination — the model cites actual documents, not fabricated facts
• Up-to-date — add new documents anytime, no retraining needed
• Domain-specific — works with your internal data (legal, medical, financial)
• Auditable — you can see exactly which documents the answer came from
• Cost-effective — cheaper than fine-tuning and much cheaper than pre-training

2.2 RAG Best Practices

3. Vector Similarity Search

At the heart of RAG is vector search: finding the most similar vectors in a database. The query and all documents are converted to vectors by an embedding model, then similarity is measured with:

$\text{cosine\_sim}(\mathbf{a}, \mathbf{b}) = \frac{\mathbf{a} \cdot \mathbf{b}}{\|\mathbf{a}\| \|\mathbf{b}\|}$

Two texts about similar topics will have cosine similarity close to 1.0, even if they use completely different words.

3.1 Popular Vector Databases

3.2 Embedding Models

The quality of your vector search depends on the embedding model. Modern embedding models are small transformers optimised for semantic similarity:

• OpenAI text-embedding-3-large — 3072 dimensions, excellent general performance
• Cohere embed-v3 — multilingual, supports semantic search and classification
• BGE / GTE (open source) — competitive with commercial models, run locally
• Nomic Embed — open source, 8K context, strong performance on MTEB benchmark
• Jina Embeddings v3 — multilingual, 8K context, multiple task adapters

4. Building AI Applications

The ecosystem of tools for building AI-powered applications has exploded. Here's the landscape:

4.1 LLM Frameworks

4.2 The AI Application Stack

5. The Open-Source Ecosystem

You don't need an API key to use powerful AI. Open-source models have reached remarkable quality, and tools like Ollama make running them locally as easy as ollama run llama3.

5.1 Key Open-Source Models (2025-2026)

5.2 Running Models Locally

5.3 When to Use Local vs Cloud

6. The Agent Architecture

An AI agent is an LLM that can take actions: call APIs, query databases, write files, browse the web, run code. The LLM serves as the "brain" that decides what to do next based on the current state and goal.

6.1 Agent Patterns

6.2 MCP — Model Context Protocol

MCP (Anthropic, 2024) is an open protocol that standardises how AI models connect to external tools and data sources. Instead of each app implementing its own tool integrations, MCP defines a universal interface:

• Resources — data the model can read (files, database records, API responses)
• Tools — actions the model can take (run query, write file, send email)
• Prompts — reusable prompt templates for common tasks

MCP servers are now supported in VS Code (Copilot), Cursor, Claude Desktop, and dozens of other tools. A single MCP server for Postgres, for example, lets any AI assistant query your database.

7. Production Considerations

7.1 The Prototype → Production Gap

Getting a demo working is easy. Making it reliable, fast, safe, and cost-effective at scale is where the real engineering happens:

7.2 Model Routing

Not every query needs GPT-4. Smart routing sends each request to the right model:

8. The New Developer Experience

AI hasn't replaced developers — it's changed what it means to develop software. Here's what matters now:

8.1 Skills That Matter More

• System design — understanding how components fit together is more important than ever when AI can generate individual components
• Problem decomposition — breaking complex problems into AI-solvable chunks
• Evaluation & debugging — knowing when AI output is wrong, and why
• Prompt engineering — communicating clearly with AI tools to get reliable results
• Domain expertise — AI amplifies what you already know. Deep knowledge in healthcare, finance, or security becomes more valuable, not less

8.2 The New Workflow

The role is shifting from author to architect + editor. You spend more time understanding what to build and verifying it works correctly, less time on the mechanical act of writing each line.

8.3 Getting Started: Your First AI Project

If you've read all five parts and want to build something, here's a practical starting path:

1. Week 1: Install Ollama, run ollama run llama3, experiment with different prompts via the CLI
2. Week 2: Build a simple chatbot with the Ollama API (Python/FastAPI or Node/Express + a simple frontend)
3. Week 3: Add RAG — load a few documents into ChromaDB, query them before generating answers
4. Week 4: Add tool calling — let your chatbot search the web or query a database using function calling
5. Week 5: Deploy it — host with Docker, add authentication, try streaming responses to the frontend

Each step builds on the last, and at the end you'll have a genuinely useful AI application with RAG, tools, and a real UI.

9. The Complete Arc — Five Parts, One Story

This series has covered the full sweep of AI — from medieval logic machines to autonomous coding agents:

1. Part 1 — Logic to Statistics: The intellectual foundations, from Llull and Leibniz through Turing, LISP, AI winters, and the statistical revolution.
2. Part 2 — Perception to Prediction: Neural nets, deep learning, CNNs, RNNs, attention, transformers, BERT, GPT, and the foundation model paradigm.
3. Part 3 — Action to Understanding: Reinforcement learning, agents, chain-of-thought, open source vs. closed, alignment, and the AGI question.
4. Part 4 — Under the Hood: Tokenizers, embeddings, training at scale, LoRA, RLHF, inference optimization, prompt engineering, and multimodal models.
5. Part 5 — AI in the Wild: Real applications, RAG, vector databases, the developer toolkit, local models, agents, deployment, and building your first AI app.

Understanding AI isn't just about knowing what a transformer is. It's about understanding the full stack — from the math behind attention to the engineering of inference servers to the design of useful applications.

The field will keep moving. New architectures will emerge. New capabilities will surprise us. But the patterns we've covered — tokenize, embed, attend, generate, retrieve, act — will remain the foundation for a long time.

Now go build something.

Further Reading & Resources

• Lewis et al. — "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks" (RAG paper, 2020)
• Anthropic — "Contextual Retrieval" blog post (2024)
• LangChain docs — docs.langchain.com
• LlamaIndex docs — docs.llamaindex.ai
• Ollama — ollama.ai
• HuggingFace Open LLM Leaderboard — model benchmarks and comparisons
• Anthropic MCP — modelcontextprotocol.io
• Simon Willison's blog — essential reading for practical AI development
• The Batch (Andrew Ng) — weekly AI newsletter with accessible explanations

← Part 1: Logic Machines to Linear Regression · Part 2: Neural Nets to Transformers · Part 3: Agents, Reasoning & AGI · Part 4: Under the Hood