LangChain: Building on Top of Large Language Models

The moment you start experimenting with modern LLMs, one realization appears quickly:

• Calling an API is easy.

• Building a reliable application around it is not. Outputs come back rather randomly if not well controlled.

Once you want structured prompts, streaming outputs, chains of reasoning, tool integrations, or agent workflows, complexity increases quickly...or how often did you ask your favourite chat AI for something and it came up with an answer in a format you did not expect or even ask for.

This is exactly the problem LangChain was designed to solve. LangChain provides a framework for composing LLM-based systems, allowing developers to combine prompts, models, tools, memory, retrieval systems, and output parsing into structured pipelines.

Instead of writing one long script full of API calls, you build small components that compose into workflows.

In this post, we will explore LangChain step by step, starting with the simplest possible example and gradually building toward a streaming terminal application that automatically writes Markdown documentation.

We are talking about LLM's but let's see what generative AI thinks about the post...

The Core Idea of LangChain

At its core, LangChain introduces a simple but powerful abstraction:

Prompt → Model → Output Parser

Each component has a clear responsibility.

• Prompt Templates structure what you ask the model.

• Models generate responses.

• Output Parsers convert model output into structured data.

These pieces can then be composed into chains.

LangChain's operator syntax makes this look surprisingly elegant and reminiscent of piping:

If you have a seen a data pipeline or functional programming, this will feel very natural.

The Baby Example

Let’s start with a minimal script.

This example simply asks a model to explain a Python package.

What happens here?

1. The prompt template defines a structured prompt. In this case with a variable parameter 'topic'

2. The model wrapper calls the LLM, we choose the specific Google Gemini model here since it has a free tier for you to test anything simple.

3. The chain connects both pieces, i.e. tells us that the prompt is fed into the model

4. invoke() runs the pipeline, where we specify the parameter 'topic' via a little dictionary.

Even this tiny script already demonstrates the main LangChain philosophy:

LLM programs should be built from composable parts.

Adding an Output Parser

LLMs return text, but most real applications need structured outputs. LangChain solves this using output parsers.

The StrOutputParser() simply converts the response into plain text. But LangChain also supports parsers for:

• JSON

• Pydantic models

• custom structured outputs

This becomes extremely useful once LLMs are used in automation pipelines.

Introducing Runnable Composition

LangChain’s modern architecture is built around Runnables.

A runnable is simply an object that can:

invoke()
stream()
batch()

Because each component follows this interface, they can be composed arbitrarily.

This is why the pipe syntax works.

prompt | model | parser

Each stage passes its result to the next.

The result is a much cleaner and more predictable data flow than an ad-hoc script. This especially makes it much easier to use for a deterministic application.

Streaming Output

One of the most powerful features of modern LLM APIs is token streaming.

Instead of waiting for the entire response, tokens arrive incrementally.

LangChain exposes this through:

chain.stream(...)

Let’s build a simple streaming example.

Now the text appears gradually, which is much nicer for interactive applications.

Building a Terminal Documentation Generator

Now we combine several ideas:

• structured prompts

• streaming

• output parsing

• terminal rendering

• saving results as Markdown

For terminal rendering we use the excellent Rich library.

The script below creates a live documentation generator that explains any Python package and saves the result as Markdown.

What This Example Demonstrates

This relatively small script already shows several fundamental ideas.

1. Prompt engineering as code

The prompt template defines the structure of the output.

## Simple Explanation
## Methods
## Example

The model becomes part of a deterministic content pipeline. In a later post we will look into how you can use Pydantic to make this even more structured.

2. Runnable pipelines

The core chain is only one line:

chain = prompt | model | parser

This declarative style is a huge improvement over manually passing strings around.

3. Streaming UX

Using chain.stream() together with rich.live.Live creates a real-time terminal UI.

Instead of waiting several seconds, users see text appear immediately.

Of course in this case this is more a gimmick than actually useful, but good enough for a demonstration of the process.

4. LLM-powered documentation automation

The script essentially creates automatic Markdown documentation for Python packages.

Given a package name like:

pandas
duckdb
langchain
polars
networkx

the script produces a easy to parse Markdown document. It is not too deep but good enough to get a quick overview whenever you are interested in learning about a specific package.

Why LangChain Became So Popular

LangChain exploded in popularity because it solved several problems at once:

1. Standardization Instead of ad-hoc API calls, developers now share common abstractions.

2. Composition Complex LLM workflows can be built from small components.

3. Integration ecosystem LangChain integrates with:

   1. vector databases

   2. APIs

   3. tools

   4. search engines

   5. file systems

4. Agent frameworks LangChain also powers more advanced systems like:

   1. autonomous agents

   2. tool-using LLMs

   3. retrieval-augmented generation

The Bigger Picture

LangChain represents a broader shift in how we think about programming with AI.

Traditional programs look like this:

input → algorithm → output

LLM applications look more like this:

input → prompt → model → tools → parsing → post-processing

LangChain provides the architecture to manage this in a very consistent way.

When “LLM Script” Becomes Interesting

Working with an LLM usually starts with a very simple idea: send a prompt, get text back, done.

For a quick experiment that works perfectly fine. But the moment you try to build something slightly more useful, the script starts growing. You add structured prompts, streaming responses, output parsing, maybe a tool call or two, perhaps a retrieval step. Before long the code is no longer a single API call but a small pipeline of moving parts.

This is exactly the situation LangChain was designed for. Instead of letting LLM applications evolve into collections of loosely connected prompt strings and API calls, LangChain provides a structured way to compose these pieces. Prompts, models, parsers, tools, and memory become components that can be wired together into a predictable workflow.

Interestingly, once you start building systems this way, the model itself often becomes the least complicated part of the program. The real engineering effort lies in orchestrating everything around it: structuring prompts, handling outputs, integrating data sources, and designing workflows that remain understandable as they grow.

In that sense LangChain is less about “using an LLM” and more about building around one. And as LLM applications continue to expand beyond simple chat prompts, frameworks that help organize this complexity will likely become just as important as the models themselves.