Logging: The Standard Way

Logging is one of those things that sits squarely in the “necessary evil” category. You rarely enjoy writing it, and it often feels like busywork when everything is running smoothly. But the moment something breaks—or behaves just a little too quietly—you’ll be very glad those log statements are there.

Python doesn’t suffer from a lack of logging options. The standard library already provides a solid, flexible logging framework, and beyond that there’s no shortage of third-party tools. Some are built for very specific use cases, others focus on adding structure, context, or consistency to what would otherwise be a loose collection of print statements. The challenge is rarely how to log, but deciding how much structure you actually need.

Letting ChatGPT decide how to visualise the content of the Post.

Logging: The Original

In this post we focus on the standard logging library that Python ships with. It already covers a surprisingly wide range of use cases. It supports different log levels, flexible formatting, multiple output targets, and fine-grained control over what gets recorded and where—often enough for everything from small scripts to fairly large applications.

The Logger

The first thing you need when using logging is a basic setup: a logger, a log level, and a destination for the output. To set up the logger is straight-forward

This initialises the logger using the name of the current module — in this case simply main. Doing so makes it easy to trace where log messages originate, especially once your codebase grows beyond a single file.

Loggers operate with different logging levels that indicate the severity of a message. The standard levels, in increasing order of importance and how they are accessed are:

These are mostly self-explanatory.

• Debug is internal chatter meant purely for developers — the “what on earth is going on here?” level.

• Info represents normal program flow: things are happening exactly as intended.

• Warning means something unexpected occurred, but it’s not serious enough to stop execution.

• Error is exactly what it sounds like: something went wrong and part of the operation failed.

• Critical is the “oh no” level — the application is in serious trouble and may be moments away from crashing and burning.

The distinction between these levels matters less when you’re printing messages locally, but becomes essential once logs are filtered, aggregated, or monitored automatically.

Basic Configuration

Now you need to tell your code what to log. The simplest way to do that is by defining a basic logging configuration.

Python’s logging module provides a basicConfig function that sets up sensible defaults in one place: the minimum log level, the output destination, and the general message format. For small scripts or early-stage projects, this is often more than enough.

In this case, any message we send to logger with a level of at least INFO will be written to the terminal using the format we defined. Each log entry will include the log level name, the name of the logger, and the message itself.

Messages below that threshold — such as DEBUG — are simply ignored. This makes it easy to dial logging verbosity up during development and back down again later, without touching the actual logging calls.

For example, if we try the following…

As expected, the call to logger.debug does not produce any output in the terminal. Its level is lower than INFO, which we configured as the minimum severity to log.

Formatting, Handling and Setup

Above, we used the basic configuration to define a log format and a minimum logging level. While this is perfectly fine for small scripts, it’s a bit coarse-grained. It applies globally and doesn’t give us much control over individual loggers.

If we want to configure things in a more deliberate and professional way — especially for a specific logger — we do this explicitly instead.

In this setup, we first set the logging level directly on our logger and then initialise a formatter. The formatter defines how each log message is rendered. In this example, we use four of the most common placeholders: a timestamp, the log level, the name of the logger, and the message itself.

There are several other useful placeholders available — such as the filename or the line number where the log call occurred. These are especially helpful during development, when you’re trying to track down where a message originated.

On its own, however, a formatter doesn’t do very much. It needs to be paired with a handler, which decides where the formatted log messages are sent. The three most common handler types are StreamHandler, which writes logs to the console, FileHandler, which writes them to a file, and RotatingFileHandler, which also writes to a file but prevents it from growing without bound.

Since their syntax is largely identical, we’ll focus on the most immediately visible option: the StreamHandler, which sends log output straight to the terminal.

At this point, aside from the initial setup, there are three important steps. First, we set the logging level on the handler, which defines which messages it will process — only logs at that level and above are passed through. Next, we attach the formatter so the handler knows how each message should look. Finally, we add the handler to our existing logger.

If we now repeat the same logging calls we used with the basic configuration earlier, we can see the effect immediately. The log output now includes timestamps, exactly as specified in our formatter, along with the log level, logger name, and message. The logging calls themselves didn’t change at all — only the configuration did.

Except Cases

The try / except construct is one of the most common patterns in Python for handling errors — at least the ones we can reasonably anticipate. Instead of letting an exception bubble up and potentially crash the program, we can intercept it, react accordingly, and continue in a controlled way.

In those situations, it’s often useful to add a bit more context than the exception alone provides. Logging is a natural fit here: it lets us record what went wrong, where it happened, and under which circumstances, without changing the program’s external behaviour. This extra information can be invaluable later.

In that case, the complete stack trace is included as well — that is, a record of where the program was in the call hierarchy when the exception occurred. Instead of just seeing the error message, you also get the chain of function calls that led up to it.

Custom Information

Up to this point, our log messages mostly just signal that something happened. Only in the case of exceptions did we attach any real additional context.

In practice, however, it’s often useful to include more information directly in the log message itself. By adding relevant context — such as parameter values, identifiers, or intermediate state — we can make log entries far more informative, even when no exception is raised.

If we try this, we immediately see that nothing special happens. The additional information isn’t recorded anywhere. That’s not because the logger ignores it — it’s because we never told the formatter to make use of it.

A formatter only renders the fields it explicitly knows about. By default, that’s things like the timestamp, level name, logger name, and message. Any extra context we attach to a log record is simply discarded unless the format string references it.

To include this additional information, we need to extend the formatter and explicitly reference those fields using their placeholder names. Only then will the extra context show up in the log output, just like the built-in attributes do. We would have to change our formatter as follows

Massive warning here: this formatter can now only format messages that include both a user_age and a reason. Any log record missing one of these fields will fail to format and raise an error instead of being logged.

We’ll see shortly how to get around this restriction by defining a custom formatter.

Other Useful Things and a Simple Prototype

There are a few other useful commands worth knowing. For example, logging.disable(level) lets you globally suppress all log messages at or below a given level, which can be handy for silencing noisy libraries during testing. The logging.shutdown() command cleanly flushes and closes all handlers — something that’s mainly relevant for long-running applications or when logging to files.

One of the most important concepts in Python’s logging system, however, is propagation. Loggers are organised hierarchically, mirroring your module structure. If you have a nested project layout, you can define a logger in each file, and they automatically form a parent–child relationship.

For example, suppose that in app/api/auth.py you define a logger using

The name of this logger will be app.api.auth. Unless configured otherwise, any log message emitted there will propagate upward to parent loggers such as app.api and app (not every one of these has to exist, eventually reaching the root logger. Each parent can handle the message using its own handlers and formatters.

This is what allows you to define logging behaviour once at a high level — for example in your application’s entry point — while still using module-specific loggers everywhere else. Individual modules don’t need to know where logs go; they just emit them. Propagation takes care of the rest.

This also means that child loggers can selectively override behaviour inherited from their parents. A child logger can choose to log at a more or less verbose level, attach its own handlers, or stop messages from propagating upward altogether.

By setting logger.propagate = False, a logger becomes self-contained: messages are handled locally and never reach parent loggers. This is useful for particularly noisy subsystems, sensitive components, or cases where duplicate log entries would otherwise occur.

Let’s now put everything we’ve discussed together into a single, explicit logger initialisation.

This sets up a logger named main and configures it to accept all log messages at DEBUG level and above. From there, the handling is split by severity: warnings and more serious messages are sent to the console, while all log messages — including debug and informational output — are written to a log file.

The Original + JSON and more structure

We now want to add a bit more structure to our logs. Instead of emitting plain text, we want log entries that can be parsed, filtered, and processed automatically. A common and very practical way to do this is to log in JSON format.

To achieve that, we bring in Python’s built-in json library and define a custom formatter. Rather than assembling a string, this formatter constructs a dictionary containing the fields we care about — timestamp, log level, logger name, message, and any additional context — and then serialises it into JSON.

We define our JSON output by working directly with the record object that the formatter receives and adding a timestamp explicitly. From there, we check whether exception information is present and, if so, include it as part of the structured output as well.

If we apply this to our prototype example from above, nothing about the logger or handlers themselves needs to change. The only thing we replace is the formatter definition used by the handlers via

The result is that our logs now arrive as a JSON dump. More importantly, this custom formatter gives us a natural hook for including any extra data we might want to attach to a log entry — exactly the mechanism we discussed earlier.

Since the formatter already has access to the full log record, it can inspect which attributes are present and selectively include them in the output. This allows us to enrich log entries with contextual information when it exists, without breaking formatting when it doesn’t.

Concretely, we can extend our JSONFormatter by adding a small piece of logic just before returning the final JSON object: we iterate over any additional attributes on the record that aren’t part of the standard logging fields and inject them into the structured output.

We’re simply making sure that we don’t overwrite or duplicate any of the predefined attributes that already exist on the log record. By filtering those out, we avoid collisions and ensure that only genuinely extra context is added to the structured output.

One important consequence of this setup is how it interacts with propagation. If only the top-level logger defines handlers — and all child loggers simply propagate their messages upward — then those child loggers will automatically emit JSON-formatted logs as well. They don’t need to know anything about JSON, custom formatters, or handlers at all; they just log messages, and the root logger’s handlers take care of formatting and output.

Logging Some Thoughts

Python’s built-in logging is the “boring but unbreakable” option. It ships everywhere, it integrates with basically every major library (most of them emit via logging), and it gives you the full toolbox: hierarchical loggers, propagation, handlers, filters, formatters, rotating files, and more.

The trade-off is ergonomics: doing anything beyond “print but nicer” tends to feel ceremony-heavy, and have to spend quite a bit of effort on handlers and formatters.