Mixle: When One Row Contains Several Kinds of Data — Part I

A customer record may contain an age, a region, a purchase count, a collection of interests, and a sequence of actions. All in a single row. These are not just different data in terms of their scale, but really different types of data.

The usual machine-learning response is to convert everything into columns, turn the categories into numbers, pad the sequences, and eventually present an algorithm with one large rectangle of floating-point values.

This is often practical. It can also hide what each field means.

A count is not merely a small continuous number. A category does not become ordered because its one-hot columns happen to sit beside one another. A set of interests is not the same object as a click sequence, even when both arrive as Python lists.

Mixle takes a different approach. It treats each field as a probabilistic object, lets those objects be combined into records and latent-variable models, and fits the resulting structure through a shared interface.

The question for this first part is:

Can we model mixed data without pretending that every field belongs to the same geometry?

We will begin with one continuous column, because that is the smallest place where the idea becomes visible. We will then build a record containing continuous, categorical, count, set-like, and sequential values and ask Mixle to find latent customer segments.

This is an exploratory introduction to Mixle 0.7.0. It is not a survey of the library’s roughly ninety distribution families, distributed engines, neural components, model distillation, or Bayesian inference machinery. The official Mixle repository and the separate Mixle notebooks repository contain considerably more machinery than one article should attempt to carry indoors.

ChatGPT’s view: traditional ML uses a data crusher; Mixle uses the right tool for each field.

The Model Is More Than Its Final Prediction

Many familiar machine-learning estimators have a narrow public interface:

features → prediction

A fitted probability distribution can answer a broader set of questions:

observation → log density (~ How likely is this observation?)
model → generated samples (~ Generate artificial observations)
observation → latent-component probabilities (~ Find missing features)
model structure → fitted parameters (~ Understand the model)

Those operations make several data-science tasks look like variations of the same problem.

A low-density row can become an anomaly candidate. Mixture responsibilities can become soft cluster assignments. Samples from the fitted model can be compared with the original data. A sequence model can estimate hidden states rather than merely forecast the next value.

Mixle’s central abstraction is therefore not “a clustering package” or “a density-estimation function”. It is a composable distribution.

A Gaussian distribution, a categorical distribution, a mixture, a sequence model, and a hidden Markov model expose compatible probabilistic behaviour. The library can then fit a nested structure with a shared optimize(...) mechanism or with the shorter probabilistic-programming interface in mixle.ppl.

Installing Only What We Need

For the examples in this series, we also use Matplotlib, NumPy, Pandas, and scikit-learn in addition to mixle itself.

Mixle provides optional extras for Torch, JAX, Numba, Spark, Dask, Ray, database connectors, and other integrations. None of those are needed here. This is deliberate, we want to keep it rather basic for now.

A distributed probabilistic model is still a probabilistic model. It is easier to decide whether the model is useful before asking a cluster to become emotionally involved.

The examples below were executed with Mixle 0.7.0. The package is developing quickly, so code written for a later release should be checked against that release’s documentation and changelog as syntax changes quickly.

The Shortest Route from Data to a Model

At the highest level, Mixle can infer a model from raw records:

The first operation scores an existing observation. The second asks the fitted model to generate three new records.

There is a great deal hidden inside optimize(records):

1. inspect the Python values and field structure;

2. propose suitable distribution families;

3. encode the observations;

4. estimate parameters;

5. preserve a fitted object that can still score and sample.

Automatic inference is useful for exploration, but it should not remove inspection from the workflow. We still need to print the fitted model, compare alternatives, hold out data, and ask whether its assumptions match the process that produced the rows. Just because it is automatic does not mean it is assumption-free.

Project 1: Did One Process Become Two?

Suppose an API normally answers in about 120 milliseconds. A smaller group of requests takes roughly 260 milliseconds because it follows a slower code path. The monitoring system stores both in one column called latency_ms.

A mean and standard deviation can summarise the column. They cannot tell us whether the broad variation comes from one noisy process or two narrower regimes.

We will generate a small synthetic dataset with that structure:

The two groups are known because we created them. But the fitting code does not receive those labels. That distinction matters, for the model this is one array of measurements, not a helpful note explaining where the slow path begins.

Fitting One Gaussian and a Two-Component Mixture

The mixle.ppl interface uses free to mark parameters that should be estimated:

Here Normal(free, free) asks Mixle to estimate both parameters of the Gaussian distribution. Then Mix([...]) wraps two such components in a latent mixture. Each observation is treated as if it came from one of the two components, but that component label is not observed.

The fitted mixture therefore needs to learn:

• the centre and spread of the fast component;

• the centre and spread of the slow component;

• the relative weight of each component;

• the probability that each observation belongs to either one.

The library selected expectation-maximisation for this structure. We can ask it to explain that decision via

This is a useful detail, but it will need a bit of extra reading in the documentation as this is usually abbreviated a bit. For example, the 'em' means mixle used an Expectation-Maximisation algorithm. Then it gives the reasons why it did choose that route. In this case because it needs to find all the parameters and no Bayesian priors were supplied. Then caveats describes the limitations of the route.

This means that the fitting route is not a decorative option chosen after the model has been specified. It follows from the structure and the parameter treatment.

It also tells us what the result does not contain. This fit returns a maximum-likelihood point estimate, not a posterior distribution over every parameter.

Comparing the Models on Data They Did Not Fit

A mixture has more parameters than a single Gaussian. It should fit the training data at least as well, even when the extra flexibility is merely memorising noise.

We therefore compare mean log density on the held-out quarter:

The run used for this article produced:

Single-Gaussian held-out mean log density: -5.638
Two-component held-out mean log density:   -4.676

Log densities are easier to compare than to admire. The larger value is better here because the held-out observations receive more probability density under the mixture. The result supports the two-regime description for this synthetic dataset.

It does not establish that every shoulder or second bump in a real latency histogram is a separate operational process. Real measurements may contain trends, censoring, changing traffic, time-of-day effects, or heavy tails that neither of these candidates describes well.

Inspecting What the Mixture Learned

The PPL fit returns a wrapper whose fitted distribution is available through .dist:

The fitted components were:

mean=119.8 ms, standard deviation=11.7 ms, weight=0.757
mean=260.8 ms, standard deviation=21.5 ms, weight=0.243

Those estimates are close to the generating values.

That is reassuring because this is a controlled demonstration. In real data, component means are not automatically named causes. A fitted “slow” component might correspond to cache misses, a particular endpoint, network congestion, or several effects compressed together.

A latent component is a statistical pattern. Giving it an operational name requires additional evidence.

Turning Responsibilities into a Soft Decision

A mixture can return the posterior probability of each component for a value:

This is a soft assignment rather than a compulsory label.

A value near 110 milliseconds should receive almost all its probability from the fast component. A value near 275 milliseconds should strongly favour the slow component. The interesting values are those near the crossing point, where the model admits that either explanation remains plausible. That uncertainty is not an inconvenience to be rounded away.

It is one of the main reasons to fit the distribution rather than immediately converting every point into a cluster number.

Drawing the Result

The density curves come directly from the fitted objects:

We can then easily plot these

Single Gaussian and two-component Mixle model

The upper panel shows why the single Gaussian struggles. To cover both groups, it spreads probability across the relatively empty middle. The mixture can place one narrow component over the main operating regime and a second over the slower group.

The lower panel shows a rapid transition in component probability. It is a model-based boundary, not a law of network engineering.

The requests have not been informed that they now belong to teams.

Why This Is More Than a Gaussian-Mixture Tutorial

A Gaussian mixture is not unusual. Scikit-learn already provides a mature implementation for ordinary numeric matrices.

Mixle becomes more distinctive when the observation is not one numeric vector with one shared distributional story.

Consider a customer record with five fields:

(
    age,                 # continuous
    region,              # categorical
    purchase_count,      # count
    interests,           # collection of labels
    session_actions,     # ordered sequence of states
)

Flattening this record is possible, but it is not neutral. We would have to decide how to encode the category, whether interest order should be ignored, how to pad sessions, how to scale counts against age, and what distance between two complete rows should mean.

A heterogeneous probability model can instead give each field its own local model and combine the field contributions at record level.

This changes the question from:

How do I force this row into one feature space?

into:

What probability model belongs to each part of the row,
and how should those parts interact?

The second question is not automatically easier, in contrast it might be quite difficult. It is usually more honest.

Project 2: Segment Customers Without Flattening the Record

We will create three synthetic customer profiles. They differ in age, region probabilities, purchase counts, interests, and session behaviour. The session uses integer states:

0 = browse
1 = compare
2 = buy

The data generator keeps the fields in their native Python forms:

REGIONS = ["north", "south", "west"]

PROFILES = [
    {
        "age": (28, 4),
        "region": [0.70, 0.20, 0.10],
        "purchases": 2,
        "interests": {
            "music": 0.80,
            "travel": 0.60,
            "sport": 0.20,
        },
        "session": [0.65, 0.25, 0.10],
    },
    {
        "age": (47, 6),
        "region": [0.10, 0.65, 0.25],
        "purchases": 7,
        "interests": {
            "music": 0.20,
            "travel": 0.40,
            "sport": 0.75,
        },
        "session": [0.15, 0.35, 0.50],
    },
    {
        "age": (36, 5),
        "region": [0.20, 0.20, 0.60],
        "purchases": 4,
        "interests": {
            "music": 0.45,
            "travel": 0.80,
            "sport": 0.40,
        },
        "session": [0.30, 0.55, 0.15],
    },
]

The important part of the sampling loop is the final record:

record = (
    float(age),
    str(region),
    int(purchases),
    interests,
    session,
)

No one-hot encoding is added. The list of interests and the ordered session remain separate fields. Mixle’s automatic structure logic inspects the values rather than requiring us to build one numeric matrix first.

For completeness, the complete generator looks as follows.

If interested you can see here line by line how the data is generated. Using a normal distribution with the age, a random choice for the region, a Poisson distribution for the purchase profile and so on.

Letting a Dirichlet-Process Mixture Use Fewer Than the Maximum

The mixed-type notebook collection includes a helper for building a Dirichlet-process mixture over heterogeneous records:

Here max_components=8 is a computational cap. It does not assert that there are eight meaningful customer segments. The fitted model can leave components effectively unused.

This is useful when the number of groups is not known in advance, but it does not remove the need to inspect stability. A different seed, concentration prior, sample size, or set of field models may change how the data is partitioned.

For our seeded run, eight components were available and three received hard assignments.

That happens to match the three generating profiles. The model was not given that answer in advance.

Recovering Component Assignments

The mixture stores component log weights and lets every component score every row.

We can combine those quantities and select the largest value:

This produces a hard label for visualisation and evaluation.

The underlying model remains probabilistic. A more cautious application could retain normalised responsibilities and treat borderline records differently from high-confidence assignments.

Evaluating Labels Without Caring About Their Names

Mixture labels are arbitrary. One run may call the youngest segment component 0. Another may call it component 6. The fitted density is unchanged by that renaming.

The adjusted Rand index compares two partitions without requiring the numeric labels to match:

The run used here produced:

Available components: 8
Occupied components:  3
Adjusted Rand index:  0.758
Cluster sizes: {0: 78, 1: 67, 2: 65}

An ARI of 0.758 indicates substantial agreement, but not perfect recovery.

That is a better teaching result than a suspiciously flawless one. The synthetic profiles overlap, and the model sees noisy individual records rather than profile definitions.

A Two-Dimensional Plot of a Five-Field Fit

For the figure, we plot only age and purchase count:

This will produce something along these line.

Known customer segments and Mixle assignments

The left panel contains the known generating labels. The right contains the fitted assignments. Some points that look misplaced in the two-dimensional projection may still make sense when region, interests, and session actions are considered.

This is an important visualisation warning.

A model fitted in a richer space should not be judged solely by the two columns that happened to fit comfortably on the page.

The chart is a projection of the analysis, not the complete evidence used by it.

When the Structure Is Known, Say So

Automatic model proposal is one route through Mixle.

It is not the only route.

When the intended structure is known, it can be specified explicitly. A simplified two-component model containing a Gaussian and a categorical field can be written with the lower-level distribution classes:

This is more verbose than optimize(data) and it also makes the intended family visible. That can be preferable when the domain already tells us which variables are counts, which categories are meaningful, and which latent structure we are willing to defend.

Automatic proposal is useful when we are exploring. Explicit structure is useful when assumptions need to survive a code review.

What the Composite Model Assumes

A component built from separate field distributions often treats those fields as conditionally independent once the latent segment is known. In plain language:

After we know the customer segment, the remaining relationship between age, region, purchase count, interests, and session is represented only through their individual field models unless an explicit dependence structure is added.

That assumption can be reasonable, but it can also be wrong.

Suppose checkout actions become more likely as purchase counts increase even within one segment. A purely independent composite may miss that relationship and become overconfident because it counts correlated evidence as if it were separate.

Mixle includes dependence modelling through automatically proposed Bayesian networks, copulas, and vines for suitable data. Those are useful additions, but they are not an instruction to accept every discovered edge as a causal statement. Dependence is not causation.

A Bayesian-network edge selected for density modelling is still a statistical representation of the joint distribution.

Where This Can Go Wrong

A family can fit while telling the wrong story

A mixture may improve held-out density because the true distribution is skewed or heavy-tailed, not because two real populations exist. Compare plausible alternatives rather than converting every component into a new persona.

Local optima still exist

Expectation-maximisation can converge to different solutions from different initialisations. Run several seeds and compare held-out fit, component stability, and interpretability.

Rare categories may be mostly estimation noise

A categorical field with many levels and few observations can produce unstable probabilities. Pool rare categories or use appropriate smoothing rather than asking a tiny sample to estimate a miniature census.

Long sequences can dominate the record likelihood

A sequence contributes multiple observations. Its likelihood scale may overwhelm shorter scalar fields. Check whether the composite objective reflects the relative importance intended for the analysis.

Missing data is not merely an empty string

Preserve missingness deliberately. None, an absent field, and a literal category called "unknown" can describe different situations.

A successful synthetic recovery proves very little about a real deployment

We created the profiles, selected the distributions, and evaluated against known labels. Real customer segments do not arrive with an answer key hidden behind the DataFrame.

When a Simpler Tool Is Better

Use SciPy when one familiar distribution needs to be fitted or evaluated.

Use scikit-learn’s GaussianMixture when the data is a conventional numeric matrix and the goal is standard Gaussian-mixture clustering.

Use ordinary Pandas summaries when the question is a mean, quantile, group count, or cross-tabulation.

Use a supervised estimator when reliable target labels exist and prediction is the actual objective.

Mixle earns more of its complexity when several of these are true:

• records contain genuinely different field types;

• a joint probability model is useful;

• latent variables or sequences belong in the model;

• scoring and sampling matter as much as prediction;

• model parts need to be composed rather than glued together through separate pipelines;

• the intended analysis benefits from one probabilistic interface across several structures.

A library should be selected because its abstraction matches the problem.

Not because its feature list has achieved escape velocity.

Final Thoughts

The first latency project used a mixture to separate two plausible operating regimes.

The second used a mixture whose observations were complete heterogeneous records rather than single numbers.

The important progression was not from a small dataset to a larger one.

It was from one probability model to a composition of several models.

That is the useful mental model for Mixle:

Give each part of the data a probability model that respects what it is, then combine those parts only at the level where the combination has meaning.

A flattened feature matrix can still be the right answer.

It just should not become the answer before the question has been allowed to keep its types.

In Part II, we will use fitted Mixle models for anomaly detection, topic discovery, synthetic-data checks, hidden-state models, and alternative compute engines.