Cloud & Deployment

A model that only runs on the laptop it was trained on is a result, not a system. The work that turns one into the other is unglamorous and it is most of the job: somewhere to run it that survives a traffic spike, a path for data to reach it that does not involve anyone emailing a spreadsheet, a way for the people who use it to say when it is wrong, and a route for that correction to reach the next version.

Running models on Kubernetes

Inference is spiky. A quality-control model sees nothing overnight and everything at shift change; a screening run is idle for a week and then wants forty GPUs for an afternoon. Kubernetes handles that shape well, which is why we default to it - on GKE, or on whichever cluster you already run.

Security is part of the architecture, not a step at the end

Scientific and industrial data is usually the most sensitive thing a company owns - process parameters, unpublished results, patient records. The infrastructure has to assume that.

• Isolation by default. Private clusters, no public endpoints on anything that does not need one, network policy between workloads rather than a flat namespace.
• Least privilege. Workload identity instead of long-lived keys, scoped service accounts, secrets in a managed store and never in an image or a repository.
• Auditability. Who called the model, with what, and what it answered - retained where that matters for review or regulation.
• Data residency. EU-region storage and processing where that is a requirement, and a clear answer about where every copy lives.
• Supply chain. Pinned dependencies, image scanning, and a base image that gets rebuilt rather than pinned to 2021.

Integration with your data

The most common reason a good model quietly stops being used is that nobody rebuilt the path that fed it. Deployment and data plumbing are the same project.

We build the collection and validation side too: ingesting from instruments, databases, object storage or an existing warehouse; validating on arrival so a schema change surfaces as an alert rather than a silent accuracy drop months later; and making sure the transformations applied at inference are the same ones applied during training, because when they diverge the failure is subtle and expensive.

Where a database is the right home for it, the model's outputs go back in alongside the inputs - predictions, confidence, model version - so the record of what was decided is queryable rather than living in a log file.

Automated processing pipelines

Almost none of this is a single job. Between an instrument writing a file and someone reading a number there is a chain of steps, and every one of them is a place where things silently stop happening. The chain is worth building as one orchestrated pipeline rather than as a folder of scripts and a cron entry nobody remembers writing.

1. 1

   Ingest

   From instruments, databases, object storage or an existing warehouse - on a schedule, or when something arrives.

2. 2

   Validate

   Schema, ranges, completeness. A row that fails is rejected and raises an alert rather than quietly passing through.

3. 3

   Transform

   The same code path that will run at inference time, so training and serving cannot drift apart.

4. 4

   Store

   Versioned, with the raw input kept alongside the derived form - because the next question is always one the current transform throws away.

5. 5

   Run

   Scoring, aggregation, retraining - whatever this pipeline exists to do.

6. 6

   Publish

   To a database, a dashboard, an API or a report, with provenance attached to every figure.

What separates a pipeline from a scheduled script is mostly what happens when something goes wrong:

The same machinery runs the training side. A retrain is a pipeline: pull the current labelled set, train, evaluate against the incumbent on a held-out set, and register the candidate - with promotion to production a deliberate step rather than an automatic one.

Active learning: the deployment improves the model

A deployed model generates the most valuable training data you will ever get, and most deployments throw it away.

Because our models carry uncertainty rather than just a prediction, we can rank incoming cases by how unsure the model was about them. Those are exactly the cases worth a human's attention - and exactly the labels that improve the next version most per hour of expert time. The loop is straightforward once it exists:

1. The model flags the cases it found hardest, instead of an operator sampling at random.
2. Someone who knows the domain resolves them, in the same tool they were already using.
3. Those resolutions land in the training store with provenance attached.
4. Retraining runs on a schedule or on a trigger, and the candidate is evaluated against the current model before anything is promoted.

Alongside it, drift monitoring watches whether today's inputs still look like the ones the model was trained on - which is what tells you a retrain is needed before performance degrades enough for someone to complain.

Applications people actually use

An API is only useful to a system. When the user is a person - an operator on a line, a researcher at a bench, a technician in a field - the model needs a front end, and often one that works where the network does not.

The point of building these ourselves is the return path. An app that shows a prediction is a demo; an app where the user can say "no, that one is a crack, not a scratch" and have it reach the training set is part of the system. Every correction, every new observation and every image the model has not seen before flows back into the same store the next training run reads from.

Where to go next