Data, Analytics and AI
MLOps and AI engineering
A model that works in a notebook has solved the part of the problem that demonstrates well; the rest is serving it, versioning it, watching it and replacing it without breaking whatever depends on it. MLOps for classic models and its younger sibling for language-model applications both belong here: deployment, drift monitoring, evaluation pipelines, cost control and rollback, the work that turns a promising result into a product.
Why this topic exists: Getting a model or an LLM application into production and keeping it there: deployment, monitoring, evaluation, cost (SFIA Machine learning and Data engineering; Applied Skills in Foundry; MLOps in the AI portal).
MLOps borrows its name and many of its habits from DevOps, then adds a complication: a model's behaviour depends on data that keeps changing after release. SFIA 9 folds the work into its machine learning skill rather than treating it as a hand-off to operations, and names deployment, monitoring, retraining and the traceability of outputs as part of that skill. AWS's AI Practitioner outline asks candidates to describe MLOps ideas such as repeatable processes, technical debt, monitoring and retraining, and to weigh technical metrics like precision and recall against business ones such as cost per user.
Classic models and language-model applications
| Concern | Classic ML model | LLM application |
|---|---|---|
| What is versioned | Training data, features, code and the trained weights | Prompts, retrieval indexes, the model version and tool definitions |
| What drifts | Input distributions and the link between inputs and labels | The provider's model, the documents retrieved and the questions users ask |
| How quality is checked | Held-out metrics, then live metrics once labels arrive | Evaluation sets of questions with graded answers, scored by people or by another model |
| Where cost sits | Training runs and serving infrastructure | Tokens per request, which grow with context length and agent steps |
| How rollback works | Redeploy the previous model artefact | Pin the previous model and prompt pair; harder once a provider retires a version |
The right-hand column is younger and far less settled. An agentic system that calls tools and takes several steps per request multiplies every row: more to version, more ways to drift, more tokens per answer and actions that cannot simply be undone. Microsoft's Applied Skills assessment for agents in Foundry tests the build side of this in a lab; running such systems over months is still learned on the job.
The working stack
- Packaging. Models ship inside containers so the library versions used in training match those in serving. PyTorch and TensorFlow both export models for this, and TensorFlow has serving and on-device paths of its own.
- Pipelines. Training and evaluation run as code, triggered by new data or a schedule, not by someone rerunning a notebook. The data side of those pipelines is what DEA-C01 examines.
- Registry and lineage. Every deployed model traces back to the data, code and parameters that produced it.
- Monitoring. Latency and errors as for any service, plus prediction distributions, drift signals and, where labels arrive later, real accuracy.
- Release control. Shadow runs, canary releases and a rehearsed way back to the previous version.
The beginner's mistake is treating deployment as the finish line. A model goes live, the dashboard shows healthy latency, and nobody notices accuracy sliding for months because ground truth arrives slowly and no one joined it back to the predictions. People reach this work from two directions: software and DevOps engineers who learn enough modelling to be dangerous, and data scientists tired of handing notebooks over a wall. Both need machine learning fundamentals, pipeline discipline from data engineering and the platform view in DevOps and CI/CD; language-model work adds generative AI and LLMs.
Next to this topic
- Data analysis fundamentalsAsk, prepare, process, analyse, share: the analyst's workflow as Google's programme and DP-900's core data concepts teach it, before any tool (SFIA Data analytics).
- SQL for analysisQuerying, joining, aggregating and windowing data is the analyst's first real tool and the language of every warehouse (DP-900 relational concepts; DMBOK Data Storage and Operations; the programming field owns database design).
- Data visualisation and business intelligenceDashboards, reports and the semantic models behind them: PL-300's four skill areas and DMBOK's Data Warehousing and BI area (Tableau, Power BI, Looker).
- Statistics and experimentationProbability, inference, regression and A/B tests separate a chart from a claim (Google's Advanced Data Analytics; SFIA Data science).
- Python for data workpandas, NumPy, notebooks and plotting: the working environment of analysis and machine learning (SFIA Data science; the AI pillar's argument for Python).
- Data engineering and pipelinesIngestion, transformation, warehouses, lakehouses and orchestration (DMBOK Data Integration and Interoperability; DEA-C01; DP-700; Databricks; Spark, Airflow, dbt).
- Data governance, quality and privacyOwnership, definitions, metadata, master data, quality rules and lawful use are DMBOK's centre and half its wheel; nobody hires for them until the numbers disagree.
- Machine learningSupervised and unsupervised learning, model evaluation, deep learning and the discipline of not fooling yourself (SFIA Machine learning; AIF-C01; Azure AI Fundamentals).
- Generative AI, LLMs and agentsFoundation models, prompting, retrieval, fine-tuning and agents: the newest layer of the field and the one AIF-C01 and Azure AI Fundamentals now test (the AI portal reads the concepts in order).
- Natural language processing and computer visionText and images are the two applied fields where classic machine learning met deep learning first, and where most production AI still lives (SFIA Machine learning; Azure AI Fundamentals AI-901 workloads).
- Responsible AI and AI governanceBias, transparency, privacy, safety and the frameworks that make them work (NIST AI RMF; AIF-C01's responsible-AI domain; SFIA 9's AI and data ethics skill; the EU AI Act).
Concepts to know
Glossary entries with the reason each one matters here.
- MLOps
The topic's own concept.
- Containers
How models ship.
- Agentic AI
The newest thing AI engineers put in production.
Certifications that test it
Vendor exams and free certificates; facts, cost and the preparation path are on each page, and the certifications hub has them all.
- AWS · DEA-C01AWS Certified Data Engineer – AssociatePipelines that feed models.
- Microsoft · Lab-based assessment · freeMicrosoft Applied Skills: Get started developing agents in Microsoft FoundryDeploying a model and an agent in Foundry.
- AWS · AIF-C01AWS Certified AI PractitionerIts first domain names MLOps, monitoring and retraining outright.
Tools of the trade
- PyTorch
Models trained in it are what gets deployed.
- TensorFlow and Keras
TensorFlow Serving and Lite are deployment paths.
Frequently asked
- Is MLOps a job title or a set of practices?
- Both, depending on the size of the organisation. Large teams hire ML platform engineers to build shared tooling; in smaller ones a data scientist or backend developer does the work beside other duties. Adverts for AI engineers usually mean someone who builds and runs applications on top of hosted language models.
- Do I need Kubernetes to deploy a model?
- Not to begin with. The managed machine learning services on the major clouds put a model behind an endpoint without it. Kubernetes becomes relevant when many models share infrastructure, and then a platform team usually owns it; containers and Kubernetes covers that ground.
- How do you monitor a model when correct answers arrive late?
- Watch what is visible at once: the distribution of inputs and of predictions compared with training time, where large shifts give early warning. Then join real outcomes to predictions as they arrive and compute accuracy on a delay. Without that join, talk of drift is guesswork.
Courses in the directory
22 courses are filed here; the top 6 by our ranking, details and the provider link on each course page.
Last reviewed 26 September 2026 · Getting Digital