# LLMOps

> LLMOps is the subset of MLOps focused on the specific operational concerns of large language models: prompt versioning, evaluation, cost control, and output observability.

Category: Production
Source: https://impetora.com/glossary/llmops
Part of: Impetora AI consulting glossary (https://impetora.com/glossary)

## What is LLMOps?

LLMOps adds practices that classical MLOps does not cover well. Prompts are first-class artefacts. Evaluation uses LLM-as-judge alongside golden datasets. Cost is metered by token, not by request. Latency is dominated by streaming and context size. Outputs are non-deterministic and need sampling and content checks. Tools include prompt registries, eval harnesses, trace viewers, and guardrail engines.

## How does LLMOps apply to enterprise AI?

Any enterprise with a generative AI feature in production needs LLMOps. Without it, the team cannot debug why a prompt regressed, why costs spiked, or why a customer received a wrong answer.

## Related terms

- [MLOps](https://impetora.com/glossary/mlops) - MLOps is the discipline of operating machine learning systems in production: versioning, deployment, monitoring, retraining, and governance.
- [Evaluation Harness](https://impetora.com/glossary/evaluation-harness) - An evaluation harness is the test framework used to measure an AI system against a fixed set of inputs, expected outputs, and metrics, run on every change.
- [Observability](https://impetora.com/glossary/observability) - Observability for AI is the ability to understand what an AI system did, why it did it, and at what cost, by inspecting its inputs, outputs, intermediate steps, and metrics.
- [Guardrails](https://impetora.com/glossary/guardrails) - Guardrails are runtime checks placed around an AI system to constrain inputs, outputs, and tool calls within safety, compliance, and business policy.

## External references

- [Anyscale LLMOps overview](https://www.anyscale.com/blog/llmops-large-language-model-operations)

---

Impetora is a custom AI consultancy and solutions partner for enterprises in regulated industries. Submit a project at https://impetora.com/intake.