DevOps School

---

Quick Definition

A service catalog is a curated, machine- and human-readable inventory of services an organization offers, including metadata, ownership, operational guarantees, provisioning interfaces, and policies.

Analogy: A service catalog is like a restaurant menu that lists dishes, ingredients, chef contact, allergens, preparation time, and ordering instructions so diners and kitchen staff understand expectations and how to request each item.

Formal technical line: A service catalog is a governed API-driven registry that exposes service metadata, provisioning endpoints, SLIs/SLOs, IAM bindings, and integration contracts for consumption by developers, CI/CD systems, and automation.

If the term has multiple meanings, the most common meaning above is the service-discovery/managed-offering catalog used by enterprises and cloud-native teams. Other meanings include:

• A procurement catalog in IT asset management.
• A product catalog for external customer-facing SaaS features.
• A data service catalog focused primarily on datasets and data lineage.

---

What is service catalog?

What it is / what it is NOT

• What it is: A controlled registry and self-service layer that documents and automates lifecycle actions for services (provision, update, decommission).
• What it is NOT: A mere list of URLs or a primitive spreadsheet; it is not a substitute for proper access control, monitoring, or runbooks.

Key properties and constraints

• Canonical metadata: owner, SLA/SLO, API endpoints, provisioning templates, cost center.
• Machine API: must be addressable via REST/GraphQL/CLI for automation.
• Governance hooks: policy checks, approval workflows, RBAC integration.
• Versioning: service offerings and their schemas must be versioned.
• Discoverability: searchable catalog with tags and dependency mapping.
• Constraints: Catalog design must balance discoverability vs noise; metadata accuracy decays without ownership commitments.

Where it fits in modern cloud/SRE workflows

• Developer self-service: request and provision platforms, databases, or feature toggles.
• CI/CD integration: pipeline steps call catalog provisioning APIs, parameterize environments.
• SRE operations: read SLOs and runbooks from catalog during incidents.
• Security/GRC: automate policy enforcement during provisioning and inventory audits.
• Observability: catalog links to dashboards and telemetry origins.

Diagram description (text only)

• Users and pipelines query the Catalog API.
• Catalog returns Service Definition and Provisioning Template.
• Provisioner invokes Infrastructure APIs (Kubernetes, cloud provider).
• Provisioned instance registers to Discovery and Observability.
• Catalog updates state and stores metadata in a versioned registry.
• Governance engine validates policy; notifications go to owners.

service catalog in one sentence

A service catalog is a single source of truth for what services exist, how to request them, who owns them, and how they behave operationally.

service catalog vs related terms (TABLE REQUIRED)

ID	Term	How it differs from service catalog	Common confusion
T1	Service registry	Focuses on runtime discovery not metadata and governance	Confused due to overlapping discovery features
T2	API gateway	Routes and enforces policies but does not inventory offerings	People assume gateway equals catalog for APIs
T3	CMDB	Broader config tracking often manual vs API-first catalog	CMDB seen as single inventory for all assets
T4	Product catalog	Customer-facing and pricing-centric, not ops-first	Product offerings may reuse internal catalog data
T5	Data catalog	Focused on datasets and lineage not runtime provisioning	Data teams mix dataset metadata with service metadata

Row Details (only if any cell says “See details below”)

• None

---

Why does service catalog matter?

Business impact (revenue, trust, risk)

• Faster time-to-market: Self-service provisioning reduces lead time for new environments and features.
• Reduced compliance risk: Automated policy checks enforce standards at request time.
• Measurable SLAs: Clear SLOs in catalog increase customer trust and set expectations.
• Cost transparency: Catalog entries can include cost templates enabling chargeback or showback.

Engineering impact (incident reduction, velocity)

• Less toil: Developers avoid manual infra requests; standard templates reduce configuration mistakes.
• Consistent deployments: Templates enforce good defaults and security settings.
• Faster incident resolution: Owners and runbooks are discoverable, reducing MTTD/MTTR.
• Increased velocity: Teams can iterate safer and faster with guarded self-service.

SRE framing (SLIs/SLOs/error budgets/toil/on-call)

• Catalog entries must include SLIs and SLOs so SREs can manage service-level objectives and error budgets.
• Runbooks and ownership reduce on-call cognitive load and toil.
• Catalog-backed provisioning allows SREs to embed monitoring templates and alerting defaults.

3–5 realistic “what breaks in production” examples

• Misconfigured DB provisioning: Wrong storage class leads to IO saturation and database outages.
• Untracked service ownership: No owner listed leads to delayed triage during alerts.
• Incompatible API contract deployed: Consumers break because a newer service version lacks backward compatibility.
• Policy bypass: Manual provisioning avoids network segmentation leading to compliance breach.
• Observability gaps: Deployed service lacks logging/metrics because catalog template omitted telemetry configuration.

---

Where is service catalog used? (TABLE REQUIRED)

Explain usage across layers and areas.

ID	Layer/Area	How service catalog appears	Typical telemetry	Common tools
L1	Edge and network	Catalog lists proxies, WAF rules, ingress templates	Request latency and error rates	Ingress controllers, API gateways
L2	Service and application	Microservice templates, runtime configs, SLOs	Request success, latency, saturation	Service mesh, service registry
L3	Data and storage	DB templates, dataset owners, retention policies	IO, query latency, freshness	Data catalog, ETL tools
L4	Cloud infra layers	VM and resource templates, IaC modules	Provision success, cost, resource usage	IaC tools, cloud consoles
L5	Kubernetes	Helm/OPA/CRD-based offerings for namespaces	Pod health, restart rates, resource requests	Helm, Operators, OPA
L6	Serverless and PaaS	Managed function templates and quotas	Invocation counts, cold starts, errors	Cloud functions, managed DBs
L7	CI/CD and pipelines	Build/test/provision actions as catalog items	Pipeline success, duration, artifact integrity	CI systems, artifact registries
L8	Observability and security	Dashboards, alert bundles, policy packs	Alert volume, false positive rate	Monitoring, SIEM, policy engines

Row Details (only if needed)

• None

---

When should you use service catalog?

When it’s necessary

• You have multiple teams provisioning shared infrastructure and need consistent configuration and governance.
• Regulatory or audit requirements require automated enforcement and traceable provisioning.
• SRE requires embedded SLOs, runbooks, and ownership for services.
• You need chargeback/showback and cost visibility for teams.

When it’s optional

• Small teams (1–3 engineers) with limited services and direct communication may not need full catalog tooling.
• Early-stage prototypes where speed beats governance temporarily.

When NOT to use / overuse it

• Avoid cataloging extremely ephemeral or experimental artifacts with high churn; the catalog becomes noisy.
• Don’t mandate heavy approval workflows for low-risk dev-only services; it blocks velocity.
• Avoid duplicating data already governed by a single-source system like specialized data catalogs unless value added.

Decision checklist

• If multiple teams and shared infra -> implement catalog with automation.
• If regulatory audits and high scale -> catalog is required.
• If single small team and rapid prototyping -> prefer lighter processes.
• If frequent scheme/contract changes -> ensure versioning and consumer compatibility checks before adopting.

Maturity ladder: Beginner -> Intermediate -> Advanced

• Beginner: Simple catalog entries in Git with basic metadata, CLI for provisioning.
• Intermediate: API-driven catalog integrated with CI/CD, basic RBAC, linked SLOs and dashboards.
• Advanced: Full platform with catalog as a product, policy-as-code, cost optimization, automated remediation, dependency mapping, and AI-assisted recommendations.

Example decisions

• Small team example: A 5-person team uses Helm charts and a README catalog in Git; they adopt a minimal API-backed catalog only when cross-team dependencies grow.
• Large enterprise example: Multiple product teams using multi-cloud require a centralized catalog with enforced policies, automated provisioning, and SLO governance tied to billing.

---

How does service catalog work?

Explain step-by-step

Components and workflow

1. Service Definition Store: Versioned repository (Git/DB) containing schema for each offering.
2. Catalog API: Exposes listing, search, request, and lifecycle actions.
3. Provisioner: Automation component that translates templates to cloud/K8s APIs.
4. Governance Engine: Policy-as-code evaluator that checks requests against rules.
5. Observability Linker: Attaches telemetry and dashboards to provisioned instances.
6. Lifecycle Manager: Tracks provisioning state, updates, scaling, decommission.
7. Notification & Approval System: For human approvals and events.

Data flow and lifecycle

• Author creates a Service Definition in the Store with metadata, templates, SLOs, and ownership.
• Consumer queries Catalog API to discover service offering and templates.
• Consumer requests provisioning via API/CLI/Portal.
• Governance engine evaluates request and approves or requires manual review.
• Provisioner executes templates against cloud/Kubernetes/managed APIs.
• Instance registers with discovery and observability; Catalog updates status and links artifacts.
• Owner manages lifecycle; decommission triggers resource cleanup and data retention actions.

Edge cases and failure modes

• Stale metadata: No owner updates lead to misleading templates.
• Provisioner drift: Manually changed resources diverge from catalog template.
• Partial provisioning: Multi-step resources succeed partially leaving orphan resources.
• Policy race: Concurrent requests violate quotas causing conflicts.

Short practical examples (pseudocode)

• Example: Requesting a database
• catalog.request(“postgres-prod”, env=”staging”, owner=”teamA”)
• governance.check(policy=db-encryption-required)
• provisioner.apply(iac_template)
• observability.attach(db-monitoring-template)
• Example: CI pipeline uses catalog to spin ephemeral env
• env = catalog.provision(“dev-namespace”, ttl=4h)
• pipeline.run(tests, env)
• catalog.decommission(env) on pipeline completion

Typical architecture patterns for service catalog

• Centralized Catalog Pattern: Single catalog service for entire org. Use when tight governance is required.
• Federated Catalog Pattern: Teams manage catalogs with shared schema and federation gateway. Use when autonomy is important.
• GitOps Catalog Pattern: Catalog definitions stored in Git with controllers applying desired state. Use when auditability and Git workflows are desired.
• Operator-based Catalog Pattern: Kubernetes Operators implement provisioning and lifecycle. Use when heavy K8s-native resources dominate.
• Managed-Service Proxy Pattern: Catalog mediates requests to managed cloud services with standardized templates. Use when relying on cloud-managed PaaS.

Failure modes & mitigation (TABLE REQUIRED)

ID	Failure mode	Symptom	Likely cause	Mitigation	Observability signal
F1	Stale metadata	Wrong owner, missing SLO	No update process	Enforce ownership rotations and metadata CI	Audit mismatch counts
F2	Partial provisioning	Orphan resources	Provisioner crash mid-flow	Transactional workflows and compensating cleanup	Resource orphan alerts
F3	Policy bypass	Noncompliant resources	Manual provisioning	Block direct infra change paths and audit	Policy violations metric
F4	High request latency	Slow catalog API	DB hotspots or heavy queries	Add caching and pagination	API latency percentiles
F5	Drift between desired vs actual	Config mismatch	Manual edits after provisioning	Periodic reconciliation jobs	Drift rate over time
F6	Explosion of catalog items	Discovery noise	Low-quality templates	Introduce tagging and approval gates	Growth rate of items
F7	Missing telemetry	Hard to debug incidents	Template omitted observability	Make telemetry required in templates	Missing monitoring links count

Row Details (only if needed)

• None

---

Key Concepts, Keywords & Terminology for service catalog

Glossary entries (40+). Each line: Term — 1–2 line definition — why it matters — common pitfall

• Service Definition — Structured metadata and templates for an offering — Central unit in a catalog — Pitfall: vague schema missing ownership
• Provisioning Template — IaC or API script to create resources — Enables automation — Pitfall: hardcoded identifiers
• Ownership — The responsible person or team for a service — Enables incident routing — Pitfall: outdated contact info
• SLI — Service level indicator measuring user-facing behavior — Basis for SLOs — Pitfall: wrong metric mapped
• SLO — Service level objective set on SLIs — Drives reliability targets — Pitfall: unrealistic targets
• Error Budget — Allowance of errors within SLO — Guides release decisions — Pitfall: ignored when breached
• Runbook — Step-by-step remediation guide — Reduces on-call toil — Pitfall: unmaintained steps
• Playbook — Higher-level operational guidance — Helps during complex incidents — Pitfall: ambiguous escalation
• Lifecycle — States a resource passes through from request to decommission — Manages automation — Pitfall: missing deprovision
• Discovery — Runtime mechanism to locate service instances — Needed for clients — Pitfall: stale registry entries
• Catalog API — Programmatic interface to the catalog — Enables automation — Pitfall: insecure endpoints
• Governance Engine — Policy evaluator for requests — Ensures compliance — Pitfall: slow policy checks
• Policy-as-code — Declarative enforcement rules in code — Testable and auditable — Pitfall: overly rigid rules
• RBAC — Role-based access control integration — Controls who can request actions — Pitfall: overly permissive roles
• Approval Workflow — Human-in-the-loop gating flow — Controls risky operations — Pitfall: blocking low-risk operations
• Versioning — Semantic versions for service definitions — Maintains compatibility — Pitfall: lacking migration plan
• Tagging — Labels for discoverability and billing — Improves queries — Pitfall: inconsistent tag schema
• Cost Template — Metadata to estimate costs — Enables chargeback — Pitfall: wrong rates
• Telemetry Link — Pointer to dashboards and metrics — Essential for debugging — Pitfall: broken links
• Observability Bundle — Preconfigured dashboards and alerts — Speeds incident response — Pitfall: noisy defaults
• Service Registry — Runtime mapping of endpoints — Differs from metadata catalog — Pitfall: conflating both roles
• Dependency Map — Graph of service dependencies — Important for blast radius analysis — Pitfall: missing indirect dependencies
• Secret Management — How credentials are provisioned — Required for secure provisioning — Pitfall: secrets in templates
• Decommission Policy — Rules for cleanup and retention — Prevents resource leaks — Pitfall: no data-retention rules
• Reconciliation Loop — Periodic checker to align actual with desired — Fixes drift — Pitfall: expensive frequency
• Webhook Integration — Event-driven hooks for actions — Enables notifications — Pitfall: unthrottled webhooks
• Audit Trail — Immutable log of changes and requests — Needed for compliance — Pitfall: insufficient retention
• TTL — Time-to-live for ephemeral resources — Controls cost and clutter — Pitfall: resource accidentally expired
• Multi-tenancy — Support for multiple teams on same platform — Enables sharing — Pitfall: noisy quotas
• Catalog Portal — Human-friendly UI to discover and request — Improves adoption — Pitfall: poor UX
• CLI Client — Developer tooling for scripting requests — Enables pipelines — Pitfall: inconsistent globals
• CRD — Custom Resource Definitions in Kubernetes for offerings — K8s-native provisioning — Pitfall: CRD complexity
• Operator — K8s controller implementing lifecycle logic — Automates stateful resources — Pitfall: operator bugs can affect many resources
• Federation — Multi-catalog cooperation model — Balances autonomy and consistency — Pitfall: sync conflicts
• Immutable Infrastructure — Deployments via declarative templates — Makes drift rare — Pitfall: lacks runbook for manual fixes
• Canary Deployment — Gradual rollout pattern tied to error budget — Reduces blast radius — Pitfall: monitoring not adapted
• Observability Coverage — Degree to which services have metrics/logs/traces — Enables diagnosis — Pitfall: inconsistent coverage
• Service Level Agreement — Formal contract often external — Tied to SLOs — Pitfall: conflicting internal SLOs
• Data Lineage — Tracing data provenance for datasets offered — Important for data catalogs — Pitfall: incomplete lineage capture
• Artifact Registry — Stores images/binaries referenced by catalog — Ensures provenance — Pitfall: expired tokens blocking deploys

---

How to Measure service catalog (Metrics, SLIs, SLOs) (TABLE REQUIRED)

Recommended SLIs and measurement table.

ID	Metric/SLI	What it tells you	How to measure	Starting target	Gotchas
M1	Catalog API success rate	API reliability for provisioning	1 – failed_requests/total_requests	99.9%	Transient retries skew rate
M2	Provision success rate	Proportion of successful provisions	successful_provisions/attempts	99%	Long running ops mask failures
M3	Median provisioning time	Speed of resource provisioning	p50 of provision durations	< 2m for simple items	Dependent on external providers
M4	Provision drift rate	Ratio of resources diverged from desired	drift_count/provisioned_count	< 1%	Drift detection frequency affects value
M5	Catalog item freshness	How current metadata is	days_since_last_update	< 90 days	Ownership practices vary
M6	SLO compliance rate	Percent of services meeting SLOs	services_meeting_SLO/total_services	95%	SLOs depend on accurate SLIs
M7	Alert noise ratio	False positive alerts from catalog defaults	false_alerts/total_alerts	< 20%	Hard to classify alerts as false+
M8	Approval latency	Time humans take to approve requests	mean approval duration	< 1 business day	Business hours and on-call affect target
M9	Cost estimation accuracy	Error between estimated and actual cost	abs(est – actual)/actual	< 15%	Cloud pricing fluctuations affect this
M10	Owner response time	Time owner responds to incident or request	median response time	< 30 mins for P1	Depends on on-call rotations

Row Details (only if needed)

• None

Best tools to measure service catalog

Tool — Prometheus

• What it measures for service catalog: API metrics, provision durations, error rates
• Best-fit environment: Kubernetes and cloud-native stacks
• Setup outline:
• Export metrics from catalog API
• Instrument provisioner with client libraries
• Create service-level metrics for SLIs
• Configure alerting rules
• Use federation for aggregated views
• Strengths:
• Querying with PromQL
• Wide exporter ecosystem
• Limitations:
• Long-term storage requires remote write
• Not ideal for high-cardinality metrics

Tool — Grafana

• What it measures for service catalog: Dashboards and alerting visualization
• Best-fit environment: Teams using Prometheus, Loki, Tempo
• Setup outline:
• Connect data sources
• Build executive, on-call, debug dashboards
• Create alerting rules and notification channels
• Strengths:
• Flexible visualization
• Alerting integrations
• Limitations:
• Dashboard sprawl without governance

Tool — Elasticsearch + Kibana

• What it measures for service catalog: Audit trails, request logs, provisioning traces
• Best-fit environment: Organizations needing log-centric analysis
• Setup outline:
• Ship logs from catalog and provisioner
• Index request and audit data
• Build Kibana visualizations
• Strengths:
• Full-text search and log correlation
• Limitations:
• Storage costs and index management

Tool — Cloud Monitoring (e.g., native provider)

• What it measures for service catalog: Provider-side resource metrics and costs
• Best-fit environment: Single cloud or heavy managed service usage
• Setup outline:
• Enable provider metrics exporters
• Link catalog items to provider resource IDs
• Configure budget alerts
• Strengths:
• Deep cloud-specific telemetry
• Limitations:
• Cross-cloud correlation is harder

Tool — ServiceNow / ITSM

• What it measures for service catalog: Request lifecycle and approvals for enterprise IT
• Best-fit environment: Large enterprises with existing ITSM
• Setup outline:
• Integrate catalog API with ITSM flows
• Map requests to tickets
• Automate status updates
• Strengths:
• Enterprise approval and audit capabilities
• Limitations:
• Heavyweight and slower for dev workflows

Recommended dashboards & alerts for service catalog

Executive dashboard

• Panels:
• Catalog availability and API success rate
• Provision success rate and average time
• Number of active catalog items and growth rate
• SLO compliance summary across services
• Monthly cost estimates by department
• Why: Provides business and leadership visibility into platform reliability and cost trends.

On-call dashboard

• Panels:
• Active P1/P2 incidents and linked catalog items
• Recent failed provisions and retry queue
• Approval requests pending and latency
• Owner contact and on-call info
• Recent changes and deployments affecting catalog
• Why: Quickly surfaces what needs immediate action and who to contact.

Debug dashboard

• Panels:
• Catalog API request timeline with traces
• Provisioner step breakdown durations
• Policy engine decision traces
• Resource drift events and reconciliation logs
• Audit trail for last 100 requests
• Why: Enables in-depth debugging and root cause analysis.

Alerting guidance

• Page vs ticket:
• Page (urgent): Provision failures causing production outage, SLO breaches, security policy violations.
• Ticket (non-urgent): Single failed dev provision, minor drift, stale metadata reminders.
• Burn-rate guidance:
• For SLOs, use burn-rate alerts: alert on 2x burn for immediate attention and 10x burn for page.
• Noise reduction tactics:
• Deduplicate alerts by grouping by service owner.
• Suppression windows for expected maintenance.
• Alert enrichment with runbook link and owner contact.

---

Implementation Guide (Step-by-step)

1) Prerequisites – Inventory of current services and owners. – Basic IaC templates for typical resources. – Authentication and RBAC framework. – Observability baseline: metrics and logging. – Decide on single vs federated catalog model.

2) Instrumentation plan – Identify SLIs for catalog operations. – Add telemetry to catalog API and provisioner. – Ensure tracing across catalog -> provisioner -> cloud API.

3) Data collection – Centralized logging for audit trails. – Telemetry ingestion to monitoring backend. – Inventory sync from provider APIs.

4) SLO design – Define SLI, SLO, and error budget per catalog-critical operation. – Start with conservative targets and iterate.

5) Dashboards – Build executive, on-call, debug dashboards. – Create service-specific dashboards via templates.

6) Alerts & routing – Implement alert policies for page vs ticket. – Integrate with paging and ticketing systems. – Route alerts to owners from catalog metadata.

7) Runbooks & automation – Attach runbooks to each catalog entry. – Automate common remediation (e.g., retry, cleanup).

8) Validation (load/chaos/game days) – Load test the catalog API and provisioner. – Run chaos experiments on provisioning paths. – Game days with cross-team exercises.

9) Continuous improvement – Regularly review item freshness and SLOs. – Collect feedback from consumers and owners. – Automate metadata quality checks.

Checklists

Pre-production checklist

• Service definition in Git with version and owner.
• Templates reviewed and tested in staging.
• Telemetry hooks instrumented.
• Policy checks added for security and cost.
• Acceptance tests for provisioning flow.

Production readiness checklist

• Monitoring dashboards created.
• Alerting and on-call routing configured.
• Runbook linked and validated.
• Cost estimation and tagging confirmed.
• Backup and retention policies set.

Incident checklist specific to service catalog

• Identify if catalog or provisioner caused incident.
• Triage: check API health, provisioner logs, and external provider.
• Notify owner and on-call.
• Execute runbook steps for common failures.
• If policy breach, isolate resources and audit changes.
• Postmortem: record timeline, root cause, remediation, and follow-up tasks.

Examples

• Kubernetes example:
• Prereq: Operator or CRD to represent ServiceDefinition.
• Instrumentation: expose metrics from controller, attach logs to EFK.
• Validation: deploy to staging namespace, assert CR reconciliation.

• Good: p50 provision < 2 minutes; SLO linked dashboard present.

• Managed cloud service example:

• Prereq: IAM roles and service accounts for catalog to call provider APIs.
• Instrumentation: vendor metrics for provisioning and cost.
• Validation: create resource via catalog and verify tags and policies applied.
• Good: Costs estimate within 15% of actual in 30 days.

---

Use Cases of service catalog

Provide 8–12 concrete use cases.

1) On-demand development namespaces – Context: Multiple devs need isolated k8s namespaces. – Problem: Manual namespace creation causes inconsistent RBAC. – Why catalog helps: Standard templates apply quotas and RBAC consistently. – What to measure: Provision time, namespace leakage, resource quota violations. – Typical tools: Helm, Operators, catalog API.

2) Managed database provisioning – Context: Teams request PostgreSQL instances. – Problem: Inconsistent backups and encryption settings. – Why catalog helps: Enforces encryption, backup, and tagging. – What to measure: Provision success, backup frequency, encryption compliance. – Typical tools: Terraform, cloud RDS APIs, secrets manager.

3) Feature-flag service provisioning – Context: Product teams need feature flag environments. – Problem: No standardized rollout strategies lead to outages. – Why catalog helps: Templates include rollout strategies and observability hooks. – What to measure: Flag rollout success, error rate after toggles. – Typical tools: Feature flagging platforms, catalog templates.

4) Data pipeline offering – Context: Teams need ETL jobs with lineage. – Problem: Unknown data owners and retention policies. – Why catalog helps: Attach owners, SLAs, and lineage to pipelines. – What to measure: Data freshness, job success rates, lineage completeness. – Typical tools: Airflow, data catalog integrations.

5) Marketplace of internal SaaS components – Context: Multiple internal reusable services (auth, billing). – Problem: Discoverability and inconsistent onboarding. – Why catalog helps: Single portal with usage guides and SDK links. – What to measure: Consumer adoption and incident rates. – Typical tools: Internal developer portals.

6) Multi-cloud standard abstractions – Context: Teams deploying across clouds require common offerings. – Problem: Divergent configs and permissions per cloud. – Why catalog helps: Provide unified templates mapped to each provider. – What to measure: Cross-cloud provisioning success, cost variance. – Typical tools: Multi-cloud IaC, abstraction layers.

7) Compliance-driven provisioning – Context: Regulated workloads need specific controls. – Problem: Manual checks miss requirements. – Why catalog helps: Policy-as-code gates and required artifacts. – What to measure: Policy violations, audit completion time. – Typical tools: Policy engines, audit logging.

8) Ephemeral test environments in CI – Context: CI pipelines spin up real infra for integration tests. – Problem: Orphaned environments increase cost. – Why catalog helps: TTL and automated teardown ensure cleanup. – What to measure: Ephemeral env count, teardown success rate. – Typical tools: CI systems integrated with catalog API.

9) Observability bootstrap for new services – Context: New services lack dashboards and alerts. – Problem: On-call cannot diagnose failures. – Why catalog helps: Attach baseline dashboard and alert bundle at provisioning. – What to measure: Coverage of metrics, alert false positive rate. – Typical tools: Monitoring templates, Grafana provisioning.

10) Cost-aware resource offerings – Context: Teams need visibility into cost for provisioning choices. – Problem: Overprovisioning leads to budget overruns. – Why catalog helps: Present cost trade-offs with each template. – What to measure: Cost variance against estimates, rightsizing events. – Typical tools: Cloud cost APIs, budgeting tools.

---

Scenario Examples (Realistic, End-to-End)

Scenario #1 — Kubernetes namespace self-service (Kubernetes scenario)

Context: Multiple product teams need dev/testing namespaces on a shared Kubernetes cluster.
Goal: Provide safe, labeled namespaces with quotas and telemetry automatically.
Why service catalog matters here: Prevents inconsistent RBAC and resource hogging while enabling developer autonomy.
Architecture / workflow: Catalog API -> Namespace CRD -> Operator applies namespace, NetworkPolicy, ResourceQuota -> Namespace registers in discovery and monitoring.
Step-by-step implementation:

• Define Namespace offering with CRD template.
• Operator reconciles CRD into namespace and ancillary resources.
• Catalog requires owner and TTL fields.
• Telemetry sidecar injects basic metrics collection.
• Decommission process triggered on TTL expiry or manual request. What to measure: Provision time, resource quota breaches, orphaned namespace count.
  Tools to use and why: Kubernetes CRD+Operator for native lifecycle, Prometheus for metrics, Grafana for dashboards.
  Common pitfalls: Missing network policies allow lateral access; TTL accidentally too short.
  Validation: Load test concurrent namespace creations, verify reconciliation and teardown.
  Outcome: Teams can self-service namespaces with minimal SRE involvement and expected defaults enforced.

Scenario #2 — Serverless function offering (Serverless/managed-PaaS scenario)

Context: Developers deploy business logic as functions using cloud-managed functions.
Goal: Standardize runtime, memory, timeouts, tracing, and cost limits.
Why service catalog matters here: Ensures security settings and observability are present without developer setup.
Architecture / workflow: Catalog portal -> Provision request with runtime and concurrency -> Catalog calls cloud API to create function with IAM and tracing -> Attach alert bundle.
Step-by-step implementation:

• Create function template with default runtime, memory, timeout, and tracing config.
• Integrate catalog with secrets manager for credentials.
• Inject monitoring and create dashboards automatically.
• Apply cost guardrails and quota enforcement. What to measure: Invocation latency, errors, cold starts, cost per invocation.
  Tools to use and why: Cloud function provider, cloud monitoring, secrets manager.
  Common pitfalls: Cold starts cause high latency; missing retry semantics break clients.
  Validation: Synthetic traffic tests and failover checks.
  Outcome: Consistent serverless deployments with traceability and predictable cost.

Scenario #3 — Incident response and postmortem (Incident-response scenario)

Context: A critical internal API fails intermittently and consumers are impacted.
Goal: Rapid identification of owner, runbook, and SLO status to restore service and learn.
Why service catalog matters here: Catalog provides owner contact, runbooks, telemetry, and dependency map for swift action.
Architecture / workflow: Alert points to catalog entry -> on-call list and runbook retrieved -> SRE follows steps -> updates ticket and records remediation in catalog audit.
Step-by-step implementation:

• Alert triggers via monitoring and includes catalog link.
• On-call uses runbook to restart service and check dependent services.
• Post-incident, catalog metadata updated with root cause and mitigation. What to measure: Time to owner contact, time to mitigation, adherence to runbook steps.
  Tools to use and why: Monitoring, incident management system, catalog for linking artifacts.
  Common pitfalls: Outdated runbooks cause wrong actions; owner unreachable.
  Validation: Game day simulation with mocked failures.
  Outcome: Faster MTTR and a documented postmortem linked to the catalog entry.

Scenario #4 — Cost vs performance template choice (Cost/performance trade-off scenario)

Context: Teams choose between SSD-backed instances or cheaper HDD for storage service.
Goal: Make cost and performance trade-offs explicit during provisioning.
Why service catalog matters here: Enables teams to choose a template with predicted cost and performance implications.
Architecture / workflow: Catalog displays options with cost estimate and expected IOPS, consumer selects template, provisioner creates storage with chosen performance tier.
Step-by-step implementation:

• Create two offerings: Premium-SSD and Standard-HDD with metadata and estimated costs.
• Implement policy to require business justification for premium choices.
• Monitor actual cost vs estimate and feedback to catalog UI. What to measure: Cost variance, performance SLI (IOPS, latency), justification compliance.
  Tools to use and why: Cloud cost APIs, monitoring for performance metrics, catalog UI.
  Common pitfalls: Estimates outdated, teams pick premium unnecessarily.
  Validation: Periodic cost audits and rightsizing recommendations.
  Outcome: Transparent trade-offs and measurable cost control.

---

Common Mistakes, Anti-patterns, and Troubleshooting

List of 20+ mistakes with symptom -> root cause -> fix.

1) Symptom: Catalog shows owner as empty -> Root cause: No enforced owner field -> Fix: Make owner required in schema and add CI validation. 2) Symptom: Frequent failed provisions -> Root cause: Provisioner lacks retries/transactional cleanup -> Fix: Add idempotent operations and retry logic with compensating cleanup. 3) Symptom: Orphaned resources after failure -> Root cause: No compensating rollback -> Fix: Implement two-phase commit style or reconciliation cleanup jobs. 4) Symptom: Alerts without runbook links -> Root cause: Templates missing observability bundle -> Fix: Make telemetry mandatory and link runbooks at template creation. 5) Symptom: Slow catalog API responses -> Root cause: Unindexed queries or heavy joins -> Fix: Add indices, caching, and pagination. 6) Symptom: High alert noise after provisioning -> Root cause: Default alert thresholds too sensitive -> Fix: Provide sane defaults and tune per-service. 7) Symptom: Drift between template and actual -> Root cause: Manual edits in infra -> Fix: Block manual edits or reconcile with automation jobs. 8) Symptom: Unknown blast radius -> Root cause: Missing dependency mapping -> Fix: Enrich catalog entries with explicit dependencies and generate graphs. 9) Symptom: Security violations found in prod -> Root cause: Approval bypass or weak policy checks -> Fix: Enforce policy-as-code and block direct provisioning routes. 10) Symptom: Cost overruns -> Root cause: Templates lacking cost estimates and quotas -> Fix: Add cost templates and enforce budgets. 11) Symptom: Low adoption of catalog -> Root cause: Poor UX or missing offerings -> Fix: Improve portal UX and prioritize popular offerings. 12) Symptom: Too many catalog items -> Root cause: No approval or lifecycle for entries -> Fix: Implement approval gates and archival policies. 13) Symptom: Stale runbooks -> Root cause: No maintenance schedule -> Fix: Require runbook review on metadata change and periodic reminders. 14) Symptom: Confused API consumers -> Root cause: Inconsistent API schema between items -> Fix: Standardize schema and publish examples. 15) Symptom: High-cardinality metrics blow up monitoring -> Root cause: Instrument per-user identifiers as labels -> Fix: Use aggregation keys and reduce label cardinality. 16) Symptom: Missing telemetry for some services -> Root cause: Optional telemetry fields allowed -> Fix: Make observability required for production items. 17) Symptom: Approval bottlenecks -> Root cause: Centralized approvals for low-risk items -> Fix: Delegated approvals and role-based thresholds. 18) Symptom: Long incident triage -> Root cause: No direct links to dashboards in catalog -> Fix: Add direct dashboard links and sample queries. 19) Symptom: Unclear SLIs -> Root cause: Metrics do not reflect user experience -> Fix: Map SLIs to user-impacting metrics and validate with users. 20) Symptom: Catalog controller crash affects cluster -> Root cause: Monolithic controller without rate limits -> Fix: Add batching, backoff, and resource limits. 21) Symptom: False positive security alerts -> Root cause: Overly broad detection rules -> Fix: Narrow rules and use contextual signals. 22) Symptom: Missing audit trails -> Root cause: Logs not persisted centrally -> Fix: Ship audit logs to centralized immutable store.

Observability pitfalls (at least 5 included above)

• Too high-cardinality labels, missing dashboard links, incomplete telemetry, absent traces across provisioning path, unpersisted audit logs.

---

Best Practices & Operating Model

Ownership and on-call

• Assign explicit owners for each catalog item and include on-call schedules.
• Owners responsible for metadata, runbooks, and SLO health.
• On-call rotations should be aware of catalog items they cover.

Runbooks vs playbooks

• Runbook: Actionable step-by-step for known failures.
• Playbook: Broader decision trees and escalation for novel incidents.
• Keep runbooks short and test them via mock incidents.

Safe deployments (canary/rollback)

• Use canary deployments authorized by error budget rules from catalog SLOs.
• Automate rollback triggers tied to burn-rate thresholds.

Toil reduction and automation

• Automate provisioning, tagging, and telemetry attachment.
• Start by automating repetitive, manual approval tasks.
• Use reconciliation loops to reduce drift-induced toil.

Security basics

• Integrate RBAC and least-privilege for catalog API.
• Use secrets manager for credentials and never store secrets in templates.
• Enforce policy-as-code for network, encryption, and IAM.

Weekly/monthly routines

• Weekly: Review pending approvals and provisioning failures.
• Monthly: Audit item freshness, SLO compliance, cost variance.
• Quarterly: Review owner assignments and runbook accuracy.

What to review in postmortems related to service catalog

• Whether catalog metadata was accurate.
• If observability artifacts were present and useful.
• If ownership and escalation were clear.
• Any provisioning process failures and mitigations.

What to automate first

• Telemetry attachment to every new service.
• Tagging and cost attribution.
• Basic policy checks for encryption and network segmentation.
• Automated TTL-based teardown for ephemeral resources.

---

Tooling & Integration Map for service catalog (TABLE REQUIRED)

ID	Category	What it does	Key integrations	Notes
I1	Infrastructure as Code	Defines provisioning templates	Cloud APIs, Terraform, Helm	Use for deterministic provisioning
I2	CI/CD	Automates catalog-driven environment creation	Git, pipelines, artifact registries	Integrate catalog API in pipelines
I3	Policy engine	Enforces policies at request time	OPA, policy-as-code, IAM	Real-time checks needed
I4	Observability	Provides metrics, logs, traces	Prometheus, Grafana, Loki	Attach bundles on provision
I5	Service registry	Runtime discovery of instances	Consul, etcd, service mesh	Complementary to catalog
I6	Secrets manager	Securely stores credentials	Vault, cloud KMS	Never store secrets in templates
I7	ITSM / Ticketing	Approval workflows and audit trails	ServiceNow, JIRA	Useful for enterprise approvals
I8	Cost management	Estimates and budgets for offerings	Cloud cost APIs, budgets	Tie catalog items to cost centers
I9	Data catalog	Data asset metadata and lineage	Glue, custom data catalogs	Integrate when offering datasets
I10	Identity & Access	Authentication and RBAC enforcement	OIDC, SSO, IAM providers	Critical for secure provisioning
I11	Artifact registry	Stores images and binaries referenced	Container registries, package repos	Ensures reproducible deploys
I12	GitOps controller	Applies Git-defined catalog state	ArgoCD, Flux	Enables auditability and Git workflows

Row Details (only if needed)

• None

---

Frequently Asked Questions (FAQs)

How do I start building a service catalog?

Start by inventorying offerings, define a simple schema with required fields, put definitions in Git, and build a minimal API or CLI to request items; iterate with one pilot team.

How do I integrate a catalog with CI/CD?

Add a step in pipelines to call the catalog API to provision ephemeral environments, pass back resource identifiers, and decommission after tests.

How do I measure SLOs for catalog-backed services?

Define SLIs that reflect user experience, instrument them in the service template, and aggregate at service and catalog levels for SLO evaluation.

What’s the difference between a service registry and a service catalog?

Registry handles runtime discovery of instances; catalog manages metadata, provisioning, ownership, and governance.

What’s the difference between CMDB and service catalog?

CMDB often aims to track a broad set of configuration items and may be manual; catalog is API-first with provisioning and lifecycle automation.

What’s the difference between product catalog and service catalog?

Product catalog targets external customers and pricing; service catalog targets internal operations, ownership, and observability.

How do I enforce security policies in the catalog?

Use policy-as-code integrated into the catalog request path and block provisioning that fails checks.

How do I keep metadata fresh?

Automate reminders, require owner reviews on changes, and add CI checks that validate metadata on PRs.

How do I handle multi-cloud offerings?

Abstract templates and map to provider-specific modules; consider a federated catalog approach or provider adapters.

How do I avoid catalog sprawl?

Implement approval gates, lifecycle archival policies, and require business justification for new entries.

How do I link runbooks to alerts?

Include runbook URL/ID as part of catalog metadata and configure alert enrichment to surface that link.

How do I set realistic SLOs for new services?

Start with conservative SLOs based on similar services, measure early, and iterate using error budget policies.

How do I automate decommission safely?

Use TTLs, staged decommission (soft delete -> retention -> hard delete), and notify owners before final delete.

How do I handle secrets during provisioning?

Integrate with a secrets manager and reference secrets by ID in templates; never inline secret values.

How do I catalog data assets differently from compute services?

Include lineage, schema, owners, and data sensitivity metadata and integrate with data cataloging tools.

How do I measure catalog adoption?

Track provision requests, active users, and reduction in manual tickets for infra requests.

How do I scale the catalog API?

Use horizontal scaling, caching for read-heavy operations, pagination, and async processing for heavy tasks.

How do I provide a good developer UX for catalog?

Offer CLI, Portal, and API clients with clear examples and templates, and make provisioning fast with good defaults.

---

Conclusion

A service catalog is an operational cornerstone that bridges developer self-service, governance, cost control, and SRE practices. When designed with clear ownership, mandatory telemetry, policy enforcement, and automation, it reduces toil and improves reliability while preserving developer velocity.

Next 7 days plan (5 bullets)

• Day 1: Inventory current services and collect owners for top 10 offerings.
• Day 2: Define minimal schema and mandatory fields (owner, SLO, runbook, cost).
• Day 3: Create Git repo for service definitions and add one pilot offering.
• Day 4: Implement minimal API/CLI to request the pilot offering and log audit events.
• Day 5–7: Integrate basic telemetry, add an owner-reviewed runbook, and run a provisioning test with a small team.

---

Appendix — service catalog Keyword Cluster (SEO)

• Primary keywords
• service catalog
• internal service catalog
• service catalog meaning
• service catalog examples
• service catalog use cases
• enterprise service catalog
• cloud service catalog
• Kubernetes service catalog
• service catalog best practices

• service catalog implementation

• Related terminology

• provisioning template
• catalog API
• service definition
• SLO for catalog
• SLI metrics catalog
• catalog ownership
• policy-as-code catalog
• catalog governance
• catalog lifecycle
• catalog observability
• catalog telemetry
• catalog operator
• catalog CRD
• catalog reconciliation
• catalog drift
• catalog runbook
• catalog playbook
• catalog portal
• catalog CLI
• catalog federation
• catalog versioning
• catalog audit trail
• catalog cost estimate
• ephemeral environment catalog
• namespace self-service
• managed database offering
• serverless catalog offering
• feature flag catalog
• catalog dependency map
• catalog SLO design
• catalog monitoring
• catalog alerts
• catalog incident response
• catalog postmortem
• catalog security controls
• catalog RBAC integration
• catalog secrets management
• catalog CI/CD integration
• catalog GitOps pattern
• catalog operator pattern
• catalog centralized vs federated
• catalog observability bundle
• catalog template testing
• catalog approval workflow
• catalog TTL cleanup
• catalog cost optimization
• catalog service registry integration
• catalog artifact registry
• catalog ITSM integration
• catalog onboarding
• catalog metadata hygiene
• catalog lineage for data
• catalog data catalog integration
• catalog compliance automation
• catalog policy engine
• catalog OPA integration
• catalog prometheus metrics
• catalog grafana dashboards
• catalog audit logs
• catalog lifecycle manager
• catalog provisioning time
• catalog success rate
• catalog drift detection
• catalog owner contact
• catalog notification hooks
• catalog webhook events
• catalog delegation model
• catalog multi-cloud templates
• catalog managed service broker
• catalog cost showback
• catalog error budget
• catalog burn rate alerting
• catalog canary deployments
• catalog rollback automation
• catalog reconciliation loop
• catalog telemetry coverage
• catalog dashboard templates
• catalog alert bundling
• catalog false positive reduction
• catalog observability pitfalls
• catalog best practices checklist
• catalog implementation guide
• catalog maturity model
• catalog beginner checklist
• catalog advanced automation
• catalog integration map
• catalog tooling
• catalog service mesh integration
• catalog operator lifecycle
• catalog CRD design
• catalog schema design
• catalog metadata schema
• catalog owner rotation
• catalog runbook validation
• catalog game day
• catalog chaos testing
• catalog SLA vs SLO
• catalog product catalog differences
• catalog CMDB differences
• catalog registry differences
• catalog marketplace internal
• catalog dev self-service
• catalog production readiness
• catalog incident checklist
• catalog pre-production checklist
• catalog production checklist
• catalog implementation roadmap
• catalog adoption metrics
• catalog UX design
• catalog portal examples
• catalog API design
• catalog idempotency
• catalog transactional provisioning
• catalog compensating actions
• catalog orphan resource cleanup
• catalog reconciliation frequency
• catalog telemetry retention
• catalog audit retention
• catalog alert grouping
• catalog deduplication
• catalog suppression windows
• catalog owner on-call
• catalog automated remediation
• catalog secrets rotation
• catalog IAM bindings
• catalog tagging strategy
• catalog cost center mapping
• catalog rightsizing recommendations
• catalog self-service patterns
• catalog federation strategies
• catalog GitOps controller
• catalog ArgoCD integration
• catalog Flux integration
• catalog Prometheus instrumentation
• catalog Grafana provisioning
• catalog log aggregation
• catalog trace context passing
• catalog auditability
• catalog compliance logs
• catalog remediation automation
• catalog sample templates
• catalog developer portal
• catalog onboarding flow
• catalog metrics SLI examples
• catalog SLO starting points
• catalog approval latency metrics
• catalog cost estimate accuracy
• catalog monitoring strategies
• catalog observability best practices
• catalog failure mode analysis
• catalog incident remediation steps
• catalog postmortem responsibilities
• catalog ownership model
• catalog automation priority