Merag Nokhiz

Systems Architect & Engineer
link terminal
July 2026

The IDP Is Now the AI Delivery Platform

Platform Engineering and AI delivery are converging. The teams that understand this early are building one pipeline for app developers, ML engineers, and agents.

Architecture aiinfrastructure

The IDP Is Now the AI Delivery Platform

July 2026 · nokhiz.github.io

TL;DR — 6 Central Insights ⚡

#Insight
1Platform Engineering has crossed the chasm — 80% of large software orgs will have a dedicated platform team by end of 2026
273% of platform teams already ship AI assistants through their IDP — the convergence is not coming, it is here
3ML pipelines were always second-class tooling; the IDP is the organizational mechanism to finally fix that
4Unified delivery means one pipeline for app developers, ML engineers, and agents — not three workflows bolted together
5Pre-deployment cost gates move FinOps from reactive dashboards to structural prevention inside the platform
6Regulated environments add a non-negotiable layer: audit trails, role-scoped AI capabilities, and model governance must live in the golden path

1. Platform Engineering’s Moment 🏗️

💡 Key Message: Platform Engineering stopped being a niche practice and became the default. Now it is inheriting a problem it was not originally designed for.

Gartner’s latest forecast: by end of 2026, 80% of large software engineering organizations will have a dedicated platform team — up from 45% in 2022.

The driver was straightforward. Developer productivity was suffering under the weight of Kubernetes complexity, cloud configuration sprawl, and inconsistent environments across teams. The Internal Developer Platform emerged as the organizational answer — a curated, self-service layer that abstracts infrastructure without hiding it entirely.

1. What 🏛️ an IDP Actually Is

An IDP is not a portal. It is not a wiki with CI/CD links bolted on.

  • Self-service provisioning — developers request environments, databases, and services through declarative templates without filing tickets
  • Golden paths — curated, opinionated workflows that encode organizational standards: security policies, cost limits, compliance controls
  • Unified observability — logs, metrics, and traces wired in by default, not configured per team
  • RBAC-enforced guardrails — what you can do depends on your role, not on who you know in the platform team

✏️ Key Rule: A good IDP makes the right thing the easy thing. It does not eliminate choice — it eliminates friction on choices that are already decided.

2. Shift ⚙️ What Changed in 2025–2026

The maturity of the IDP market crossed a threshold. Backstage became the de facto foundation. Port, Cortex, and Humanitec built opinionated layers on top. The conversation shifted from “should we build a platform?” to “how do we measure its impact?”

Platform teams began reporting in business terms: developer onboarding time, deployment frequency, change failure rate, mean time to restore. The platform became a product with customers — the internal engineering organization.

2. AI Is Now a Platform Problem 🤖

💡 Key Message: AI workloads are not special. They have the same infrastructure needs as everything else — plus a set of requirements the classic IDP was never designed for.

73% of platform teams are now shipping AI assistants through their IDP. The question is no longer whether AI ends up inside the platform — it already has. The question is whether the platform was designed to handle it. Most were not. The gap is structural:

  • App delivery assumes deterministic code — same input, same output, test coverage proves correctness
  • ML delivery assumes probabilistic models — same input, different output distributions, evaluation not unit testing
  • Agent delivery assumes goal-directed runtime behavior — the system plans and iterates, it does not just execute

Three delivery models. Three different risk profiles. Three different failure modes. Running them through the same pipeline without accounting for these differences creates invisible risk — AI features that ship fast but fail in ways the standard SLO stack cannot detect.

%%{init: {'theme': 'base', 'themeVariables': {'primaryColor': '#1e2433', 'primaryTextColor': '#f0f4ff', 'primaryBorderColor': '#3a4460', 'lineColor': '#6b7fa3', 'secondaryColor': '#252d3d', 'tertiaryColor': '#1a2030', 'background': '#161c2d', 'mainBkg': '#1e2433', 'nodeBorder': '#3a4460', 'clusterBkg': '#252d3d', 'titleColor': '#f0f4ff', 'edgeLabelBackground': '#252d3d', 'fontFamily': 'monospace'}}}%%
graph TD
    subgraph CLASSIC["Classic IDP (2023–2024)"]
        direction TB
        A["🖥️ App Developer
        deterministic code"]
        B["🛤️ Golden Path
        CI/CD · RBAC · Observability"]
        C["☁️ Cloud Infrastructure
        K8s · Services · Databases"]
        A --> B --> C
    end

    subgraph AI_IDP["AI-Ready IDP (2026)"]
        direction TB
        D["🖥️ App Developer
        deterministic code"]
        E["🔬 ML Engineer
        probabilistic models"]
        F["🤖 Agent Workload
        goal-directed runtime"]
        G["🛤️ Unified Platform
        CI/CD · Eval · Cost Gate · RBAC · Traces"]
        H["☁️ Cloud Infrastructure
        K8s · GPU · Vector DB · Model Registry"]
        D --> G
        E --> G
        F --> G
        G --> H
    end

    style A fill:#1e2433,stroke:#3a4460,color:#f0f4ff
    style B fill:#252d3d,stroke:#3a4460,color:#f0f4ff
    style C fill:#1a2030,stroke:#3a4460,color:#f0f4ff
    style D fill:#1e2433,stroke:#3a4460,color:#f0f4ff
    style E fill:#252d3d,stroke:#3a4460,color:#f0f4ff
    style F fill:#1a2d1a,stroke:#3a6a3a,color:#f0f4ff
    style G fill:#2a1f3d,stroke:#5a3a7a,color:#f0f4ff
    style H fill:#1a2030,stroke:#3a4460,color:#f0f4ff

3. The Unified Pipeline 🔄

💡 Key Message: By end of 2026, mature platforms will offer a single delivery pipeline serving app developers, ML engineers, and agents. “Unified” is an architectural decision — not a product feature.

3. Problem 🛠️ ML Was Always Second-Class Tooling

In most enterprises, ML delivery still runs through ad hoc tooling stitched together by individual data science teams. A Jupyter notebook for experiments. A manual handoff to an MLflow instance nobody fully owns. A Kubernetes manifest written by whoever happened to know YAML that week. A monitoring dashboard configured once and never updated.

This is not an ML problem. It is a platform problem. The platform team never claimed ML delivery as their domain, so it ended up owned by everyone and maintained by no one.

📌 Example: An automotive OEM running quality-control models in production had five separate deployment scripts for five models — each written by a different contractor, none integrated with the central IDP, none reporting to the standard SRE stack. Incidents were discovered by downstream teams, not by alerts.

4. Design ⚙️ What a Unified Pipeline Actually Requires

Unifying app, ML, and agent delivery on one platform requires explicit decisions about four concerns:

ConcernApp DeliveryML DeliveryAgent Delivery
ValidationUnit / integration testsEval datasets + accuracy metricsGoal completion + safety checks
VersioningGit commit SHAModel artifact + data versionPrompt version + tool manifest
ObservabilityLatency · error rate · throughputDrift · accuracy degradationTask completion · token cost · reasoning traces
Rollback triggerError rate spikeMetric regression thresholdCost threshold or trust boundary violation

The platform team that ignores these differences will ship a unified portal with fragmented delivery underneath. The one that builds for them owns AI delivery end-to-end.

4. What the AI-Ready IDP Needs 🏗️

💡 Key Message: Four capabilities separate an IDP that supports AI from one that was retrofitted to tolerate it.

5. Capability 🛠️ AI Workload Onboarding

Onboarding an AI workload should follow the same self-service path as onboarding an application. Templates for model serving (vLLM, TGI, Azure ML endpoints), vector database provisioning, and GPU node pools — available through the same catalog as a PostgreSQL instance or a Kafka topic.

Teams that require a ticket to provision an A100 node group will slow down every AI project in the organization. The platform makes it self-service or the platform gets bypassed — and bypasses become shadow infrastructure.

6. Capability 💰 Pre-Deployment Cost Gates

FinOps has operated as a reactive discipline: dashboards showing what you spent, alerts when you went over budget, quarterly reviews that arrive too late to change anything.

The IDP changes this structurally. Pre-deployment cost gates — policy checks that evaluate projected token spend, GPU hour consumption, or inference cost per request before a workload ships — move cost control from reporting to prevention.

📋 Note: This is not optional in 2026. The extreme case is an organization burning its entire annual AI budget before Q2. The moderate case is a team that ships a fine-tuned model with no inference cost estimate and discovers the problem six weeks into production traffic. Both are preventable. Neither is a budgeting failure — both are platform failures.

7. Capability 🔍 Non-Deterministic Observability

Standard SRE metrics — p99 latency, error rate, request throughput — are necessary but insufficient for AI workloads. A model can respond in 200ms with content that is factually wrong, off-tone, or in violation of a compliance policy. None of that shows up in a latency histogram.

The AI-ready IDP instruments three additional layers:

  • Evaluation pipelines — scheduled runs against golden datasets to detect model drift before users report it
  • Reasoning traces — for agent workloads, structured logs of goal decomposition, tool calls, and intermediate decisions
  • Cost-per-task tracking — token consumption attributed to specific agent tasks, not aggregated at service level where it becomes unactionable

5. The Regulated Environment Constraint 🔐

💡 Key Message: In regulated industries, the AI-ready IDP is not a nice-to-have — it is the only governance mechanism that actually works at platform scale.

MedTech, automotive OEMs, and financial services carry a constraint that consumer tech does not: every AI system that touches a regulated process must produce an audit trail. Not a log file. An auditable, immutable, role-attributed record of what decision was made, with which model version, on which data, approved by whom.

That requirement cannot be bolted on after deployment. It has to be part of the golden path — the template every team uses when they onboard an AI workload to the IDP. If the template does not enforce it, some teams will implement it correctly and most will not.

The platform team in a regulated environment is the enforcement mechanism. The IDP golden path is where compliance policy becomes operational reality — anything outside it is outside the governance perimeter.

%%{init: {'theme': 'base', 'themeVariables': {'primaryColor': '#1e2433', 'primaryTextColor': '#f0f4ff', 'primaryBorderColor': '#3a4460', 'lineColor': '#6b7fa3', 'secondaryColor': '#252d3d', 'tertiaryColor': '#1a2030', 'background': '#161c2d', 'mainBkg': '#1e2433', 'nodeBorder': '#3a4460', 'clusterBkg': '#252d3d', 'titleColor': '#f0f4ff', 'edgeLabelBackground': '#252d3d', 'fontFamily': 'monospace'}}}%%
graph LR
    subgraph ONBOARD["AI Workload Onboarding"]
        direction TB
        REQ["📋 Developer Request
        model type · use case · env"]
        GATE1["💰 Cost Gate
        projected spend vs. budget policy"]
        GATE2["🔐 Compliance Check
        audit trail · data class · RBAC"]
        TMPL["🛤️ Golden Path Template
        serving · eval pipeline · observability"]
    end

    subgraph RUNTIME["Runtime"]
        direction TB
        SERVE["🤖 Model / Agent
        serving endpoint"]
        OBS["🔍 Observability
        drift · traces · cost-per-task"]
        AUDIT["📁 Audit Log
        immutable · role-attributed"]
    end

    REQ --> GATE1
    GATE1 -->|"approved"| GATE2
    GATE2 -->|"cleared"| TMPL
    TMPL --> SERVE
    SERVE --> OBS
    SERVE --> AUDIT

    style REQ fill:#1e2433,stroke:#3a4460,color:#f0f4ff
    style GATE1 fill:#2d1f1a,stroke:#7a5a3a,color:#f0f4ff
    style GATE2 fill:#2d1a1a,stroke:#7a3a3a,color:#f0f4ff
    style TMPL fill:#2a1f3d,stroke:#5a3a7a,color:#f0f4ff
    style SERVE fill:#1a2d1a,stroke:#3a6a3a,color:#f0f4ff
    style OBS fill:#252d3d,stroke:#3a4460,color:#f0f4ff
    style AUDIT fill:#1a2030,stroke:#3a4460,color:#f0f4ff

6. The Bottom Line 🚀

💡 Key Message: The IDP’s next mandate is AI delivery. Teams that recognize this now build the platform once. Teams that do not will rebuild it after the AI sprawl becomes ungovernable.

AI delivery is not a separate domain that lives next to platform engineering — it is landing inside the platform, whether the platform was designed for it or not.

The IDP was built to make app delivery repeatable and governed. That mandate did not change. The surface area did.

Tags: platform-engineering · ai · infrastructure · idp