adaptive-model-cascade — vibes-plug Skill

English | Bahasa Indonesia

---

<a name="english"></a>

English

Orchestration & Integration

Connects and orchestrates with llm-finops-router, frontier-ai-models-expert, context-window-engineer, kv-cache-prefix-optimizer, llm-observability-expert, ai-llm-integration-expert, vercel-ai-sdk-expert, and test-time-compute-optimizer to implement cost-optimal model selection across all AI-powered features.

Description

Production architecture for routing AI requests to the optimal model tier based on real-time task complexity assessment. Instead of sending every request to the most expensive frontier model, this skill implements a cascade: try the fastest/cheapest model first, evaluate output quality via automated judges, and escalate to more capable (expensive) models only when the cheaper tier fails the quality gate. Achieves 40-60% cost reduction for typical production workloads where 60-70% of requests are simple enough for smaller models.

Trigger Conditions

Activate this skill when:

• Running AI features in production with significant token spend (>$500/month).
• Building applications where response quality varies by task complexity (some queries need GPT Astra 6, others need only Flash).
• Implementing cost optimization for multi-model AI architectures.
• Designing fallback and retry strategies across model providers.

---

Core Concepts & Patterns

1. The Model Cascade Architecture

USER REQUEST
      │
      ▼
┌──────────────────────┐
│ COMPLEXITY SCORER    │  Classify request complexity: LOW / MEDIUM / HIGH / ULTRA
│ (Fast heuristic or   │
│  lightweight model)   │
└──────┬───────────────┘
       │
       ├── LOW ──────────► Gemini 4 Flash-Lite / Claude Haiku ($0.025/1M)
       │                        │
       │                   ┌────▼────┐
       │                   │ QUALITY  │  Pass? → Return response
       │                   │ GATE     │  Fail? → Escalate to MEDIUM
       │                   └─────────┘
       │
       ├── MEDIUM ───────► Gemini 4 Flash / Claude Sonnet ($0.30/1M)
       │                        │
       │                   ┌────▼────┐
       │                   │ QUALITY  │  Pass? → Return response
       │                   │ GATE     │  Fail? → Escalate to HIGH
       │                   └─────────┘
       │
       ├── HIGH ─────────► Gemini 4 Pro / Claude Opus ($3-15/1M)
       │                        │
       │                   ┌────▼────┐
       │                   │ QUALITY  │  Pass? → Return response
       │                   │ GATE     │  Fail? → Escalate to ULTRA
       │                   └─────────┘
       │
       └── ULTRA ────────► GPT Astra 6 + Extended Thinking ($15/1M)
                                │
                           Return response (final tier, no escalation)

2. Complexity Scorer (TypeScript Implementation)

export type ComplexityLevel = 'LOW' | 'MEDIUM' | 'HIGH' | 'ULTRA';

export interface ComplexitySignals {
  tokenCount: number;
  hasCodeGeneration: boolean;
  requiresReasoning: boolean;
  hasMultiStep: boolean;
  domainSpecificity: 'general' | 'technical' | 'specialized';
  requiresCreativity: boolean;
  hasToolCalls: boolean;
  contextSize: number;
}

export function scoreComplexity(signals: ComplexitySignals): ComplexityLevel {
  let score = 0;

  if (signals.tokenCount > 2000) score += 1;
  if (signals.tokenCount > 8000) score += 1;
  if (signals.hasCodeGeneration) score += 2;
  if (signals.requiresReasoning) score += 2;
  if (signals.hasMultiStep) score += 1;
  if (signals.domainSpecificity === 'technical') score += 1;
  if (signals.domainSpecificity === 'specialized') score += 2;
  if (signals.requiresCreativity) score += 1;
  if (signals.hasToolCalls) score += 1;
  if (signals.contextSize > 100_000) score += 1;
  if (signals.contextSize > 500_000) score += 2;

  if (score <= 2) return 'LOW';
  if (score <= 5) return 'MEDIUM';
  if (score <= 8) return 'HIGH';
  return 'ULTRA';
}

3. Quality Gate Evaluator

export interface QualityGateResult {
  passed: boolean;
  score: number;
  reasons: string[];
}

export async function evaluateQualityGate(
  request: string,
  response: string,
  tier: ComplexityLevel
): Promise<QualityGateResult> {
  const checks: Array<{ name: string; passed: boolean }> = [];

  // Structural completeness: does the response address all parts of the request?
  const hasSubstance = response.length > 50 && !response.includes("I don't know");
  checks.push({ name: 'substance', passed: hasSubstance });

  // Code validity: if code was requested, does the output contain valid-looking code blocks?
  if (request.toLowerCase().includes('code') || request.toLowerCase().includes('function')) {
    const hasCodeBlocks = /```[\s\S]+```/.test(response);
    checks.push({ name: 'code_blocks', passed: hasCodeBlocks });
  }

  // Refusal detection: did the model refuse or hedge excessively?
  const refusalPatterns = /i('m| am) (unable|not able|can't|cannot)\s+(to|help)/i;
  const noRefusal = !refusalPatterns.test(response);
  checks.push({ name: 'no_refusal', passed: noRefusal });

  // Length adequacy: response should be proportional to request complexity
  const minExpectedLength = tier === 'LOW' ? 100 : tier === 'MEDIUM' ? 300 : 500;
  const adequateLength = response.length >= minExpectedLength;
  checks.push({ name: 'adequate_length', passed: adequateLength });

  const passedCount = checks.filter((c) => c.passed).length;
  const score = passedCount / checks.length;
  const failedReasons = checks.filter((c) => !c.passed).map((c) => c.name);

  return {
    passed: score >= 0.75,
    score,
    reasons: failedReasons,
  };
}

4. Cascade Router with Escalation

import { generateText } from 'ai';

export interface ModelTier {
  level: ComplexityLevel;
  modelId: string;
  provider: string;
  costPer1MTokens: number;
  maxRetries: number;
}

const MODEL_TIERS: ModelTier[] = [
  { level: 'LOW', modelId: 'gemini-4-flash-lite', provider: 'google', costPer1MTokens: 0.025, maxRetries: 1 },
  { level: 'MEDIUM', modelId: 'claude-5.5-sonnet', provider: 'anthropic', costPer1MTokens: 3.0, maxRetries: 1 },
  { level: 'HIGH', modelId: 'gemini-4-pro', provider: 'google', costPer1MTokens: 1.25, maxRetries: 1 },
  { level: 'ULTRA', modelId: 'gpt-astra-6', provider: 'openai', costPer1MTokens: 15.0, maxRetries: 2 },
];

const TIER_ORDER: ComplexityLevel[] = ['LOW', 'MEDIUM', 'HIGH', 'ULTRA'];

export async function cascadeRequest(
  prompt: string,
  signals: ComplexitySignals,
  systemPrompt: string
): Promise<{ text: string; model: string; totalCost: number; escalations: number }> {
  const startTier = scoreComplexity(signals);
  const startIndex = TIER_ORDER.indexOf(startTier);
  let escalations = 0;

  for (let i = startIndex; i < MODEL_TIERS.length; i++) {
    const tier = MODEL_TIERS[i];

    const result = await generateText({
      model: getModelProvider(tier.provider, tier.modelId),
      system: systemPrompt,
      prompt,
    });

    const quality = await evaluateQualityGate(prompt, result.text, tier.level);

    if (quality.passed || i === MODEL_TIERS.length - 1) {
      return {
        text: result.text,
        model: tier.modelId,
        totalCost: calculateCost(result.usage, tier.costPer1MTokens),
        escalations,
      };
    }

    escalations++;
  }

  throw new Error('All model tiers exhausted');
}

function calculateCost(usage: { promptTokens: number; completionTokens: number }, costPer1M: number): number {
  return ((usage.promptTokens + usage.completionTokens) / 1_000_000) * costPer1M;
}

5. Cost Savings Projection

Workload Distribution	Without Cascade	With Cascade	Savings
60% simple, 30% medium, 10% complex	$1,500/month (all Opus)	$450/month	70%
40% simple, 40% medium, 20% complex	$2,000/month (all Pro)	$800/month	60%
20% simple, 50% medium, 30% complex	$3,000/month (mixed)	$1,500/month	50%

---

Best Practices

1. Start with the Cheapest Tier: Always try the fastest model first. Escalation costs less than defaulting to the most expensive model.
2. Quality Gates Must Be Fast: The evaluator should add <200ms overhead. Use heuristic checks, not another LLM call, for the gate.
3. Log Every Escalation: Track escalation frequency per feature to identify tasks that always require frontier models (route directly).
4. Cache Before Cascade: Apply kv-cache-prefix-optimizer patterns to reduce base cost at every tier.
5. A/B Test Tier Boundaries: Use feature-flag-analytics-expert to experiment with complexity score thresholds.

---

Common Pitfalls to Avoid

Anti-Pattern	Consequence	Remedy
Sending everything to the frontier model	5-10x unnecessary cost	Implement complexity scoring and cascade
Using another LLM call as the quality gate	Gate cost exceeds savings from cascading	Use fast heuristic checks (regex, length, structure)
No fallback for model provider outages	Total feature outage when one provider is down	Cascade across providers, not just model sizes
Static tier assignment without telemetry	Suboptimal routing as usage patterns evolve	Monitor and adjust complexity thresholds weekly

---

Integration with Other Skills (MANDATORY)

• llm-finops-router — Feed cascade cost data to the FinOps dashboard for ROI tracking.
• llm-observability-expert — Log all cascade decisions, escalations, and quality gate results.
• frontier-ai-models-expert — Reference model capabilities for tier assignment.
• context-window-engineer — Route to larger-context models when context exceeds smaller model limits.
• kv-cache-prefix-optimizer — Apply cache optimization at each tier to minimize base cost.
• vercel-ai-sdk-expert — Implement cascade routing using Vercel AI SDK's multi-provider support.

Referenced By Orchestrators (MANDATORY)

• brainstorming — Added to "AI & LLM Integration" matrix row.
• zero-to-prod-orchestrator — Integrated in Phase 4 (Backend APIs, Microservices & AI Agents).

---

<a name="bahasa-indonesia"></a>

Bahasa Indonesia

Integrasi Orkestrasi

Terhubung dan mengorkestrasi dengan llm-finops-router, frontier-ai-models-expert, context-window-engineer, kv-cache-prefix-optimizer, llm-observability-expert, ai-llm-integration-expert, vercel-ai-sdk-expert, dan test-time-compute-optimizer untuk mengimplementasikan pemilihan model optimal dari segi biaya di seluruh fitur berbasis AI.

Deskripsi

Arsitektur produksi untuk merutekan permintaan AI ke tier model optimal berdasarkan penilaian kompleksitas tugas secara real-time. Alih-alih mengirim setiap permintaan ke model frontier termahal, skill ini mengimplementasikan kaskade: coba model tercepat/termurah dulu, evaluasi kualitas output via penilai otomatis, dan eskalasi ke model lebih capable (mahal) hanya jika tier murah gagal melewati gerbang kualitas. Menghemat 40-60% biaya untuk beban kerja produksi tipikal.

Kondisi Pemicu

Aktifkan skill ini ketika:

• Menjalankan fitur AI di produksi dengan pengeluaran token signifikan (>$500/bulan).
• Membangun aplikasi di mana kualitas respons bervariasi menurut kompleksitas tugas.
• Mengimplementasikan optimasi biaya untuk arsitektur AI multi-model.

---

Konsep Inti & Pola Praktik

1. Arsitektur Kaskade Model

Empat tier berurutan: LOW (Flash-Lite/Haiku), MEDIUM (Flash/Sonnet), HIGH (Pro/Opus), ULTRA (Astra 6 + Extended Thinking). Setiap tier memiliki gerbang kualitas. Jika gagal, eskalasi ke tier berikutnya.

2. Penilaian Kompleksitas

Skor numerik berdasarkan sinyal: jumlah token, kebutuhan generasi kode, penalaran, langkah multi-step, spesifisitas domain, kreativitas, tool calls, dan ukuran konteks.

3. Gerbang Kualitas

Evaluasi cepat berbasis heuristik (<200ms): substansi respons, blok kode valid, deteksi penolakan, dan kecukupan panjang. Tidak menggunakan panggilan LLM tambahan.

---

Praktik Terbaik

1. Mulai dari Tier Termurah: Selalu coba model tercepat dulu.
2. Gerbang Kualitas Harus Cepat: Evaluator harus menambahkan <200ms overhead.
3. Log Setiap Eskalasi: Lacak frekuensi eskalasi per fitur.
4. Cache Sebelum Kaskade: Terapkan pola kv-cache-prefix-optimizer untuk mengurangi biaya dasar di setiap tier.

---

Integrasi dengan Skill Lain (WAJIB)

• llm-finops-router — Kirim data biaya kaskade ke dashboard FinOps.
• llm-observability-expert — Log semua keputusan kaskade, eskalasi, dan hasil gerbang kualitas.
• frontier-ai-models-expert — Referensi kemampuan model untuk penugasan tier.
• context-window-engineer — Rutekan ke model berkonteks lebih besar saat konteks melebihi batas model kecil.

Direferensikan oleh Orchestrator (WAJIB)

• brainstorming — Ditambahkan ke baris "AI & LLM Integration" pada Matriks Orkestrasi.
• zero-to-prod-orchestrator — Diintegrasikan di Fase 4 (Backend APIs, Microservices & AI Agents).