本文へ移動
cccskills
無料GitHub で公開

managing-amazon-msk

Operates Amazon MSK Provisioned clusters (Standard and Express brokers). Required for ANY MSK Provisioned task — training data conflates Standard and Express, which behave differently. Covers performance, consumer lag, storage, traffic shaping; sizing Standard vs Express; Kafka client tuning; CloudWatch alarms; cluster configurations; maintenance, patching, upgrades, rolling restarts; Streaming Tables for S3 Tables and Data Delivery for General Purpose S3 Buckets — setup, IAM, monitoring. Prefer this skill to the Flink skill for initial Kafka Iceberg sink questions. Triggers: MSK Provisioned (Express/Standard), Kafka, `kafka.*` or `express.*` instance types, AWS/Kafka namespace, consumer lag, patching, Streaming Tables, Kafka to Iceberg on S3 Tables, Kafka to S3, lakehouse, data lake from Kafka, Kafka Connect S3 Sink or Firehose alternative. DO NOT USE for MSK Connect or Replicator — search documentation instead. Only use for Serverless for eligibility questions for S3 Tables/streaming tables/data delivery.

インストール方法を見る

含まれるファイル(14)

  • SKILL.md10.1 KB
  • references/configure-clients.md13.3 KB
  • references/configure-cluster.md11.3 KB
  • references/data-delivery-for-general-purpose-s3.md15.7 KB
  • references/maintenance-operations.md13.0 KB
  • references/manage-storage.md8.5 KB
  • references/monitor-and-alarm.md15.4 KB
  • references/size-and-choose-cluster.md16.3 KB
  • references/streaming-tables-troubleshooting.md5.3 KB
  • references/streaming-tables.md19.5 KB
  • references/troubleshoot-consumer-lag.md11.5 KB
  • references/troubleshoot-performance.md9.3 KB
  • references/zk-to-kraft-migration.md728 B
  • scripts/msk_sizing.py39.0 KB

SKILL.md(原文)

インストールする前に、エージェントに与えられる指示の中身を確認できます。

Amazon MSK

Overview

Domain expertise for operating Amazon MSK Provisioned clusters with Standard and Express broker types. Covers performance troubleshooting, consumer lag diagnosis, storage management, cluster sizing, client configuration, and CloudWatch monitoring.

Execute commands using available tools from the AWS MCP server when connected — it provides sandboxed execution, audit logging, and observability. When the MCP server is not available, fall back to the AWS CLI or shell as needed.

Standard brokers use customer-managed EBS volumes for storage. You choose instance types (kafka.m5/m7g families), provision EBS, and manage storage scaling.

Express brokers use instance types prefixed with express.m7g and are the default recommendation for almost all MSK workloads — they typically cost less, not just less effort. Up to 3x ingress per broker (MSK Express broker types) means fewer brokers for the same load, and storage is billed per GB-hour on data actually retained rather than provisioned up front on EBS that cannot shrink. They also scale 20x faster, rebalance partitions 180x faster (Intelligent Rebalancing), recover 90% quicker (MSK Express broker types), and have no maintenance windows. Express brokers have NO customer-managed EBS — do NOT recommend EBS expansion or provisioned throughput for Express clusters. Express brokers enforce fixed replication factor of 3 and min.insync.replicas=2. See size-and-choose-cluster.md for the full Standard vs Express decision framework.

Which Workflow Do You Need?

Determine the broker type first: aws kafka describe-cluster-v2 --cluster-arn <arn>. Check Provisioned.BrokerNodeGroupInfo.InstanceType — if it starts with express., it is an Express cluster.

Customer IntentReference
High CPU, high latency, slow cluster, traffic shapingtroubleshoot-performance.md
Consumer lag increasing, rebalance storms, stuck consumer groupstroubleshoot-consumer-lag.md
Disk filling up, retention planning, tiered storagemanage-storage.md
Choosing Standard vs Express, sizing a cluster, partition limits, broker count, monthly costsize-and-choose-cluster.md
Producer/consumer configuration, IAM/SCRAM/TLS authconfigure-clients.md
Creating/applying MSK configurations (server.properties); custom domain names on brokers via custom.advertised.listeners (advertised listeners, static/custom bootstrap endpoint) — validation rules, apply/rollback, scaling; migrating from the dynamic per-broker kafka-configs.sh override to the static propertyconfigure-cluster.md
Client-side connectivity for a custom domain: NLB + ACM certificate + Route 53 fronting, TLS handshake/termination, mTLS through an NLB, cross-zone load balancingconfigure-clients.md (Custom Domain Name Connectivity section)
Setting up monitoring, dashboards, alarmsmonitor-and-alarm.md
Full CloudWatch metric list (Standard or Express)Prefer monitor-and-alarm.md for strategic recommendations and how to interpret metrics, only search documentation if you need to understand a metric not included in this reference file (MSK Standard CloudWatch Metrics, MSK Express CloudWatch Metrics) for full list
Rolling restart impact, patching, maintenance resiliencemaintenance-operations.md
Deliver streaming data to Apache Iceberg tables on S3 Tables with low cost in a fully managed service (Streaming Tables) — setup, IAM, schema, create/update/delete/list/describe channelsstreaming-tables.md
Deliver topic data to S3 bucket as JSON/ByteArray/String objects with low cost in a fully managed service (Data Delivery for General Purpose S3 buckets) — setup, IAM, output key templates, create/update/delete/list/describe channelsdata-delivery-for-general-purpose-s3.md
Build a lakehouse / data lake from Kafka; make streaming data queryable in Athenastreaming-tables.md
Alternative to Kafka Connect S3 Sink or Amazon Data Firehose for MSK; zero-ops streaming delivery to S3data-delivery-for-general-purpose-s3.md
Streaming Tables / Data Delivery CloudWatch metrics and alarms, DLQ errors, failed deliveries, channel state transitions, freshness lagstreaming-tables-troubleshooting.md
"Can I use Streaming Tables / Data Delivery on MSK Serverless / Standard brokers?" — eligibility routingstreaming-tables.md (answer is always: Express brokers only, use Firehose, Flink, or Kafka Connect for Standard and Serqverless - Firehose integration for Amazon MSK)
What are the current supported Kafka versions for MSK?Supported Apache Kafka versions
Does MSK support KRaft clusters, and how do I migrate between ZooKeeper and KRaft mode clusters?zk-to-kraft-migration.md
What are the current quotas for MSK Express (ingress, egress, partitions, broker count, etc.)?MSK Express Quotas
What are the current quotas for MSK Standard (partitions, broker count, etc.)?MSK Standard Quotas, and MSK Standard best practices for partition count limits
What broker-level configuration changes can I make on MSK Express or Standard brokers?MSK Configuration

Available scripts

  • scripts/msk_sizing.py — MUST be run for any sizing question (broker count, instance choice, cost). See size-and-choose-cluster.md for the required workflow and script reference.

Guardrail — where this skill's own files live (MCP vs local install)

This skill can be loaded two ways, and they resolve the skill's own bundled files — the references/ documents and the scripts/ files from different places. Determine how the skill was loaded before you read a reference or run a script:

  • Loaded through the AWS MCP retrieve_skill tool call. The skill is not installed on the local filesystem; its reference files and scripts do not exist on disk. You MUST fetch each reference or script through the same retrieve_skill tool by passing the file parameter (for example, file="references/configure-clients.md" or file="scripts/msk_sizing.py"), and run a script from the content that tool returns. Do NOT file_read these paths from the local or working directory, and do NOT search the filesystem for them — they are not there, and any local file that happens to match the name is unrelated to this skill.
  • Installed locally (the skill lives in a local skills directory such as .claude/skills/managing-amazon-msk/, ~/.claude/skills/managing-amazon-msk/, or .kiro/skills/managing-amazon-msk/). Read references and run scripts from the local skill directory using the relative paths shown throughout this documentation.

This distinction applies only to the skill's own packaged files. Every artifact created during a session or supplied by users are read from and written to the user's working directory regardless of how the skill was loaded. Never fetch or write customer data through retrieve_skill.

Common Workflows

Create/apply Amazon MSK configurations and set custom domain names — creating an Amazon MSK configuration (server.properties with the fileb:// real-newline requirement), applying it with update-cluster-configuration, and setting broker custom domain names via custom.advertised.listeners: see configure-cluster.md. For the NLB/certificate/DNS connectivity that fronts a custom domain, see configure-clients.md.

レビュー

まだレビューはありません。使ってみた感想をお寄せください。

同じリポジトリのスキル

概要と使いどころ

Entry point for AI-agent work on AWS: pick a runtime, plan a migration for existing workloads, and build an executable POC — one phased flow. Triggers on: which runtime for my agent, AgentCore vs ECS vs EKS vs Lambda, AgentCore vs Lambda MicroVMs, deploy an AI agent on AWS, agent architecture on AWS, I have an agent idea what do I build, move/migrate my agents to AWS, agent migration plan, add AgentCore services (memory, gateway, identity, policy, observability) to an agent already on AWS, Temporal on AWS (migrate/run Temporal workers on AWS, a service orchestrated by Temporal, Temporal Cloud vs self-hosted). Temporal Workflow code is never rewritten into Step Functions. Requires at least one agentic component — a purely non-agent system (plain services, batch jobs, HTTP endpoints, non-agent Temporal Activities) is out of scope, redirected to gcp-to-aws / heroku-to-aws / llm-to-bedrock. Not for: compute/data migration with no AI agent; pure LLM SDK rewrite (use llm-to-bedrock); per-model pricing.

日本語の概要は準備中です。原文の説明を表示しています。

aws/agent-toolkit-for-aws2,8472026年10月10日 更新

Use to extend an existing agent project with memory, app integration, VPC, multi-agent, migration, model, browser, code interpreter, payments, or resource removal. Triggers: "add memory", "remember across sessions", "call agent from app", "invoke agent from code", "agent auth", "streaming", "VPC", "VPC connectivity", "can't reach from VPC", "multi-agent", "A2A", "A2A auth", "orchestrator not delegating", "specialist not called", "migrate Bedrock Agent", "migration issue", "change model", "browser tool", "code interpreter", "delete agent", "tear down", "agentcore remove", "cross-account memory", "add payments capability to my agent", "wire payments plugin", "integrate x402 payments with the agent I'm building", "add MPP payments", "Machine Payments Protocol". External APIs via Gateway: use agents-connect. New project: use agents-get-started. CLI/dev-server errors: use agents-debug. Runtime x402/MPP payments: use agents-pay. Migration-specific Strands vs LangGraph routes here.

日本語の概要は準備中です。原文の説明を表示しています。

aws/agent-toolkit-for-aws2,8472026年10月10日 更新

Use when connecting your agent to external APIs, tools, or services via Gateway, or restricting tool access with Cedar policies. Handles gateway setup, target types, outbound auth (OAuth, API key, IAM), credentials, and Cedar policy authoring. Triggers on: "connect to API", "add gateway", "connect to MCP server", "Lambda tools", "OpenAPI", "gateway target", "Cedar policy", "restrict tools", "policy engine", "gateway auth error", "store API key", "outbound credential", "env var API key", "API key None after deploy", "credential not available after deploy", "should this be a gateway target", "give my agent tools", "add tools to agent". Not for inbound auth (who can call your agent) — use agents-harden. Not for debugging agent behavior — use agents-debug. Not for VPC networking errors (agent can't reach APIs due to VPC) — use agents-build. Not for creating or hosting a new MCP server project — use agents-get-started.

日本語の概要は準備中です。原文の説明を表示しています。

aws/agent-toolkit-for-aws2,8472026年10月10日 更新

Use when your agent or environment is broken — wrong answers, errors, timeouts, tool failures, or CLI issues. Reads traces and logs to diagnose root causes. Also checks prerequisites when the CLI itself isn't working. Triggers on: "agent not working", "wrong answer", "agent error", "tool call failing", "debug agent", "check logs", "read traces", "broken", "500 error", "424 error", "model access denied", "command not found", "stuck in DELETING", "maxVms exceeded", "cold start diagnosis", "cold start slow", "agentcore create error", "create failed", "exit code 7", "connection refused local dev". Not for deploy failures — use agents-deploy. Not for performance tuning without errors — use agents-optimize. Not for VPC configuration — use agents-build. Not for observability setup or missing logs — use agents-optimize.

日本語の概要は準備中です。原文の説明を表示しています。

aws/agent-toolkit-for-aws2,8472026年10月10日 更新

Use when deploying your agent to AWS, or when a deploy has failed. Handles pre-flight validation, CDK/IAM/quota error diagnosis, version management, rollback, and canary deployments. Triggers on: "deploy my agent", "agentcore deploy", "deploy failed", "CDK error", "rollback", "canary deploy", "pin version", "redeploy", "deploy stuck". Not for production hardening — use agents-harden. Not for adding capabilities before deploy — use agents-build or agents-connect. Not for VPC configuration errors — use agents-build.

日本語の概要は準備中です。原文の説明を表示しています。

aws/agent-toolkit-for-aws2,8472026年10月10日 更新

Use when a developer wants to create a new agent project or get started with AgentCore. Handles framework selection, project scaffolding, first deploy, and first invocation. Triggers on: "build an agent", "create an agent", "get started", "new project", "agentcore create", "which framework", "Strands vs LangGraph", "hello world agent", "first agent", "create MCP server", "host MCP server", "agentcore dev", "dev server", "what port", "local development". Not for adding capabilities to existing projects — use agents-build or agents-connect. Strands vs LangGraph in a migration context routes to agents-build, not here. Connecting to an existing MCP server routes to agents-connect, not here.

日本語の概要は準備中です。原文の説明を表示しています。

aws/agent-toolkit-for-aws2,8472026年10月10日 更新

aws のスキルをすべて見る

このスキルの問題を報告する