本文へ移動
cccskills
無料GitHub で公開

connecting-to-data-source

Create and troubleshoot AWS Glue connections to JDBC databases (Oracle, SQL Server, PostgreSQL, MySQL, RDS), Redshift, Snowflake, and BigQuery. Gathers connection hints from user, discovers existing connections and RDS/Redshift candidates, registers credentials in Secrets Manager or IAM DB auth, configures VPC, and tests. Triggers on: connect to database, set up Glue connection, register data source, connect to Snowflake/BigQuery/RDS, connection timeout, test connection, troubleshoot connection. Do NOT use for moving data (use ingesting-into-data-lake), creating tables (use creating-data-lake-table), queries (use querying-data-lake), catalog exploration (use exploring-data-catalog), or SaaS (Salesforce, ServiceNow, SAP, MongoDB, Kafka).

aws/agent-toolkit-for-aws
インストール方法を見る

含まれるファイル（8）

  • SKILL.md8.7 KB
  • references/bigquery-setup.md2.6 KB
  • references/credential-security.md4.3 KB
  • references/discovery.md3.6 KB
  • references/jdbc-setup.md3.5 KB
  • references/network-setup.md3.9 KB
  • references/snowflake-setup.md2.6 KB
  • references/troubleshooting.md7.6 KB

SKILL.md（原文）

インストールする前に、エージェントに与えられる指示の中身を確認できます。

Connect to Data Source

Register an external data source with AWS Glue so downstream skills (ingesting-into-data-lake) can move data from it. A Glue connection stores the network config, driver, and credential reference for one source. Create once per source, reuse across jobs.

Philosophy

A connection is a named pipe, not a pipeline. This skill produces a tested, reusable Glue connection. It does not move data.

Common Tasks

You MUST execute commands using AWS MCP server tools when connected -- they provide validation, sandboxed execution, and audit logging. Fall back to AWS CLI only if MCP is unavailable. You MUST explain each step before executing.

Workflow

1. Verify Dependencies and Context

  • You MUST check whether AWS MCP tools or AWS CLI are available and inform the user if missing
  • You MUST confirm target AWS region and verify credentials with aws sts get-caller-identity

2. Classify the Source

Ask the user which source type they want to connect to, or infer from hints:

User says...Source typeConnection typeReference
"Oracle", "SQL Server", "Postgres", "MySQL", "RDS <engine>"JDBC databaseJDBCjdbc-setup.md
"Redshift", "my cluster", "my data warehouse on AWS"RedshiftJDBCjdbc-setup.md (Redshift section)
"Snowflake"SnowflakeSNOWFLAKEsnowflake-setup.md
"BigQuery", "Google analytics warehouse"BigQueryBIGQUERYbigquery-setup.md

If the user names DynamoDB or a local file, stop and tell them: DynamoDB is read directly by Glue without a connection, and local files belong in the ingesting-into-data-lake skill's local-upload workflow.

3. Gather Connection Hints from the User

You MUST ask for hints the user can provide -- do not guess.

For all sources:

  • Desired connection name (lowercase, hyphens: oracle-prod-sales, snowflake-analytics)
  • Existing Secrets Manager secret, or create one
  • Is source reachable from a Glue VPC (same, peered, VPN, Direct Connect)

JDBC: hostname/endpoint, port, database, whether RDS/Aurora/self-managed, IAM DB auth enabled (Aurora/RDS MySQL/Postgres), SSL required.

Snowflake: account identifier, warehouse, role, default database, auth (password, key-pair, OAuth).

BigQuery: GCP project ID, location, whether service account JSON is provisioned.

4. Discover Existing Connections and Candidate Sources

Check what exists before creating.

Existing Glue connections:

aws glue get-connections --filter ConnectionType=<TYPE> --region <REGION>

If a suitable one exists, confirm and skip to Step 7.

Candidate sources in account (JDBC/Redshift only):

  • RDS: aws rds describe-db-instances
  • Aurora: aws rds describe-db-clusters
  • Redshift: aws redshift describe-clusters

Present candidates to user; let them pick. See discovery.md.

5. Register Credentials

You MUST encourage AWS Secrets Manager over plaintext passwords. You SHOULD prefer IAM database authentication where supported (Aurora/RDS MySQL and PostgreSQL, Redshift). See credential-security.md.

  • You MUST confirm with user before creating a new Secrets Manager secret
  • You MUST NOT write plaintext credentials into chat or logs
  • For IAM DB auth, no secret is needed

6. Create the Glue Connection

Follow the source-specific reference for connection properties:

aws glue create-connection --connection-input '<JSON>' --region <REGION>

Private sources require PhysicalConnectionRequirements (SubnetId, SecurityGroupIdList, AvailabilityZone). See network-setup.md.

7. Test the Connection

You MUST test before handing off. Testing is two-phase: a quick API check, then an engine-level verification.

Phase A: Glue TestConnection (network and credential sanity check)

aws glue test-connection --connection-name <NAME> --region <REGION>

This validates that Glue can reach the source and authenticate. It does NOT prove the connection works end-to-end with the query engine the user plans to use.

Phase B: Engine-level verification

After TestConnection passes, verify the connection works with the user's intended engine by running a minimal query through it:

  • Glue ETL (default): Run a smoke-test Glue job that reads one row via the connection. See troubleshooting.md.
  • Athena: If the user plans to query via Athena with a federated connector, run a SELECT 1 through the Athena connection to confirm the Lambda-based connector can reach the source.
  • Glue Crawler: If the user plans to crawl the source, run a test crawl on a single table.

Phase B catches issues that TestConnection misses: driver compatibility at job runtime, catalog configuration, Spark-level serialization, and engine-specific auth flows (e.g., Snowflake SNOWFLAKE type works in ETL but not via JDBC crawlers).

On success in both phases, tell user the connection name is ready for ingesting-into-data-lake. On failure in either phase, Step 8.

8. Troubleshoot (only if test failed)

Diagnose in order: network, credentials, driver. See troubleshooting.md.

Constraints:

  • You MUST check VPC routing, security groups, and S3 VPC endpoint before blaming credentials
  • You MUST verify Glue role can read the Secrets Manager secret
  • You MUST NOT rotate credentials without user confirmation

Argument Routing

  • No args: Walk through Steps 1-7 interactively
  • Source type keyword (e.g., snowflake, oracle): Skip to Step 2 with the type prefilled
  • Existing connection name: Skip to Step 7 (test) then Step 8 if failing
  • Hostname or RDS endpoint: Skip to Step 4 with the candidate prefilled

Gotchas

  • Glue's SNOWFLAKE connection type is distinct from JDBC configured for Snowflake. You MUST use SNOWFLAKE for Spark ETL jobs; do not use JDBC.
  • Connection names are immutable. Choose carefully.
  • PhysicalConnectionRequirements.AvailabilityZone MUST match the subnet's AZ or the connection fails at job runtime, not creation time.
  • IAM database authentication tokens expire in 15 minutes. The Glue job generates a fresh token on each connection; do not cache.
  • An S3 VPC gateway endpoint MUST exist in the VPC used by private-source connections. Without it, Glue jobs cannot read their scripts or write results to S3.

Troubleshooting

ErrorLikely causeFix
Connect timed outVPC routing, SG rule, or NAT gateway missingSee troubleshooting.md
Access denied for user / ORA-01017Credentials wrong, Secrets Manager access missing, or IAM DB auth misconfiguredSee troubleshooting.md
No suitable driver foundCustom driver JAR not set or wrong class nameSee troubleshooting.md
SSL handshake failedJDBC_ENFORCE_SSL mismatch between Glue and sourceSee troubleshooting.md
UnableToFindVpcEndpointS3 VPC endpoint missingCreate S3 gateway endpoint in the connection's VPC

References

レビュー

まだレビューはありません。使ってみた感想をお寄せください。

ログインしてレビューを書く

同じリポジトリのスキル

概要と使いどころ

agent-advisor

無料

Entry point for AI-agent work on AWS: pick a runtime, plan a migration for existing workloads, and build an executable POC — one phased flow. Triggers on: which runtime for my agent, AgentCore vs ECS vs EKS vs Lambda, AgentCore vs Lambda MicroVMs, deploy an AI agent on AWS, agent architecture on AWS, I have an agent idea what do I build, move/migrate my agents to AWS, agent migration plan, add AgentCore services (memory, gateway, identity, policy, observability) to an agent already on AWS, Temporal on AWS (migrate/run Temporal workers on AWS, a service orchestrated by Temporal, Temporal Cloud vs self-hosted). Temporal Workflow code is never rewritten into Step Functions. Requires at least one agentic component — a purely non-agent system (plain services, batch jobs, HTTP endpoints, non-agent Temporal Activities) is out of scope, redirected to gcp-to-aws / heroku-to-aws / llm-to-bedrock. Not for: compute/data migration with no AI agent; pure LLM SDK rewrite (use llm-to-bedrock); per-model pricing.

日本語の概要は準備中です。原文の説明を表示しています。

aws/agent-toolkit-for-aws2,8432026年10月10日 更新

agents-build

無料

Use to extend an existing agent project with memory, app integration, VPC, multi-agent, migration, model, browser, code interpreter, payments, or resource removal. Triggers: "add memory", "remember across sessions", "call agent from app", "invoke agent from code", "agent auth", "streaming", "VPC", "VPC connectivity", "can't reach from VPC", "multi-agent", "A2A", "A2A auth", "orchestrator not delegating", "specialist not called", "migrate Bedrock Agent", "migration issue", "change model", "browser tool", "code interpreter", "delete agent", "tear down", "agentcore remove", "cross-account memory", "add payments capability to my agent", "wire payments plugin", "integrate x402 payments with the agent I'm building", "add MPP payments", "Machine Payments Protocol". External APIs via Gateway: use agents-connect. New project: use agents-get-started. CLI/dev-server errors: use agents-debug. Runtime x402/MPP payments: use agents-pay. Migration-specific Strands vs LangGraph routes here.

日本語の概要は準備中です。原文の説明を表示しています。

aws/agent-toolkit-for-aws2,8432026年10月10日 更新

agents-connect

無料

Use when connecting your agent to external APIs, tools, or services via Gateway, or restricting tool access with Cedar policies. Handles gateway setup, target types, outbound auth (OAuth, API key, IAM), credentials, and Cedar policy authoring. Triggers on: "connect to API", "add gateway", "connect to MCP server", "Lambda tools", "OpenAPI", "gateway target", "Cedar policy", "restrict tools", "policy engine", "gateway auth error", "store API key", "outbound credential", "env var API key", "API key None after deploy", "credential not available after deploy", "should this be a gateway target", "give my agent tools", "add tools to agent". Not for inbound auth (who can call your agent) — use agents-harden. Not for debugging agent behavior — use agents-debug. Not for VPC networking errors (agent can't reach APIs due to VPC) — use agents-build. Not for creating or hosting a new MCP server project — use agents-get-started.

日本語の概要は準備中です。原文の説明を表示しています。

aws/agent-toolkit-for-aws2,8432026年10月10日 更新

agents-debug

無料

Use when your agent or environment is broken — wrong answers, errors, timeouts, tool failures, or CLI issues. Reads traces and logs to diagnose root causes. Also checks prerequisites when the CLI itself isn't working. Triggers on: "agent not working", "wrong answer", "agent error", "tool call failing", "debug agent", "check logs", "read traces", "broken", "500 error", "424 error", "model access denied", "command not found", "stuck in DELETING", "maxVms exceeded", "cold start diagnosis", "cold start slow", "agentcore create error", "create failed", "exit code 7", "connection refused local dev". Not for deploy failures — use agents-deploy. Not for performance tuning without errors — use agents-optimize. Not for VPC configuration — use agents-build. Not for observability setup or missing logs — use agents-optimize.

日本語の概要は準備中です。原文の説明を表示しています。

aws/agent-toolkit-for-aws2,8432026年10月10日 更新

agents-deploy

無料

Use when deploying your agent to AWS, or when a deploy has failed. Handles pre-flight validation, CDK/IAM/quota error diagnosis, version management, rollback, and canary deployments. Triggers on: "deploy my agent", "agentcore deploy", "deploy failed", "CDK error", "rollback", "canary deploy", "pin version", "redeploy", "deploy stuck". Not for production hardening — use agents-harden. Not for adding capabilities before deploy — use agents-build or agents-connect. Not for VPC configuration errors — use agents-build.

日本語の概要は準備中です。原文の説明を表示しています。

aws/agent-toolkit-for-aws2,8432026年10月10日 更新

agents-get-started

無料

Use when a developer wants to create a new agent project or get started with AgentCore. Handles framework selection, project scaffolding, first deploy, and first invocation. Triggers on: "build an agent", "create an agent", "get started", "new project", "agentcore create", "which framework", "Strands vs LangGraph", "hello world agent", "first agent", "create MCP server", "host MCP server", "agentcore dev", "dev server", "what port", "local development". Not for adding capabilities to existing projects — use agents-build or agents-connect. Strands vs LangGraph in a migration context routes to agents-build, not here. Connecting to an existing MCP server routes to agents-connect, not here.

日本語の概要は準備中です。原文の説明を表示しています。

aws/agent-toolkit-for-aws2,8432026年10月10日 更新

aws のスキルをすべて見る

このスキルの問題を報告する