clickhouse-managed-postgres-rca
MUST USE when investigating performance issues on a ClickHouse-managed Postgres instance. Provides an evidence-based RCA workflow that scrapes the Prometheus endpoint for system signal, pulls per-digest evidence from the Slow Query Patterns API, and recommends (does not apply) a fix.
By clickhouse · 6,019 installs
npx skills add clickhouse/agent-skills --skill clickhouse-managed-postgres-rca
Source repository · Upstream listing
ClickHouse Managed Postgres RCA
When to use
Trigger whenever a user reports slowness, high CPU, low
throughput, cache thrash, or any unexplained pain on a
ClickHouse managed Postgres instance.
What you have access to
Two APIs on https://api.clickhouse.cloud (HTTP Basic auth
using a ClickHouse Cloud API key/secret pair):
Prometheus metrics — operation postgresInstancePrometheusGet
under the Prometheus tag. Returns Prometheus exposition format.
System and workload metrics for one Postgres service.
Slow Query Patterns — operation slowQueryPatternsGetList
under the Postgres tag. Returns per digest latency, IO, and
call statistics for normalized query patterns. Beta.
Both endpoints require an organizationId and a serviceId as
path parameters. The user must supply both, plus the API
key/secret pair.
What you do NOT have
Query plans / EXPLAIN output.
Per table scan type counters ( seq scan / idx scan ).
Autovacuum or last ANALYZE timestamps.
Reason from IO and timing signals, not from a plan tree.
Workflow
Six steps, in order. Do not skip ahead.
Steps 2 and 3 only share auth — no data dependency between
them. Run them in parallel (background curls, & + wait ) to
cut wall time from sequential ~2s to ~1s.
1. Discover the live API shape
These endpoints are Beta — paths, params, and JSON field names
can shift. Follow rules/openapi discovery.md to:
1. Fetch the OpenAPI spec from https://api.clickhouse.cloud/v1 .
2. Locate the two operations by operationId :
postgresInstancePrometheusGet (Prometheus tag)
slowQueryPatternsGetList (Postgres tag)
3. Resolve their path templates, required query parameters,
and (for the slow query endpoint) the response schema.
4. Build a session scoped role map from the schema property
descriptions: { semantic role → actual field name } .
Use the resolved names in every subsequent request and citation.
Never hardcode field names from memory.
2. Scrape Prom once for system gauges
Follow rules/prometheus scrape.md . One scrape, no wait.
You're after gauges (current values) that don't need a delta:
CacheHitRatio , ActiveConnections , MemoryUsedPercent ,
FilesystemUsedPercent .
A CacheHitRatio well below ~95% on a workload that should
fit in cache is a real signal on its own. Climbing
ActiveConnections toward the pool ceiling is a real signal
on its own. These don't need rate of change.
A second scrape for counter deltas is opt in , used only
when Step 4 triage points at write congestion (where deadlock
and rollback rates matter and the Slow Query Patterns API
can't substitute). For the read path case (the most common
RCA shape) the single scrape is enough.
3. Pull top slow query patterns
Request the slow query patterns. Follow
rules/slow query patterns fields.md for the fields that
matter and how to read them. This is the primary diagnostic —
it returns per pattern accumulated totals (call count, runtime,
blocks, rows) over the window you request, which is the
"rate of change" data you'd otherwise derive from two Prom
scrapes — but per query and without waiting.
If no patterns return a meaningful totalDurationUs , the
report may be overstated or the issue isn't query shaped.
Stop and tell the user what you looked at.
4. Triage: pick the right heuristic
Follow rules/triage.md . Match the combined Prom + slow query
signal to one of the heuristic shapes. Each shape points to a
specific heuristic file:
rules/heuristic full scan.md — read path full scan.
rules/heuristic hot loop.md — N+1 / hot loop from the app.
rules/heuristic write congestion.md — deadlocks, slow
writes, high rollback rate.
If the signal does not match any shape cleanly, do not invent
a hypothesis. Surface the top patterns and ask the user which
workload they recognize. New heuristics are welcome as PRs.
5. Reason, then recommend
Use the format in rules/output template.md . Always include:
symptom, evidence, hypothesis (noting any alternative cause
you cannot rule out from this surface alone), short term fix,
and long term follow ups.
6. Do not apply the fix
Follow rules/recommend only.md . Never run DDL. Never call
pg cancel backend or pg terminate backend . Write the
recommendation, explain why, and let the human apply it.
Full Compiled Document
For the complete guide with every rule expanded in a single
context load: AGENTS.md .