top of page
  • Writer's picturekyle Hailey

Warning: PQ not monitoring in Datadog nor Query Insights (nor Performance Insights)

Say you want to test out the speed of Parallel Query on Postgres You kick off a query ... well how do you know it is running in parallel?

You can do an explain and get some idea:

explain select count(*) from foo;
                                                                            QUERY PLAN                                                                             
 Finalize Aggregate  (cost=29503078.89..29503078.90 rows=1 width=8)
   ->  Gather  (cost=29503078.67..29503078.88 rows=2 width=8)
         Workers Planned: 2
         ->  Partial Aggregate  (cost=29502078.67..29502078.68 rows=1 width=8)

No I want to crank up the parallelism so I set the values to 64:

SET maintenance_work_mem TO '64 GB';
SET max_parallel_maintenance_workers TO 64;
SET max_parallel_workers_per_gather TO 64;
SET max_parallel_workers TO 64;
SLTER TABLE foo SET (parallel_workers = 64);
set force_parallel_mode=on;

Now after I kick the query off, how do I really make sure it's running in parallel?

I mean you can check pg_stat_statement, but how about using a nice graphic tool?

Here is PASH-Viewer, which is great fat client tool

But let's look at an enterprise monitor , Datadog which is my go to tool. Unfortunately it make sit look like there is only one thread running. where as in PASH viewer we could see that there were 64 threads

Now as far as Query Insights goes, I have no idea of what is even showing

Now I'm doing this on Google, not Amazon RDS, but I recall that Performance Insights on RDS only shows one thread when running parallel query. So all 3 enterprise tools are broken, but PASH viewer works

(it would be easy to fix the enterprise tools ... my guess is they are filtering only "client backend" for backend_type )

81 views0 comments


bottom of page