6

TL; DR: Is there anyway to query a postgres instance for how many cores it has available for processing?

Postgres has a good reputation with respect to parallelizing multiple queries efficiently across available cores.

I'm writing an application with a CPU intensive data transformation pipeline to be executed by postgres on a large number of collections of source data. Each pipeline will be the work of a single connection, and I hope to exploit parallelism by establishing multiple Connections and performing the work (which occurs in temporary tables scoped within single Connections).

I'd like to tune how many concurrent Connections to maintain with the database. My inclination is to use a multiplier over available cores, something like 1.5x, in order not to bog down the DBMS with waiting Connections but still ensure that it always has work to do.

However, the DBMS will be an AWS RDS instance selected by users. I don't know in advance how many cores will be available. Is there anyway to query the database to find out?

2
  • I have not found such a thing in Postgres documentation. You might be able to adjust the workload to hit a target CPU usage or number of running tasks from OS monitoring. With AWS, possibly using CloudWatch data. Commented Apr 24, 2016 at 22:53
  • Thank you very much for looking. I haven't found anything either. Trying via AWS monitoring services is an interesting idea, I'll have to look into that. Commented Apr 25, 2016 at 1:41

2 Answers 2

5
create temp table cores(num_cores integer);
copy cores(num_cores) from program 'grep processor /proc/cpuinfo|wc -l';
select * from cores;

In psql as postgres

This doesn't work on PaaS offerings like AWS and Azure.

4
  • 2
    In case you don't want to keep this table around forever: create temporary table cores(num_cores integer); which will be dropped when the connection is dropped. Commented Mar 8, 2022 at 21:11
  • 1
    Make sure to change the quotes from ‘grep processor /proc/cpuinfo|wc -l’ to 'grep processor /proc/cpuinfo|wc -l'
    – Gourneau
    Commented Nov 22, 2022 at 4:37
  • Does not work on AWS RDS.
    – scravy
    Commented May 13, 2023 at 7:30
  • Doesn't work in Azure :/ PaaS limitations
    – Rafs
    Commented Aug 2 at 15:57
0

on windows (yes, i know postgresql on windows is an abomination :-) )

you can do

create temporary table cores(num_cores integer);
copy cores(num_cores) from program 'echo %NUMBER_OF_PROCESSORS%';
select * from cores;

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .