4

TL; DR: Is there anyway to query a postgres instance for how many cores it has available for processing?

Postgres has a good reputation with respect to parallelizing multiple queries efficiently across available cores.

I'm writing an application with a CPU intensive data transformation pipeline to be executed by postgres on a large number of collections of source data. Each pipeline will be the work of a single connection, and I hope to exploit parallelism by establishing multiple Connections and performing the work (which occurs in temporary tables scoped within single Connections).

I'd like to tune how many concurrent Connections to maintain with the database. My inclination is to use a multiplier over available cores, something like 1.5x, in order not to bog down the DBMS with waiting Connections but still ensure that it always has work to do.

However, the DBMS will be an AWS RDS instance selected by users. I don't know in advance how many cores will be available. Is there anyway to query the database to find out?

Many thanks.

2
  • I have not found such a thing in Postgres documentation. You might be able to adjust the workload to hit a target CPU usage or number of running tasks from OS monitoring. With AWS, possibly using CloudWatch data. Apr 24 '16 at 22:53
  • Thank you very much for looking. I haven't found anything either. Trying via AWS monitoring services is an interesting idea, I'll have to look into that. Apr 25 '16 at 1:41
2
create table cores(num_cores integer);
copy cores(num_cores) from program ‘grep processor /proc/cpuinfo|wc -l’;
select * from cores;

In psql as postgres (Not tested in AWS)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.