Re: PSQL performance - TPS

2019-08-02 Thread Imre Samu
> Application as a whole is expected to give an throughput of 100k
transactions per sec.
> On this env(8core cpu, 16GB) what is the TPS that we can expect?

as a reference - maybe you can reuse/adapt  the "TechEmpower Framework
Benchmarks" tests - and compare your PG9.6+hardware results.

The new TechEmpower Framework Benchmarks  [2019-07-09 Round 18]
* reference numbers:
https://www.techempower.com/benchmarks/#section=data-r18&hw=ph&test=update
* source code: https://github.com/TechEmpower/FrameworkBenchmarks
* PG11 config:
https://github.com/TechEmpower/FrameworkBenchmarks/blob/master/toolset/databases/postgres/postgresql.conf
* java frameworks:
https://github.com/TechEmpower/FrameworkBenchmarks/tree/master/frameworks/Java

> We have tested with a simple Java code firing insert

As I see - There are lot of java framework - and sometimes 10x difference
in performance :
https://www.techempower.com/benchmarks/#section=data-r18&hw=ph&test=update

"Responses per second at 20 updates per request, Dell R440 Xeon Gold + 10
GbE"
( "Intel Xeon Gold 5120 CPU (14c28t) , 32 GB of memory, and an enterprise
SSD. Dedicated Cisco 10-gigabit Ethernet switch")
*  java + PG11 results:   low:126 -> high:21807

"Responses per second at 20 updates per request, Azure D3v2 instances"
*  java + PG11 results:  low:329 -> high:2975

best,
  Imre



Shital A  ezt írta (időpont: 2019. aug. 1., Cs,
5:11):

> Hello,
>
> We are working on development of an application with postgresql 9.6 as
> backend. Application as a whole is expected to give an throughput of 100k
> transactions per sec. The transactions are received by DB from component
> firing DMLs in ad-hoc fashion i.e. the commits are fired after random
> numbers of transaction like 2,3,4. There is no bulk loading of records. DB
> should have HA setup in active passive streaming replication. We are doing
> a test setup on a 8-core machine having 16 GB RAM. Actual HW will be
> better.
>
> Need help in:
> 1. On this env(8core cpu, 16GB) what is the TPS that we can expect? We
> have tested with a simple Java code firing insert and commit in a loop on a
> simple table with one column. We get 1200 rows per sec. If we increase
> threads RPS decrease.
>
> 2. We have tuned some DB params like shared_buffers, sync_commit off, are
> there any other pointers to tune DB params?
>
>
> Thanks.
>


Strange runtime partition pruning behaviour with 11.4

2019-08-02 Thread Thomas Kellerer
I stumbled across this question on SO: 
https://stackoverflow.com/questions/56517852

Disregarding the part about Postgres 9.3, the example for Postgres 11 looks a 
bit confusing. 

There is a script to setup test data in that question: 

 start of script 

create table foo (
foo_id integer not null,
foo_name varchar(10),
constraint foo_pkey primary key (foo_id) 
);  

insert into foo
  (foo_id, foo_name) 
values
  (1, 'eeny'),
  (2, 'meeny'),
  (3, 'miny'),
  (4, 'moe'),
  (5, 'tiger'), 
  (6, 'toe');

create table foo_bar_baz (
foo_id integer not null,
bar_id integer not null,
bazinteger not null,
constraint foo_bar_baz_pkey primary key (foo_id, bar_id, baz),
constraint foo_bar_baz_fkey1 foreign key (foo_id)
references foo (foo_id)
) partition by range (foo_id) 
;

create table if not exists foo_bar_baz_0 partition of foo_bar_baz for 
values from (0) to (1);
create table if not exists foo_bar_baz_1 partition of foo_bar_baz for 
values from (1) to (2);
create table if not exists foo_bar_baz_2 partition of foo_bar_baz for 
values from (2) to (3);
create table if not exists foo_bar_baz_3 partition of foo_bar_baz for 
values from (3) to (4);
create table if not exists foo_bar_baz_4 partition of foo_bar_baz for 
values from (4) to (5);
create table if not exists foo_bar_baz_5 partition of foo_bar_baz for 
values from (5) to (6);

with foos_and_bars as (
select ((random() * 4) + 1)::int as foo_id, bar_id::int
from generate_series(0, 1499) as t(bar_id)
), bazzes as (
select baz::int
from generate_series(1, 1500) as t(baz)
)
insert into foo_bar_baz (foo_id, bar_id, baz) 
select foo_id, bar_id, baz 
from bazzes as bz 
  join foos_and_bars as fab on mod(bz.baz, fab.foo_id) = 0;

 end of script 

I see the some strange behaviour similar to to what is reported in the comments 
to that question: 

When I run the test query immediately after populating the tables with the 
sample data:

explain analyze 
select count(*) 
from foo_bar_baz as fbb 
  join foo on fbb.foo_id = foo.foo_id 
where foo.foo_name = 'eeny'

I do see an "Index Only Scan  (never executed)" in the plan for the 
irrelevant partitions: 

  https://explain.depesz.com/s/AqlE

However once I run "analyze foo_bar_baz" (or "vacuum analyze"), Postgres 
chooses to do a "Parallel Seq Scan" for each partition:

  https://explain.depesz.com/s/WwxE

Why does updating the statistics mess up (runtime) partition pruning? 


I played around with random_page_cost and that didn't change anything. 
I tried to create extended statistics on "foo(id, name)" so that the planner 
would no, that there is only one name per id. No change. 

I saw the above behaviour when running this on Windows 10 (my Laptop) or CentOS 
7 (a test environment on a VM) 

On the CentOS server default_statistics_target is set to 100, on my laptop it 
is set to 1000

In both cases the Postgres version was 11.4

Any ideas? 

Thomas