RE: pgbench results arent accurate

[Greg Clough]

> I installed a new postgres 9.6 on both of my machines.

Where is your storage?  Is it local, or on a SAN?  A SAN will definitely have a 
cache, so possibly there is another layer of cache that you’re not accounting 
for.

Greg Clough.



This e-mail, including accompanying communications and attachments, is strictly 
confidential and only for the intended recipient. Any retention, use or 
disclosure not expressly authorised by IHSMarkit is prohibited. This email is 
subject to all waivers and other terms at the following link: 
https://ihsmarkit.com/Legal/EmailDisclaimer.html

Please visit www.ihsmarkit.com/about/contact-us.html for contact information on 
our offices worldwide.

Re: pgbench results arent accurate

[Mark Kirkwood]

If you have not amended any Postgres config parameters, then you'll get 
checkpoints approx every 5 min or so. Thus using a Pgbench run time of 
5min is going sometimes miss/sometimes hit a checkpoint in progress - 
which will hugely impact test results.


I tend to do Pgbench runs of about 2x checkpoint_timeout - (i.e 10 min 
for default configurations). Also for increased repeatability, I do a 
manually triggered checkpoint immediately before each run.


regards

Mark

On 13/12/18 1:53 AM, Mariel Cherkassky wrote:

Hey,
I installed a new postgres 9.6 on both of my machines. I'm trying to 
measure the differences between the performances in each machine but 
it seems that the results arent accurate.

I did 2 tests :

1)In the first test the scale was set to 100 :
pgbench -i -s 100 -U postgres -d bench -h machine_name
pgbench -U postgres -d bench  -h machine_name -j 2 -c 16 -T 300
RUN    TPS - machine1   TPS-machine2
1   697 555
2   732 861
3   784 842





2)In this test the scale was set to 1 :
pgbench -i -s 1 -U postgres -d bench -h machine_name
pgbench -U postgres -d bench --progress=30 -h machine_name -j 2 -c 16 
-T 300

RUN TPS-MACHINE1TPS-MACHINE2
1   103 60
2   63  66
3   74  83
4   56  61
5   75  53
6   73  60
7   62  53


In both cases after the initalization I restarted the database and 
cleared the cashe(echo 1 > /proc/sys/vm/drop_caches) one time. During 
all the runs I didnt shutdown the machine.


Now, I was hopping the the tps will be almost the same in each machine 
for all the runs. In other words, I wanted to see that the tps in 
machine1 during all the tps are almost the same but I see that the 
values arent accurate.


Any idea what might cause the differences in every run ?