SHARED LOCKS , EXCLUSIVE LOCKS, ACCESS EXCLUSIVE LOCKS

2021-04-04 Thread aditya desai
Hi,
We have few select queries during which we see SHARED LOCKS and EXCLUSIVE
LOCKS on tables. Can these locks cause slowness? Is there any way to reduce
the locks?

What must be causing ACCESS EXCLUSIVE LOCKS when the application is running
select queries? Is it AUTOVACUUM?

Regards,
Aditya.


Re: SELECT Query taking 200 ms on PostgreSQL compared to 4 ms on Oracle after migration.

2021-04-04 Thread aditya desai
Noted thanks!!

On Sun, Apr 4, 2021 at 4:19 PM Pavel Stehule 
wrote:

>
>
> ne 4. 4. 2021 v 12:39 odesílatel aditya desai  napsal:
>
>> Hi Pavel,
>> Notes thanks. We have 64 core cpu and 320 GB RAM.
>>
>
> ok - this is probably good for max thousand connections, maybe less (about
> 6 hundred). Postgres doesn't perform well, when there are too many active
> queries. Other databases have limits for active queries, and then use an
> internal queue. But Postgres has nothing similar.
>
>
>
>
>
>
>
>> Regards,
>> Aditya.
>>
>> On Sat, Apr 3, 2021 at 11:21 PM Pavel Stehule 
>> wrote:
>>
>>>
>>>
>>> so 3. 4. 2021 v 19:45 odesílatel aditya desai 
>>> napsal:
>>>
 Yes. I have made suggestions on connection pooling as well. Currently
 it is being done from Application side.

>>>
>>> It is usual - but the application side pooling doesn't solve well
>>> overloading. The behaviour of the database is not linear. Usually opened
>>> connections are not active. But any non active connection can be changed to
>>> an active connection (there is not any limit for active connections), and
>>> then the performance can be very very slow. Good pooling and good setting
>>> of max_connections is protection against overloading. max_connection should
>>> be 10-20 x CPU cores  (for OLTP)
>>>
>>> Regards
>>>
>>> Pavel
>>>
>>>
>>>
>>>
 On Sat, Apr 3, 2021 at 11:12 PM Pavel Stehule 
 wrote:

>
>
> so 3. 4. 2021 v 19:37 odesílatel aditya desai 
> napsal:
>
>> Hi Justin/Bruce/Pavel,
>> Thanks for your inputs. After setting force_parallel_mode=off
>> Execution time of same query was reduced to 1ms from 200 ms. Worked like 
>> a
>> charm. We also increased work_mem to 80=MB. Thanks
>>
>
> super.
>
> The too big max_connection can cause a lot of problems. You should
> install and use pgbouncer or pgpool II.
>
>
> https://scalegrid.io/blog/postgresql-connection-pooling-part-4-pgbouncer-vs-pgpool/
>
> Regards
>
> Pavel
>
>
>
>
>> again.
>>
>> Regards,
>> Aditya.
>>
>> On Sat, Apr 3, 2021 at 9:14 PM aditya desai 
>> wrote:
>>
>>> Thanks Justin. Will review all parameters and get back to you.
>>>
>>> On Sat, Apr 3, 2021 at 9:11 PM Justin Pryzby 
>>> wrote:
>>>
 On Sat, Apr 03, 2021 at 11:39:19AM -0400, Tom Lane wrote:
 > Bruce Momjian  writes:
 > > On Sat, Apr  3, 2021 at 08:38:18PM +0530, aditya desai wrote:
 > >> Yes, force_parallel_mode is on. Should we set it off?
 >
 > > Yes.  I bet someone set it without reading our docs:
 >
 > >
 https://www.postgresql.org/docs/13/runtime-config-query.html#RUNTIME-CONFIG-QUERY-OTHER
 >
 > > --> Allows the use of parallel queries for testing purposes
 even in cases
 > > --> where no performance benefit is expected.
 >
 > > We might need to clarify this sentence to be clearer it is
 _only_ for
 > > testing.
 >
 > I wonder why it is listed under planner options at all, and not
 under
 > developer options.

 Because it's there to help DBAs catch errors in functions
 incorrectly marked as
 parallel safe.

 --
 Justin

>>>


Re: SHARED LOCKS , EXCLUSIVE LOCKS, ACCESS EXCLUSIVE LOCKS

2021-04-04 Thread Amine Tengilimoglu
Hi;

It's normal to see locks on tables during queries. These are usually locks
used automatically by postgres as a result of the operations you perform on
your database. You should check the document for the lock modes postgres
uses.

Lock causes slowness if it causes other queries to wait. You can see the
queries waiting for lock from pg_locks view.  Access Exclusive Lock
completely locks the table, does not allow read and write operations,
blocks queries.

Commands such as  drop table, truncate, reindex, vacuum full, alter table
use this lock. And autovacuum  uses a weaker lock on the table, not using
an exclusive lock.

aditya desai , 4 Nis 2021 Paz, 13:42 tarihinde şunu
yazdı:

> Hi,
> We have few select queries during which we see SHARED LOCKS and EXCLUSIVE
> LOCKS on tables. Can these locks cause slowness? Is there any way to reduce
> the locks?
>
> What must be causing ACCESS EXCLUSIVE LOCKS when the application is
> running select queries? Is it AUTOVACUUM?
>
> Regards,
> Aditya.
>


Re: SHARED LOCKS , EXCLUSIVE LOCKS, ACCESS EXCLUSIVE LOCKS

2021-04-04 Thread Justin Pryzby
On Sun, Apr 04, 2021 at 04:12:14PM +0530, aditya desai wrote:
> Hi,
> We have few select queries during which we see SHARED LOCKS and EXCLUSIVE
> LOCKS on tables. Can these locks cause slowness? Is there any way to reduce
> the locks?
> 
> What must be causing ACCESS EXCLUSIVE LOCKS when the application is running
> select queries? Is it AUTOVACUUM?

I suggest to review all the logging settings, and consider setting:
log_destination = 'stderr,csvlog'   

  
log_checkpoints = on

   
log_lock_waits  = on

   
log_min_messages= info  

  
log_min_error_statement = notice

   
log_temp_files  = 0 

   
log_min_duration_statement  = '9sec'

   
log_autovacuum_min_duration = '99sec'   

  

You should probably set up some way to monitor logs.
We set log_destination=csvlog and import them into the DB.
Then I have nagios checks for slow queries, errors, many tempfiles, etc.
https://www.postgresql.org/docs/current/runtime-config-logging.html#RUNTIME-CONFIG-LOGGING-CSVLOG
https://www.postgresql.org/message-id/[email protected]

-- 
Justin