Re: Install different directory issues

2019-11-27 Thread Justin
Hi Thomas did you make sure the search paths and the paths in the config files were updated to find postgresql files. I've seen this not get updated by the install scripts On Wed, Nov 27, 2019 at 2:20 PM Thomas Carter wrote: > I installed using the Postgres Installer package available on the

Re: Install different directory issues

2019-11-27 Thread Justin
ry to start postgres from the command prompt. it will throw errors giving you an idea what the problem is on debian logs are located in /var/log/postgresql/versionnumber/ On Wed, Nov 27, 2019 at 3:58 PM Thomas Carter wrote: > Hi Justin, > I expect this is the case since the traditional i

Re: what causes xact_rollback for a database in pg_stat_database to increment?

2019-12-04 Thread Justin
Hi Hu Log_statement = all can miss some statements sent to Postgresql from the manual https://www.postgresql.org/docs/current/runtime-config-logging.html *Note* *Statements that contain simple syntax errors are not logged even by the log_statement = all setting, because the log message is emit

Re: Schema Copy

2019-12-05 Thread Justin
Hi Sonam As long as the edited sql script has been changed from oldschema.tables or oldschema.functions etc... to newschema.functions, newschema.functios etc... This does not move data On Thu, Dec 5, 2019 at 5:07 AM Sonam Sharma wrote: > Can someone please help in schema copy in same datab

Re: Schema Copy

2019-12-05 Thread Justin
newschema On Thu, Dec 5, 2019 at 9:31 AM Sonam Sharma wrote: > Hi Justin, > > What can be done to move the data.. > > On Thu, Dec 5, 2019, 7:57 PM Justin wrote: > >> Hi Sonam >> >> As long as the edited sql script has been changed from >> oldschema

Re: Date created for tables

2019-12-05 Thread Justin
Hi Tom can't we get access to this information in a backwards way by using pg_xact_commit_timestamp() then query the system catalog tables xmin entry for the relevant object??? this requires turning on pg_xact_commit_timestamp https://www.postgresql.org/docs/current/runtime-config-replication.ht

Re: upgrade and migrate

2019-12-06 Thread Justin
Off topic but food for thought given the jump in versions 8.3 to (9.6 or greater.) List of major changes i can think of to watch out for that can bite without warning.. SQL operators~=~ for Like were drop in 8.4 => for hstore was drop in 9.0 in 9.1 standard_conforming_string is ON by default

Re: Query with correlated join having slow performance

2019-12-09 Thread Justin
Hi Saket The first filter condition seems to be duplicated it appears this can be simplified from and ( pdtaltrelt0_.status_typ_dbky=102 and ( pdtaltrelt0_.rule_status_typ_dbky is null ) or pdtaltrelt0_.status_typ_dbky in ( 19 ) or pdtaltrelt0_.status_typ_dbky in (20 )

Re: tcp keep alive don't work when the backend is busy

2019-12-10 Thread Justin
Hi Oner It appears that you looking for a way to detect and kill of idle connections or process that are running for a long time Correct?? If that is the case use statement_timeout setting and then use Pg_Agent and this script to kill off idle connections SELECT pg_terminate_backend(pid) FROM p

Re: server will not start (Debian)

2019-12-10 Thread Justin
/docs/current/runtime-config-statistics.html On Tue, Dec 10, 2019 at 10:32 AM stan wrote: > On Tue, Dec 10, 2019 at 08:55:02AM -0500, Justin wrote: > > Hi Stan > > > > Check security make sure V12 postgres has the correct credentials > > > OK, > > postg

Re: Is there an equivalent to sp_getapplock, sp_releaseapplock in PostgreSql?

2019-12-13 Thread Justin
Hi Ityas Advisory locks do not act like the locks in MSSQL, Postgresql will ignore advisory locks for other transactions, its up the applications layer to poll/ obey/ enforce advisory locks You can do the type of locking as in MSSQL with Postgresql but its avoided in practice as that is the po

Re: Race condition while creating a new partition

2019-12-16 Thread Justin
Hi Andrei, My gut reactions is Yes this is a deadlock caused by a race condition, the error from psycopg2 tells us that. Question becomes what is causing these two process to collide, are both processes 33 and 37 python code, As both are trying to access the same resource 16453 i would assume

Re: Race condition while creating a new partition

2019-12-16 Thread Justin
Hi Andrei General speaking any DDL (Create, Alter Drop .etc) commands issue exclusive locks automatically, so anything this transaction touches starts getting exclusive locks. Assuming this is a multi-threading app these two threads are sending commands all but at the same time. The Exclusive

Re: Tuple concurrency issue in large objects

2019-12-18 Thread Justin
I have a question reading through this email chain. Does Large Objects table using these functions work like normal MVCC where there can be two versions of a large object in pg_largeobject . My gut says no as moving/copying potentially 4 TB of data would kill any IO. I can not find any document

Re: Tuple concurrency issue in large objects

2019-12-18 Thread Justin
y X chunks" On Wed, Dec 18, 2019 at 11:12 AM Tom Lane wrote: > Justin writes: > > I have a question reading through this email chain. Does Large Objects > > table using these functions work like normal MVCC where there can be two > > versions of a large object in pg_

Re: Tuple concurrency issue in large objects

2019-12-18 Thread Justin
:12 PM Tom Lane wrote: > Justin writes: > > I now see what is causing this specific issue... > > The update and row versions is happening on 2kb chunk at a time, That's > > going to make tracking what other clients are doing a difficult task. > > Yeah, it

Re: unanalyze a foreign table

2019-12-23 Thread Justin
I do not know of way to undo an analyze once its committed. I do not know the danger in deleting an entry in pg_statistic What you can do in the future is make copy of the Statics for this table, analyze, if it negatively affect results put the copy back. Another option is to do begin ; ANALYZE

Re: When should parameters be passed as text v. binary?

2020-01-04 Thread Justin
As noted by Adrian what is the USE CASE As a general rule one wants to use the format the data is being stored in. every time data is cast to another type its going to eat those all so precious CPU cycles. (all the horror of electrons turned into infrared beams) converting Bytea type to a string

Re: UPDATE many records

2020-01-06 Thread Justin
There are several ways to actually do this If you have Postgresql 11 or higher we now have Create Procedure that allows committing transactions, one draw back is it can not parallel from inside the procedure https://www.postgresql.org/docs/11/sql-createprocedure.html https://severalnines.com/dat

Re: UPDATE many records

2020-01-06 Thread Justin
7320 > Work: 907-474-5172 > cell: 907-328-9145 > > On Jan 6, 2020, at 10:05 AM, Justin wrote: > > There are several ways to actually do this > > If you have Postgresql 11 or higher we now have Create Procedure that > allows committing transactions, one d

Re: UPDATE many records

2020-01-06 Thread Justin
What was the HD wait time ? What tool is being use to monitor the server resources?? It appears based on this information there is allot more going on than a simple Update command Moving code out of the trigger probably not going to improve performance, unless there is allot of code that does

Re: UPDATE many records

2020-01-06 Thread Justin
worth looking into If its a one time run or every 12 months who cares, Start the update on friday night, go in on Saturday to check it On Mon, Jan 6, 2020 at 5:38 PM Israel Brewster wrote: > > On Jan 6, 2020, at 12:49 PM, Justin wrote: > > What was the HD wait time ? What tool is bei

Re: Encrypted connection SQL server fdw

2020-01-10 Thread Justin
By loading data meaning this is a one time deal or only used to refresh data stored in the postgresql database??? A possible solution would be to setup a vpn tunnel, or ipsec connection to server. then run FDW through that connection. Not idea and will slow things down. The other option is to u

Re: Worse performance with higher work_mem?

2020-01-14 Thread Justin
Updating the stats can be done via vacuum or analyze command, https://www.postgresql.org/docs/12/sql-analyze.html. To just analyze a table typically does not take much time. and can be scheduled to run so the stats update instead of waiting on auto-vacuum to deal with it which could be some tim

Re: Is it possible to replicate through an http proxy?

2020-01-14 Thread Justin
Another solution to the problem instead of logical replication would be utilize wal_shipping and have the edge servers replay the wal using the restore_command The wal files can be downloaded from from HTTP server via a proxy and placed on the edge servers wal_archive directory to be replayed se

Re: Can I do this?

2020-01-16 Thread Justin
Hi Stan in you code sample there are "(" mis-matched, "MAX(" matches to "= project_key)"; it should be MAX(NULLIF(regexp_replace(report_no, '\D','','g'), '')::numeric) I do exactly what you do, and you are correct sequences are not a good fit I typically do something like this for selec

Re: Can I drop a NOT NUL constrain on an existing table?

2020-01-16 Thread Justin
Alter table mytable drop constraint name_of_constraint https://www.postgresql.org/docs/current/sql-altertable.html On Thu, Jan 16, 2020 at 9:43 AM stan wrote: > I see how to do this if it is a "dcleared" constraint, but this was just > defined in the table createion as inL > > report_no

Re: Can I do this?

2020-01-16 Thread Justin
it does not, but the odds the same user will run this command by this id in two different sessions at the same time are very low. this type of code exist for PO,SO, Invoices, to assign the next line item # in many apps. On Thu, Jan 16, 2020 at 10:40 AM Michael Nolan wrote: > > > On Thu, Jan

Re: Is there a GoTo ?

2020-01-16 Thread Justin
Hi İlyas As noted by other there is no GOTO you can move the goto code into the else statement or move that code into a new function call and return that function or just return null in the else if that is all the it is doing CREATE OR REPLACE FUNCTION test(i integer) RETURNS integer AS $$BE

Re: @@TRANCOUNT ?

2020-01-22 Thread Justin
Postgresql does not support Transaction in the same way mssql, it does support nesting transactions in a limited way from version 11+ https://docs.microsoft.com/en-us/sql/t-sql/functions/trancount-transact-sql?view=sql-server-ver15 https://www.postgresql.org/docs/current/sql-begin.html https://w

Re: calculating the MD5 hash of role passwords in C

2020-01-22 Thread Justin
Not sure what your after but here is more information regarding how to store passwords in Postgresql, not related to database roles but for storing passwords for things like websites... https://www.postgresql.org/docs/current/pgcrypto.html section F.25.2.XXX On Wed, Jan 22, 2020 at 2:41 PM Ma

Re: sensible configuration of max_connections

2020-02-07 Thread Justin
Hi Chris Withers As stated each connection uses X amount of resources and its very easy to configure Postgresql where even small number of connections will each up all the RAM WorkMem is the biggest consumer of resources lets say its set to 5 megs per connection at 1000 connections that 5,000 me

Re: Fwd: sensible configuration of max_connections

2020-02-07 Thread Justin
On Fri, Feb 7, 2020 at 1:56 PM Sam Gendler wrote: > Benchmarks, at the time, showed that performance started to fall off due > to contention if the number of processes got much larger. I imagine that > the speed of storage today would maybe make 3 or 4x core count a pretty > reasonable place to

Re: How to restore roles without changing postgres password

2020-02-11 Thread Justin
pg_dumpall creates an SQL file which is just a simple text file you can then edit sql removing postgres user from the file This can be automated in a script that searches the generated sql file for the postgres user replacing it with a blank/empty line or adds -- to the bringing of the line whi

Re: How to restore roles without changing postgres password

2020-02-11 Thread Justin
HI Tom Not a bad idea, would want to extend this to all the roles on the server not just postgres I've edited the global dump many times removing/editing table spaces, comment old users, etc.. On Tue, Feb 11, 2020 at 5:46 PM Tom Lane wrote: > "Andrus" writes: > > How to create backup scri

Re: JIT on Windows with Postgres 12.1

2020-02-12 Thread Justin
Here is a link to build LLVM on windows http://llvm.org/docs/GettingStartedVS.html On Wed, Feb 12, 2020 at 2:55 PM Tom Lane wrote: > Andres Freund writes: > > On 2020-02-12 09:39:19 +0100, Josef Šimánek wrote: > >> I'm not aware of any PG 12 Windows installer with JIT/LLVM enabled. > > > It's

Re: How to fix 0xC0000005 exception in Postgres 9.0

2020-02-20 Thread Justin
This error is really common and is caused by windows via UAC, SFC, Firewarll, AV etc.. for it spontaneously to appear means system config change happened or update occurred. Common sources windows updates, anti-viruses changes, firewall changes. If its not the above 4 figuring out what causin

Re: How to fix 0xC0000005 exception in Postgres 9.0

2020-02-20 Thread Justin
having chased this error several times over the last 20 years, if its not windows update, AV update, firewall config change, UAC or a driver update it is nearly impossible to to figure out what is causing it. One can try to run sysinternal app such as process explorer or depends to look over the

Re: How to fix 0xC0000005 exception in Postgres 9.0

2020-02-21 Thread Justin
is this happening from any client or just a specific client running ODBC? are the clients running AV if so are the AV versions the same? Given this is killing a Linux server, sounds like ODBC is sending back garabage data to the server crashing it. There are several settings in OBDC, to change

Re: How to fix 0xC0000005 exception in Postgres 9.0

2020-02-21 Thread Justin
forgot to say publish the Linux logs it may have more details what is going on On Fri, Feb 21, 2020 at 12:27 PM Justin wrote: > is this happening from any client or just a specific client running ODBC? > are the clients running AV if so are the AV versions the same? > > Given this

Re: How to fix 0xC0000005 exception in Postgres 9.0

2020-02-21 Thread Justin
Yes publish the ODBC logs seeing the Linux logs tells us the ODBC client crashing NOT the client process. if the linux postgrsql client process crashes it typically will cause the postgresql postmaster to panic and restart as it has to assume it corrupt share memory. On Fri, Feb 21, 2020 at

Re: Connections dropping while using Postgres backend DB with Ejabberd

2020-02-25 Thread Justin
Hi Dipanjan Please do not post to all the postgresql mailing list lets keep this on one list at a time, Keep this on general list Am i reading this correctly 10,000 to 50,000 open connections. Postgresql really is not meant to serve that many open connections. Due to design of Postgresql each c

Re: Connections dropping while using Postgres backend DB with Ejabberd

2020-02-25 Thread Justin
resources long before reaching 50K Something is off here I would be looking into how this test actually works, how the connections are opened, and commands it sends to Postgresql On Tue, Feb 25, 2020 at 2:12 PM Dipanjan Ganguly wrote: > Hi Justin, > > Thanks for your insight. > &g

Re: Postgresql 12.x on Windows (vs Linux)

2020-03-02 Thread Justin
Hi Robert I've used Postgresql on windows for years. Yes there are performance differences between windows and linux and the gap has gotten bigger with JIT Common performance hits Shared Buffers JIT not supported Windows can be aggressive flushing its disk cache, https://docs.microsoft.com/en-

Re: Performance Problem

2020-03-04 Thread Justin
taking a quick glance at config file I do not see any modifications to any key settings shared_buffers, efffecttive cache size work_mem meaning the server is running at the default settings which results in horrible performance here is a website that gives suggested config changes based on serve

Re: Rules versus triggers

2020-03-07 Thread Justin
Hi Stan Rules actual are able to rewrite the SQL query sent to postgresql. Most everyone suggestion is avoid rules. Triggers are just like every other databases Triggers firing off code for Insert/Update/Delete/Truncate event https://www.postgresql.org/docs/current/sql-createtrigger.html On Sa

Re: Rules versus triggers

2020-03-07 Thread Justin
9AM -0500, Justin wrote: > > Hi Stan > > > > Rules actual are able to rewrite the SQL query sent to postgresql. Most > > everyone suggestion is avoid rules. > > > > Triggers are just like every other databases Triggers firing off code for > >

Re: Who mades the inserts?

2020-03-09 Thread Justin
Hi DD By default Postgresql does not collect this level of detail information to tell you which database has a high load at X point in time. You can infer which database has this high load without increasing logging Select * from pg_stat_database this dumps total inserts, update, scans etc..

Re: Force WAL cleanup on running instance

2020-03-11 Thread Justin
Question everyone isn't this a problem with the order of operations? switching the wal files then running checkpoint means the Checkpoint can cross wal files, so the previous wal file can not be deleted??? To my understanding the order operations should be Checkpoint which flushes everything

Re: Need help debugging slow logical replication

2023-02-07 Thread Justin
On Tue, Feb 7, 2023 at 6:38 PM sunyuc...@gmail.com wrote: > Hi there, > > I am using PG 14.14 on both primary and secondary DB on AWS, setup > using a logical replication, I'm having trouble with huge replication > lag. > > My setup is as follows: > > P1 - physical - P1-R > | (logical) > P2 - p

Re: Need help debugging slow logical replication

2023-02-07 Thread Justin
On Tue, Feb 7, 2023 at 8:07 PM sunyuc...@gmail.com wrote: > Hi Justin: > > - i checked that I have 2 tables using replication identity FULL, but > one table is empty and one table has only 1 row > - 7 tables using index > - overall I have ~100 tables in the publication:

Re: Trigger questions

2023-05-04 Thread Justin
On Thu, May 4, 2023 at 9:49 AM DAVID ROTH wrote: > 1) Can I create a trigger on a view? > 2) Do triggers cascade? > > Say I have an insert trigger on a table. > And, I have an insert trigger on a view that references this table > If I do an insert on the view, will both triggers fire? > Can not

Re: Logical replication claims to work, not working - new tables

2024-01-21 Thread Justin
On Sat, Jan 20, 2024, 5:43 PM Chris Angelico wrote: > PostgreSQL 15 on Debian, both ends of replication. > > I'm doing logical replication in a bit of a complex setup. Not sure > how much of this is relevant so I'll give you a lot of detail; sorry > if a lot of this is just noise. > > * Bidirecti

Re: Partitioning options

2024-02-08 Thread Justin
Hi Sud, Would not look at HASH partitioning as it is very expensive to add or subtract the number of partitions. Would probably look at a nested partitioning using customer ID using range or list of IDs then by transaction date, Its easy to add partitions and balance the partitions segments.

Re: Partitioning options

2024-02-11 Thread Justin
, 2024 at 10:25 PM Justin wrote: > >> Hi Sud, >> >> Would not look at HASH partitioning as it is very expensive to add or >> subtract the number of partitions. >> >> Would probably look at a nested partitioning using customer ID using >> range o

Re: Partitioning options

2024-02-20 Thread Justin
om > > Alec > > > Hi Alec, would need to see the DDL of the partitions and the queries accessing these partitions to have an opinion Thank you Justin

Re: Local replication "slot does not exist" after initial sync

2024-02-25 Thread Justin
On Sun, Feb 25, 2024 at 1:11 PM Mike Lissner wrote: > Sorry, two more little things here. The publisher logs add much, but > here's what we see: > > STATEMENT: START_REPLICATION SLOT > "pg_20031_sync_17418_7324846428853951375" LOGICAL F1D0/346C6508 > (proto_version '2', publication_names '"compas

Re: Storing and comparing columns of cryptographic hashes?

2024-04-09 Thread Justin
has many drawbacks compared to btree. None of the above queries are possible with GIN indexes or using array columns without a lot more code. Arrays are not data sets if the design needs to access a specific hash value for update,delete, append new values, an array probably not the best solution. Hope this helps Justin

Re: replication slot "pg_1015733_sync_1014718_7358407382484881476" does not exist

2024-04-17 Thread Justin
Hi Avi, Based on the slot name this is an initial sync worker being created by the Logical Replication supervisor. Subscriber started an initial sync either failed to create the slot and now thinks it exists and keeps trying to drop it on the publisher or another process dropped the slot on the p

Re: Questions on logical replication

2024-06-08 Thread Justin
On Sat, Jun 8, 2024 at 1:41 PM Koen De Groote wrote: > What I'm trying to do is upgrade a PG11 database to PG16, using logical > replication. > > The PG11 has an active and a standby, there are a handful of databases. On > particular one has a few tables just over 100GB, then a few 100 tables nea

Re: Questions on logical replication

2024-06-12 Thread Justin
On Tue, Jun 11, 2024 at 5:43 PM Koen De Groote wrote: > > If there are any errors during the replay of WAL such as missing indexes > for Replica Identities during an Update or Delete this will cause the main > subscriber worker slot on the publisher to start backing up WAL files > > And also if

Re: Questions on logical replication

2024-06-13 Thread Justin
o low on resources that Logical Replication is problematic one can create a binary replica, promote it and convert it to logical replication skipping the initial sync. Then upgrade that server. There is a minor outage required to convert a binary replica to a logical replica. I've done it in u

Re: --frokbackend process

2024-06-26 Thread Justin
Hi Satyajit: Can't tell what is going on from the task manager list. Looks like a normal task list for PostgreSQL. Keep in mind PG is a process based application NOT a threaded application. Meaning postgresql starts/forks a new process for every connection. This means we can see each sessions

Re: Logical Replication Delay

2024-09-21 Thread Justin
Hi Ramakrishna, 4GB of WAL generated per minute is a lot. I would expect the replay on the subscriber to lag behind because it is a single process. PostgreSQL 16 can create parallel workers for large transactions, however if there is a flood of small transactions touching many tables the single

Re: Logical Replication Delay

2024-09-23 Thread Justin
ng LR will execute it. I can see an LR worker being AHEAD of other workers trying to reference ROWs that do not exist yet. Which can be dealt with by making sure the Triggers that reference other tables are in the same publication and subscription. Thanks Justin On Mon, Sep 23, 2024 at 12:

Re: Synchronize the dump with a logical slot with --snapshot

2024-09-28 Thread Justin
ql.org/docs/17/functions-admin.html#FUNCTIONS-SNAPSHOT-SYNCHRONIZATION Then you can create the logical replication slot with using that slotname option https://www.postgresql.org/docs/17/sql-createsubscription.html#SQL-CREATESUBSCRIPTION-PARAMS-WITH-SLOT-NAME and no sync option. Then you tell pg_dump to use that snapshot name snapshot with this option --snapshot=snapshotname https://www.postgresql.org/docs/current/app-pgdump.html Once pg_restore is done on the destination , you can create a subscription using that slotname option probably and specify copy_data = false. Keep in mind the WAL will build up during this process, not sure what the benefit would be just allowing logical replication to do the initial sync. Thanks Justin

Re: Suddenly all queries moved to seq scan

2024-11-20 Thread Justin
On Wed, Nov 20, 2024 at 9:09 AM Sreejith P wrote: > > > > Queries were taking 20 ms started taking 60 seconds. So have done SQL > analyse to understand about query plan. There we found that query planner > taking seq scan instead in index scan. > > I would like to add one ore point. A delete que

Re: Need help in logical replication

2025-01-13 Thread Justin
Hi Divyansh, Go to the subscriber and look for errors in the PostgreSQL logs. When creating a subscription the default action is to sync the tables. Is the subscriber table empty?? Thank you, On Mon, Jan 13, 2025 at 7:30 AM Divyansh Gupta JNsThMAudy < ag1567...@gmail.com> wrote: > Hii PostgreS

Re: find replication slots that "belong" to a publication

2025-04-06 Thread Justin
ed is how far behind it is. The pg_wal_lsn_diff can be used to figure out how far behind a slot is https://www.postgresql.org/docs/13/view-pg-replication-slots.html https://www.postgresql.org/docs/13/monitoring-stats.html#MONITORING-PG-STAT-REPLICATION-VIEW https://www.postgresql.org/docs/current/functions-admin.html#FUNCTIONS-ADMIN-BACKUP Hope this answers your question Justin

Re: find replication slots that "belong" to a publication

2025-04-07 Thread Justin
I can't think of a way to link publication to a replication slot I agree using pg_state_activity is the only way to do that however you don't know if the subscriber is momentary disconnected due network error or disconnected due to an error in replication such as duplicated key SELECT true f

Re: Logical decoding

2025-02-21 Thread Justin
On Thu, Feb 20, 2025 at 12:04 AM Jethish Jethish wrote: > Hi everyone, > > Is there is any option to perform logical decoding on an active > replication slot. > I'm trying to decode a replication slot but it throughs an error as below. > > ERROR: replication slot "my_sub" is active for PID 252572

circular wait not triggering deadlock ?

2018-03-08 Thread Justin Pryzby
00:00:00 /usr/pgsql-10/bin/pg_repack -E warning --no-kill-backend -d ts -i eric_enodeb_cell_20180304_site_idx -Ss oldindex Note, I believe last night our backup job would've run for longer and processed (many) more tables than usual, looping around pg_dump --snapshot. Is it a bug that this isn't caught by a deadlock detector and cancelled? Thanks, Justin

Re: circular wait not triggering deadlock ?

2018-03-08 Thread Justin Pryzby
On Thu, Mar 08, 2018 at 01:57:06PM -0500, Tom Lane wrote: > Justin Pryzby writes: > > Running pg10.2, I have a handful of maintenance jobs run in the middle of > > the > > night, which appear to have gotten stuck waiting on each other.. > > > ts=# SELECT granted,

Re: circular wait not triggering deadlock ?

2018-03-08 Thread Justin Pryzby
On Thu, Mar 08, 2018 at 03:05:36PM -0500, Tom Lane wrote: > Justin Pryzby writes: > > On Thu, Mar 08, 2018 at 01:57:06PM -0500, Tom Lane wrote: > >> PID 20488 is evidently waiting for PID 6471 to finish its transaction. > >> What's that one doing? > > >

relkind='p' has no pg_stat_user_tables

2018-05-03 Thread Justin Pryzby
node. Is there any good workaround other than making stampfiles or making my own "last analyzed" table? Thanks, Justin

Re: relkind='p' has no pg_stat_user_tables

2018-05-03 Thread Justin Pryzby
On Thu, May 03, 2018 at 07:44:24AM -0700, Adrian Klaver wrote: > On 05/03/2018 07:14 AM, Justin Pryzby wrote: > >I (finally) realized that my script for ANALYZEing parents of table > >hierarchies > >every month or so was looping around the same parent tables every night due

Re: relkind='p' has no pg_stat_user_tables

2018-05-03 Thread Justin Pryzby
hich > >>are created using separate CREATE TABLE commands. The partitioned table is > >>itself empty. A data row inserted into the table is routed to a partition > >>based on the value of columns or expressions in the partition key. ... " > > > >Yeah, but I t

Re: relkind='p' has no pg_stat_user_tables

2018-05-03 Thread Justin Pryzby
On Thu, May 03, 2018 at 11:15:19AM -0700, Adrian Klaver wrote: > On 05/03/2018 10:38 AM, Justin Pryzby wrote: > >On Thu, May 03, 2018 at 09:31:12AM -0700, Adrian Klaver wrote: > >>On 05/03/2018 09:20 AM, Alvaro Herrera wrote: > >>>>https://www.postgresql.org/do

postgres=# VACUUM FULL pg_statistic => ERROR: missing chunk number 0 for toast value .. in pg_toast_2619

2018-05-19 Thread Justin Pryzby
ombination of REINDEX/VACUUM/ANALYZE, and the only complication was me needing to realize the right combination of affected DB(s). Thanks, Justin

Re: postgres=# VACUUM FULL pg_statistic => ERROR: missing chunk number 0 for toast value .. in pg_toast_2619

2018-05-19 Thread Justin Pryzby
On Sat, May 19, 2018 at 11:08:23AM -0400, Tom Lane wrote: > Justin Pryzby writes: > > I'll defer fixing this for awhile in case someone wants me to save a copy of > > the relation/toast/index. From last time, I recall this just needs the > > right > > combinatio

Re: postgres=# VACUUM FULL pg_statistic => ERROR: missing chunk number 0 for toast value .. in pg_toast_2619

2018-05-19 Thread Justin Pryzby
On Sat, May 19, 2018 at 11:24:57AM -0500, Justin Pryzby wrote: > On Sat, May 19, 2018 at 11:08:23AM -0400, Tom Lane wrote: > > Justin Pryzby writes: > > > I'll defer fixing this for awhile in case someone wants me to save a copy > > > of > > > the relati

Re: postgres=# VACUUM FULL pg_statistic => ERROR: missing chunk number 0 for toast value .. in pg_toast_2619

2018-05-19 Thread Justin Pryzby
On Sat, May 19, 2018 at 02:39:26PM -0400, Tom Lane wrote: > Hm, so was the timeout error happening every time through on that table, > or just occasionally, or did you provoke it somehow? I'm wondering how > your 9s timeout relates to the expected completion time. I did not knowingly provoke it :

Re: postgres=# VACUUM FULL pg_statistic => ERROR: missing chunk number 0 for toast value .. in pg_toast_2619

2018-05-25 Thread Justin Pryzby
On Sat, May 19, 2018 at 02:39:26PM -0400, Tom Lane wrote: > Justin Pryzby writes: > > [pryzbyj@database ~]$ while :; do for db in `psql postgres -Atc "SELECT > > datname FROM pg_database WHERE datallowconn"`; do for t in pg_statistic > > pg_attrdef

\d t: ERROR: XX000: cache lookup failed for relation

2018-05-26 Thread Justin Pryzby
@kgfs66telsadb ~]$ psql --port 5678 postgres -x psql (11beta1) Type "help" for help. ... postgres=# \set VERBOSITY verbose postgres=# \d t ERROR: XX000: cache lookup failed for relation 8096742 LOCATION: flatten_reloptions, ruleutils.c:11065 Justin

notes from transition to relkind='p'

2018-06-01 Thread Justin Pryzby
ld vs parent relative to NO/INHERIT. . And actually, having both ALTER TABLE DE/TACH vs NO/INHERIT is itself messy: we ended up having branches (both shell and python) to handle both cases (at least for a transitional period, but probably we'll need to continue handling both into the indeterminate future). Cheers, Justin

Re: notes from transition to relkind='p'

2018-06-04 Thread Justin Pryzby
On Sun, Jun 03, 2018 at 12:13:49PM -0700, Adrian Klaver wrote: > On 06/01/2018 03:14 PM, Justin Pryzby wrote: > >Before I forget any more, this is a brain of issues/considerations/concerns Should have said brain DUMP Keep in mind, I've phrased these as notes, maybe useful to someo

Re: Code of Conduct plan

2018-06-04 Thread Justin Clift
ely, people who _need_ rules in order to feel comfortable will start to stick around. Neither group is intrinsically right nor wrong. They just operate internally differently, and have different needs. Adding a CoC will change the quantity-of-fules mix _slightly_, depending on how in-your-face people are with it. Our Community will naturally adjust it's makeup over time to reflect this change. Mentioning the above, as I hope we're going into this "eyes wide open". ;) + Justin -- "My grandfather once told me that there are two kinds of people: those who work and those who take the credit. He told me to try to be in the first group; there was less competition there." - Indira Gandhi

Re: Code of Conduct plan

2018-06-04 Thread Justin Clift
On 4 Jun 2018, at 18:24, Justin Clift wrote: > Adding a CoC will change the quantity-of-fules mix _slightly_, depending on > how in-your-face people > are with it. s/quantity-of-fules/quantity-of-rules/ Interesting typo though. :) -- "My grandfather once told me that there ar

Re: Microsoft buys GitHub, is this a threat to open-source

2018-06-04 Thread Justin Clift
ere are more options. Gitea (Open Source GitHub clone) is pretty good: https://gitea.io It's also very efficient resource wise (unlike GitLab), so can run effectively on tiny hardware. Even Raspberry Pi level can do a decent job for small scale stuff. Naturally, anyone with team-sized nee

Re: Code of Conduct plan

2018-06-04 Thread Justin Clift
s the occasional teen who does meaningful stuff with Open Source. So, "we are all adults here" might not actually be 100% correct. :D + Justin

Re: Microsoft buys GitHub, is this a threat to open-source

2018-06-07 Thread Justin Clift
, we seem to be fairly off topic now... + Justin

Re: Code of Conduct plan

2018-06-08 Thread Justin Clift
her or not they come about in the PG Community or not is a different matter. My point being that arbitration isn't necessarily automatically the right direction. I'd probably leave it up to the CoC team/people to figure it out. :) + Justin

ALTER TABLE SET (toast.asdf) is not rejected

2018-06-11 Thread Justin Pryzby
, but found this one. https://www.postgresql.org/message-id/flat/20090114144332.GF24156%40alvh.no-ip.org Justin

Re: ALTER TABLE SET (toast.asdf) is not rejected ... if a table has no toast

2018-06-15 Thread Justin Pryzby
On Mon, Jun 11, 2018 at 11:47:59AM -0400, Alvaro Herrera wrote: > On 2018-Jun-11, Justin Pryzby wrote: > > > I noticed that this is accepted: > > > > postgres=# ALTER TABLE t SET (toast.asdf=128); > > ALTER TABLE > > > > I thought since "toast&q

pg_stat_activity.query_start in the future?

2018-08-17 Thread Justin Pryzby
postgres=# \! date Fri Aug 17 11:10:58 EDT 2018 Justin

OOM with many sorts

2019-07-08 Thread Justin Pryzby
uot; of partitions. It seems to me that would require tiny work_mem, which would be devastating to some workloads. This is not a contrived test case, it's one of our reports run across

Re: postgres=# VACUUM FULL pg_statistic => ERROR: missing chunk number 0 for toast value .. in pg_toast_2619

2019-08-30 Thread Justin Pryzby
ing CLUSTER. I imagine that's related issue. I haven't seen this in awhile (but stopped trying to reproduce it long ago). A recently-deployed update to this maintenance script is probably why it's now doing CLUSTER. On Fri, May 25, 2018 at 08:49:50AM -0500, Justin Pryzby wrote: &g

v12 and pg_restore -f-

2019-10-06 Thread Justin Pryzby
istribution we're running. Unfortunately, I can't think of anything portable across *OS* or useful to include in documentation. In the worst case, someone might need to call pg_restore differently based on its version. Justin

Fwd: PG12 autovac issues

2020-03-17 Thread Justin King
and why it suddenly started when we moved from PG10 > PG12. The configs and workload are essentially the same between versions. We realize we could simply increase the autovacuum_freeze_max_age, but that doesn't seem to actually resolve anything -- it just pushes the problem out. Has anyone seen anything similar to this? Thanks very much for the consideration. Justin King http://flightaware.com/

Re: Fwd: PG12 autovac issues

2020-03-17 Thread Justin King
On Tue, Mar 17, 2020 at 5:39 PM Adrian Klaver wrote: > > On 3/17/20 3:22 PM, Justin King wrote: > > Apologies, I accidentally sent this to the pgsql-admin list initially > > but intended it go here: > > > > We have a database that isn't overly large (~20G

  1   2   >