Re: is libpq and openssl 1.1.* compatible?
Thank you everyone who posted answers! I went back to openssl-1.0.2. On Mon, Feb 26, 2018 at 1:15 AM, Michael Paquier wrote: > On Mon, Feb 26, 2018 at 12:30:38AM -0700, Konstantin Izmailov wrote: > > Let me ask this differently: can Visual Studio 2013/2017 compile libpq > with > > openssl 1.1 support? Under Windows? > > The answer to this question should be yes. (Please note that no > Windows buildfarm machines use openssl 1.1.0 as far as I can see, I > myself build stuff with 1.0.2 on Windows.) > > > I have not been able to find an answer by googling before asking this > > question here. Can someone share the compiled library and changes in the > > source if needed for the openssl 1.1? Magnus please. > > No changes should be needed as far as I know in the scripts in > src/tools/msvc. The set of APIs present in 1.1.0 is the same whatever > the platform so the compatibility is the same, and the dependent > libraries should be ssleay32.lib and libeay32.lib whose location depend > on your installation of OpenSSL. > -- > Michael >
COPY command returns "ERROR: invalid XML content"
Hi,
I'm using libpq (v10) to import lots of xml files into a PG10 table. I
noticed if number of records imported exceeds 2100 then the following error
is returned:
ERROR: invalid XML content
DETAIL: line 1: Couldn't find end of Start Tag
timeBasedFileNamingAndTriggerin line 1
logFile.%d{-MM-dd}.%i.html
Re: COPY command returns "ERROR: invalid XML content"
Tomas, thank you for your reply! I cannot upload 2100+ xml files. Some of
them are huge.
I'm not sure if libpq is using libxml2 on Windows. In debugger I see very
strange behavior of pqsecure_write. It seems like it stops sending data
from provided buffer after 9,100,000 bytes.
I hoped that someone came across similar issue, and bring some insight. I
continue researching the issue.
On Mon, Oct 7, 2019 at 5:13 AM Tomas Vondra
wrote:
> On Sun, Oct 06, 2019 at 08:45:40PM -0700, Konstantin Izmailov wrote:
> >Hi,
> >I'm using libpq (v10) to import lots of xml files into a PG10 table. I
> >noticed if number of records imported exceeds 2100 then the following
> error
> >is returned:
> >ERROR: invalid XML content
> >DETAIL: line 1: Couldn't find end of Start Tag
> >timeBasedFileNamingAndTriggerin line 1
>
> >logFile.%d{-MM-dd}.%i.html >
>
> My guess is this is an issue/limitation in libxml2, which we use to
> parse and process XML. What libxml2 version you have installed? Can you
> share an example of a XML document to reproduce the issue?
>
> regards
>
> --
> Tomas Vondra http://www.2ndQuadrant.com
> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
>
Re: COPY command returns "ERROR: invalid XML content"
Please ignore this thread. After several days of debugging I found bug in
my application. It was misalignment of data when internal buffers
reallocated. After the application fix it all works as expected. Sorry for
the false alarm.
On Mon, Oct 7, 2019 at 7:03 PM Konstantin Izmailov wrote:
> Tomas, thank you for your reply! I cannot upload 2100+ xml files. Some of
> them are huge.
>
> I'm not sure if libpq is using libxml2 on Windows. In debugger I see very
> strange behavior of pqsecure_write. It seems like it stops sending data
> from provided buffer after 9,100,000 bytes.
>
> I hoped that someone came across similar issue, and bring some insight. I
> continue researching the issue.
>
> On Mon, Oct 7, 2019 at 5:13 AM Tomas Vondra
> wrote:
>
>> On Sun, Oct 06, 2019 at 08:45:40PM -0700, Konstantin Izmailov wrote:
>> >Hi,
>> >I'm using libpq (v10) to import lots of xml files into a PG10 table. I
>> >noticed if number of records imported exceeds 2100 then the following
>> error
>> >is returned:
>> >ERROR: invalid XML content
>> >DETAIL: line 1: Couldn't find end of Start Tag
>> >timeBasedFileNamingAndTriggerin line 1
>>
>> >logFile.%d{-MM-dd}.%i.html> >
>>
>> My guess is this is an issue/limitation in libxml2, which we use to
>> parse and process XML. What libxml2 version you have installed? Can you
>> share an example of a XML document to reproduce the issue?
>>
>> regards
>>
>> --
>> Tomas Vondra http://www.2ndQuadrant.com
>> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
>>
>
programmatically retrieve details of a custom Postgres type
Hello, I was unable to find how to get column names, sizes and types for a given composite type. Example. For a type defines as: CREATE TYPE inventory_item AS ( name text, supplier_id integer, price numeric ); I have a plpgsql stored proc that returns SETOF inventory_item (i.e. there is no table with a column of this type). I looked into the pg_type table but it only contains oid and typrelid for the inventory_item type. I need a query that returns information about structure of the composite type, i.e.: ColumnName | ColumnType | ColumnSize name | text| -1 supplier_id | integer | 4 price | numeric | 16 Is this possible? I'm executing queries via libpq... Thank you!
Re: programmatically retrieve details of a custom Postgres type
Thank you, Pavel and Tom! It works great! On Thu, Nov 10, 2022 at 9:30 PM Tom Lane wrote: > I wrote: > > For a composite type, pg_type.typrelid links to pg_class and pg_attribute > > entries that work much like a table. > > Actually, you could reverse that: for a table, pg_type.typrelid links to > pg_class and pg_attribute entries that work much like a composite type. > > For both relations and composite types, there are pg_class and pg_type > entries that (by convention only) have the same names and namespaces. > They cross-link to each other via pg_class.reltype and pg_type.typrelid. > The associated pg_attribute entries have attrelid matching the pg_class > OID. The catalog entries for the two cases are pretty nearly > indistinguishable except for pg_class.relkind. Again, it's only by > convention that we consider that the pg_type entry is primary for a > composite type but pg_class is primary for a relation. > > Of course, a relation has some underlying storage (for most relkinds), > and it will likely have associated entries in other catalogs that a > composite type won't. But the core catalog entries are about the same. > > regards, tom lane >
assembling PGresults from multiple simultaneous queries (libpq, singlerowmode)
Hi, I'm experimenting with Postgres 10 and protocol v3. I noticed that the Postgres allows executing multiple queries simultaneously (I basically commented out a check that prevents sending another query in libpq while previous result(s) reading is not complete). Things appear like working, but I just wanted to ask if anyone else tried the same (logically separate results from multiple simultaneous queries)? Afaik libpq was not designed for that scenario, but it seems the Postgres server supports it. Any thought or information will be appreciated. Thank you! Konstantin
Re: assembling PGresults from multiple simultaneous queries (libpq, singlerowmode)
Yes, Andres, I meant "pipelining", just couldn't choose correct word. Thank you for the answer(s)! I also made changes in my own copy of libpq, and they work fine. I think the pipelining support is needed in libpq. Btw, how can I get the patch code? I want to compare your approach with mine. I couldn't figure out how to get the patch from the link. Thanks so much! On Sun, Apr 7, 2019 at 12:21 PM Andres Freund wrote: > Hi, > > On 2019-04-07 20:57:56 +0200, Pavel Stehule wrote: > > ne 7. 4. 2019 v 20:47 odesÃlatel Konstantin Izmailov > > napsal: > > > > > Hi, > > > I'm experimenting with Postgres 10 and protocol v3. I noticed that the > > > Postgres allows executing multiple queries simultaneously (I basically > > > commented out a check that prevents sending another query in libpq > while > > > previous result(s) reading is not complete). Things appear like > working, > > > but I just wanted to ask if anyone else tried the same (logically > separate > > > results from multiple simultaneous queries)? > > > > > > > > Postgres cannot to execute simultaneous queries in one session. So > queries > > should be executed in serial form every time. > > I think what Konstantin is really talking about is pipelining > (i.e. sending multiple queries without waiting for the results > inbetween, thereby reducing latency), and that is actually supported by > postgres. Some drivers make fairly extensive use of it (e.g. the pgjdbc > driver). > > As for libpq: No, that's currently not supported. There is a patch that > I hope to get into v13 however: https://commitfest.postgresql.org/20/1317 > > Greetings, > > Andres Freund >
Re: assembling PGresults from multiple simultaneous queries (libpq, singlerowmode)
Never mind, I found the link to the github in emails from the link. Thanks again! Konstantin On Sun, Apr 7, 2019 at 1:28 PM Konstantin Izmailov wrote: > Yes, Andres, I meant "pipelining", just couldn't choose correct word. > Thank you for the answer(s)! > > I also made changes in my own copy of libpq, and they work fine. I think > the pipelining support is needed in libpq. Btw, how can I get the patch > code? I want to compare your approach with mine. I couldn't figure out how > to get the patch from the link. > > Thanks so much! > > > On Sun, Apr 7, 2019 at 12:21 PM Andres Freund wrote: > >> Hi, >> >> On 2019-04-07 20:57:56 +0200, Pavel Stehule wrote: >> > ne 7. 4. 2019 v 20:47 odesÃlatel Konstantin Izmailov >> > napsal: >> > >> > > Hi, >> > > I'm experimenting with Postgres 10 and protocol v3. I noticed that the >> > > Postgres allows executing multiple queries simultaneously (I basically >> > > commented out a check that prevents sending another query in libpq >> while >> > > previous result(s) reading is not complete). Things appear like >> working, >> > > but I just wanted to ask if anyone else tried the same (logically >> separate >> > > results from multiple simultaneous queries)? >> > > >> > > >> > Postgres cannot to execute simultaneous queries in one session. So >> queries >> > should be executed in serial form every time. >> >> I think what Konstantin is really talking about is pipelining >> (i.e. sending multiple queries without waiting for the results >> inbetween, thereby reducing latency), and that is actually supported by >> postgres. Some drivers make fairly extensive use of it (e.g. the pgjdbc >> driver). >> >> As for libpq: No, that's currently not supported. There is a patch that >> I hope to get into v13 however: https://commitfest.postgresql.org/20/1317 >> >> Greetings, >> >> Andres Freund >> >
Re: assembling PGresults from multiple simultaneous queries (libpq, singlerowmode)
Got it! Thanks!
is libpq and openssl 1.1.* compatible?
I got the latest libpq (from pg 10.1), and tried to compile it with openssl 1.1. I got errors: libpq10\fe-secure-openssl.c(1582): error C2037: left of 'ptr' specifies undefined struct/union 'bio_st' libpq10\fe-secure-openssl.c(1582): error C2198: 'pqsecure_raw_read' : too few arguments for call libpq10\fe-secure-openssl.c(1612): error C2037: left of 'ptr' specifies undefined struct/union 'bio_st' libpq10\fe-secure-openssl.c(1612): error C2198: 'pqsecure_raw_write' : too few arguments for call libpq10\fe-secure-openssl.c(1671): error C2027: use of undefined type 'bio_method_st' openssl-1.1.0g\inc32\openssl\bio.h(244) : see declaration of 'bio_method_st' libpq10\fe-secure-openssl.c(1674): error C2027: use of undefined type 'bio_method_st' I wonder if I'm not doing smth right, or libpq is not compatible with openssl 1.1? Please help. Thank you!
Re: is libpq and openssl 1.1.* compatible?
Let me ask this differently: can Visual Studio 2013/2017 compile libpq with openssl 1.1 support? Under Windows? I have not been able to find an answer by googling before asking this question here. Can someone share the compiled library and changes in the source if needed for the openssl 1.1? Magnus please. On Sun, Feb 25, 2018 at 10:44 PM, Michael Paquier wrote: > On Sun, Feb 25, 2018 at 09:06:39PM -0800, Adrian Klaver wrote: > > On 02/25/2018 08:36 PM, Konstantin Izmailov wrote: > >> I wonder if I'm not doing smth right, or libpq is not compatible with > >> openssl 1.1? Please help. > > The answer to the second question is that libpq is compatible with > OpenSSL 1.1.0 since this commit: > commit: 593d4e47db7af1a3a5dd6b6b1971f181b5566dbd > author: Heikki Linnakangas > date: Thu, 15 Sep 2016 14:42:29 +0300 > Support OpenSSL 1.1.0. > > This applies down to 9.5. Porting got a bit more difficult with 9.4 and > older versions. As 1.0.2 is LTS, meaning that it will become EOL after > 1.1.0, porting 1.1.0 does not matter much as most application are going > to use 1.0.2 in priority (I do so for one). > > > Do you have openssl-dev(el) installed? > > Most likely that's the problem. Debian does this package split for > example. > -- > Michael >
