Re: Better, consistent instrumentation for postgreSQL using a similar API as Oracle
On Mon, 2021-10-04 at 21:51 -0400, Mladen Gogala wrote: > > On 10/4/21 02:34, Laurenz Albe wrote: > > On Fri, 2021-10-01 at 15:06 -0500, Jeff Holt wrote: > > > TLDR; If I spend the time necessary to instrument the many functions that > > > are the equivalent > > > of the Oracle counterparts, would anyone pull those changes and use them? > > > Specifically, for those who know Oracle, I'm talking about implementing: > > > 1. The portion of the ALTER SESSION that enables extended SQL trace > > > 2. Most of the DBMS_MONITOR and DBMS_APPLICATION_INFO packages > > > 3. Instrument the thousand or so functions that are the equivalent of > > > those found in Oracle's V$EVENT_NAME > > > 4. Dynamic performance view V$DIAG_INFO > > > > Anything that improves user experience in that respect is welcome, but > > consider > > that each database has different approaches to solve the same problems. > > > > Before you go to the length of implementing a lot of stuff, check in with > > the -hackers list and discuss your ideas. > > > > Please be a lot more specific than in this e-mail. While it is certainly > > fine to sketch your ambitios vision, focus on one specific thing you can > > imagine implementing and come up with a design for that. > > > > Note that "Oracle has it" is not a good enough reason for a PostgreSQL > > feature. We think we can do better than they do (at least in many > > respects). > > Also, don't assume that everyone on the -hackers list will be familiar with > > certain PostgreSQL features. > > > > One think that you should keep in mind is that Oracle has to provide > > different > > features in that area because they are not open source. In PostgreSQL, I > > can > > simply read the code or attach a debugger to a backend, and when it comes to > > profiling, "perf" works pretty well. So there is less need for these > > things. > > > > I don't want to discourage you, but contributing to PostgreSQL can be a > > lengthy > > and tedious process. On the upside, things that make it into core are > > usually > > fairly mature. > > > > Laurenz, you are obviously not aware who are you talking to. Let me > introduce you: Cary Millsap and Jeff Holt are authors of the "Optimizing > Oracle for Performance", one of the most influential books in the entire > realm of Oracle literature. I have never heard of Jeff Holt, but then there are a lot of wonderful and smart people I have never heard of. I tend to be respectful in my conversation, regardless if I know the other person or not. > Haughty lectures about "Oracle has it" not being good enough could > hardly be more out of place here. I have no idea how you arrive at the conclusion that I was delivering a haughty lecture. Somebody asked if PostgreSQL would consider applying patches he is ready to write, somebody who seems not to be familiar with the way PostgreSQL development works, so I tried to give helpful pointers. > To put it as politely as is possible in this case, shut your pie hole. I think you have just disqualified yourself from taking part in this conversation. I recommend that you don't embarrass Jeff Holt by trying to champion him. Yours, Laurenz Albe
Re: Better, consistent instrumentation for postgreSQL using a similar API as Oracle
Em ter., 5 de out. de 2021 às 01:04, Mladen Gogala escreveu: > As for Jeff Holt, I believe that a person of his stature needs to be > taken seriously and not lectured "how are things done in Postgres > community". I am rather confused by the thinly veiled hostility toward > Oracle. In my opinion, Postgres community should be rather welcoming to > Oracle people like Frits Hoogland, Frank Pachot or Jeff Holt. > I think that you're a little mistaken, the hostility of the "gurus" is not exactly against Oracle guys, but rather towards anyone who is not a "committer". Just follow the pgsql-hackers list, and you'll see that newbies are very unwelcome, whether they're really newbies like me, or they're really teachers. regards, Ranier Vilela
Re: Better, consistent instrumentation for postgreSQL using a similar API as Oracle
Comments in-line: On 10/5/21 04:26, Laurenz Albe wrote: have never heard of Jeff Holt, but then there are a lot of wonderful and smart people I have never heard of. I tend to be respectful in my conversation, regardless if I know the other person or not. That much is apparent. However, that's no excuse. Haughty lectures about "Oracle has it" not being good enough could hardly be more out of place here. I have no idea how you arrive at the conclusion that I was delivering a haughty lecture. Somebody asked if PostgreSQL would consider applying patches he is ready to write, somebody who seems not to be familiar with the way PostgreSQL development works, so I tried to give helpful pointers. Your tone of voice did. Plus, you took it on yourself to explain "how things are done in the Postgres community". I always use hints and Josh Berkus as an example "how things are done in the Postgres community" and why is the Postgres progress so slow. You have just provided me another perfect example of the "community spirit". To put it as politely as is possible in this case, shut your pie hole. I think you have just disqualified yourself from taking part in this conversation. I recommend that you don't embarrass Jeff Holt by trying to champion him. If you are under impression that I want to take part in a conversation, then you're sorely mistaken. And I have to adjust my style of writing to make things even more clear. As for Jeff, I don't need to 'champion him'. He did that all by himself. In his place, I would simply ignore both this topic and you, Mr. Postgres Community. -- Mladen Gogala Database Consultant Tel: (347) 321-1217 https://dbwhisperer.wordpress.com
Re: Better, consistent instrumentation for postgreSQL using a similar API as Oracle
On Fri, Oct 1, 2021 at 1:06 PM Jeff Holt wrote: > Now looking closely at postgreSQL, I see an opportunity to more quickly > implement Oracle's current feature list. > > I've come to this point because I see many roadblocks for users who want to > see a detailed "receipt" for their response time. I have heard of method R. Offhand it seems roughly comparable to something like the Top-down Microarchitecture Analysis Method that low level systems programmers sometimes use, along with Intel's pmu-tools -- at least at a very high level. The point seems to be to provide a workflow that can plausibly zero in on low-level bottlenecks, by providing high level context. Many tricky real world problems are in some sense a high level problem that is disguised as a low level problem. And so all of the pieces need to be present on the board, so to speak. Does that sound accurate? One obvious issue with much of the Postgres instrumentation is that it makes it hard to see how things change over time. I think that that is often *way* more informative than static snapshots. I can see why you'd emphasize the need for PostgreSQL to more or less own the end to end experience for something like this. It doesn't necessarily follow that the underlying implementation cannot make use of infrastructure like eBPF, though. Fast user space probes provably have no overhead, and can be compiled-in by distros that can support it. There hasn't been a consistent effort to make that stuff available, but I doubt that that tells us much about what is possible. The probes that we have today are somewhat of a grab-bag, that aren't particularly useful -- so it's a chicken-and-egg thing. It would probably be helpful if you could describe what you feel is missing in more general terms -- while perhaps giving specific practical examples of specific scenarios that give us some sense of what the strengths of the model are. ISTM that it's not so much a lack of automation in PostgreSQL. It's more like a lack of a generalized model, which includes automation, but also some high level top-down theory. -- Peter Geoghegan
Re: Better, consistent instrumentation for postgreSQL using a similar API as Oracle
Comments in-line On 10/5/21 16:24, Peter Geoghegan wrote: On Fri, Oct 1, 2021 at 1:06 PM Jeff Holt wrote: Now looking closely at postgreSQL, I see an opportunity to more quickly implement Oracle's current feature list. I've come to this point because I see many roadblocks for users who want to see a detailed "receipt" for their response time. I have heard of method R. Offhand it seems roughly comparable to something like the Top-down Microarchitecture Analysis Method that low level systems programmers sometimes use, along with Intel's pmu-tools -- at least at a very high level. The point seems to be to provide a workflow that can plausibly zero in on low-level bottlenecks, by providing high level context. Many tricky real world problems are in some sense a high level problem that is disguised as a low level problem. And so all of the pieces need to be present on the board, so to speak. Does that sound accurate? Yes, that is pretty accurate. It is essentially the same method described in the "High Performance Computing" books. The trick is to figure what the process is waiting for and then reduce the wait times. All computers wait at the same speed. One obvious issue with much of the Postgres instrumentation is that it makes it hard to see how things change over time. I think that that is often *way* more informative than static snapshots. I can see why you'd emphasize the need for PostgreSQL to more or less own the end to end experience for something like this. It doesn't necessarily follow that the underlying implementation cannot make use of infrastructure like eBPF, though. Fast user space probes provably have no overhead, and can be compiled-in by distros that can support it. There hasn't been a consistent effort to make that stuff available, but I doubt that that tells us much about what is possible. The probes that we have today are somewhat of a grab-bag, that aren't particularly useful -- so it's a chicken-and-egg thing. Not exactly. There already is a very good extension for Postgres called pg_wait_sampling: https://github.com/postgrespro/pg_wait_sampling What is missing here is mostly the documentation. This extension should become a part of Postgres proper and the events should be documented as they are (mostly) documented for Oracle. Oracle uses trace files instead. However, with Postgres equivalence of files and tables, this is not a big difference. It would probably be helpful if you could describe what you feel is missing in more general terms -- while perhaps giving specific practical examples of specific scenarios that give us some sense of what the strengths of the model are. ISTM that it's not so much a lack of automation in PostgreSQL. It's more like a lack of a generalized model, which includes automation, but also some high level top-down theory. I am not Jeff and my opinion is not as valuable and doesn't carry the same weight, by far. However, I do believe that we may not see Jeff Holt again on this group so I am providing my opinion instead. At least I would, in Jeff's place, be reluctant to return to this group. -- Mladen Gogala Database Consultant Tel: (347) 321-1217 https://dbwhisperer.wordpress.com
Re: Better, consistent instrumentation for postgreSQL using a similar API as Oracle
Jeff Holt is probably pretty embarrassed there's some blowhard making a scene using his name in a casual mailing list thread. On Tue, Oct 5, 2021 at 5:28 PM Mladen Gogala wrote: > Comments in-line > > On 10/5/21 16:24, Peter Geoghegan wrote: > > On Fri, Oct 1, 2021 at 1:06 PM Jeff Holt wrote: > >> Now looking closely at postgreSQL, I see an opportunity to more quickly > implement Oracle's current feature list. > >> > >> I've come to this point because I see many roadblocks for users who > want to see a detailed "receipt" for their response time. > > I have heard of method R. Offhand it seems roughly comparable to > > something like the Top-down Microarchitecture Analysis Method that low > > level systems programmers sometimes use, along with Intel's pmu-tools > > -- at least at a very high level. The point seems to be to provide a > > workflow that can plausibly zero in on low-level bottlenecks, by > > providing high level context. Many tricky real world problems are in > > some sense a high level problem that is disguised as a low level > > problem. And so all of the pieces need to be present on the board, so > > to speak. > > > > Does that sound accurate? > Yes, that is pretty accurate. It is essentially the same method > described in the "High Performance Computing" books. The trick is to > figure what the process is waiting for and then reduce the wait times. > All computers wait at the same speed. > > One obvious issue with much of the Postgres instrumentation is that it > > makes it hard to see how things change over time. I think that that is > > often *way* more informative than static snapshots. > > > > I can see why you'd emphasize the need for PostgreSQL to more or less > > own the end to end experience for something like this. It doesn't > > necessarily follow that the underlying implementation cannot make use > > of infrastructure like eBPF, though. Fast user space probes provably > > have no overhead, and can be compiled-in by distros that can support > > it. There hasn't been a consistent effort to make that stuff > > available, but I doubt that that tells us much about what is possible. > > The probes that we have today are somewhat of a grab-bag, that aren't > > particularly useful -- so it's a chicken-and-egg thing. > > Not exactly. There already is a very good extension for Postgres called > pg_wait_sampling: > > https://github.com/postgrespro/pg_wait_sampling > > What is missing here is mostly the documentation. This extension should > become a part of Postgres proper and the events should be documented as > they are (mostly) documented for Oracle. Oracle uses trace files > instead. However, with Postgres equivalence of files and tables, this is > not a big difference. > > > > > > It would probably be helpful if you could describe what you feel is > > missing in more general terms -- while perhaps giving specific > > practical examples of specific scenarios that give us some sense of > > what the strengths of the model are. ISTM that it's not so much a lack > > of automation in PostgreSQL. It's more like a lack of a generalized > > model, which includes automation, but also some high level top-down > > theory. > > I am not Jeff and my opinion is not as valuable and doesn't carry the > same weight, by far. However, I do believe that we may not see Jeff Holt > again on this group so I am providing my opinion instead. At least I > would, in Jeff's place, be reluctant to return to this group. > > > -- > Mladen Gogala > Database Consultant > Tel: (347) 321-1217 > https://dbwhisperer.wordpress.com > > > >
Re: Better, consistent instrumentation for postgreSQL using a similar API as Oracle
On 10/5/21 20:02, Tim wrote: Jeff Holt is probably pretty embarrassed there's some blowhard making a scene using his name in a casual mailing list thread. Wow! What a contribution to the discussion! Calling me a blowhard, all while top-posting at the same time. Your post will be remembered for generations to come. Or not. Laurenz will probably tell you that we don't top-post in Postgres community. He's good with rules, regulations and the way things are done in Postgres community. -- Mladen Gogala Database Consultant Tel: (347) 321-1217 https://dbwhisperer.wordpress.com
Installation of PostgreSQL on fedora zVM
Hi All, Have anyone tried to install PostgreSQL on the VM provisioned on IBM Z? If yes please could you share the installation instructions or point to the blog etc. Thanks in advance. Thanks & Regards, Manish
