Re: [Pdns-users] public DoH/DoT dnsdist 1.9.8 exited on signal 11

These crashes continued on an almost daily basis until we updated to 1.9.9 in early May, since then we have never seen a dnsdist crash. So maybe it was related to CVE-2025-30194 or some other changes in 1.9.9 after all, not sure. best regards, Christoph

Re: [Pdns-users] public DoH/DoT dnsdist 1.9.8 exited on signal 11

dnsdist v1.9.7 crashed today as well, so did nginx on the same host. I no longer believe it is a dnsdist issue but I do not understand why it crashes on two separate servers.. best regards, Christoph ___ Pdns-users mailing list Pdns-users@mailman.pow

Re: [Pdns-users] public DoH/DoT dnsdist 1.9.8 exited on signal 11

Hi Winfried, Winfried via Pdns-users: My two cents. Because crashes with dnsdist are really very unusual, I wouldn't rule out hardware errors. Especially because we've had that happen ourselves. In the end, bad memory was the cause of rare crashes. So I would recommend rebooting the server a few

Re: [Pdns-users] public DoH/DoT dnsdist 1.9.8 exited on signal 11

Hi Remi, Remi Gacogne wrote: I'm not aware of any bug in 1.9.8 that could cause a crash, no. It's hard to narrow it down with the information you have, unfortunately. Did you upgrade recently from a previous version of dnsdist (in which case I can look at the diff since the previous version)?

[Pdns-users] public DoH/DoT dnsdist 1.9.8 exited on signal 11

Hi, our public DoH/DoT dnsdist instance just crashed. It is the first time I see a dnsdist crash. Unfortunately we do not have any core dump. Jan 29 22:48:09 kernel: pid 75804 (dnsdist), jid 0, uid 208: exited on signal 11 (no core dump - bad address) Our dnsdist runs behind an nginx and forw

[Pdns-users] rec_control show-yaml experience

Hi, I want to share my experience when trying to upgrade from recursor 5.1 to 5.2 on debian 12 using the powerdns repo. Maybe it is of use for others running into the same problems. After reading https://docs.powerdns.com/recursor/upgrade.html https://docs.powerdns.com/recursor/appendices/yam

Re: [Pdns-users] FreeBSD Recursor 5.0.7 -> 5.1.1 upgrade warning (ships new empty yml config)

fair warning if you are upgrading Recursor on FreeBSD via pkg upgrade: 5.0.7 -> 5.1.1 The FreeBSD powerdns-recursor package ships a default recursor.yml config file but does not convert the recursor.conf to recursor.yml In our case this resulted in a broken recursor service because it tried

[Pdns-users] Recursor 4.8.x Debian 12 repo

Hi, for regression testing we would like to downgrade our recursor to version 4.8.x but we noticed that there is no rec-48 debian repo on https://repo.powerdns.com/debian/dists/ for Debian Bookworm. Is this on purpose or will there be a recursor 48 repo for Debian 12 in the future? thank yo

[Pdns-users] DNSdist 1.9.0-alpha1 released dnsdist_server_healthcheckfailurestimeout

Hi Remi, We have made a lot of small improvements since 1.8.x as well, like adding Lua bindings to access selectors and actions, more fields of a DNS header in Lua actions, and adding metrics for health-check events. thanks a lot! We are already running it on one server to get the new healthc

Re: [Pdns-users] Error prio events with loglevel 2

Thanks for looking into this. I've filed it as a github issue now. As a workaround I'm now trying to block these DNS queries in dnsdist, so they do not reach recursor and the logs: addAction(QTypeRule(qtype from the logs), RCodeAction(DNSRCode.NOTIMP)) best regards, Christoph

Re: [Pdns-users] Error prio events with loglevel 2

level="0" I noticed this part of the log entry just now. Which could mean that this is an emergency loglevel entry and therefore expected at loglevel 2. Is there any other way to avoid logging qnames in these events? best regards, Christoph ___ Pdns-

[Pdns-users] Error prio events with loglevel 2

Hello, we changed our recursor loglevel from 3 to 2 with the intention to avoid logging these events because they contain qnames: msg="qtype unsupported" error="Cannot push task" subsystem="taskq" level="0" prio="Error" tid="6" ts="..." name="..." netmask="" qtype="TYPE65535" but these event

Re: [Pdns-users] Recursor Cache Sizing: Is more always better?

Hi Winfried, My recommendation is to limit the TTL to 12 or 6 hours and find out how many cache entries are created during this time. Increase that by 50% and that's your value. thanks for your recommendation. I've played a bit with this to see what max-cache-entries values this procedure wou

Re: [Pdns-users] Recursor Cache Sizing: Is more always better?

Another word of advice: see https://docs.powerdns.com/recursor/performance.html#threading-and-distribution-of-queries in particular the "imbalance" section. Thanks for the pointer, changing this had a significant positive impact. This feels like an important metric to monitor. I was not able

Re: [Pdns-users] Recursor forwarder DoT configuration

If you need DNSEC validation you must use recursor, dnsdist cannot do that. Others might reflect on the dnsdist cache performance and hit ratio's compared to recursor's packet cache and/or record cache. Do note that dnsdist cache is more like the recursor's packet cache. Thanks for confirming

Re: [Pdns-users] Recursor Cache Sizing: Is more always better?

Agrreed, I think that general rules are hard to give for cache sizing, as each site and its users are different. Do remember that the packet cache was changed in 4.9.0, it is now shared between threads. This means that its performance and behaviour wrt hit ratio etc did change as well. The differe

[Pdns-users] Recursor Cache Sizing: Is more always better?

Hi, if you have 20 or 100 GB of free RAM what is a good approach to choose the different Recursor's cache sizes? Is larger always better or is there a sweet spot between cache size, cache lookup time, cache management overhead and CPU usage? How does upstream latency fit into the equation? In

Re: [Pdns-users] Recursor forwarder DoT configuration

I do wonder about the purpose of the recursor in the recursor -> dnsdist -> upstream-recursive case. You might as well use dnsdist -> upstream-recursive With a caching dnsdist. Unless you need recursor specific functionality, of course. It was my impression that dnsdist was meant for smaller

Re: [Pdns-users] Recursor forwarder DoT configuration

Thanks a lot for the fast reply, very much appreciated! best regards, Christoph ___ Pdns-users mailing list Pdns-users@mailman.powerdns.com https://mailman.powerdns.com/mailman/listinfo/pdns-users

[Pdns-users] Recursor forwarder DoT configuration

Hello! I'm looking for documentation about configuring recursor to talk DoT to a recursive resolver. This minimal config works: dot-to-port-853=yes forward-zones-recurse=.=1.1.1.1:853;1.0.0.1:853 but compared to DNSdist newServer() configuration options I'm not sure about: - does it validate

Re: [Pdns-users] CPU Usage Regression in Recursor 4.9.1?

Would it be possible to give me some stats on the aggresisve cache on the node(s) showing the issue? Sure, I'll send them directly to you. best regards, Christoph ___ Pdns-users mailing list Pdns-users@mailman.powerdns.com https://mailman.powerdns.c

Re: [Pdns-users] Rcode 3 NXDOMAIN for existing CNAME

Hi Peter Thomassen, Since this is the background of the DNS query I find your suggestion a valid solution for the problem that lego could implement. I agree! Thanks for clearing this up, I was on the wrong track about what the goal of that query was. I looked at the pcap again - the one you

Re: [Pdns-users] Rcode 3 NXDOMAIN for existing CNAME

 >> However, I doubt this is a reasonable approach for your ACME  >> client. Sounds like a simple enough solution to me, can you elaborate why you doubt it is reasonable? My understanding is that ACME is about whether there is a TXT RRset with the challenge record; if it is not there, it's irr

[Pdns-users] Rcode 3 NXDOMAIN for existing CNAME

Hi, let me start with some context information: We are using the lego [1] ACME client with a DNS challenge and desec.io as our DNS provider and run into a problem that results in the failure of certificate renewals. According to the lego developer the root cause is a DNS issue [2] in our envir

[Pdns-users] encrypted zone transfer / RFC9103 support

Hi, are encrypted zone transfers (and TSIG + source IP ACLs) supported by PowerDNS by running dnsdist with DoT in front of an authoritative NS? thanks, Christoph https://dnsprivacy.org/implementation_status/#xfrxot-implementation-status https://www.rfc-editor.org/rfc/rfc9103 https://datatrack

Re: [Pdns-users] troubleshooting dnsdist -> recursor instability

Hi Remi, Clearly unexpected. You might be able to get more information about what is going by setting setVerboseHealthChecks [1] in dnsdist. very good point, thank you I'll enable it! One unrelated thing I noticed in your configuration is that you are setting maxInFlight to 1000 on newServe

Re: [Pdns-users] troubleshooting dnsdist -> recursor instability

Hi, thanks for the pointers. A coincidence helped getting a bit closer to the root cause maybe: Due to a linux kernel update the server had to be rebooted, after the reboot the problem disappeared. Stacked graphs of irate(pdns_recursor_sys_msec... irate(pdns_recursor_user_msec... show that the

[Pdns-users] troubleshooting dnsdist -> recursor instability

Hi, we have the following setup running on debian 11 bare metal servers and doing about 250 qps: dnsdist -> recursor1 (running on the same machine as dnsdist/localhost) -> recursor2 (on a second machine on the same network) Full config files are found at the end of this email. The lo

Re: [Pdns-users] Will DoT disappear in favor of DoQ for recursor to auth?

> No plans. > > Currently, Recursor does not support outgoing DoQ. If/when we start > supporting outgoing DoQ it would not *imply* dropping outgoing DoT. Thanks for your reply. > So it seems DoT is only supported on v4. Also not that the domain > listed for an IP is the first name the lead to a

[Pdns-users] Will DoT disappear in favor of DoQ for recursor to auth?

Hi, does the PowerDNS team have any specific plans to remove DoT support for recursor to authoritative queries in favor of DoQ in PowerDNS Recursor? thanks, Christoph related links: https://blog.powerdns.com/2022/06/13/probing-dot-support-of-authoritative-servers-just-try-it/ https://datatracke

[Pdns-users] Recursor: Error writing TCP answer - broken pipe

Hello, I get about 2000 of these log events per day: pdns-recursor[11727]: Error writing TCP answer to 109.70.100.132:31192: Broken pipe 109.70.100.132 is the IP address of an dnsdist instance. setup: DoH/DoT clients -> dnsdist -> recursors Is there anything that can be optimized to avoid t

Re: [Pdns-users] Zone to Cache: url method support in powerdns repo package

msg="Unable to load zone into cache, will retry" subsystem="ztc" level=0 ts="1636499834.251" exception="url method configured but libcurl not compiled in" refresh="60" zone="." installed from your debian repo, version: 4.6.0~beta1-1pdns.bullseye Thanks for the report, will fix soon. Can you t

[Pdns-users] Zone to Cache: url method support in powerdns repo package

Otto Moerbeek via Pdns-users: * A new Zone to Cache[1] function that will retrieve a zone (using AXFR, HTTP, HTTPS or a local file) periodically and insert the contents into the record cache, allowing the cache to be always hot for a zone. This can be used for the

[Pdns-users] overall cache hit rate in a dnsdist -> recursor environment

Hi, I'd like to calculate the overall percentage of queries that got completely answered with a cached entry, regardless of the type of cache (packet cache or not) and regardless of whether the cache was in dnsdist or Recursor. simple setup: clients -> dnsdist -> Recursor Would you say this i

Re: [Pdns-users] resource-limits metrics

Hi Remi, Remi Gacogne wrote: This counter is increased when we encounter a network issue that does not seem to be caused by the remote end but by a problem on our side, like if the recursor runs out of file descriptors or if we don't have a network route to contact a given IP address, for exam

[Pdns-users] resource-limits metrics

Hi, while going over the list of prometheus metrics available in PowerDNS Recursor I found this one: resource-limits counts number of queries that could not be performed because of resource limits https://docs.powerdns.com/recursor/metrics.html#resource-limits I guess if this value is inc

[Pdns-users] Injection Attacks Reloaded: Validating hostnames?

Hi, in the light of a recent Usenix paper "Injection Attacks Reloaded: Tunnelling Malicious Payloads over DNS" [1], we tested our dnsdist -> PowerDNS Recursor setup [2] with the following results: (quoting from their test page [3]) Special character filtering These tests will test if your r