Hi Robby, A valid question, non-stop forwarding and failover for voice services. Normally the session sync is fast enough for most traffic. However, with powerdns I appear to be finding the limits of how fast a session can sync. Which seems like a luxury issue have IMHO. It’s not a major problem, client will rerequest a DNS entry if the first one fails, and it’s only a tiny amount of traffic that’s erroneously dropped.
I think for now I’ll just permit the egress traffic based on the source port. If it was inbound traffic I would be less inclined to do so. -- Best regards, Djerk Geurts > On 12 May 2025, at 15:28, pdns-users-requ...@mailman.powerdns.com wrote: > > Send Pdns-users mailing list submissions to > pdns-users@mailman.powerdns.com > > To subscribe or unsubscribe via the World Wide Web, visit > https://mailman.powerdns.com/mailman/listinfo/pdns-users > or, via email, send a message with subject or body 'help' to > pdns-users-requ...@mailman.powerdns.com > > You can reach the person managing the list at > pdns-users-ow...@mailman.powerdns.com > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Pdns-users digest..." > > > Today's Topics: > > 1. Recursor too fast? (Djerk Geurts) > 2. Re: Recursor too fast? (Robby Pedrica) > 3. Re: No response from pdns-recursor for some clients > (Robby Pedrica) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Mon, 12 May 2025 15:04:44 +0100 > From: Djerk Geurts <dj...@maizymoo.com> > To: pdns-users@mailman.powerdns.com > Subject: [Pdns-users] Recursor too fast? > Message-ID: <6ccd76d8-cc96-4e2a-be86-b8f486c2e...@maizymoo.com> > Content-Type: text/plain; charset="utf-8" > > An odd statement possibly, but I?m looking for a way to solve a problem (even > if it?s a temporary solution). > > The DC firewalls have changed and the recursors are located in a DMZ behind > two HA firewalls in active/active mode. So far so good. The firewalls sync > their state tables, so asymmetric return traffic works fine. Except when the > recursor replies so quickly that the sync hasn?t updated the state table yet > for the return packets. As a result we?re seeing a few drops among a lot of > perfectly fine traffic. > > I have a few things I can do: > > 1) permit all outbound traffic with source udp/53 from the recursors. Not > ideal, but low risk. > 2) raise a support ticket with the firewall vendor. Will do this, but not > holding my breath for a solution (if any) > 3) delay DNS replies a millisecond or so. Not ideal as this introduces delay. > > Thoughts? > > -- > Best regards, > Djerk Geurts > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > <http://mailman.powerdns.com/pipermail/pdns-users/attachments/20250512/62791407/attachment-0001.htm> > > ------------------------------ > > Message: 2 > Date: Mon, 12 May 2025 15:26:57 +0100 > From: Robby Pedrica <rpedr...@gmail.com> > To: pdns-users@mailman.powerdns.com > Subject: Re: [Pdns-users] Recursor too fast? > Message-ID: <1d8adb52-6f8e-42e1-8473-f4591bf50...@gmail.com> > Content-Type: text/plain; charset="utf-8"; Format="flowed" > > Question Djerk: why are you running your firewalls in active/active? > This is an unusual configuration that has many challenges, including the > one you've just mentioned. > > Regards > > Robby > > > On 2025/05/12 15:04, Djerk Geurts via Pdns-users wrote: >> An odd statement possibly, but I?m looking for a way to solve a >> problem (even if it?s a temporary solution). >> >> The DC firewalls have changed and the recursors are located in a DMZ >> behind two HA firewalls in active/active mode. So far so good. The >> firewalls sync their state tables, so asymmetric return traffic works >> fine. Except when the recursor replies so quickly that the sync hasn?t >> updated the state table yet for the return packets. As a result we?re >> seeing a few drops among a lot of perfectly fine traffic. >> >> I have a few things I can do: >> >> 1) permit all outbound traffic with source udp/53 from the recursors. >> Not ideal, but low risk. >> 2) raise a support ticket with the firewall vendor. Will do this, but >> not holding my breath for a solution (if any) >> 3) delay DNS replies a millisecond or so. Not ideal as this introduces >> delay. >> >> Thoughts? >> >> -- >> Best regards, >> *Djerk Geurts* >> >> >> _______________________________________________ >> Pdns-users mailing list >> Pdns-users@mailman.powerdns.com >> https://mailman.powerdns.com/mailman/listinfo/pdns-users > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > <http://mailman.powerdns.com/pipermail/pdns-users/attachments/20250512/e21e6a6b/attachment-0001.htm> > > ------------------------------ > > Message: 3 > Date: Mon, 12 May 2025 15:28:53 +0100 > From: Robby Pedrica <rpedr...@gmail.com> > To: Otto Moerbeek <o...@drijf.net> > Cc: All about using and deploying powerdns > <pdns-users@mailman.powerdns.com> > Subject: Re: [Pdns-users] No response from pdns-recursor for some > clients > Message-ID: <d96d5bed-1292-409a-a56e-f51fadba5...@gmail.com> > Content-Type: text/plain; charset="utf-8"; Format="flowed" > > > On 2025/05/08 15:25, Otto Moerbeek wrote: >> The logs (in your original post) are redacted. So we cannot correlate >> the log lines with your config. If posting unredacted logs is not >> possible we cannot help you here. >> >> -Otto > Thanks Otto, I'll check internally if we can share otherwise, thanks for > your help and assume this request is closed. > > Regards > > Robby >> >> On Thu, May 08, 2025 at 03:00:37PM +0100, Robby Pedrica wrote: >> >>> >>> On 2025/04/30 12:41, Otto Moerbeek wrote: >>>> On Tue, Apr 29, 2025 at 03:18:44PM +0100, Robby Pedrica via Pdns-users >>>> wrote: >>>> >>>>> Hi pdns community >>>>> >>>>> I've got an odd issue where some clients do not get a response from either >>>>> of my 2 recursors. Both are v5.1.4 deployed via docker with fairly std >>>>> configs. Generally the logs will indicate if something is not in the >>>>> allowed-from list but these clients don't show there. For all intents and >>>>> purposes, the recursors work normally and well for all my other clients. >>>>> >>>> Since you left out specifics, it's not possible for us to see what is >>>> going wrong. Please read >>>> https://blog.powerdns.com/2016/01/18/open-source-support-out-in-the-open >>>> and try again with no information edited except secrets like >>>> passwords. >>>> >>>> -Otto >>> Hi Otto >>> >>> 1 - thank you very much for your reply >>> >>> 2 - my apologies for the delayed response however I've been travelling the >>> last week >>> >>> 3 - I intimately understand the requirement to provide as much information >>> as possible as I provide support myself; in this case, I spent a significant >>> amount of time troubleshooting and collecting information on the issue - |I >>> thought I had provided everything relevant but it's clear from your reply >>> that that is not the case; however what is not clear, is what I left out >>> (and the provided link does not assist with specifics either). >>> >>> I'm going to make the assumption that you are referring to the recursor.yml >>> configuration file, and therefore provide that here in full (minus secrets): >>> >>> /// >>> ######### SECTION incoming ######### >>> incoming: >>> ? listen: >>> ? - 0.0.0.0 >>> ? - '::' >>> ? allow_from: >>> #??? - 0.0.0.0/0 >>> ? - 127.0.0.1 >>> ? - 172.0.0.0/8 # docker networks >>> ? - 10.10.10.0/24 # client subnet >>> >>> ##### The load factor used when PowerDNS is distributing queries to worker >>> threads >>> #?? distribution_load_factor: 0.0 >>> ##### Launch this number of distributor threads, distributing queries to >>> other threads >>> #?? distributor_threads: 0 >>> ? port: 53 >>> ? proxy_protocol_from: [105.55.55.33/32] >>> ? use_incoming_edns_subnet: true >>> ##### Maximum number of requests handled concurrently per TCP connection >>> #?? max_concurrent_requests_per_tcp_connection: 10 >>> ##### Maximum number of simultaneous TCP clients >>> ? max_tcp_clients: 128 >>> >>> ######### SECTION logging ######### >>> logging: >>> ? common_errors: true >>> ? disable_syslog: false >>> #?? facility: '' >>> ? loglevel: 6 >>> ##### Suppress logging of questions and answers >>> ? quiet: false >>> >>> ######### SECTION nod ######### >>> nod: >>> ##### Log newly observed domains. >>> ? log: true >>> ##### Track newly observed domains (i.e. never seen before). >>> #?? tracking: false >>> >>> ######### SECTION outgoing ######### >>> outgoing: >>> ? edns_subnet_allow_list: !override >>> ? - 0.0.0.0/0. >>> ? max_busy_dot_probes: 50 >>> >>> ######### SECTION packetcache ######### >>> packetcache: >>> ##### Disable packetcache >>> #?? disable: false >>> >>> ######### SECTION recursor ######### >>> recursor: >>> ? daemon: false >>> ? etc_hosts_file: /etc/hosts >>> ? hint_file: /etc/named.root.txt >>> ? lua_config_file: /etc/proxy-map.lua >>> ##### Launch this number of threads listening for and processing TCP queries >>> #?? tcp_threads: 1 >>> ##### Launch this number of threads >>> ? threads: 4 >>> ##### string reported on version.pdns or version.bind >>> #?? version_string: '*runtime determined*' >>> ? write_pid: true >>> >>> ######### SECTION webservice ######### >>> webservice: >>> ? address: 0.0.0.0 >>> ? allow_from: !override >>> ? - 10.10.11.0/24 >>> ? api_key: --- >>> ##### Amount of logging in the webserver (none, normal, detailed) >>> ? loglevel: normal >>> ? password: --- >>> ? port: 8082 >>> ? webserver: true >>> >>> ######### SECTION dnssec ######### >>> dnssec: >>> ? log_bogus: false >>> ? max_dnskeys: 2 >>> ? validation: process >>> >>> ######### SECTION ecs ######### >>> ecs: >>> ##### List of client netmasks for which EDNS Client Subnet will be added >>> ? add_for: >>> ? - 0.0.0.0/0 >>> ? - ::/0 >>> /// >>> >>> The related proxy-map.lua: >>> >>> /// >>> -- protobufServer("10.10.11.50:514" , "maxQueuedEntries=100", >>> "logQueries=true", "logResponses=true", "logMappedFrom=false") >>> protobufServer("10.10.11.50:514") >>> >>> -- AE >>> addProxyMapping("10.10.10.0/24", "41.55.55.33") >>> /// >>> >>> I can't provide less sanitised information in the pcap and logs as that >>> would expose sensitive information (which I think is reasonably sanitised). >>> But let me know on this point in any case. >>> >>> If you are however referring to something else, then I would appreciate you >>> specifying the additional information that you would require to assist me in >>> collecting that info. >>> >>> Appreciate your time >>> >>> Robby >>> >>>>> Design: >>>>> >>>>> client ---> firewall --- ipsec vpn --- firewall ---> recursor ---> >>>>> internet >>>>> >>>>> Troubleshooting: >>>>> >>>>> - check for blocks due to allow_from (nothing listed for these clients) >>>>> - check local firewall rules (nothing special or different for specific >>>>> clients) >>>>> - tcpdump on the recursor hosts show queries hitting those hosts >>>>> - pcaps on both firewalls show good traffic >>>>> - the start of the logs show the ACL for allow_from is correct >>>>> >>>>> PDNS-rec Config: >>>>> ------------------------ >>>>> >>>>> // >>>>> /######### SECTION incoming ######### >>>>> incoming: >>>>> ? listen: >>>>> ? - 0.0.0.0 >>>>> ? - '::' >>>>> ? allow_from: >>>>> ? - x.x.x.x/y >>>>> ? - etc. >>>>> >>>>> ? port: 53 >>>>> ? proxy_protocol_from: [a.a.a.a/b] >>>>> ? use_incoming_edns_subnet: true >>>>> ? max_tcp_clients: 128/ >>>>> // >>>>> >>>>> >>>>> PDNS-rec docker config: >>>>> --------------------------------- >>>>> >>>>> // >>>>> /--- >>>>> version: '2.0' >>>>> services: >>>>> ? recursor: >>>>> ??? image: powerdns/pdns-recursor-51:latest >>>>> ??? restart: always >>>>> ??? ports: >>>>> ????? - "53:53" >>>>> ????? - "53:53/udp" >>>>> ????? - "8082:8082" >>>>> ??? logging: >>>>> ????? driver: "syslog" >>>>> ??? volumes: >>>>> ????? - ./recursor.yml:/etc/powerdns/recursor.yml >>>>> ????? - ./named.root.txt:/etc/named.root.txt >>>>> ????? - ./proxy-map.lua:/etc/proxy-map.lua/ >>>>> // >>>>> >>>>> PDNS-rec logs: >>>>> --------------------- >>>>> >>>>> recursor_1? | Apr 29 13:53:49 PowerDNS Recursor 5.1.4 (C) PowerDNS.COM BV >>>>> recursor_1? | Apr 29 13:53:49 Using 64-bits mode. Built using gcc 10.2.1 >>>>> 20210110 on Apr? 8 2025 10:17:24 by root@localhost. >>>>> recursor_1? | Apr 29 13:53:49 PowerDNS comes with ABSOLUTELY NO WARRANTY. >>>>> This is free software, and you are welcome to redistribute it according to >>>>> the terms of the GPL version 2. >>>>> recursor_1? | Apr 29 13:53:49 msg="Processing main YAML settings" >>>>> subsystem="config" level="0" prio="Notice" tid="0" ts="1745934829.121" >>>>> path="/etc/powerdns/recursor.yml" >>>>> recursor_1? | Apr 29 13:53:49 msg="YAML config found and processed" >>>>> subsystem="config" level="0" prio="Notice" tid="0" ts="1745934829.121" >>>>> configname="/etc/powerdns/recursor.yml" >>>>> recursor_1? | Apr 29 13:53:49 msg="Enabling IPv4 transport for outgoing >>>>> queries" subsystem="config" level="0" prio="Notice" tid="0" >>>>> ts="1745934829.123" >>>>> recursor_1? | Apr 29 13:53:49 msg="Setting access control" >>>>> subsystem="config" level="0" prio="Info" tid="0" ts="1745934829.125" >>>>> acl="allow-from" addresses="x.x.x.x/y a.a.a.a/b etc." >>>>> recursor_1? | Apr 29 13:53:49 msg="Will not send queries to" >>>>> subsystem="config" level="0" prio="Notice" tid="0" ts="1745934829.132" >>>>> addresses="127.0.0.0/8 10.0.0.0/8 100.64.0.0/10 169.254.0.0/16 >>>>> 192.168.0.0/16 172.16.0.0/12 ::1/128 fc00::/7 fe80::/10 0.0.0.0/8 >>>>> 192.0.0.0/24 192.0.2.0/24 198.51.100.0/24 203.0.113.0/24 240.0.0.0/4 ::/96 >>>>> ::ffff:0:0/96 100::/64 2001:db8::/32 0.0.0.0 ::" >>>>> >>>>> PDNS-rec host pcap: >>>>> ------------------------------ >>>>> >>>>> tcpdump -i any -v 'host <client-ip>' >>>>> tcpdump: listening on any, link-type LINUX_SLL (Linux cooked), capture >>>>> size >>>>> 262144 bytes >>>>> 14:01:49.419703 IP (tos 0x0, ttl 124, id 45946, offset 0, flags [none], >>>>> proto UDP (17), length 83) >>>>> ??? <client-hostname>.65424 > <recursor-hostname>.domain: 16579+ [1au] A? >>>>> canary.officeapps.live.com. (55) >>>>> 14:01:49.419758 IP (tos 0x0, ttl 123, id 45946, offset 0, flags [none], >>>>> proto UDP (17), length 83) >>>>> ??? <client-hostname>.65424 > 172.24.0.2.domain: 16579+ [1au] A? >>>>> canary.officeapps.live.com. (55) >>>>> 14:01:49.419766 IP (tos 0x0, ttl 123, id 45946, offset 0, flags [none], >>>>> proto UDP (17), length 83) >>>>> ??? <client-hostname>.65424 > 172.24.0.2.domain: 16579+ [1au] A? >>>>> canary.officeapps.live.com. (55) >>>>> >>>>> Any ideas on what could be wrong or what I'm missing here is appreciated. >>>>> >>>>> Regards >>>>> >>>>> Robby >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Pdns-users mailing list >>>>> Pdns-users@mailman.powerdns.com >>>>> https://mailman.powerdns.com/mailman/listinfo/pdns-users > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > <http://mailman.powerdns.com/pipermail/pdns-users/attachments/20250512/4080c0e8/attachment.htm> > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > Pdns-users mailing list > Pdns-users@mailman.powerdns.com > https://mailman.powerdns.com/mailman/listinfo/pdns-users > > > ------------------------------ > > End of Pdns-users Digest, Vol 268, Issue 6 > ******************************************
_______________________________________________ Pdns-users mailing list Pdns-users@mailman.powerdns.com https://mailman.powerdns.com/mailman/listinfo/pdns-users
