Hi,

I'm also controlling the backend servers and no such rate limit exists. As I mentioned, each backend server can easily handle 60k QpS and the numbers of queries that should be reaching the backend servers from the performance testing, should not be more than 100-200 QpS per backend server.

The reason I sent my dnsdist configuration is so that people -more experienced than me- could spot obvious omissions or errors.

cheers

On 7/24/21 11:41 AM, Winfried Angele via Pdns-users wrote:
Maybe there is some kind of ratelimt in the backend. Keep in mind that from the backend point of view all queries come from the same source

Winfried

Am 24. Juli 2021 10:28:49 MESZ schrieb Winfried Angele via Pdns-users <pdns-users@mailman.powerdns.com>:

    Right. Therefore I would recommend rather to troubleshoot the backend

    Winfried


    Am 23. Juli 2021 20:56:55 MESZ schrieb Dez C via Pdns-users
    <pdns-users@mailman.powerdns.com>:

        Hi,

        On 7/23/21 4:14 PM, Winfried Angele via Pdns-users wrote:
        Does this only happen with DoH frontends? Did you try with
        UDP frontends as well? Sounds like a bottleneck on your
        backends imo.

        I'm only using dnsdist for DoH so I haven't tried with UDP/53.
        I doubt it'll make a difference because the problem manifests
        itself when dnsdist is trying to query the backends
        (eventhough most replies already exist in dnsdist's cache)

        cheers


        Winfried


        Am 23. Juli 2021 13:32:39 MESZ schrieb Yannis via Pdns-users
        <pdns-users@mailman.powerdns.com>:

            hello,

            We're using dnsdist (1.5.1 on Ubuntu 20.04, 16 cores, 32GB RAM) as 
a DoH
            proxy/LB with normal DNS/53 resolvers as backend. This is a test
            installation and we're trying to figure out the performance. It can
            barely handle 1.5k QpS, which I consider pretty low (each backend
            resolver can easily handle >60k QpS). It seems that each time the
            queries rate is higher than ~1.5k, all backend servers are marked 
"DOWN"
            until the rate goes below 1k. I understand that dnsdist marks the
            servers down because it's not receiving a response on its 
healthcheck
            query and I wonder why.

            Should I increase "checkTimeout" and "checkInterval"? Should I use a
            large number for "sockets"? Am I missing other tuning options or 
maybe
            something more important?

            Here's the relevant config (addresses, etc changed)

            setLocal('0.0.0.0:5300')
            addLocal('[::1]:5300')
            controlSocket('local_public_address:xxxx')
            setKey("XXX")
            setConsoleACL('x.x.x.x/24')
            NotRule(MaxQPSRule(50000))
            setMaxUDPOutstanding(65535)
            setMaxTCPClientThreads(128)
            setMaxTCPQueuedConnections(10000)
            setMaxTCPConnectionDuration(600)
            PrimaryCache = newPacketCache(30000000, { keepStaleData=true,
            maxTTL=86400, minTTL=0, numberOfShards=8, maxNegativeTTL=600, 
staleTTL=60 })
            getPool(""):setCache(PrimaryCache)
            addDOHLocal('10.2.3.4', 'cert.pem', 'key.key', "/dns-query", {
            reusePort=true, minTLSVersion='tls1.2' })
            addDOHLocal('10.2.3.4', 'cert.pem', 'key.key', "/dns-query", {
            reusePort=true, minTLSVersion='tls1.2' })
            addDOHLocal('10.2.3.4', 'cert.pem', 'key.key', "/dns-query", {
            reusePort=true, minTLSVersion='tls1.2' })
            addDOHLocal('10.2.3.4', 'cert.pem', 'key.key', "/dns-query", {
            reusePort=true, minTLSVersion='tls1.2' })
            addDOHLocal('2001:DB8::443', 'cert.pem', 'key.key', "/dns-query", {
            reusePort=true, minTLSVersion='tls1.2' })
            addDOHLocal('2001:DB8::443', 'cert.pem', 'key.key', "/dns-query", {
            reusePort=true, minTLSVersion='tls1.2' })
            addDOHLocal('2001:DB8::443', 'cert.pem', 'key.key', "/dns-query", {
            reusePort=true, minTLSVersion='tls1.2' })
            addDOHLocal('2001:DB8::443', 'cert.pem', 'key.key', "/dns-query", {
            reusePort=true, minTLSVersion='tls1.2' })
            newServer({address="2001:DB8::62", qps=10000})
            newServer({address="2001:DB8::61", qps=10000})
            newServer({address="2001:DB8::60", qps=10000})
            newServer({address="2001:DB8::59", qps=10000})
            newServer({address="2001:DB8::58", qps=10000})
            newServer({address="2001:DB8::57", qps=10000})
            newServer({address="2001:DB8::56", qps=10000})
            newServer({address="2001:DB8::55", qps=10000})
            newServer({address="2001:DB8::48", qps=10000})
            newServer({address="2001:DB8::47", qps=10000})
            newServer({address="10.10.10.62", qps=10000})
            newServer({address="10.10.10.61", qps=10000})
            newServer({address="10.10.10.60", qps=10000})
            newServer({address="10.10.10.59", qps=10000})
            newServer({address="10.10.10.58", qps=10000})
            newServer({address="10.10.10.57", qps=10000})
            newServer({address="10.10.10.56", qps=10000})
            newServer({address="10.10.10.55", qps=10000})
            newServer({address="10.10.10.48", qps=10000})
            newServer({address="10.10.10.47", qps=10000})
            setServerPolicy(roundrobin)

            thanks in advance, I'd appreciate any input :)
            
------------------------------------------------------------------------
            Pdns-users mailing list
            Pdns-users@mailman.powerdns.com
            https://mailman.powerdns.com/mailman/listinfo/pdns-users  
<https://mailman.powerdns.com/mailman/listinfo/pdns-users>


        _______________________________________________
        Pdns-users mailing list
        Pdns-users@mailman.powerdns.com
        https://mailman.powerdns.com/mailman/listinfo/pdns-users


--
Diese Nachricht wurde von meinem Android-Gerät mit K-9 Mail gesendet.

_______________________________________________
Pdns-users mailing list
Pdns-users@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/pdns-users

_______________________________________________
Pdns-users mailing list
Pdns-users@mailman.powerdns.com
https://mailman.powerdns.com/mailman/listinfo/pdns-users

Reply via email to