Hi Otto, thanks for the pointer! AFAICT it covers my patches as well, looks a lot more complicated, though. I'll take a closer look at it.
Is there any reason, why it hasn't been merged yet? Any cases that would break that needed to be avoided? Thanks, Michael. -- Michael Rommel, Erlangen, Germany > On 23 Jun 2020, at 08:16, Otto Moerbeek <o...@drijf.net> wrote: > > On Mon, Jun 22, 2020 at 10:11:30PM +0200, Michael Rommel via Pdns-users > wrote: > >> >> Dear all, >> >> a while ago (2020-03-01) I asked about setting up domains with LUA >> createForward() >> records. >> >> I suceeded in setting it up and found some peculiarities, which I would like >> to >> discuss here (in parallel I consider to submit PRs for some issues in Github >> and >> would appreciate guidance, whether it makes sense to open them). >> >> There are four (4) questions in this mail and sorry for the length, but I >> wanted >> to make it explicit with all possible information provided from the get-go. >> >> The setup for the proof-of-concept is a MASTER/SLAVE setup with sqlite3 as >> backend. The used version is 4.3.0-1pdns.bionic from >> http://repo.powerdns.com/ubuntu bionic-auth-43. >> >> The demo setup has essentially these domains and records (taken from the >> master): >> >> sqlite> select * from records; >> 1|1|example.com|SOA|ns1.example.com ra-dns-admin.example.com 3 10380 3600 >> 604800 3600|86400|||0||1 >> 2|1|example.com|NS|ns1.example.com|86400|||0||1 >> 3|1|example.com|NS|ns2.example.com|86400|||0||1 >> 4|1|ns1.example.com|A|104.41.128.19|86400|||0||1 >> 5|1|ns2.example.com|A|52.148.215.179|86400|||0||1 >> 7|1|*.11111111.1001.example.com|LUA|A "createForward()"|60|||0||1 >> 8|1|*.22222222-2002.example.com|LUA|A "createForward()"|60|||0||1 >> 9|2|33333333-3003.example.com|SOA|ns1.example.com ra-dns-admin.example.com 2 >> 10380 3600 604800 3600|86400|||0||1 >> 10|2|*.33333333-3003.example.com|LUA|A "createForward()"|60|||0||1 >> >> sqlite> select * from domains; >> 1|example.com|||MASTER|2| >> 2|33333333-3003.example.com|||MASTER|2| >> >> Other tables available on request, I'll try to be as brief as possible. >> >> The intended use is a DNS resolver for approx. 200.000 devices (more >> later), each device shall have one of those wildcard createForward() >> records and an accompanying _acme-challenge TXT record to obtain a Let's >> Encrypt certificate for that record. >> >> >> Q 1: Structure of the domain/subdomains / current implementation limitations >> ==== >> >> Currently the implementation of the LUA createForward() is in a way that >> accepts the wildcard only as being directly underneath the domain in >> question. In the example setup above, the 4.3 version: >> >> - will not resolve the record ip10203040.22222222-2002.example.com >> - will resolve the record ip10203040.33333333-3003.example.com >> >> because only the latter one is directly beneath the domain. In my use case >> that would mean to create 200.0000 additional entries in the domain table >> (the NS records for a proper DNS delegation can be omitted here, because >> all live on the same server). Each domain would only have two entries. >> >> Even with a less aggressive SOA refresh time, that would mean, that pdns >> would check all of those 200K domains within one hour. Since they mostly >> stay the same, there is no AXFR involved, but the checking imposes a load >> on the database and logging (tuneable of course). With PGSQL later this >> will certainly bearable, but I think a multi-level structure might be >> better suited. Hence the first patch: >> >> I suggest changing the line 616 in lua-record.cc to >> >> if(parts.size()<4) { >> >> This would retain the behaviour of accepting questions like: >> >> 192.168.1.1.33333333-3003.example.com >> >> but would enable additionally questions like: >> >> ip10203040.22222222-2002.example.com >> ip10203040.11111111.1001.example.com >> >> letting me subdivide the domain without the need for separate subdomains >> just for the resolution purpose. >> >> It would be breaking for setups where the top level domain also has a >> wildcard record and it is not wished that subdomains are resolved: >> >> *.example.com|LUA|A "createForward()" >> >> And ip10203040.test.example.com shall NOT be resolved. With the patch, it >> would. >> >> Shall I submit a PR with this or do you have better ideas for an >> implementation. >> >> >> Q 2: Does it make sense to subdivide the domain >> ==== >> >> The patch above allows me to structure the domain like the example >> 1001.11111111.example.com or vice versa. This would result in >> >> ~ 850 records like 11111111.example.com, each with >> 1 - 10.000 records underneath it like 1001.11111111.example.com each with: >> *.1001.11111111.example.com LUA "createForward()" and >> _acme-challenge.1001.11111111.example.com TXT "token from LE" >> The 850 records would be full domains with their entry in the domains >> table, but the 10.000 entries below would not be separate domains. >> >> This means that once a new device needs a certificate, two records would be >> created and in the worst case a domain with 20.000 entries would be needed >> to AXFR by the SLAVE (or via native replication later). >> >> But the refresh would only check the SOA for 850 records between pdns and >> its backend db. >> >> Would you consider a different solution / structure or does that make sense >> to you? >> >> >> Q 3: SERVFAIL with special questions >> ==== >> >> Currently there is a strange behaviour with createForward(). I would >> consider this a bug, but am open to corrections. >> >> The implementation skips the first two octets, then parses the remainder >> with sscanf. This leads to a problem, when someone asks a question like >> >> 192-168-3-4.33333333-3003.example.com >> >> which leads to a SERVFAIL, because the string returned from the function is >> 2.4294967295.104.4294967293 = 2 . -1 . 0x68 . -3 >> which then cannot be put into the answer packet. >> >> ; <<>> DiG 9.11.3-1ubuntu1.12-Ubuntu <<>> +norecurse @172.24.46.11 >> 192-168-3-4.33333333-3003.example.com >> ; (1 server found) >> ;; global options: +cmd >> ;; Got answer: >> ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 10082 >> ;; flags: qr aa; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1 >> >> ;; OPT PSEUDOSECTION: >> ; EDNS: version: 0, flags:; udp: 1232 >> ;; QUESTION SECTION: >> ;192-168-3-4.33333333-3003.example.com. IN A >> >> ;; Query time: 1 msec >> ;; SERVER: 172.24.46.11#53(172.24.46.11) >> ;; WHEN: Mon Jun 22 19:46:50 UTC 2020 >> ;; MSG SIZE rcvd: 66 >> >> root:/home/rommel/configuration# tail -100 /var/log/syslog |grep pdns >> Jun 22 19:46:50 CertifVM01 pdns_server[1276]: Remote 172.24.46.11 wants >> '192-168-3-4.33333333-3003.example.com|A', do = 0, bufsize = 1232 (4096): >> packetcache MISS >> Jun 22 19:46:50 CertifVM01 pdns_server[1276]: Lua record >> (192-168-3-4.33333333-3003.example.com|A) reported: Parsing record content >> (try 'pdnsutil check-zone'): unable to parse IP address >> Jun 22 19:46:50 CertifVM01 pdns_server[1276]: Exception building answer >> packet for 192-168-3-4.33333333-3003.example.com/A (Parsing record content >> (try 'pdnsutil check-zone'): unable to parse IP address) sending out servfail >> >> My patch suggestion would be to add in the check for values above 255, like: >> if(sscanf(parts[0].c_str()+2, "%02x%02x%02x%02x", &x1, &x2, &x3, &x4)==4) { >> if(x1<=0xff && x2<=0xff && x3<=0xff && x4<=0xff) >> return >> std::to_string(x1)+"."+std::to_string(x2)+"."+std::to_string(x3)+"."+std::to_string(x4); >> } >> >> Would you agree that this might be better to fall through to returning >> "0.0.0.0" >> rather than SERVFAIL? >> >> >> Q 4: Adding the (forgotten) ability to parse the dash delimited decimal >> questions >> ==== >> >> In addition to the hexadecimal notation, I would really like to see the >> proper resolving of entries like 192-168-3-4.33333333-3003.example.com. >> >> These additional lines below the hex portion would allow this: >> >> if(sscanf(parts[0].c_str(), "%u-%u-%u-%u", &x1, &x2, &x3, &x4)==4) { >> if(x1<=0xff && x2<=0xff && x3<=0xff && x4<=0xff) >> return >> std::to_string(x1)+"."+std::to_string(x2)+"."+std::to_string(x3)+"."+std::to_string(x4); >> } >> >> Anyone interested in this PR? >> >> >> Thanks everybody, who has read through this monster to this point. Any >> suggestions or corrections or improvements. >> >> Michael. > > I must say I did not read your complete post in all detail, but you > should take a look at (and preferably test!) > https://github.com/PowerDNS/pdns/pull/9101 > > I believe it covers a lot (all?) of your issues. > > -Otto > >> >> >> >> -- >> Michael Rommel, Erlangen, Germany >> >> _______________________________________________ >> Pdns-users mailing list >> Pdns-users@mailman.powerdns.com >> https://mailman.powerdns.com/mailman/listinfo/pdns-users _______________________________________________ Pdns-users mailing list Pdns-users@mailman.powerdns.com https://mailman.powerdns.com/mailman/listinfo/pdns-users
