Hey Maria, Thanks for following up - I hope you had a good trip! I found it very easy to navigate the git repo, and found with git blame and git log within a few minutes. I don't think there's much there, but I thought about it over the weekend, and actually maybe two things might be useful:
Context: In the upgrade from Bird1.6.8 to Debian's older Bird2.0.7 we also noticed that BFD needs us to set 'accept ipv4'; and 'accept ipv6' because we'd like to continue to use two processes, one for each address family. The docs show this feature for BFD, but they don't *say as of* *which version *the feature is added, so after a few syntax errors in 2.0.7 we found that the docs are always at HEAD, so a quick search in the git history revealed that we needed 2.0.8 or higher. => For docs, it may be useful to keep a snapshot of previous versions when new versions are cut, so that folks can scan back in the history to find when a feature appeared or got changed, or removed. I wasn't aware of this wiki article <https://gitlab.nic.cz/labs/bird/-/wikis/transition-notes-to-bird-2>; it shows pretty much all the things we found (channels, 'device routes'), thanks for writing that, Ondrej (and five years ago, even!) => I am wondering if other folks have noticed duplicate kernel routes for connected networks, we settled on the pattern of ipvX { import none; export where source != RTS_DEVICE; }; In short: Bird2 gets 5 stars on Yelp from me as a long time user :) groet, Pim On Sun, May 21, 2023 at 4:56 PM Maria Matejka <[email protected]> wrote: > Hello! > > You're right, the information you found and described here is correct. > Kudos to you for being able to dig it all out yourself. > > We're all travelling this weekend with a limited time and internet access > so we couldn't reply instantly as we sometimes do. Yet the fact that it > apparently didn't take you excessive time, I'd interpret it that our > development documentation is quite good. > > Anyway, to pickup on this, is there anything we could do better to help > you finding the needed information faster and in more convenient way? > > Thank you for sharing. > Maria > > > On 21 May 2023 12:23:25 CEST, Pim van Pelt via Bird-users < > [email protected]> wrote: > >> Hoi, >> >> To close out my monologue -- I sent https://gerrit.fd.io/r/c/vpp/+/38854 >> to make VPP's Linux Controlplane plugin aware of NLM_F_REPLACE messages. >> Rolled that out at AS8283 this morning, and our duplicate FIB entry issue >> is gone. Nothing to see here, moving along :) >> >> groet, >> Pim >> >> On Sat, May 20, 2023 at 11:50 PM Pim van Pelt <[email protected]> wrote: >> >>> Hoi, >>> >>> I think I've found the answer to my question by taking a look at git >>> history in netlink handling. >>> >>> This commit: >>> >>> commit 8235c4747dcc92de2ea991f78cdf9c6b8fa7f522 >>> >>> Author: Ondrej Zajicek (work) <[email protected]> >>> >>> Date: Mon Jul 15 16:23:18 2019 +0200 >>> >>> >>> Netlink: Use route replace for IPv4 >>> >>> >>> Started using NL_OP_REPLACE for IPv4, but it kept it disabled for IPv6, >>> and then this commit: >>> >>> >>> commit 722daa950046a7ad307fd7aca8e0506f30b3d000 >>> >>> Author: Ondrej Zajicek <[email protected]> >>> >>> Date: Mon Jul 25 00:11:40 2022 +0200 >>> >>> >>> Netlink: Simplify handling of IPv6 ECMP routes >>> >>> >>> started using for IPv6 as well, where this commit: >>> >>> >>> commit ddb1bdf2819ce69248d5a51e71d803f13548b217 >>> >>> Author: Ondrej Zajicek <[email protected]> >>> >>> Date: Tue Jul 26 18:45:20 2022 +0200 >>> >>> >>> Netlink: Restrict route replace for IPv6 >>> >>> >>> added a nice guard in nl_allow_replace() -- this explains the replace >>> semantics (which 'ip monitor route' does not show), and answers my >>> question. For my application, I'll have to take a good look at consuming >>> messages with flag NLM_F_CREATE|NLM_F_REPLACE set; and otherwise perhaps >>> add the ability to Bird2/Bird3 to holdback and >>> issue NL_OP_DELETE + NL_OP_ADD. >>> >>> For the curious, the application is Vector Packet Processing [ref >>> <https://ipng.ch/s/articles/2021/09/02/vpp-5.html>] which consumes >>> Netlink messages from the Linux kernel, and uses them to program a >>> userspace dataplane, see [Linux Control Plane >>> <https://s3-docs.fd.io/vpp/23.06/developer/plugins/lcp.html>] for >>> details. Until now, this system consumes RTM_NEWROUTE and RTM_DELROUTE but >>> is not yet capable of consuming this replacing logic. I'll take a look at >>> adding that. >>> >>> groet, >>> Pim >>> >>> >>> >>> On Sat, May 20, 2023 at 11:10 PM Pim van Pelt <[email protected]> wrote: >>> >>>> Hoi, >>>> >>>> As a quick followup why I'm asking about versions -- on a Bird2.0.7, I >>>> do see the delete-before-insert: >>>> >>>> root@chgtg0:~# ip -6 monitor route | grep 2001:678:d78::6 >>>> >>>> >>>> # Raise OSPFv3 cost to prefer tf-0-0 >>>> >>>> *Deleted* 2001:678:d78::6 via fe80::21b:21ff:febd:c718 dev >>>> xe0-3.3102.20 proto bird metric 32 pref medium >>>> >>>> 2001:678:d78::6 via fe80::6eb3:11ff:fe20:e0c4 dev tf0-0 proto bird >>>> metric 32 pref medium >>>> >>>> >>>> # Lower OSPFv3 cost to prefer xe0-3.3102.20 again >>>> >>>> *Deleted* 2001:678:d78::6 via fe80::6eb3:11ff:fe20:e0c4 dev tf0-0 >>>> proto bird metric 32 pref medium >>>> >>>> 2001:678:d78::6 via fe80::21b:21ff:febd:c718 dev xe0-3.3102.20 proto >>>> bird metric 32 pref medium >>>> >>>> groet, >>>> Pim >>>> >>>> On Sat, May 20, 2023 at 10:51 PM Pim van Pelt <[email protected]> wrote: >>>> >>>>> Hoi folks, >>>>> >>>>> At Coloclue AS8283, we upgraded from Bird1.6.8 to Bird2.0.12 this >>>>> week. We use two separate processes, one for IPv4 and one for IPv6 - and >>>>> 2.0.7 in Debian is missing the ability to select 'accept ipv4' and 'accept >>>>> ipv6' in BFD, so we installed backports and version 2.0.12). >>>>> >>>>> I am wondering if Bird2 later than 2.0.7 perhaps has an optimization >>>>> when swapping routes? I would expect a swap to be "delete + add" but I am >>>>> seeing only "add with new nexthop" appear in Netlink. >>>>> >>>>> Considering the following topology with link names and OSPFv3 costs >>>>> associated: >>>>> >>>>> dcg-1 bond0.130 ---- bond0.130 eun-2 >>>>> >>>>> | 2000 | >>>>> >>>>> enp1s0f3 enp1s0f2 >>>>> >>>>> | | >>>>> >>>>> | 10 10 | >>>>> >>>>> | | >>>>> >>>>> enp1s0f3 enp1s0f3 >>>>> >>>>> | 1000 | >>>>> >>>>> dcg-2 eno2.3469 ---- eno2.3469 eun-3 >>>>> >>>>> If I restart the OSPFv3 protocol, I see that the topology settles in >>>>> the expected way. What I observed with bird 2.0.12 is that there is a >>>>> deletion of the currently selected route followed by one addition, when >>>>> the >>>>> shortest path reveales (dcg1 - dcg2 - eun3 - eun2, ospf_metric1 is 1020, >>>>> this is fine): >>>>> >>>>> root@dcg-1:~# birdc -s /run/bird/bird6.ctl restart ospf1 >>>>> >>>>> root@dcg-1:~# ip -6 monitor route | grep 2a02:898:0:300::3 >>>>> >>>>> Deleted 2a02:898:0:300::3 via fe80::669d:99ff:feb1:31af dev bond0.130 >>>>> proto bird metric 32 pref medium >>>>> >>>>> 2a02:898:0:300::3 via fe80::669d:99ff:feb1:3910 dev enp1s0f3 proto >>>>> bird metric 32 pref medium >>>>> >>>>> Now I lower the cost of the dcg-1 -- eun-2 link from 2000 to 100, so >>>>> that it becomes preferred (cost ospf_metric is 120): >>>>> >>>>> root@dcg-1:~# birdc -s /run/bird/bird6.ctl reconfigure ospf1 >>>>> >>>>> root@dcg-1:~# ip -6 monitor route | grep 2a02:898:0:300::3 >>>>> >>>>> *[[ HERE ]]* >>>>> >>>>> 2a02:898:0:300::3 via fe80::669d:99ff:feb1:31af dev bond0.130 proto >>>>> bird metric 32 pref medium >>>>> >>>>> I would expect this new addition of the installed route on bond0.130 >>>>> to be *preceded by a deletion* of the previous route from enp1s0f3, >>>>> but this is not the case (marked in red with [[ HERE ]]). >>>>> >>>>> To anyone's knowledge: *Has this behavior changed between 2.0.7 and >>>>> 2.0.12 ?* >>>>> >>>>> groet, >>>>> Pim >>>>> -- >>>>> Pim van Pelt <[email protected]> >>>>> PBVP1-RIPE - http://www.ipng.nl/ >>>>> >>>> >>>> >>>> -- >>>> Pim van Pelt <[email protected]> >>>> PBVP1-RIPE - http://www.ipng.nl/ >>>> >>> >>> >>> -- >>> Pim van Pelt <[email protected]> >>> PBVP1-RIPE - http://www.ipng.nl/ >>> >> >> >> -- Pim van Pelt <[email protected]> PBVP1-RIPE - http://www.ipng.nl/
