for those that can reproduce this manually, getting that wireshark would be helpful. If we can pin this as a bug in OVS then we can figure out how to get it fixed there.
for the ODL csit that sees this, we do collect the ovs log but there was nothing I saw that made me think OVS was in the wrong. example here: https://logs.opendaylight.org/releng/jenkins092/netvirt-csit-1node-openstack-mitaka-upstream-learn-carbon/178/archives/control/ovs-vswitchd.log.gz JamO On 04/06/2017 12:54 PM, Anil Vishnoi wrote: > In my opinion, this is partially a controller issue and partially problem > with the switch. The patches that jozef mention, it > basically stop sending the slave role, but it doesn't mean that this issue > won't occur when controller sends master role. > First issue here is that switch is not accepting the role (it can be slave or > master role), and when switch doesn't accept > the role, controller just disconnect from the device and wait for the > connection reattempt. Whatever you are seeing is > expected behavior. > > Not sending slave role will workaround the issue, but what if controller > sends master role and switch doesn't accept it ? You > will end up in the same situation here. I would suggest to please collect the > wireshark from the switch that is not accepting > the role request and see what error message it's sending, that will help us > to figure out whether it's an issue with ovs or > controller should attempt anything else in case of failure. > > > > On Thu, Apr 6, 2017 at 10:14 AM, Abhijit Kumbhare <[email protected] > <mailto:[email protected]>> wrote: > > The summary is here: > > > https://wiki.opendaylight.org/view/OpenDaylight_OpenFlow_Plugin:Split_connection_layer > > <https://wiki.opendaylight.org/view/OpenDaylight_OpenFlow_Plugin:Split_connection_layer> > > Yes - it will be merged in the master first - we will get it in Boron > only after it has been soaked well in Carbon first. > Anil has been reviewing the set of patches since January - since then > Shuva has reviewed them over the last weekend, > followed by tests from Luis and now Prasanna is reviewing as one last > check (he reviewed 8 of those yesterday & hopefully > remaining will be done Friday). The tests with Luis have indicated things > are better than before (in terms of number of > failures in the tests) - but there are still 2 remaining cluster > failures. Bug 6459 is kept open to track the 2 remaining > cluster failures. We are targeting merge of the patches before the branch > cutoff - but hopefully on Friday. > > > > On Thu, Apr 6, 2017 at 12:49 AM, Kochba, Alon <[email protected] > <mailto:[email protected]>> wrote: > > Thanks guys.____ > > __ __ > > Could we have a summary of the patches and have them merged in master > first before we start talking about Boron?____ > > What is this waiting on? Branch cutting is around the corner, we > should allow some soaking time after merge to see > there are no new issues.____ > > __ __ > > --alon____ > > __ __ > > *From:*[email protected] > <mailto:[email protected]> > [mailto:[email protected] > <mailto:[email protected]>] > *On Behalf Of *Jozef Bacigál > *Sent:* Thursday, 6 April 2017 09:49 > *To:* Abhijit Kumbhare <[email protected] > <mailto:[email protected]>>; Srikanth Lingala > <[email protected] <mailto:[email protected]>> > *Cc:* odl netvirt dev <[email protected] > <mailto:[email protected]>>; > [email protected] > <mailto:[email protected]>; > [email protected] <mailto:[email protected]>; > Gorja Gorja <[email protected] > <mailto:[email protected]>>; Aizer, Koby <[email protected] > <mailto:[email protected]>>; Veera.Reddy B > <[email protected] <mailto:[email protected]>> > > > *Subject:* Re: [openflowplugin-dev] [netvirt-dev] FW: br-int become > slave and disconnect from ODL____ > > __ __ > > Let me check today if I am able to pick the fix from the changes and > put in on the standing code in boron. It may or > may not be possible. I will let you know.____ > > __ __ > > Jozef____ > > __ __ > > *From:*Abhijit Kumbhare [mailto:[email protected]] > *Sent:* Thursday, April 6, 2017 7:45 AM > *To:* Srikanth Lingala <[email protected] > <mailto:[email protected]>> > *Cc:* Aizer, Koby <[email protected] <mailto:[email protected]>>; > Jozef Bacigál <[email protected] > <mailto:[email protected]>>; odl netvirt dev > <[email protected] > <mailto:[email protected]>>; > [email protected] > <mailto:[email protected]>; > [email protected] <mailto:[email protected]>; > Veera.Reddy B <[email protected] <mailto:[email protected]>>; Gorja Gorja > <[email protected] > <mailto:[email protected]>> > *Subject:* Re: [openflowplugin-dev] [netvirt-dev] FW: br-int become > slave and disconnect from ODL____ > > __ __ > > I believe there are several patches by Jozef for the split connection > handler. Since it is not really a new feature - > rather a fix (albeit with a lot of code change) - it may be possible > to backport to stable/boron but only Jozef can > answer how easy it would be to backport as code may be depending on > changes in Carbon.____ > > __ __ > > __ __ > > On Wed, Apr 5, 2017 at 10:23 PM, Srikanth Lingala > <[email protected] <mailto:[email protected]>> wrote:____ > > Thanks for the info.____ > > I am also using stable/boron. Can I cherry pick these patches to > stable/boron?____ > > Can you provide me the link to the patch file you are > referring?____ > > ____ > > ____ > > Regards,____ > > Srikanth.____ > > ____ > > *From:*Aizer, Koby [mailto:[email protected] > <mailto:[email protected]>] > *Sent:* Wednesday, April 05, 2017 10:53 PM > *To:* Jozef Bacigál <[email protected] > <mailto:[email protected]>>; Abhijit Kumbhare > <[email protected] <mailto:[email protected]>> > *Cc:* odl netvirt dev <[email protected] > <mailto:[email protected]>>; Srikanth > Lingala <[email protected] > <mailto:[email protected]>>; [email protected] > <mailto:[email protected]>; > [email protected] > <mailto:[email protected]>____ > > > *Subject:* Re: [openflowplugin-dev] [netvirt-dev] FW: br-int > become slave and disconnect from ODL____ > > ____ > > Sounds great thanks. ____ > > Can we get it cherry picked to stable/boron as well? We've been > hitting this quite a lot after OVS restarts. ____ > > ____ > > Thanks, ____ > > Koby____ > > ____ > > > ----------------------------------------------------------------------------------------------------------------------------- > > *From:*Jozef Bacigál <[email protected] > <mailto:[email protected]>> > *Sent:* Apr 5, 2017 7:55 PM > *To:* Abhijit Kumbhare; Aizer, Koby > *Cc:* odl netvirt dev; Srikanth Lingala; > [email protected] > <mailto:[email protected]>; > [email protected] <mailto:[email protected]> > *Subject:* Re: [openflowplugin-dev] [netvirt-dev] FW: br-int > become slave and disconnect from ODL____ > > ____ > > Yes it should.____ > > ____ > > Jozef____ > > > ----------------------------------------------------------------------------------------------------------------------------- > > *Od:*Abhijit Kumbhare <[email protected] > <mailto:[email protected]>> > *Odoslané:* 5. apríla 2017 18:43 > *Komu:* Aizer, Koby > *Kópia:* odl netvirt dev; Srikanth Lingala; > [email protected] > <mailto:[email protected]>; > [email protected] <mailto:[email protected]> > *Predmet:* Re: [openflowplugin-dev] [netvirt-dev] FW: br-int > become slave and disconnect from ODL____ > > ____ > > Jozef, ____ > > ____ > > Do you think this will be fixed when the split connection patches > get merged?____ > > ____ > > Abhijit____ > > ____ > > On Wed, Apr 5, 2017 at 9:27 AM, Aizer, Koby <[email protected] > <mailto:[email protected]>> wrote:____ > > + OFP____ > > ____ > > Hi Srikanth, ____ > > ____ > > I think you're hitting [1], which I believe has a pending > patch in master. ____ > > ____ > > Koby____ > > ____ > > [1] https://bugs.opendaylight.org/show_bug.cgi?id=8087 > <https://bugs.opendaylight.org/show_bug.cgi?id=8087>____ > > ____ > > ____ > > > ----------------------------------------------------------------------------------------------------------------------------- > > *From:*Srikanth Lingala <[email protected] > <mailto:[email protected]>> > *Sent:* Apr 5, 2017 6:00 PM > *To:* odl netvirt dev; [email protected] > <mailto:[email protected]> > *Subject:* [netvirt-dev] FW: br-int become slave and > disconnect from ODL____ > > ____ > > +netvirt-dev and ovs-discuss____ > > ____ > > *From:*Srikanth Lingala > *Sent:* Tuesday, April 04, 2017 11:11 PM > *To:* '[email protected] > <mailto:[email protected]>' > <[email protected] > <mailto:[email protected]>>; > [email protected] > <mailto:[email protected]> > *Subject:* br-int become slave and disconnect from ODL____ > > ____ > > Hi,____ > > I am trying to integrate Openstack with ODL.____ > > I am using ODL Boron stable release and OVS 2.6.1. I started > ODL karaf using the below command:____ > > ____ > > /opt/distribution-karaf-0.5.3-SNAPSHOT/bin# ./karaf -of13 > clean____ > > Apache Karaf starting up. Press Enter to open the shell > now...____ > > 100% > [========================================================================]____ > > Karaf started in 273s. Bundle stats: 389 active, 401 total____ > > ____ > > ________ ________ > .__ .__ .__ ______ > > \_____ \ ______ ____ ____ \______ \ _____ ___.__.| > | |__| ____ | |___/ |_____ > > / | \\____ \_/ __ \ / \ | | \\__ \< | || > | | |/ ___\| | \ __\____ > > / | \ |_> > ___/| | \| ` \/ __ \\___ || > |_| / /_/ > Y \ |____ > > \_______ / __/ \___ >___| /_______ (____ / > ____||____/__\___ /|___| /__|____ > > \/|__| \/ \/ \/ \/\/ > /_____/ \/____ > > ____ > > ____ > > Hit '<tab>' for a list of available commands____ > > and '[cmd] --help' for help on a specific command.____ > > Hit '<ctrl-d>' or type 'system:shutdown' or 'logout' to > shutdown OpenDaylight.____ > > opendaylight-user@root>____ > > ____ > > And, executed below command to set manager to ODL:____ > > ____ > > #> ovs-vsctl set-manager tcp:10.20.0.26:6640 > <http://10.20.0.26:6640>____ > > ____ > > ODL able to add br-int to OVS. Following are the OVS > details:____ > > ____ > > root@node-4446ubuntu:~# ovs-vsctl show____ > > 9bf5a6cb-e906-4205-afb2-14bb31510a51____ > > Manager "tcp:10.20.0.26:6640 <http://10.20.0.26:6640>"____ > > is_connected: true____ > > Bridge br-int____ > > Controller "tcp:10.20.0.26:6653 > <http://10.20.0.26:6653>"____ > > fail_mode: secure____ > > Port br-int____ > > Interface br-int____ > > type: internal____ > > ____ > > Following are the ovs-vswitchd logs:____ > > ____ > > 2017-04-04T22:30:46.005Z|01281|connmgr|INFO|br-int: added > primary controller "tcp:10.20.0.26:6653 > <http://10.20.0.26:6653>"____ > > > 2017-04-04T22:30:46.005Z|01282|rconn|INFO|br-int<->tcp:10.20.0.26:6653 > <http://10.20.0.26:6653>: > connecting...____ > > 2017-04-04T22:30:46.009Z|01283|netdev_linux|WARN|br-int: > removing policing failed: Operation not supported____ > > > 2017-04-04T22:30:46.576Z|01284|rconn|INFO|br-int<->tcp:10.20.0.26:6653 > <http://10.20.0.26:6653>: connected____ > > > 2017-04-04T22:31:51.646Z|01300|rconn|ERR|br-int<->tcp:10.20.0.26:6653 > <http://10.20.0.26:6653>: no response > to inactivity probe after 5 seconds, disconnecting____ > > > 2017-04-04T22:31:52.646Z|01301|rconn|INFO|br-int<->tcp:10.20.0.26:6653 > <http://10.20.0.26:6653>: > connecting...____ > > > 2017-04-04T22:31:52.675Z|01302|rconn|INFO|br-int<->tcp:10.20.0.26:6653 > <http://10.20.0.26:6653>: connected____ > > > 2017-04-04T22:32:02.646Z|01303|rconn|ERR|br-int<->tcp:10.20.0.26:6653 > <http://10.20.0.26:6653>: no response > to inactivity probe after 5 seconds, disconnecting____ > > > 2017-04-04T22:32:03.645Z|01304|rconn|INFO|br-int<->tcp:10.20.0.26:6653 > <http://10.20.0.26:6653>: > connecting...____ > > > 2017-04-04T22:32:03.665Z|01305|rconn|INFO|br-int<->tcp:10.20.0.26:6653 > <http://10.20.0.26:6653>: connected____ > > > 2017-04-04T22:32:13.646Z|01306|rconn|ERR|br-int<->tcp:10.20.0.26:6653 > <http://10.20.0.26:6653>: no response > to inactivity probe after 5 seconds, disconnecting____ > > > 2017-04-04T22:32:14.646Z|01307|rconn|INFO|br-int<->tcp:10.20.0.26:6653 > <http://10.20.0.26:6653>: > connecting...____ > > > 2017-04-04T22:32:15.645Z|01308|rconn|INFO|br-int<->tcp:10.20.0.26:6653 > <http://10.20.0.26:6653>: connection > timed out____ > > > 2017-04-04T22:32:15.646Z|01309|rconn|INFO|br-int<->tcp:10.20.0.26:6653 > <http://10.20.0.26:6653>: waiting 2 > seconds before reconnect____ > > > 2017-04-04T22:32:17.646Z|01310|rconn|INFO|br-int<->tcp:10.20.0.26:6653 > <http://10.20.0.26:6653>: > connecting...____ > > > 2017-04-04T22:32:19.646Z|01311|rconn|INFO|br-int<->tcp:10.20.0.26:6653 > <http://10.20.0.26:6653>: connection > timed out____ > > > 2017-04-04T22:32:19.647Z|01312|rconn|INFO|br-int<->tcp:10.20.0.26:6653 > <http://10.20.0.26:6653>: waiting 4 > seconds before reconnect____ > > > 2017-04-04T22:32:23.646Z|01313|rconn|INFO|br-int<->tcp:10.20.0.26:6653 > <http://10.20.0.26:6653>: > connecting...____ > > > 2017-04-04T22:32:27.646Z|01314|rconn|INFO|br-int<->tcp:10.20.0.26:6653 > <http://10.20.0.26:6653>: connection > timed out____ > > > 2017-04-04T22:32:27.647Z|01315|rconn|INFO|br-int<->tcp:10.20.0.26:6653 > <http://10.20.0.26:6653>: continuing > to retry connections in the background but suppressing > further logging____ > > ____ > > I am able to see the following ODL errors in the Karaf > logs:____ > > ____ > > 2017-04-05 04:01:08,675 | INFO | entLoopGroup-7-3 | > ConnectionAdapterImpl | 363 - > org.opendaylight.openflowjava.openflow-protocol-impl - > 0.8.3.SNAPSHOT | Hello received / branch____ > > 2017-04-05 04:01:08,684 | INFO | entLoopGroup-7-3 | > DeviceManagerImpl | 368 - > org.opendaylight.openflowplugin.impl - 0.3.3.SNAPSHOT | > ConnectionEvent: Device connected to controller, > Device:/10.20.0.26:54319 <http://10.20.0.26:54319>, > NodeId:Uri [_value=openflow:167725479026029]____ > > 2017-04-05 04:01:08,685 | INFO | entLoopGroup-7-3 | > SalRoleServiceImpl | 368 - > org.opendaylight.openflowplugin.impl - 0.3.3.SNAPSHOT | > SetRole called with input:SetRoleInput > [_controllerRole=BECOMESLAVE, _node=NodeRef > [_value=KeyedInstanceIdentifier{targetType=interface > > org.opendaylight.yang.gen.v1.urn.opendaylight.inventory.rev130819.nodes.Node, > path=[org.opendaylight.yang.ge > > <http://org.opendaylight.yang.ge>n.v1.urn.opendaylight.inventory.rev130819.Nodes, > > org.opendaylight.yang.gen.v1.urn.opendaylight.inventory.rev130819.nodes.Node[key=NodeKey > [_id=Uri > [_value=openflow:167725479026029]]]]}], augmentation=[]]____ > > 2017-04-05 04:01:08,685 | INFO | entLoopGroup-7-3 | > SalRoleServiceImpl | 368 - > org.opendaylight.openflowplugin.impl - 0.3.3.SNAPSHOT | > Requesting state change to BECOMESLAVE____ > > 2017-04-05 04:01:08,685 | INFO | entLoopGroup-7-3 | > SalRoleServiceImpl | 368 - > org.opendaylight.openflowplugin.impl - 0.3.3.SNAPSHOT | > RoleChangeTask called on > device:openflow:167725479026029 OFPRole:BECOMESLAVE____ > > 2017-04-05 04:01:08,685 | INFO | entLoopGroup-7-3 | > RoleService | 368 - > org.opendaylight.openflowplugin.impl - 0.3.3.SNAPSHOT | > getGenerationIdFromDevice called for device: > openflow:167725479026029____ > > 2017-04-05 04:01:18,695 | WARN | pool-36-thread-1 | > DeviceContextImpl | 368 - > org.opendaylight.openflowplugin.impl - 0.3.3.SNAPSHOT | New > role BECOMESLAVE was not propagated to device > openflow:167725479026029 during 10 sec____ > > 2017-04-05 04:01:18,695 | ERROR | pool-36-thread-1 | > SalRoleServiceImpl | 368 - > org.opendaylight.openflowplugin.impl - 0.3.3.SNAPSHOT | > SetRoleService set Role BECOMESLAVE for Node: > KeyedInstanceIdentifier{targetType=interface > > org.opendaylight.yang.gen.v1.urn.opendaylight.inventory.rev130819.nodes.Node, > path=[org.opendaylight.yang.ge > > <http://org.opendaylight.yang.ge>n.v1.urn.opendaylight.inventory.rev130819.Nodes, > > org.opendaylight.yang.gen.v1.urn.opendaylight.inventory.rev130819.nodes.Node[key=NodeKey > [_id=Uri > [_value=openflow:167725479026029]]]]} fail . Reason > java.util.concurrent.CancellationException: Task was > cancelled.____ > > 2017-04-05 04:01:18,696 | WARN | pool-36-thread-1 | > DeviceManagerImpl | 368 - > org.opendaylight.openflowplugin.impl - 0.3.3.SNAPSHOT | Was > not able to set role SLAVE to device on node > openflow:167725479026029____ > > ____ > > Can anyone let me know what could be the issue?____ > > ____ > > Regards,____ > > Srikanth.____ > > > _______________________________________________ > openflowplugin-dev mailing list > [email protected] > <mailto:[email protected]> > > https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev > > <https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev>____ > > ____ > > __ __ > > > > _______________________________________________ > netvirt-dev mailing list > [email protected] > <mailto:[email protected]> > https://lists.opendaylight.org/mailman/listinfo/netvirt-dev > <https://lists.opendaylight.org/mailman/listinfo/netvirt-dev> > > > > > -- > Thanks > Anil > > > _______________________________________________ > netvirt-dev mailing list > [email protected] > https://lists.opendaylight.org/mailman/listinfo/netvirt-dev > _______________________________________________ openflowplugin-dev mailing list [email protected] https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev
