Skip to main content
Question

Failures attempting to fix consitency across nodes

  • April 13, 2025
  • 1 reply
  • 24 views

focaccio
Forum|alt.badge.img

I’m trying not to need to re-image a node, but the networking does not seem to be cooperating.

Here is the error I’m getting:

Errors in cluster networking
Bond configuration is not consistent across nodes. No bond found on host:192.168.86.70, but present on host:192.168.86.170

Here is .170

[root@NTNX-895b5886-A ~]# ip -4 a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
10: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 192.168.86.170/24 brd 192.168.86.255 scope global br0
       valid_lft forever preferred_lft forever
15: virbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    inet 192.168.5.1/24 brd 192.168.5.255 scope global virbr0
       valid_lft forever preferred_lft forever
[root@NTNX-895b5886-A ~]# 
[root@NTNX-895b5886-A ~]# ovs-vsctl list bridge |grep name
name                : br2.local
name                : br0.local
name                : br.microseg
name                : br1
name                : br1.local
name                : brSpan
name                : br0
name                : br.mx
name                : br.nf
name                : br.dmx
name                : br2
[root@NTNX-895b5886-A ~]# ovs-vsctl list-ports br2.local
br2.local.d
[root@NTNX-895b5886-A ~]# ovs-vsctl list-ports br0.local
br0.local.d
[root@NTNX-895b5886-A ~]# ovs-vsctl list-ports br.microseg
br.microseg.d
br.microseg.u
[root@NTNX-895b5886-A ~]# ovs-vsctl list-ports br1
br1-arp
br1-dhcp
br1-up
br1.u
[root@NTNX-895b5886-A ~]# ovs-vsctl list-ports br1.local
br1.local.d
[root@NTNX-895b5886-A ~]# ovs-vsctl list-ports brSpan
[root@NTNX-895b5886-A ~]# ovs-vsctl list-ports br0
br0-arp
br0-dhcp
br0-up
br0.u
vnet0
vnet2
[root@NTNX-895b5886-A ~]# ovs-vsctl list-ports br.mx
br.mx.d
br.mx.u.br0
br.mx.u.br1
br.mx.u.br2
[root@NTNX-895b5886-A ~]# ovs-vsctl list-ports br.nf
br.nf.d
br.nf.u
[root@NTNX-895b5886-A ~]# ovs-vsctl list-ports br.dmx
br.dmx.d.br0
br.dmx.d.br1
br.dmx.d.br2
br.dmx.u
[root@NTNX-895b5886-A ~]# ovs-vsctl list-ports br2
br2-arp
br2-dhcp
br2.u
eth0
vnet4
[root@NTNX-895b5886-A ~]# ovsdb-client dump
<ouput omitted>
676261b1-8a07-4acf-8de6-8d8fd4866c78 "a0:36:9f:1f:c1:18" 0              false           balance-tcp 0            []     {}           false       [50a1474a-f1b5-4af2-a142-66d0c5749b9a, cb13ef58-5cb7-4562-86ad-2de1ed75e39c] active []  br0-up        {lacp-fallback-ab=True, lacp-time=fast} false     []  {}              {}          {}         {}     []  []     []
7d91390d-8e61-4a88-8c75-0731e8f51535 "a0:36:9f:1f:c1:1a" 0              false           balance-tcp 0            []     {}           false       [17e55f0e-c671-4065-8bc5-cb95f466d564, 53ec5f8d-de7e-4d3a-a316-19d2910bb25d] active []  br1-up        {lacp-fallback-ab=True, lacp-time=fast} false     []  {}              {}          {}         {}     []  []     []

and here is the .70

[root@NTNX-244131ad-A ~]# ip -4 a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
10: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 192.168.86.70/24 brd 192.168.86.255 scope global br0
       valid_lft forever preferred_lft forever
15: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
    inet 192.168.5.1/24 brd 192.168.5.255 scope global virbr0
       valid_lft forever preferred_lft forever
[root@NTNX-244131ad-A ~]# ovs-vsctl list bridge |grep name
name                : brSpan
name                : br1
name                : br.mx
name                : br0
name                : br0.local
name                : br.dmx
name                : br.nf
name                : br.microseg
name                : br1.local
name                : br1-up
name                : br1-up.local
[root@NTNX-244131ad-A ~]# 
[root@NTNX-244131ad-A ~]# ovsdb-client dump
<output omitted>
6adb357e-5e3d-44d6-a8e4-4fdc09fc90e7 []                  0              false           []          0            []     {}           false       [717665f9-d270-484d-8397-ba11bef7edbd]                                                                             []     []  br1-up          {}                                      false     []  {}              {}          {}         {}     []  []     []

52edc5f3-c290-4af6-836d-65240d5bd478 "a0:36:9f:8d:f2:20" 0              false           balance-tcp 0            []     {}           false       [1ec4ec6b-c3d2-4166-96b8-5395dd0babec, c1ee5617-4d42-4c6d-aabc-50bf4bb84bae, d4ca1e3b-cae9-47a3-b7e1-6c8aa4ddf46c] active []  br0-up          {lacp-fallback-ab=True, lacp-time=fast} false     []  {}              {}          {}         {}     []  []     []


 

 

Is the problem that in .170 that the br1-up is a port in bridge br1 and by contract on .70 that br1-up is itself a bridge?

 

I think I will just need to re-AHV the NTNX-244131ad-A node...

This topic has been closed for replies.

  • Voyager
  • April 23, 2025

Your networking needs to be IDENTICAL for number and name of bonds on each node.  IPs can be different, but the same bonds need to be on each node.  You have br2 on your 170 node and not on your 70 node.  Also, unless you are changing the bond mode, avoid ovs-vsctl for configuration, do it through prism element in the virtual networking configuration, otherwise the nutanix will get temperamental.   

[root@NTNX-895b5886-A ~]# ovs-vsctl list bridge |grep name name : br2.local name : br0.local name : br.microseg name : br1 name : br1.local name : brSpan name : br0 name : br.mx name : br.nf name : br.dmx name : br2

[root@NTNX-244131ad-A ~]# ovs-vsctl list bridge |grep name name : brSpan name : br1 name : br.mx name : br0 name : br0.local name : br.dmx name : br.nf name : br.microseg name : br1.local name : br1-up name : br1-up.local