Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

l2layer announcer not work fine for ipv6 #2685

Open
8 tasks done
btkinghome opened this issue Feb 13, 2025 · 0 comments
Open
8 tasks done

l2layer announcer not work fine for ipv6 #2685

btkinghome opened this issue Feb 13, 2025 · 0 comments
Labels

Comments

@btkinghome
Copy link

MetalLB Version

all

Deployment method

Operator

Main CNI

cilium

Kubernetes Version

v1.28.12

Cluster Distribution

No response

Describe the bug

layer2 announcer not work fine When use metallb in k8s ipv6 network.

speaker can announce vip ndp info success when the globalunicat ipv6 address is the first addres of the intreface, if the ipv6 globalunicat ipv6 address is not the first address, speak will get a error: "ipv6-icmp bind: cannot assign requested address" ,because speaker use the link-local address as the source IPv6 address for NDP communications. the os will refuse this op.

finally,the real ipv6 interface can't be the partner of the map “keepNDP“,init failed

of course,there is other problem in internal/layer2/announcer.go

  1. send arp package to interface which only have ipv6 address
  2. send ndp package to interface which only have ipv4 address

To Reproduce

  1. config ipv6 for os
  2. install k8s
  3. install metallb
  4. keep ipv6 address is not the first ipaddress of the interface(The first one from the bottom up)
  5. View the metallb speaker log

Expected Behavior

  1. only announce arp to interface which only have ipv4address
  2. only announce ndp to interface which only have ipv6address
  3. announce arp and ndp to interface which have dualstack address
  4. send ndp package use globalunicat address as the source IPv6 address for NDP communications.

Additional Context

code Analysis
--------------------internal/layer2/announcer.go------
if ifi.Flags&net.FlagBroadcast != 0 { // ---> only use broadcast can't distinguish interface who should use the ARP or NDP
keepARP[ifi.Index] = true
}

	for _, a := range addrs {
		ipaddr, ok := a.(*net.IPNet)
		if !ok {
			continue
		}
		if ipaddr.IP.To4() != nil || !ipaddr.IP.IsLinkLocalUnicast() {   //  --> Linux usually config a localunicat ipv6 address for interface, so this interface also will be the keepNDP item also it only config a ipv4 address.
			continue
		}
		keepNDP[ifi.Index] = true
		break
	}

-----------------------internal/layer2/ndp..go----------------------------------------
func newNDPResponder(logger log.Logger, ifi *net.Interface, ann announceFunc) (*ndpResponder, error) {
// Use link-local address as the source IPv6 address for NDP communications.
conn, _, err := ndp.Dial(ifi, ndp.LinkLocal) // ------> LinkLocal raw6 listen op is not allowed by all os.
if err != nil {
return nil, fmt.Errorf("creating NDP responder for %q: %s", ifi.Name, err)
}

ret := &ndpResponder{
	logger:              logger,
	intf:                ifi.Name,
	hardwareAddr:        ifi.HardwareAddr,
	conn:                conn,
	closed:              make(chan struct{}),
	announce:            ann,
	solicitedNodeGroups: map[string]int64{},
}
go ret.run()
return ret, nil

}

so, i changed the source code

-------------------internal/layer2/announcer.go------

             for _, a := range addrs {
		ipaddr, ok := a.(*net.IPNet)
		if !ok {
			continue
		}

		if ipaddr.IP.To4() != nil && ipaddr.IP.IsGlobalUnicast() && ifi.Flags&net.FlagBroadcast != 0{
			level.Debug(l).Log("event", "announced interface", "interface", ifi.Name, "ip", ipaddr.IP)
			keepARP[ifi.Index] = true
		}

		if ipaddr.IP.To4() == nil && ipaddr.IP.To16() != nil && ipaddr.IP.IsGlobalUnicast() {
			level.Debug(l).Log("event", "announced interface", "interface", ifi.Name, "ip", ipaddr.IP)
			keepNDP[ifi.Index] = true
		}
	}

	if !keepARP[ifi.Index] && !keepNDP[ifi.Index] {
		continue
	}

-----------------------internal/layer2/ndp..go----------------------------------------
func newNDPResponder(logger log.Logger, ifi *net.Interface, ann announceFunc) (*ndpResponder, error) {
// Use link-local address as the source IPv6 address for NDP communications.
conn, _, err := ndp.Dial(ifi, ndp.Global)
........
}

Except the code change,also add a config for the metallb operator values.json to filter some interface.

ps: i can't found this config document in the metallb website ^_^


configInline: |
announcedInterfacesToExclude:

  • lo
  • lxc.*

I've read and agree with the following

  • I've checked all open and closed issues and my request is not there.
  • I've checked all open and closed pull requests and my request is not there.

I've read and agree with the following

  • I've checked all open and closed issues and my issue is not there.
  • This bug is reproducible when deploying MetalLB from the main branch
  • I have read the troubleshooting guide and I am still not able to make it work
  • I checked the logs and MetalLB is not discarding the configuration as not valid
  • I enabled the debug logs, collected the information required from the cluster using the collect script and will attach them to the issue
  • I will provide the definition of my service and the related endpoint slices and attach them to this issue
@btkinghome btkinghome added the bug label Feb 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant