I’m running Keepalived on our loadbalancers for many years and I’m really happy with it. Today I run into an issue that took me some time to solve. I thought I’d share it 🙂
In my current setup I have a pair of Debian Squeeze boxes running version 1.1.20. Since I’m rolling out ipv6 at the moment, I need to upgrade to 1.2.2. Fortunately, Debian provides this version in Squeeze-Backports.
So, I decided to upgrade the Backup loadbalancer first (lb-1). It was a simple ‘apt-get’ procedure to get it installed. But soon these errors popped up in my syslog:
Jun 16 21:12:25 lb-1 Keepalived_vrrp: bogus VRRP packet received on bond1 !!! Jun 16 21:12:25 lb-1 Keepalived_vrrp: VRRP_Instance(CLOUD_MGT_GW) ignoring received advertisment... Jun 16 21:12:25 lb-1 Keepalived_vrrp: receive an invalid passwd!
No messages were to be found in the primary loadbalancer, lb-0. The two loadbalancers weren’t talking to each other any more. I did’t try a failover, as this apparently wouldn’t work. To be sure, I stopped keepalived on the lb-1.
Using tcpdump I found the problem: version 1.1.20 uses a password in its broadcast advertisement that was truncated to the first 7 characters, while the version 1.2.2 uses the full length password, as configured in /etc/keepalived/keepalived.conf. This of course did not match, and so they refused to talk to each other.
The solution was simple: I changed the password in the version 1.2.2 loadbalancer, to be 7 characters long. Then restarted keepalived, and all was working again. After upgrading both loadbalancers, I changed back the password to the longer version and since the versions are now both 1.2.2 it still worked 🙂
Hello, I work in a French company and our servers are all Debian Squeze we want to implement Keepalived for Load Balancing can you please send me your e-mail for ask you some questions of your configuration. My email is juan.diaz[ at ]insa-lyon[ . ]fr. Thank you very much!
Why not posting your question in the comments? I might be able to help you then. Otherwise, a great mailinglist with a lot of people that can help with keepalived is at: http://www.keepalived.org/listes.html
We want to utilize keepalived for load balancing in our enterprise. Our architecture has 3 servers one load balancer and 2 real servers (Postfix). We use Direct Routing and Round Robin.
We have tried for the interfaces’ configuration:
To activate ip_forwarding at the load balancer and deactivate it at the real servers. After that we deactivated the ARP answers from the 2 real servers. After that we tried to give to each real server one loopback interface with the load balancer’s ip address (lo:0 /32)
Our problem is that the request that comes to the load balancer don’t get out from him, but the load balancer sends and receives the healthcheckes with the 2 real servers.
We use Linux Debian Squeeze v6.0.4 and Keepalived v1.1.20 (03/24,2012).
We would like to know which you think the problem is.
This are the tables ipvsadm from the load balancer:
[email protected]:# ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.60.211.50:smtp rr
-> 10.60.211.7:smtp Local 1 0 0
-> 10.60.211.23:smtp Route 1 0 0
[email protected]:~# ipvsadm –list –stats –numeric
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Conns InPkts OutPkts InBytes OutBytes
-> RemoteAddress:Port
TCP 10.60.211 50:25 56 2326 0 117680 0
-> 10.60.211.7:25 28 2242 0 113424 0
-> 10.60.211.23:25 28 84 0 4256 0
Thank you very much
Sorry I haven’t been able to get back to you any sooner. Now that I finished our cloud migration project I’ve some more spare time 🙂
Did you already find a solution for your problem?
This very helpful.
Glad to hear that, thanks!
Thank you for sharing this! Fixed the same issue in minutes thanks to your article.
Thanks a lot buddy! Problem solved!
Greetings from Brazil / RS.
This post just saved me a day of pain. Thanks!