Dell N3000 – Firmware upgrade breaks BGP

I have 2 stacks of Dell N3048 switches. I upgraded one of them on the weekend from 6.2.7.2 to 6.3.0.56 and found that two of my BGP sessions didn’t come up any more – after establishing and receiving the routes the BGP session would cease with this logged:

<191> May  9 07:57:07 level14-stack-5 BGP[BGP Protocol]: bgptcp.c(653) 15582 %% [VRF ""] Sent BGP NOTIFICATION to neighbor 192.168.39.253 with error code 6 (Cease) and subcode 0 (None).

After enabling debug logging for BGP it didn’t help narrow down the cause at all – I couldn’t see anything wrong with the routes being advertised to it and I compared the routes to the other stack and verified that all routes being advertised to this upgraded stack were the same. I decided to filter the routes and only advertise each prefix one by one to find which specific route was causing the issue. I decided to start with any /32’s advertised to the switch and got lucky on the first try – the prefix that was causing this was 192.168.39.253/32 (which belongs to the BGP neighbor itself). After filtering out that route my BGP session comes up again as normal.

I have opened a case with Dell about this and will update this with what they say. If you are running one of the N3000 switches on the latest firmware and come across this I recommend filtering the BGP neighbors IP from outgoing routes as a workaround.