and this was a gotcha that stole a couple of hours...
Answer... it didn't stop working, I just couldn't see it...
bringing up eth1 when I'd cloned eth0's config:
/etc/sysconfig/network-scripts/ifcfg-eth0
/etc/sysconfig/network-scripts/ifcfg-eth1
and neglected to remove this line:
GATEWAY=AA.BB.CC.DD
When eth1 came up after eth0, it would over-write the default route.
I realised what the problem was when I noticed that a terminal logged in from another host on the same subnet was still working, the one I'd been using from my desktop was being disconnected - the default gateway was obviously missing or wrong.
Before (bad):
# netstat -nr
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
192.168.98.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
172.16.98.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth1
0.0.0.0 192.168.98.98 0.0.0.0 UG 0 0 0 eth1
# route add default gw 172.16.98.1
suddenly it starts working again...
# netstat -nr
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
192.168.98.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
172.16.98.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth1
0.0.0.0 172.16.98.1 0.0.0.0 UG 0 0 0 eth0
0.0.0.0 192.168.98.98 0.0.0.0 UG 0 0 0 eth1
But really we want:
# netstat -nr
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
192.168.98.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
172.16.98.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth1
0.0.0.0 172.16.98.1 0.0.0.0 UG 0 0 0 eth0