How to isolate a router for maintenance

We want to isolate one of the routers for maintenance or in case one of the ISPs are doing maintenance on their side.

Normally our inbound and outbound traffic goes through both routers (active/active).

If for maintenance we want to isolate one of the routers so that no inbound and outbound traffic goes through it we follow these steps.

Here we assume that we want to isolate Router0 (Cogent) and the command examples are for this router.

If we shut down the BGP session with the ISP the inbound traffic from that ISP will stop all inbound traffic will go through the other router.

We also shut down the iBGP session with the other router. So basically all the IPv4 and IPv6 BGP sessions are shut down.

  1. Make sure both IPv4 and IPv6 eBGP sessions on the other router (Router1-HE) are up and running.# show ip bgp summary
    # show ipv6 bgp summary
  2. Shut down both IPv4 and IPv6 eBGP with Cogent in Router0.
    Router0# set protocols bgp 3xxxx neighbor 199.x.x.x shutdown
    Router0# set protocols bgp 3xxxx neighbor 26xx:xxxx::2 shutdown

  3. Shutdown both IPv4 and IPv6 iBGP between the routers.
    Router0# set protocols bgp 3xxxx neighbor 199.x.x.x shutdown
    Router0# set protocols bgp 3xxxx neighbor 26xx:xxxx::2 shutdown

  4. Disable all ports on the other side of the links to Router0: Core0-p01, Core1-p01 and Router1-eth1. This will kill all OSPF and OSPFv3 neighbor adjacencies with Router0.
    Also when the corresponding ports on Router0 go up and down, they have no effect on neighboring devices. IMPORTANT: Make sure not to disable the port that is connected to ISP.
    Otherwise the router cannot be accessed remotely and it can only be fixed by going to data center and connecting a console and keyboard to it.

    SSH@core0(config)# interface ethernet 1/1/1
    SSH@core0(config)# disable
    SSH@core1(config)# interface ethernet 1/1/1
    SSH@core1(config)# disable
    Router1# set interfaces ethernet eth1 disable

  5. Verify that there is no neighborship adjacency in Router0

    Router0# show ip ospf neighbors
    Router0# show ipv6 ospfv3 neighbor

  6. Save Router0 config, so that BGP doesn’t start after possible reboot.

  7. Disable the Cacti alerts regarding the isolated router.

In order for bringing back the isolated router into production we do same steps in reverse:
First we enable OSPF by enabling the interfaces that were disabled. Then we start the BGP sessions that were shut down in the previous step.

  1. Enable the ports that were disabled  on Core0 and Core1 and Router1

    Router1# delete interfaces ethernet eth1 disable
    Router1# commit
    SSH@core0(config)# interface ethernet 1/1/1
    SSH@core0(config)# enable
    SSH@core1(config)# interface ethernet 1/1/1
    SSH@core1(config)# enable
  2. Verify the OSPF neighbor adjacency to be all in state FULL

    Router0## show ip ospf neighbors
    Router0## show ipv6 ospfv3 neighbor

  3. Start BGP all the BGP sessions on Router0

    Router0# delete protocols bgp 3xxxx neighbor 38.x.x.x shutdown
    Router0# delete protocols bgp 3xxxx neighbor 20xx:xxx:x:xxx::1:1 shutdown
    Router0# delete protocols bgp 3xxxx neighbor 199.x.x.x shutdown
    Router0# delete protocols bgp 3xxxx neighbor 26xx:xxxx::2 shutdown
    Router0#  commit

  4. Verify the BGP peerings:

    # show ip bgp summary
    # show ipv6 bgp summary

  5. If everything is ok, save all the configurations on routers and core switches