LAN cable operation to switch internet line on L2 SW cascade connection

Question

We have 2 internet lines A and B, and 2 independent network segments on the lines.

All PCs are connected to L2 switches (SWAA, SWAB, ..., SWBA, SWBB, ...), and the switches are connected to another L2 switches (SWA, SWB). SWA, SWB are connected to a router for each internet line A and B. All switches have 8 ports, we use 1 port for uplink, and others for downlink. Switches are unmanaged, just cheap models.

Internet Line A Router
|
SWA----------------------------
|    |    |    |    |    |    |
SWAA SWAB SWAC SWAD
|    |-   |-   |-
|
|------------------------
|   |   |   |   |   |   |
PC1 PC2 PC3 PC4 PC5 PC6 PC7


Internet Line B Router
|
SWB----------------------------
|    |    |    |    |    |    |
SWBA SWBB SWBC
|    |-   |- 
|
|------------------------
|   |   |   |   |   |   |
PC1 PC2 PC3 PC4 PC5 PC6 PC7

When an internet line (suppose line B) gets trouble, we pull the all cables inserted to SWB out, and insert them into SWA like following, as temporary troubleshooting operation.

Internet Line A
|
SWA----------------------------
|    |    |    |    |    |    |
SWAA SWAB SWAC SWAD SWBA SWBB SWBC
|    |-   |-   |-   |-   |-   |-
|
|------------------------
|   |   |   |   |   |   |
PC1 PC2 PC3 PC4 PC5 PC6 PC7

But sometimes the troubleshooting operation doesn't work well, after the operation, all PCs (not only PCs from SWB, but PCs on SWA originally) get network problems.

Line A and B routers provide DHCP using different subnet, like 192.168.11.* on line A router, and 102.168.12.* on line B router. So all PCs which are moved should reset their IP to get new IP by DHCP.

I'm doubting the problem is due to many broadcasts after the operation, or arp table mismatch on routers, but still not sure what the main reason is.

Do you have any idea about the following things?

1. The reason of the problem.

2. Idea how to make sure the reason.

3. Better network topology to avoid the problem.

Answer 1

If get this correctly, you make your clients move over to the other default gateway by letting them update their DHCP lease. This isn't very reliable. To find out the exact reason for your problems you'll need to do some analysis including packet capturing. Do the clients have the right IP addresses? Does ARP work? Is the gateway pingable?

A layer 3 topology for serious use would be a central (redundant) router with both Internet uplinks connected. In normal operation, loads are balanced between both links (by load, by protocol, by user group, ...) and when a link fails, all connections go out the other.

Additionally, using these deep cascades with cheap switches introduces several risks - cheap, consumer switches are not as reliable as business-grade hardware and unmanaged switches allow you no control at all over your network. Also, implementing different security zones is at least awkward.

For the layer 2 topology, you should aim for a central 'core' switch and connect all client switches to it in a single layer ("collapsed core"). If you need redundancy use two core switches with a redundancy protocol (e.g. RSTP) and connect the client switches to both.