1

I am setting up a new network. I have 4 routers in a full mesh. Edge1, Edge2, Client1, Client2.

This are Mikrotik Cloud Core Routers (not that it matters). They are set up with OSPF (distributing connected and static routes) and MPLS, and BGP.

Edge1 has an upstream provider that provides a full BGP feed.

Edge2 has an upstream provider with just a gateway address.

Client1 has a downstream client, will provide full BGP feed to.

Client2 has a downstream client, will provide a partial BGP feed to.

All four routers have a iBGP feed with the other 3 routers. BGP settings for all are the same, redistribute other BGP. Don't redistribute OSPF / Connected / Static. Though I have tried changing those with no success.

That is the basic setup, more specifics below. All four routers can ping all their IP addresses just fine. Any router to any router, perfect (as far as I can tell) network.

Each router can ping it's external Gateway, but other routers in the mesh can not.

For instance.

Edge1's upstream provider

NOC1 - Gateway Address A.B.238.65/29

Edge1's IP addresses A.B.238.66

Edge1 can ping A.B.238.65. Edge2, Client1, Client2 can all ping A.B.238.66 (so they know the route to that lan), but they can not ping A.B.238.65

This goes for every router. Any of the four routers can ping my side of any external lan attached to any of the four routers. Each router that is directly attached to that network can ping the external gateway on that network. But none of the other routers can ping that external gateway.

The second part of this is that I have my IP block I am announcing. It is a /24. We'll call it Z.X.100.0/24. If I announce that block to my BGP upstream guy, I can ping and access all 4 of my routers from the internet. So they can all respond once they have the routes from BGP. BUT they still can't ping the external gateways. The above mentioned problem still exists.



Detailed Configuration (Sorry, they are configured through the UI, so no scripts):

::Shorthand::

R1 -> Edge1

R2 -> Edge2

R3 -> Client1

R4 -> Client2


Edge1:

Eth1ToUpStream - A.B.238.66/29

Eth2ToR2 - 192.168.1.17/30

Eth3ToR3 - 192.168.1.13/30

Eth4ToR4 - 192.168.1.10/30

Loopback - Z.X.100.1/32 <- Z.X.100.0/24 is what I am announcing upstream as well.

OSPF (Just one area):

Redistribute Connected

Redistrubute Static

Connected to all interfaces

Networks:

(Tried both all the specific /30 networks or just the aggregate /24s. Left as /24s)

192.168.1.0/24

Z.X.100.0/24

A.B.238.64/29

MPLS:

Transport Address:

Z.X.100.1 (with LDP to each of the other three in the mesh)

Failure Case:

Try to ping Z.X.100.66 (via Client1/R3) fails

ping to Z.X.100.65 (via Client1/R3) works

Try to ping C.D.176.65 (via Edge2/R2) fails

ping C.D.176.69 (via Edge2/R2) works


Edge2:

Eth1ToUpStream - C.D.176.69/29

Eth2ToR1 - 192.168.1.18/30

Eth3ToR3 - 192.168.1.21/30

Eth4ToR4 - 192.168.1.6/30

Loopback - Z.X.100.2/32 <- Z.X.100.0/24 is what I am announcing upstream as well.

OSPF (Just one area):

Redistribute Connected

Redistrubute Static

Connected to all interfaces

Networks:

(Tried both all the specific /30 networks or just the aggregate /24s. Left as /24s)

192.168.1.0/24

Z.X.100.0/24

C.D.176.64/29

MPLS:

Transport Address:

Z.X.100.2 (with LDP to each of the other three in the mesh)

Failure Case:

Try to ping Z.X.100.66 (via Client1/R3) fails

ping to Z.X.100.65 (via Client1/R3) works

Try to ping A.B.238.66 (via Edge1/R1) fails

ping to A.B.238.65 (via Edge1/R1) works


Client1:

Eth1ToClient - Z.X.100.65/29

Eth2ToR1 - 192.168.1.14/30

Eth3ToR2 - 192.168.1.22/30

Eth4ToR4 - 192.168.1.2/30

Loopback - Z.X.100.3/32 <- Z.X.100.0/24 is what I am announcing upstream as well.

OSPF (Just one area):

Redistribute Connected

Redistrubute Static

Connected to all interfaces

Networks:

(Tried both all the specific /30 networks or just the aggregate /24s. Left as /24s)

192.168.1.0/24

Z.X.100.0/24

MPLS:

Transport Address:

Z.X.100.3 (with LDP to each of the other three in the mesh)

Failure Case:

Try to ping C.D.176.65 (via Edge2/R2) fails

ping C.D.176.69 (via Edge2/R2) works

Try to ping A.B.238.66 (via Edge1/R1) fails

ping to A.B.238.65 (via Edge1/R1) works


Client2:

Eth1ToClient - Z.X.100.69/29

Eth2ToR1 - 192.168.1.9/30

Eth3ToR2 - 192.168.1.5/30

Eth4ToR3 - 192.168.1.1/30

Loopback - Z.X.100.4/32 <- Z.X.100.0/24 is what I am announcing upstream as well.

OSPF (Just one area):

Redistribute Connected

Redistrubute Static

Connected to all interfaces

Networks:

(Tried both all the specific /30 networks or just the aggregate /24s. Left as /24s)

192.168.1.0/24

Z.X.100.0/24

MPLS:

Transport Address:

Z.X.100.4 (with LDP to each of the other three in the mesh)

Failure Case:

Try to ping C.D.176.65 (via Edge2/R2) fails

ping C.D.176.69 (via Edge2/R2) works

Try to ping A.B.238.66 (via Edge1/R1) fails

ping to A.B.238.65 (via Edge1/R1) works

Try to ping Z.X.100.66 (via Client1/R3) fails

ping to Z.X.100.65 (via Client1/R3) works


Additional information. If it helps.

A traceroute from outside the network to Z.X.100.66 fails 192.168.1.14 is the last hop

A traceroute from outside the network to Z.X.100.65 succeeds 192.168.1.14 is not listed in the hops.

  • RouterOS has a CLI. The command "/ip route print" is your friend. Make sure the destination you are trying to reach has a route back toward the source via the interface you expect. The source IP should be the outgoing interface's IP address unless otherwise specified. – Jeff Loughridge Jan 23 '15 at 14:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.