SixXS route collector <-> AS1741 flapping

Pekka Savola pekkas at netcore.fi
Mon Mar 26 12:55:17 CEST 2007 

Hi,

This problem is still ongoing.  Added ipv6-ops@ list to get this on 
the record.  It's interesting to see 50-60% packet loss ongoing for 
about 10 days and no one doing anything about it.  Jeroen has been 
responsive but that doesn't seem to help when transits are messing up. 
Yep.. it's IPv6! :-/

As Jeroen suggested, this issue is most likely with AS6830.  Pinging 
AS9009 from here is lossless (previous hop), but pinging Concepts 
(path to us through AS6830?) results in major packet loss.

Can Concepts push UPC broadband to get this fixed?  They're probably 
paying for the transit ... and even if not, this seems like an 
unacceptable (dis-)service and deactivating or deprefering that 
session (there are other paths) might be appropriate.

(In the longer term I think NORDUnet will come to peer at AMS-IX but 
that's going to take a while; it would be good if GEANT did that as 
well, but I don't think it's going to change anything right now..)

On Fri, 23 Mar 2007, Jeroen Massar wrote:
> Pekka Savola wrote:
>> Hello,
>>
>> This is still going on and no indication of resolution (at least to
>> me).  Has anything been done about this?  It seems like a good time for
>> a ipv6-ops@ rant...
>
> [I deserved that pun, unfortunately I got no enable in this path :( ]
>
> The only solution I see at the moment is to avoid Geant, which doesn't
> connect well to a lot of commercial networks, as mentioned also in my
> rants. (I really would love to see Geant&Abilene&TEIN providing full
> transit to their NREN's as it would definitely make connectivity for the
> NREN's much much much better, that is if those RENT's would actually
> connect to commercial networks, which they unfortunately don't :( )
>
> Between noc.sixxs.net and 2001:708:10:60::4:225
>
> Currently the route back is:
> 12871 6830 3356 20965 2603 1741
> Concepts -> AORTA -> Level3 -> GEANT -> NORDUnet -> FUNet
>
> The other way around is:
> 1741 2603 20965 3356 3257 9009 12871
> FUNet -> NORDUnet -> GEANT -> Level3 -> Tiscali -> RealRoot -> Concepts
>
> The paths are assymetric, so packets might go missing somewhere where we
> can't easily find out that they are going missing.
>
> If I do the same from BIT.nl (AS12859) then the traffic flows directly
> over NTT into Funet:
> 12859 2914 2603 1741
> BIT -> NTT -> NORDUnet -> FUnet
>  1741 2603 2914 12859
> which is exactly the other way around, thus there are good transits
> available for your route too. Clearly NTT doesn't link up directly with
> AORTA nor RealRoot, from which Concepts gets transit.
>
> The problem seems to be that inside AORTA the routes are flapping
> between nl -> be -> uk (which causes packetloss) and nl -> uk which
> gives 0% packetloss (except for a host in Geant dropping ICMP it seems)
>
> Steven can you pretty please look at this, it is most likely causing
> other problems too.
>
> Greets,
> Jeroen
>
>> On Fri, 16 Mar 2007, Jeroen Massar wrote:
>>> Pekka Savola wrote:
>>>> Hi Jeroen,
>>>>
>>>> Our (AS1741) SixXS route collector peering has been flapping
>>>> significantly, for example about 30 times today, 35 times yesterday. It
>>>> seemed to have started around 18:48 UTC on Wednesday (14.3).
>>>>
>>>> Our upstream NORDUnet has transit from both Geant2 (currently used) and
>>>> Sprint (through Tiscali, currenty de-prefered), and the issues are seen
>>>> through Geant2.
>>>
>>> It seems that there is a problem between AORTA and GEANT and therefor
>>> also Concepts which uses AORTA as a transit at the moment.
>>>
>>> CC'd Steven van Steen @ AORTA + Richard Hartensveld @ Concepts as they
>>> run those networks.
>>>
>>> According to other people on #networker (http://unfix.org/networker/)
>>> connectivity over GEANT is fine when not flowing over AORTA.
>>>
>>> Btw, http://www.sixxs.net/tools/traceroute/ allows one to traceroute
>>> from the machine where grh.sixxs.net runs on. Those traceroutes towards
>>> your hosts are flaky to say the least though.
>>>
>>> Greets,
>>> Jeroen
>>>
>>> --
>>>
>>> rest of the message kept for info at sixxs.net, Concepts & AORTA copy.
>>>
>>>> Pinging 2001:838:1:1:210:dcff:fe20:7c7c gives significant packet loss,
>>>> for example (at our end, you can ping for example 2001:708::1):
>>>>
>>>> # ping6 2001:838:1:1:210:dcff:fe20:7c7c
>>>> PING 2001:838:1:1:210:dcff:fe20:7c7c(2001:838:1:1:210:dcff:fe20:7c7c) 56
>>>> data bytes
>>>> 64 bytes from 2001:838:1:1:210:dcff:fe20:7c7c: icmp_seq=0 ttl=47
>>>> time=94.4 ms
>>>> 64 bytes from 2001:838:1:1:210:dcff:fe20:7c7c: icmp_seq=1 ttl=47
>>>> time=92.9 ms
>>>> 64 bytes from 2001:838:1:1:210:dcff:fe20:7c7c: icmp_seq=2 ttl=47
>>>> time=92.3 ms
>>>> 64 bytes from 2001:838:1:1:210:dcff:fe20:7c7c: icmp_seq=7 ttl=47
>>>> time=93.3 ms
>>>> 64 bytes from 2001:838:1:1:210:dcff:fe20:7c7c: icmp_seq=10 ttl=47
>>>> time=93.2 ms
>>>> 64 bytes from 2001:838:1:1:210:dcff:fe20:7c7c: icmp_seq=14 ttl=47
>>>> time=97.0 ms
>>>> 64 bytes from 2001:838:1:1:210:dcff:fe20:7c7c: icmp_seq=17 ttl=47
>>>> time=95.1 ms
>>>> 64 bytes from 2001:838:1:1:210:dcff:fe20:7c7c: icmp_seq=18 ttl=47
>>>> time=94.4 ms
>>>>
>>>> --- 2001:838:1:1:210:dcff:fe20:7c7c ping statistics ---
>>>> 19 packets transmitted, 8 received, 57% packet loss, time 18009ms
>>>> rtt min/avg/max/mdev = 92.395/94.133/97.071/1.459 ms, pipe 2
>>>> now and then.
>>>>
>>>> The traceroute is pretty ugly:
>>>>
>>>> # /usr/sbin/traceroute6 2001:838:1:1:210:dcff:fe20:7c7c
>>>> traceroute to 2001:838:1:1:210:dcff:fe20:7c7c
>>>> (2001:838:1:1:210:dcff:fe20:7c7c) from 2001:708:10:60::4:225, 30 hops
>>>> max, 16 byte packets
>>>>  1  csc1-f20-funet-m.ipv6.funet.fi (2001:708:10:60::1)  0.404 ms  0.322
>>>> ms  0.329 ms
>>>>  2  csc4-g1200-csc1.ipv6.funet.fi (2001:708:0:f000:0:10:40:1)  0.636 ms
>>>> 0.56 ms  0.57 ms
>>>>  3  helsinki0-g2100-csc4.ipv6.funet.fi (2001:708:0:f000:0:1000:40:1)
>>>> 0.786 ms  0.636 ms  0.577 ms
>>>>  4  fi-gw2.nordu.net (2001:948:0:f035::1)  11.386 ms  7.745 ms  7.68 ms
>>>>  5  se-ov.nordu.net (2001:948:0:f037::2)  7.768 ms  7.84 ms  7.681 ms
>>>>  6  dk-gw2.nordu.net (2001:948:0:f03f::2)  15.885 ms  15.862 ms
>>>> 15.851 ms
>>>>  7  dk-gw.nordu.net (2001:948:0:f032::2)  16.349 ms  16.287 ms
>>>> 16.348 ms
>>>>  8  2001:798:15:10aa::d (2001:798:15:10aa::d)  29.335 ms  29.471 ms
>>>> 29.343 ms
>>>>  9  so-6-2-0.rt1.fra.de.geant2.net (2001:798:cc:1401:2201::1)  33.464
>>>> ms  33.523 ms  33.478 ms
>>>> 10  so-6-0-0.rt1.pra.cz.geant2.net (2001:798:cc:1301:1401::1)  41.435
>>>> ms  41.584 ms  41.317 ms
>>>> 11  so-7-2-0.rt1.vie.at.geant2.net (2001:798:cc:1001:1301::2)  48.093
>>>> ms  47.962 ms  47.958 ms
>>>> 12  2001:1900:5:2::5 (2001:1900:5:2::5)  71.327 ms  71.284 ms  71.172 ms
>>>> 13  2001:1900:5:1::1 (2001:1900:5:1::1)  78.792 ms  78.652 ms  78.677 ms
>>>> 14  2001:7f8:4:1::cb9:1 (2001:7f8:4:1::cb9:1)  69.288 ms  69.549 ms
>>>> 69.293 ms
>>>> 15  so-0-1-0.lon11.ip6.tiscali.net (2001:668:0:2::640)  69.326 ms  69.25
>>>> ms  69.304 ms
>>>> 16  2001:7f8:4::2331:1 (2001:7f8:4::2331:1)  69.569 ms  69.464 ms
>>>> 69.342 ms
>>>> 17  ge0-0-44.br0.nlams1.realroute.net (2001:1598:a:3::1)  78.416 ms
>>>> 77.581 ms  77.557 ms
>>>> 18  2001:1598:b031:2::1 (2001:1598:b031:2::1)  77.8 ms  78.095 ms
>>>> 78.168 ms
>>>> 19  * * *
>>>> 20  *
>>>>
>>>> Looking glass shows the following path:
>>>> http://www.csc.fi/funet/status/tools/lg.cgi
>>>>
>>>> inet6.0: 884 destinations, 1575 routes (795 active, 0 holddown, 93
>>>> hidden)
>>>> + = Active Route, - = Last Active, * = Both
>>>>
>>>> 2001:838::/32      *[BGP/170] 05:05:41, MED 300, localpref 50, from
>>>> 2001:708::10:100:1
>>>>                       AS path: 2603 20965 3356 3257 9009 12871 I
>>>>                   > to fe80::290:69ff:fe76:4e17 via ge-0/0/0.0
>>>>                     [BGP/170] 4d 11:20:02, MED 400, localpref 50, from
>>>> 2001:948:0:f006::c
>>>>                       AS path: 2603 20965 3356 3257 9009 12871 I
>>>>                       to 2001:948:0:f036::1 via so-1/1/0.0
>>>>                   > to 2001:948:0:f045::1 via so-3/1/0.0
>>>>
>>>> Our upstream NORDUnet (AS2603) seems to have another path as well
>>>> (through upstreams Sprint and Tiscali), but that isn't being preferred:
>>>>
>>>> [BGP/170] 5d 22:18:33, MED 63, localpref 100
>>>>                       AS path: 6175 3257 9009 12871 I
>>>>                   > to 2001:7f8:d:fb::139 via ge-3/1/0.16
>>>>
>>>
>>>
>>>
>>
>
>
>

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

More information about the ipv6-ops mailing list