6to4 relay at AS26943 (Your.org) blackholing >1280B traffic

[Kevin - Your.Org]

On Feb 25, 2008, at 2:44 PM, Pekka Savola wrote:

> Hi,
>
> On Mon, 25 Feb 2008, Kevin - Your.Org wrote:
>> I'm not completely sure the problem is on our end from what I've  
>> just looked at, but I'll gladly look into this further with you.
>
> This started working about 30-60 minutes ago, maybe you or someone  
> else at your org fixed something, because when a succesful  
> traceroute earlier went like this:
>
> $ traceroute6 gap.netcore.fi 1232
> traceroute to gap.netcore.fi (2002:5872:3460::1), 30 hops max, 1232  
> byte packets
> 1  netcore-gw.ipv6.eunet.fi (2001:670:86:3001::2)  40.147 ms  64.671  
> ms  91.880 ms
> 2  ge0-0-0-997.bbr1.esp1.fi.v6.eunetip.net (2001:670:3:8::1)   
> 111.293 ms  133.066 ms  157.701 ms
> 3  as0-0.bbr1.sto1.se.v6.eunetip.net (2001:670:3:4::116)  185.953  
> ms  209.308 ms  232.101 ms
> 4  2001:670:3:4::2be (2001:670:3:4::2be)  288.335 ms  309.986 ms   
> 344.195 ms
> 5  ge-0.3.0.core1.ams.bb6.your.org (2001:7f8:1::a502:6943:1)   
> 357.126 ms  382.098 ms  403.501 ms
> 6  2002:5872:3460::1 (2002:5872:3460::1)  438.259 ms !X  442.325 ms ! 
> X  443.619 ms !X
>
> .. now there's an stf extra hop:
>
> $ traceroute6 gap.netcore.fi
> traceroute to gap.netcore.fi (2002:5872:3460::1), 30 hops max, 40  
> byte packets
> 1  netcore-gw.ipv6.eunet.fi (2001:670:86:3001::2)  17.524 ms  21.440  
> ms  24.667 ms
> 2  ge0-0-0-997.bbr1.esp1.fi.v6.eunetip.net (2001:670:3:8::1)  28.633  
> ms  30.455 ms  35.247 ms
> 3  as0-0.bbr1.sto1.se.v6.eunetip.net (2001:670:3:4::116)  43.658 ms   
> 48.121 ms  50.301 ms
> 4  2001:670:3:4::2be (2001:670:3:4::2be)  87.304 ms  89.850 ms   
> 92.919 ms
> 5  ge-0.3.0.core1.ams.bb6.your.org (2001:7f8:1::a502:6943:1)  95.556  
> ms  100.303 ms  102.945 ms
> 6  stf.ams.bb6.your.org (2001:4978:2:410:211:43ff:fee8:2c76)   
> 107.032 ms  94.145 ms  95.260 ms
> 7  2002:5872:3460::1 (2002:5872:3460::1)  102.909 ms  101.645 ms   
> 103.216 ms
>
> It seems as if 6to4 interface had been accidentally enabled on  
> core1.ams.bb6 or there were some kind of icmpv6 filtering going on  
> that would affect traceroute.

Nothing was changed on our end and core1.ams.bb6.your.org is a Juniper  
that doesn't even support 6to4, so I honestly have no idea what could  
have been causing it. Our outbound filters for 2002:: say that if the  
internal next-hop isn't our 6to4 relay box not to announce 2002:: at  
all (to prevent that kind of problem from happening) as a double  
safety net.

I notice the latency has really dropped on the second traceroute, I  
wonder what was going on there, and if that was somehow relevant?

I did notice that the 6to4 relay was configured with a rather low icmp  
rate limit (100pps). It doesn't look like it was ever hit(nothing in  
syslog saying it was), but I'll bump it up higher now just to be safe.

In any case, let me know if you see the problem crop up again and I'll  
have a look. I will add "icmp too big" to my automated every 5 minute  
test to make sure our relay is working, too.

-- Kevin