Strange IPv6 reachability problem

[Fred Baker]

I noticed a similar problem with one of the perhaps-1000 sites I tested on 7 June. As near as they could tell, their site was fine; from my perspective, I could open a web page (browser or curl) but would only get every other data segment, and as a result went off dupacking. They eventually decided it had to do with one of their vendors (an upstream network, I imagine, but they didn't say). When they routed around it, the problem went away.

Could be happening with you too.

On Jun 19, 2011, at 3:19 PM, Seth Mattinen wrote:

> I'm having a strange IPv6 problem that I hope someone here can help with or at least generate some ideas on how to resolve. I have a strange partial reachability issue with 2001:500:61:28::70 (debian mirror) and a couple others, but I'll just focus on this one for clarity. Both working and trouble host are sourced from 2607:fe70::/which I announce via BGP and route around in my network.
> 
> I can reach it fine from my daily I-work-here-in-SSH server:
> 
> source: 2607:fe70:0:beef::a
> dest: 2001:500:61:28::70
> 
> PING 2001:500:61:28::70(2001:500:61:28::70) 56 data bytes
> 64 bytes from 2001:500:61:28::70: icmp_seq=1 ttl=57 time=81.3 ms
> 64 bytes from 2001:500:61:28::70: icmp_seq=2 ttl=57 time=81.3 ms
> 64 bytes from 2001:500:61:28::70: icmp_seq=3 ttl=57 time=82.4 ms
> 64 bytes from 2001:500:61:28::70: icmp_seq=4 ttl=57 time=81.3 ms
> 64 bytes from 2001:500:61:28::70: icmp_seq=5 ttl=57 time=81.3 ms
> 
> --- 2001:500:61:28::70 ping statistics ---
> 5 packets transmitted, 5 received, 0% packet loss, time 4015ms
> rtt min/avg/max/mdev = 81.304/81.547/82.434/0.511 ms
> 
> 
> I *can not* reach it from a server at my parent's house:
> 
> source: 2607:fe70:1002:0:202:b3ff:fe30:f633
> dest: 2001:500:61:28::70
> 
> PING 2001:500:61:28::70(2001:500:61:28::70) 56 data bytes
> 
> --- 2001:500:61:28::70 ping statistics ---
> 5 packets transmitted, 0 received, 100% packet loss, time 4013ms
> 
> 
> Now, here's the part that confuses me. After trying a whole bunch of basic things I ran nfdump and found something I can't explain:
> 
> nfdump filter:
> proto icmp6 and host 2001:500:61:28::70
> Aggregated flows 3
> Top 500 flows ordered by flows:
> Date flow start          Duration  Proto      Src IP Addr      Dst IP Addr   Packets    Bytes      bps    Bpp Flows
> 2011-06-19 14:55:00.264     4.012  ICMP6 2001:50..:28::70 2607:fe..beef::a         5      520     1036    104     1
> 2011-06-19 14:55:00.760     4.016  ICMP6 2607:fe..beef::a 2001:50..:28::70         5      520     1035    104     1
> 2011-06-19 14:56:15.724     4.016  ICMP6 2607:fe..30:f633 2001:50..:28::70         5      520     1035    104     1
> 
> Summary: total flows: 3, total bytes: 1560, total packets: 15, avg bps: 157, avg pps: 0, avg bpp: 104
> Time window: 2011-06-19 14:55:00 - 2011-06-19 14:56:19
> Total flows processed: 329214, Blocks skipped: 0, Bytes read: 23941692
> Sys: 0.288s flows/second: 1143036.7  Wall: 0.324s flows/second: 1014236.3
> 
> For the one that works there's flow records in each direction as expected. For the one that doesn't there's no return traffic coming back to me. So I'm assuming the problem is not with me since if it was, I'd at least see a flow record coming back to me if something in my network was dropping it. But there's no record of return traffic. Netflow is sampled at the routers that connect to each upstream.
> 
> I tried from a second server on 2607:fe70:10::/64 and it dodn't work either. Every server/router I try from within 2607:fe70::/48 works fine.
> 
> Any ideas?
> 
> ~Seth