Some very nice broken IPv6 networks at Google and Akamai (Was: Some very nice IPv6 growth as measured by Google)

[Tore Anderson]

* Nick Hilliard

> On 09/11/2014 11:00, Tore Anderson wrote:
> > Only if Google and Akamai are universally broken, which does not
> > seem to have been the case. I tested Google from the RING at 23:20
> > UTC yesterday:
> 
> did you do a control run on a known working site?

No. I feel that 250+ successes vs 10 failures is enough to conclude
that Akamai and Google are *not* universally broken, far from it. Thus
refuting the claim that «Google and Akamai IPv6 are currently broken,
enabling IPv6 thus breaks connectivity to those sites».

Whatever broke, it must have been much more local than that, or only
occurring under certain conditions (e.g., tunnels dependent on PMTUD).

> Not all ring nodes have working ipv6.

Exactly. That's a likely explanation for (some of) the 10 failures.

I redid the tests now, and the failing nodes were:

beanfield01.ring.nlnog.net
bluezonejordan01.ring.nlnog.net
claranet02.ring.nlnog.net
hosteam01.ring.nlnog.net
keenondots01.ring.nlnog.net
maxitel01.ring.nlnog.net
nicchile01.ring.nlnog.net
occaid01.ring.nlnog.net
popsc01.ring.nlnog.net
rackfish01.ring.nlnog.net
robtex01.ring.nlnog.net

Of these, only three were able to ping 2a02:c0::1 which I know should
respond fine. The other ones got various "no route to host",
"destination beyond scope of source", and stuff like that.

The three that had working IPv6 connectivity were:

hosteam01.ring.nlnog.net
nicchile01.ring.nlnog.net
occaid01.ring.nlnog.net

hosteam01 and occaid01 have defective local DNS, they can't resolve
anything it seems. So nothing to do with Google and Akamai there.

nicchile01 is the only one that looks interesting, as it works for
Google but not Akamai:

redpilllinpro at nicchile01:~$ wget -6 --header "User-Agent: foo" -O /dev/null http://www.akamai.com/images/img/banners/entertainment-home-page-banner-932x251.jpg
--2014-11-09 12:03:41--  http://www.akamai.com/images/img/banners/entertainment-home-page-banner-932x251.jpg
Resolving www.akamai.com (www.akamai.com)... 2600:1419:7:185::22d9, 2600:1419:7:189::22d9
Connecting to www.akamai.com (www.akamai.com)|2600:1419:7:185::22d9|:80... failed: Connection refused.
Connecting to www.akamai.com (www.akamai.com)|2600:1419:7:189::22d9|:80... failed: Connection refused.

However, tcpdump reveals that this isn't Akamai's doing, as it's
ICMP errors originating from a NIC Chile-owned IP address.

12:06:19.388093 IP6 2001:1398:32:177::40 > 2001:1398:3:120:200:1:120:28: ICMP6, destination unreachable, unreachable port, 2600:1419:7:185::22d9 tcp port 80, length 88
12:06:19.389095 IP6 2001:1398:32:177::40 > 2001:1398:3:120:200:1:120:28: ICMP6, destination unreachable, unreachable port, 2600:1419:7:189::22d9 tcp port 80, length 88

Perhaps they have firewalled out Akamai for some reason?

In any case. I summary I see *zero* evidence of ubiquitous IPv6
problems with Google and Akamai. So ISPs should not worry about
deploying IPv6, at least if they're doing it native and don't
expose themselves to PMTUD breakage.

Tore