Peering traceroute hops

Bennmann

Hello,

I am currently on a 30 trial for 1 gig cable, hoping to switch from AT&T Fiber due to cost.

I am on day 1. So far the throughput speeds are nearly equivalent in all my testing.

But latency is 4-5x worse, with what seems like an enormously bad number of netops.charter hops for even basic things like 8.8.8.8 dns server. Again, just 4-5x worse ping (18ms vs like 90-100ms).

The main game I like to play (Apex Legends) is 15ms ping on AT&T fiber. On Spectrum it's 60ms to the same game server.

I really want to switch to spectrum, it's cheaper.

Do I need to submit a ticket with Spectrum with various traceroutes showing peering and request a tier 2-3 dig into my peering and traceroutes data?

HT_Greenfield

I beg your pardon for talking out of turn, here, but in order to help the man to help you to gage your expectations:

1. Spectrum netops routing notwithstanding, the only way it would be a peering issue is if there's a reasonably significant difference between the first hop off of Spectrum netops versus the last hop on it.
2. 60 ms is nevertheless good if not excellent as "… Ping under 100ms will keep your game running well, …"
3. What is the difference according to Gameplay > Performance Display?
4. Is there any discernible gameplay difference?

Ref.: 🔗 https://help.ea.com/en/help/apex-legends/apex-legends/ping-data-center/

P.S.: It's all Greek to me but, if you're seeing the same kind of proportional difference on every hop, then i take it that it could simply come down to the difference between FTTH PHY versus DOCSIS PHY.

William_M

Hi @Bennmann, welcome to Spectrum and our community here!

That's a much larger increase than expected just from switching from fiber to coaxial cable. If you share your traceroute results here we can review them.

Bennmann

Some back of envelope theoretical impact:

I play the game at 270FPS (theoretical best case, some judder), each frame is about 3.7ms.

A difference of 40ms is about 10 in game frames (theoretical maximum impact).

Some people might posit that 10 frames or 40ms difference are not measurable in human performance, however I do have some third party sources which provide evidence lag can and does make a difference if that will help explain.

Example spectrum peering: tracert -h 15 3.213.116.90

Tracing route to ec2-3-213-116-90.compute-1.amazonaws.com [3.213.116.90] over a maximum of 15 hops:

1 2 ms 2 ms 3 ms 192.168.0.1

2 * * * Request timed out.
3 11 ms 17 ms 13 ms int-0-7-1-15.dtr01sghlga.netops.charter.com [96.34.78.18]
4 14 ms 14 ms 18 ms lag-103.mcr11nwnngase.netops.charter.com [69.144.252.118]
5 63 ms 40 ms 32 ms lag-21.rcr01mnchtnem.netops.charter.com [96.34.13.228]
6 31 ms 25 ms 29 ms lag-10.rcr01sghlgaao.netops.charter.com [96.34.13.230]
7 109 ms 131 ms 37 ms lag-810.bbr01atlnga.netops.charter.com [96.34.15.19]
8 31 ms 33 ms 28 ms lag-803.prr02atlnga.netops.charter.com [96.34.3.37]
9 * * * Request timed out
.… etc


Example AT&T fiber peering:tracert -h 15 3.213.116.90

Tracing route to ec2-3-213-116-90.compute-1.amazonaws.com [3.213.116.90] over a maximum of 15 hops:

1 2 ms 2 ms 2 ms dsldevice.attlocal.net [192.168.1.254]
2 2 ms 2 ms 2 ms 45-19-108-1.lightspeed.tukrga.sbcglobal.net [45.19.108.1]
3 3 ms 2 ms 7 ms 107.212.168.236
4 8 ms 10 ms 5 ms 12.242.113.44
5 * * * Request timed out
.…etc

Why are so many lag netops.charter peers used in this routing/peering? Each one adds latency.

James_M

Hi @Bennmann -

For the example you provided, the 109ms and 63ms on separate hops are consistent with congestion, since the resolve after the next hop. Here is a link to Ping and Traceroute FAQ's that might also be helpful.

Can you provide a FULL traceroute to the game server for us to take a look? We need to see the traceroute until the "trace complete", and not just "…etc", so we can also see the total number of hops.

Thanks!

Bennmann

Hello James,

Spectrum max 30 hops (Windows 11 cmd limit):

tracert -w 200 3.213.116.90

Tracing route to ec2-3-213-116-90.compute-1.amazonaws.com [3.213.116.90]
over a maximum of 30 hops:

1 3 ms 2 ms 2 ms 192.168.0.1
2 * * * Request timed out.
3 15 ms 12 ms 18 ms int-0-7-1-15.dtr01sghlga.netops.charter.com [96.34.78.18]
4 17 ms 18 ms 17 ms lag-103.mcr11nwnngase.netops.charter.com [69.144.252.118]
5 28 ms 29 ms 26 ms lag-21.rcr01mnchtnem.netops.charter.com [96.34.13.228]
6 25 ms 25 ms 24 ms lag-10.rcr01sghlgaao.netops.charter.com [96.34.13.230]
7 * 125 ms 56 ms lag-810.bbr01atlnga.netops.charter.com [96.34.15.19]
8 29 ms 29 ms 27 ms lag-803.prr02atlnga.netops.charter.com [96.34.3.37]
9 * * * Request timed out.
10 * * * Request timed out.
11 * * * Request timed out.
12 * * * Request timed out.
13 * * * Request timed out.
14 * * * Request timed out.
15 * * * Request timed out.
16 * * * Request timed out.
17 * * * Request timed out.
18 * * * Request timed out.
19 * * * Request timed out.
20 * * * Request timed out.
21 * * * Request timed out.
22 * * * Request timed out.
23 * * * Request timed out.
24 * * * Request timed out.
25 * * * Request timed out.
26 * * * Request timed out.
27 * * * Request timed out.
28 * * * Request timed out.
29 * * * Request timed out.
30 * * * Request timed out.

Trace complete.

——————

Example AT&T fiber peering:tracert -w 200 3.213.116.90

Tracing route to ec2-3-213-116-90.compute-1.amazonaws.com [3.213.116.90] over a maximum of 15 hops:

1 1 ms 2 ms 1 ms dsldevice.attlocal.net [192.168.1.254]
2 8 ms 5 ms 5 ms 45-19-108-1.lightspeed.tukrga.sbcglobal.net [45.19.108.1]
3 7 ms 4 ms 6 ms 107.212.168.236
4 7 ms 5 ms 9 ms 12.242.113.44
5 * * * Request timed out

6 * * * Request timed out

7 * * * Request timed out

8 * * * Request timed out

9 * * * Request timed out.

10 * * * Request timed out.

11 * * * Request timed out.

12 * * * Request timed out.

13 * * * Request timed out.

14 * * * Request timed out.

15 * * * Request timed out.

16 * * * Request timed out.

17 * * * Request timed out.

18 * * * Request timed out.

19 * * * Request timed out.

20 * * * Request timed out.

21 * * * Request timed out.

22 * * * Request timed out.

23 * * * Request timed out.

24 * * * Request timed out.

25 * * * Request timed out.

26 * * * Request timed out.

27 * * * Request timed out.

28 * * * Request timed out.

29 * * * Request timed out.

30 * * * Request timed out.

Trace complete.

None of the peers past lag netops.charter will populate (this is expected behavior for a consumer/end user).

The open questions around lag netops.charter remain completely unanswered, and getting more trace route information is not really helpful.

It may still be best to involve a tier 2/3 on this request.

Mo_S

There is no standard number of hops either within or outside our network to route traffic successfully to the destination. Routing is dynamic with network management and can change if a better route is available. As the latency shown is around 30ms at the last hop before leaving our network, that would be well within normal. Do you have traceroutes to other sites that show successfully reaching the destination since these do not?

Bennmann

I had congestion earlier in this ticket, obviously peak times is Friday/Saturday. My congestion example was a Saturday.

The latest example is off peak Monday morning data.

Based on the analysis so far probably gonna switch back to AT&T, seems like Charter netops lag servers get congested on peak times vs AT&T fiber.

You might want to let your netops lag team know in an internal ticket about lag-810 instance. Maybe they will do a peak time analysis based on this data, or maybe not. Maybe they will adjust the instance sizing or maybe not.

James_M

Sounds good. We will archive this post, but if you decide to post additional examples, or if you see a new issue, let us know in a new post.

HT_Greenfield

It turns out that it's ultimately up to whoever's operating an EC2 instance whether what's routing-wise good for the goose is also supposed to be good for the gander. I can't even ping the one you've cited, by the way.

🔗 https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Internet_Gateway.html

I fold. Over and out. ☘️