Dynamic CDN Comparison and Benchmark

This document presents a comparative benchmark of different Content Delivery Networks (CDNs) based on tests conducted from Silicon Valley (SV), USA, and Shanghai (SHA), China, to Guangzhou (GZ), China, and Seoul (SEL), South Korea. The tests were performed using both direct connections and several CDN providers.

Test Setup

• Tool: Playwright with default settings.
• Procedure: Each test was executed five times to provide each CDN ample opportunity to cache content.
• Test Website: Generated from the Flutter game template
  (flutter/samples/game_template) with the following modifications:
  • Fonts:
    • The default theme font was replaced with a larger NotoSansSC-VariableFont_wght.ttf.
    • The Roboto font was replaced with a local version by modifying the main.dart.js file to prevent third-party resource requests.

Test Flow Diagram

+------------+          +--------------+          +---------------+
|   Client   |  <--->   |     CDN      |  <--->   | Origin Server |
| (SV / SHA) |          |  (or Direct) |          |  (GZ / SEL)   |
+------------+          +--------------+          +---------------+

---

Test Results

1. Silicon Valley (SV) to Seoul (SEL)

Client	CDN	Server	Time (seconds)
SV	GC	SEL	4.78
SV	CF	SEL	5.12
SV	FL	SEL	5.30
SV	CFT	SEL	5.95
SV	Direct	SEL	6.80

Observations:

• The performance across different modes is fairly consistent.
• GC is notably the fastest, while even a direct connection (without caching) is only 14% slower than CFT.

2. Silicon Valley (SV) to Guangzhou (GZ)

Client	CDN	Server	Time (seconds)
SV	FL	GZ	5.17
SV	CF	GZ	8.85
SV	Direct	GZ	9.27

Observations:

• Increased latency is expected due to the greater distance.
• CF’s slower performance may be due to uncached resources (e.g., index.html), while FL’s uncharacteristically fast result suggests possible aggressive caching.that cached un-cachable content.
• Note: CFT and GC were excluded from this test because they lack IPv6 origin support, and the GZ server is IPv6-only.

3. Shanghai (SHA) to Guangzhou (GZ)

Client	CDN	Server	Time (seconds)
SHA	Direct	GZ	7.57
SHA	FL	GZ	120.00
SHA	CF	GZ	∞

Observations:

• Despite both SHA and GZ being located in China, Direct access should be faster, but the bandwidth limitations of the residential network hosting the GZ server might be a factor.
• FL and CF perform poorly,likely due to the challenging network environment in China for accessing overseas infrastructure, with CF particularly suffering from abuse..

4. Shanghai (SHA) to Seoul (SEL)

Client	CDN	Server	Time (seconds)
SHA	GC	SEL	6.73
SHA	Direct	SEL	7.40
SHA	CFT	SEL	8.47
SHA	FL	SEL	70.00
SHA	CF	SEL	∞

Observations:

• GC again shows impressive performance, remaining robust despite the challenging network conditions associated with cross-border traffic.
• Direct access to SEL was also reasonable, comparable to the direct SHA to GZ connection, despite the different network enviroment and locations.
• CFT, while slightly slower, remains usable.
• FL and CF are not viable options for this route in China.

---

Conclusions

Amazon CloudFront (CFT)

• Pros:
  • Free quotas and overall stability.
  • Acceptable response times.
  • No need to change your domain nameservers.
• Cons:
  • Complex configuration for HTTPS (requires your origin to provide both a TLS certificate for its own domain and a TLS certificate for the domain pointing to CFT, or a custom cache policy to remove the host header).
  • Potential need for additional firewall rules (and possibly extra costs).
  • A steep learning curve—services like ACM and CloudWatch need time to master.

Gcore (GC)

• Pros:
  • Simpler configuration compared to CFT.
  • Fast and consistent performance.
• Cons:
  • Limited documentation and community support.
  • Requires changing domain nameservers.
  • Configuration changes can take several minutes to take effect without clear status indicators.
  • HTTPS configuration requires adjustments (setting SNI and Host headers to match the origin).

Cloudflare (CF)

• Pros:
  • A viable option when both server and visitors are overseas.
  • Feature-rich, mature community, and free plans.
• Cons:
  • Poor performance in tests initiated from within China, due to network restrictions or potential abuse issues.

Fastly (FL)

• Issues:
  • Despite supporting IPv6 origins, Not recommended when compared to CF.CF offers comparable performance.

---

Important Considerations

• IPv6 Support:
  CFT and GC do not support IPv6 origins.

• Caching Behavior:

  • FL defaults to caching everything and is difficult to configure.
  • GC also caches aggressively but can be tweaked with easily configuration adjustments.

• WebSocket Support:
  FL requires a paid plan to enable WebSocket support.

• Configuration Propagation:
  GC shows a delay (taking several minutes) for configuration changes to take effect without clear progress indications.

Abbreviations:

• SV: Silicon Valley, USA
• SEL: Seoul, South Korea
• GZ: Guangzhou, China
• SHA: Shanghai, China
• CF: Cloudflare
• CFT: CloudFront
• FL: Fastly
• GC: Gcore

---

Additional Observations

After the tests, there was an attempt to resolve the absence of a public IPv4 address on the server by pointing the origin for CFT or GC to Cloudflare. Despite multiple configuration attempts, success was only achieved by:

• Changing the cache policy in CFT, or
• Modifying the SNI and Host headers in GC to match the origin domain.

While this workaround might seem superfluous, it could be beneficial for setups lacking a public IPv4 address. Note that with this method, GC would require the use of a separate domain since both Cloudflare and GC demand their domains be pointed to their own nameservers.