River Message Propagation Debugging Report

Date: January 27, 2025
Machine: technic
River Version: v0.1.10 (with subscription fix)
Freenet Version: v0.1.21

Executive Summary

River v0.1.10 addresses a critical subscription bug where room creators never subscribed to their own rooms. While this fix resolves the issue in local gateway testing, the production gateway on nova fails to store contracts. Testing shows PUT operations reach the gateway but subsequent GET operations fail with "Contract state not found in store". The production gateway's --skip-load-from-network flag may be preventing local contract storage.

Problem Description

Original Issue: In River chat rooms, Alice (room creator) could not see messages from Bob (invited user), while Bob could see all messages including Alice's.

Root Cause Analysis

Primary Issue (FIXED in v0.1.10)

• Location: /home/ian/code/freenet/river/cli/src/api.rs
• Problem: After creating a room, Alice only performed a PUT operation but never subscribed to receive updates
• Impact: Alice missed all contract updates, including new messages from other users

Fix Implementation

// Added in River v0.1.10
// Subscribe to the room we just created
info!("Subscribing to room contract: {}", contract_key.id());
let subscribe_request = ContractRequest::Subscribe {
    key: contract_key.clone(),
    summary: None,
};

let client_request = ClientRequest::ContractOp(subscribe_request);
web_api.send(client_request).await
    .map_err(|e| anyhow!("Failed to send subscribe request: {}", e))?;

Test Results

1. Automated Integration Test ✅

• Test: multi-machine-test with enhanced River test
• Configuration: Fresh local gateway on technic
• Result: SUCCESS - All messages visible to both users
• Key Success Factors:
  • Local gateway (no network hops)
  • Fresh state (no stale data)
  • River v0.1.10 with subscription fix

2. Manual Test with Local Gateway ✅

• Configuration:
  • Local gateway on port 31338
  • Two peers (ports 52619, 52620)
  • River v0.1.10
• Test Steps:
  1. Alice creates room: GcTcCDGbsFLJLwqzAxknxxGD5PL868i9DadsLmr4NVL8
  2. Alice creates invitation
  3. Bob accepts invitation
  4. Alice sends: "Hello from Alice in manual test!"
  5. Bob sends: "Hello from Bob in manual test!"
• Result: SUCCESS - Both users see all messages

3. Manual Test with Production Gateway ❌

• Configuration: Connected peers to nova.locut.us:31337 (automatic key download worked after removing --skip-load-from-network)
• Result: FAILED - GET operations timeout
• Finding:
  • PUT operations succeed and reach the gateway
  • GET operations fail with "Contract state not found in store"
  • The production gateway is not storing contracts, possibly due to --skip-load-from-network configuration

Key Discoveries

1. Gateway Architecture Impact

• Local Gateway: Message propagation works in our tests
• Remote Gateway: May introduce issues (speculation based on observed failures):
  • Network latency between machines
  • UDP connectivity problems (historical issues with vega)
  • Additional routing complexity
• Important: We cannot confirm these are the actual causes without testing

2. Test Infrastructure

• Automated tests use local gateways (success path)
• Manual tests attempted to use production gateway (failure path)
• Machine references updated from vega → nova due to UDP issues

3. River UI Analysis

• Web UI correctly uses subscribe: true in GET operations
• No subscription issues found in UI code
• UI follows same pattern as fixed CLI code

Configuration Details

Working Local Gateway Setup

# Gateway (port 31338)
freenet network \
  --skip-load-from-network \
  --is-gateway \
  --network-port 31338 \
  --ws-api-port 50519

# Peer 1 (Alice)
freenet network \
  --network-port 52619 \
  --ws-api-port 52619 \
  --skip-load-from-network

# Peer 2 (Bob)  
freenet network \
  --network-port 52620 \
  --ws-api-port 52620 \
  --skip-load-from-network

River Commands Used

# Create room (Alice)
riverctl room create --name "Test Room" --nickname "Alice"

# Create invitation (Alice)
riverctl invite create <room-id>

# Accept invitation (Bob)
riverctl invite accept --nickname "Bob" <invitation-code>

# Send messages
riverctl message send <room-id> "Hello!"

# List messages
riverctl message list <room-id>

Recommendations

Immediate Actions

1. Restart production gateway on nova.locut.us
2. Monitor gateway health with automated checks
3. Deploy River v0.1.10 to all users

Medium-term Improvements

1. Gateway redundancy: Multiple production gateways for failover
2. Health checks: Automated monitoring and alerting
3. Network diagnostics: Better error messages for connectivity issues

Long-term Architecture

1. Peer-to-peer fallback: Allow direct peer connections when gateway unavailable
2. Gateway mesh: Distributed gateway network for resilience
3. Local caching: Improve offline message availability

Conclusion

River v0.1.10 fixes the identified subscription bug where room creators didn't subscribe to their own rooms. This fix has been verified to work in local gateway scenarios. However, we cannot definitively state that all message propagation issues are resolved because:

1. The production gateway is currently unreachable
2. Previous manual tests with the production gateway failed
3. We have not been able to test the fix across network boundaries

While local testing suggests the remaining issues may be connectivity-related, this remains speculation until we can perform comprehensive testing with a functioning production gateway.

Version History

• River v0.1.9: Had subscription bug (room creators didn't subscribe)
• River v0.1.10: Fixed subscription issue, deployed to crates.io
• Freenet v0.1.21: Latest version with River support

Test Artifacts

• Automated test logs: /home/ian/code/freenet/freenet-testing-tools/multi-machine-test/test-results/
• Manual test configs: /tmp/manual-* directories
• Production test attempts: /tmp/prod-test-* directories