Exploring whether/how mavsim-viewer (lightweight C/Raylib 3D vehicle visualizer) and PlotJuggler (2D time-series plotting tool) could work together.
This is an open-ended exploration — not a fixed proposal. The embedding idea (mavsim-viewer as a PlotJuggler plugin) was just one angle considered among many. The real question is: what's the most valuable thing that could emerge from combining these tools?
Prompted by PlotJuggler PR #1295 — a WebSocket DataStreamer plugin (merged Feb 2026). That PR turned out to be inbound-only (data INTO PlotJuggler), not useful for driving an external visualizer.
- Adds a WebSocket client plugin, not a DDS bridge
- Unidirectional: server → PlotJuggler for data; PlotJuggler → server only for control commands (subscribe, pause, heartbeat)
- Wire format: 16-byte header (magic
0x42524A50), zstd-compressed ROS 2 CDR payloads - No mechanism to stream data OUT of PlotJuggler
- Merged in 5 days with zero human review discussion
The StatePublisherZMQ plugin is the only existing way to get data out:
| Aspect | Detail |
|---|---|
| Transport | ZMQ REP socket on tcp://*:6665 |
| Pattern | REQ/REP polling (not streaming PUB/SUB) |
| Protocol | Text-based: [get_data_names] → names, [get_data][name1;name2] → values |
| Data | Flat string → double key-value pairs |
| Trigger | Fires on timeline scrub or playback |
We asked four different personas to think about this problem from scratch — not to evaluate a specific proposal, but to imagine what would be most valuable.
Verdict: The sync'd 3D+2D workflow has value, but only for specific scenarios.
- Shines for VTOL transitions and tailsitter debugging where Euler plots lie (gimbal lock). For standard multirotor work, PlotJuggler alone is sufficient. Maybe 1-in-10 debug sessions would benefit.
- The actually valuable data to visualize is setpoint vs. estimate — a ghost/wireframe vehicle at the commanded attitude overlaid on the actual. That's a mavsim-viewer feature, independent of PlotJuggler.
- Frame convention concern: PlotJuggler's ZMQ emits flat doubles with no frame metadata. Quaternion convention (Hamilton w,x,y,z) is especially dangerous — silent corruption if key naming changes.
- Game-changer idea: Build strip charts directly into mavsim-viewer. The codebase is 70% there — ULog parser, playback with seek, Raylib draws lines trivially. 2-3 user-selected time series synced to the playback cursor. One binary, one file, one clock. No IPC. "FlightPlot tried this in Java and failed. Your C/Raylib stack could actually pull it off."
Verdict: The standalone tool has more untapped potential than any integration.
- The only real persona for integration is the PX4 flight test engineer who lives in PlotJuggler post-flight and mentally reconstructs the 3D flight.
- Setup friction is the #1 killer for any multi-tool workflow. Build from source + configure connection = lose 90% of users before they see value.
- Competitive gap is real but narrow: Foxglove could do synchronized 2D+3D but doesn't support ULog well. That's the beachhead.
- Prioritized roadmap:
- Ship standalone replay with clickable timeline bar, drag-and-drop ULog, binary downloads
- Build multi-vehicle synchronized replay (
--replay v1.ulg v2.ulg v3.ulg v4.ulg) — the moat, nobody else can do it, architecture is 80% there - Only then consider external data source integration
- "Get 50 real users before optimizing for integration."
Verdict: Many ways to connect these tools. Embedding is one option, but the simplest approach ships fastest.
We explored the full spectrum of integration approaches — embedding was just one angle among several:
| Approach | Effort | Risk | UX | Notes |
|---|---|---|---|---|
| PJ StatePublisher spawns mavsim-viewer as child process, sends UDP | 1 week | Low | Good (separate window) | Uses existing binary unchanged |
| Standalone bridge process translating PJ ZMQ → mavsim-viewer UDP | 1 week | Low | Good | Fully decoupled, no PJ plugin needed |
| Shared memory + QWidget blit | 3 weeks | Medium | Great (docked) | Embedded feel without GL conflicts |
| Offscreen Raylib render in QWidget | 4 weeks | High | Great (docked) | macOS GLFW-in-Qt conflict is #1 risk |
| WebGL via Emscripten in QWebEngineView | 3 weeks | Medium | OK | Raylib has WASM support, but heavy |
The embedding path (rows 3-5) is technically feasible but carries significant platform risk — GLFW calls [NSApp run] on macOS, conflicting with Qt's Cocoa ownership. That alone eats a week of debugging.
Key insight: The simplest viable approach — a PJ plugin that spawns mavsim-viewer and sends HIL_STATE_QUATERNION over localhost UDP on timeline scrub — gives 95% of the UX with 10% of the integration risk. ~300 lines of C++.
Verdict: Define the protocol, not the integration.
He's seen this exact pattern three times (SGI flight sim 1993, DIS/HLA 2004, game studio 2011). The answer is always the same: don't couple tools together — agree on a data format and get out of each other's way.
Important context he was given: MAVLink is already a well-established standard for vehicle state. PX4's SITL and SIH (Simulation-In-Hardware) already publish full vehicle state over MAVLink — HIL_STATE_QUATERNION carries attitude, position, velocity at high rate. This is a battle-tested protocol used by thousands of developers and dozens of tools (QGroundControl, Gazebo, jMAVSim, AirSim, etc.). Any "flight state bus" concept should build on MAVLink, not reinvent it.
His refined take: Given that MAVLink already IS the protocol, the problem simplifies dramatically:
- mavsim-viewer already speaks MAVLink — it listens on UDP and renders
HIL_STATE_QUATERNION. This is the bus. It already exists. - The missing piece is a PlotJuggler → MAVLink bridge: when you scrub PJ's timeline, it packs the current attitude/position values into a
HIL_STATE_QUATERNIONmessage and sends it to mavsim-viewer's UDP port. That's ~200 lines of code. - Any other tool that speaks MAVLink can participate — SITL, SIH, a ROS bridge, a Python script, a hardware vehicle. The protocol is already universal.
- Replay becomes trivial: a
mavlink-replayutility reads a.ulg, converts to MAVLink messages, publishes at original rate. Any MAVLink-aware tool gets replay for free.
Why not reinvent the wire format:
"You already have the protocol. MAVLink has been battle-tested for 15+ years across thousands of vehicles. It has message definitions, versioning, CRC validation, multi-system addressing. The
vehicle_idissysid. Thevehicle_typeis inHEARTBEAT. Don't build a new thing when the existing thing already does the job."
His wild idea still stands: time-travel debugging. Fork the timeline at a crash point, inject different control inputs, run a forward simulation. Publish the hypothetical state on a different MAVLink sysid. See the crashed vehicle and the "what-if" ghost diverging in 3D while PlotJuggler overlays both attitude traces. mavsim-viewer already supports 16 simultaneous vehicles with color coding.
"The Unix philosophy got this right forty years ago: write programs that do one thing well, and write programs to work together. The 'work together' part isn't 'embed one inside the other.' It's 'agree on a data format and get out of each other's way.' You already agreed on one — it's called MAVLink."
All four reviewers, approaching from completely different angles, converge:
- The standalone tool has the most untapped value — clickable timeline, drag-and-drop, multi-vehicle replay
- Multi-vehicle synchronized replay is the unique moat — nobody else does this, the architecture is 80% there
- Any external integration should be additive, not required — mavsim-viewer must remain fully functional on its own
- MAVLink is already the protocol — no need to invent a new wire format; the bridge to PlotJuggler is just "pack current values into HIL_STATE_QUATERNION and send UDP"
- Keep the coupling minimal — whether it's a PJ plugin, a bridge process, or embedded rendering, the less these projects depend on each other, the better
- Now: Ship standalone ULog replay to "just works" quality (clickable timeline, drag-and-drop, binary downloads)
- Next: Multi-vehicle synchronized replay (
--replay v1.ulg v2.ulg v3.ulg v4.ulg) - Then: Build a PlotJuggler → MAVLink bridge (separate repo, ~200 lines, translates PJ timeline scrub to HIL_STATE_QUATERNION over UDP)
- Maybe: Embedded strip charts inside mavsim-viewer for self-contained 2D+3D analysis
- Explore: Setpoint-vs-estimate ghost overlay for GNC debugging