Skip to content

Instantly share code, notes, and snippets.

@Dwite
Last active February 18, 2026 12:05
Show Gist options
  • Select an option

  • Save Dwite/19d5bc9dfba81b1ea332c0f39fc38cb2 to your computer and use it in GitHub Desktop.

Select an option

Save Dwite/19d5bc9dfba81b1ea332c0f39fc38cb2 to your computer and use it in GitHub Desktop.
Mobile Release Management: Evaluation Proposal — Tramline vs Runway

Mobile Release Management: Evaluation Proposal

TL;DR

We release iOS + Android weekly (104 releases/year). Our release process has several pain points that slow us down and create risk:

  • No centralized release visibility — release status, rollout percentage, and health are scattered across Play Console, App Store Connect, and Crashlytics. There's no single place to see "where are we?"
  • No automated crash response — if a bad build ships Friday, it can reach 100% of users before anyone checks Monday. We have no auto-halt.
  • Android rollouts are manual — iOS has auto staged rollout, but Android requires someone to manually advance percentages in Play Console. A centralized tool handles both platforms in one place.
  • iOS release coordination overhead — iOS team manually spins off release branches and manages Asana release projects every cycle. Repetitive weekly work that a tool can automate.
  • Release pilot coordination is ad-hoc — who manages this week's release? It's Slack threads and tribal knowledge, not a managed rotation.
  • Release health is opaque — checking crash rates requires opening Crashlytics separately and manually correlating with the current release version.

We evaluated Runway ($24K/year) and Tramline ($1,200/year). Both solve these problems. We recommend a free 7-week evaluation of Tramline, then $100/month if it proves valuable. Tramline is open source (Apache 2.0), so if it works for us, we can also fork and customize it for our needs or self-host with minimal effort. Runway stays as a premium fallback if we outgrow Tramline.

Runway Tramline Savings
Annual cost ~$24,000 $1,200 $22,800 (95%)
Evaluation cost N/A $0 (free tier)

The Problem

104 releases per year (weekly, both platforms). Today's process:

Pain Point Current State Impact
Release visibility Check Play Console, App Store Connect, and Crashlytics separately No single source of truth; status lives in Slack threads
Android rollout Someone manually clicks percentages in Play Console iOS has auto staged rollout; Android doesn't — creates asymmetry and manual work
iOS release branch + Asana coordination iOS team manually spins off release branches and manages Asana release projects each cycle Repetitive overhead every week; easy to miss steps
Crash monitoring Open Crashlytics, find the right release version, check crash rates Disconnected from rollout decisions; reactive, not proactive
Auto-halt Doesn't exist A bad build can silently roll out to 100% of users over a weekend
Release pilot "Who's managing this one?" in Slack No managed rotation; knowledge silos, uneven workload
Slack updates Manual posts about release status Easy to forget; team lacks real-time awareness

Version bumps and store submissions are already automated. The remaining pain is coordination: release branch management, Asana project setup, rollout monitoring, and cross-platform visibility. A centralized tool handles all of this in one place — with a dashboard, managed rotation, and automated safety net.


What We Need

Requirement Priority Why Tramline Runway
Current release status + stage Must have Single source of truth Yes Yes
Live rollout percentage Must have See both platforms at a glance Yes Yes
Auto rollout progression (Android) Must have Match iOS auto-stage; stop manual Play Console clicks Yes (auto-advance every 24h if healthy) Yes
Auto halt on crash spike Must have Safety net — stop bad builds before 100% rollout Yes (polls every 5 min) Yes
Release health + crash metrics Must have Transparent, easy view of release quality Yes (Crashlytics via BigQuery) Yes
Release pilot rotation Must have Clear ownership; managed handoff between releases Yes (Release Pilots) Yes
Slack notifications Must have Real-time team awareness without checking dashboards Yes Yes
What's included in release (commits/PRs) Must have Quick answer to "what shipped?" Yes Yes
Was Asana ticket in this release? Nice to have Ticket-level tracking, not just commits No (commits/PRs only) Yes (native Asana)
True rollback (re-sign previous build) Nice to have Instant rollback vs hotfix-forward No Yes

Both tools cover all must-haves. Runway's advantages (Asana integration, true rollback) are nice-to-haves that don't justify a 20x price premium.

Asana workaround for Tramline: If PR titles reference Asana task IDs (e.g., feat/PROJ-123-add-cart), the release changelog is searchable by task. Alternatively, Tramline's webhook API can power a custom Asana bridge (~half day of work).


Options Evaluated

Runway — $24,000/year

The market leader. YC W21, $33.5M raised (a16z), 18 employees. Used by DoorDash, Mercari, Chick-fil-A.

  • Broadest integrations (13+ crash tools, 13+ CI systems, native Asana/Jira/Linear)
  • True rollback, feature flag monitoring, SOC 2 certified
  • But: ~$1K/app/month, closed source (full vendor lock-in), API restricted to Enterprise tier
  • For our stack: most premium integrations don't apply — we use GitHub Actions (both support it), Firebase Crashlytics (same BigQuery setup on both), and neither supports Bitdrift

Tramline — $1,200/year

Open-source alternative. Founded 2022, 3 employees, $125K raised. Used by Groww (14M+ users), Speak (10M+ downloads).

  • 95% cheaper, covers all core release management features
  • Apache 2.0 open source — no vendor lock-in, fork rights, auditable
  • If it works well, we can fork and customize for our specific needs (e.g., add Bitdrift integration, Asana bridge) or self-host on our own infrastructure with minimal effort (~$30/month)
  • API access on all plans including free
  • But: 3-person team (bus factor), no Asana integration, no true rollback, smaller integration surface

Self-Hosted Tramline — ~$360/year

Same codebase on our infrastructure (Apache 2.0).

  • Cheapest option, full control, can customize freely
  • But: poor self-hosting docs, no versioned releases, we own maintenance
  • Not recommended as starting point — better to evaluate cloud first, self-host later if needed

DIY — $0

Custom GitHub Actions + Slack bot for staged rollouts and auto-halt.

  • Covers ~80% of value (rollout automation, crash monitoring, Slack alerts)
  • But: no dashboard, no release pilot management, no cross-platform view, no release history
  • Risk: if monitoring script breaks silently, nobody catches a bad rollout

Why Runway's Premium Doesn't Apply to Us

Runway Advantage Why It Doesn't Matter for Us
13+ crash monitoring integrations We use Firebase Crashlytics — same BigQuery setup on both tools
DataDog, Sentry, New Relic support We use Firebase + Bitdrift — neither tool supports Bitdrift
13+ CI/CD integrations We use GitHub Actions — both support it
Amazon/Samsung/Huawei stores We only ship to App Store + Play Store
Asana integration Real gap, but solvable via PR naming or webhook (~0.5 day)
True rollback Nice, but hotfix-forward works — not worth $22.8K/year

How These Tools Work (Security Note)

Neither tool accesses our source code or modifies our CI. They orchestrate via APIs using credentials we provide:

Connection Method What It Does
GitHub GitHub App (OAuth) Read branches, create tags, trigger existing workflows
Google Play Service account key Upload builds, manage rollout %, halt/resume
App Store Connect .p8 API key Submit builds, manage phased release
Slack Slack App webhook Notifications and alerts
Crashlytics BigQuery service account Read crash-free rates (no direct Firebase API exists)

Tramline is open source — we can audit exactly what it does with our credentials. Runway is closed source but SOC 2 Type 2 certified.


Risk Assessment

Risk Impact Mitigation
Tramline folds (3-person team, $125K funding) Medium Apache 2.0 — we fork and self-host or migrate to Runway
Missing feature we later need Low Open API + webhooks on all plans; can fork and add features; Runway as upgrade path
Development gaps (Oct 2025 had zero commits) Low SaaS remained operational during the code pause
Firebase BigQuery setup complexity Low ~1 hour setup, same requirement on any tool including Runway
Bitdrift not supported Low Custom webhook integration ~1 day, required regardless of tool

Recommended Path

Phase 1: Free Evaluation — $0, 7 Weeks

Sign up for Tramline Hobby (free: 2 apps, 15 releases/app).

Week Goal
1-2 Connect GitHub, CI, Play Store, App Store Connect, Slack. Run first release through Tramline
3-4 Set up Crashlytics via BigQuery. Test auto staged rollout for Android. Verify auto-halt works
5-7 Full team adoption. Set up release pilot rotation. Measure time saved per release. Collect feedback

15 releases per app covers 7 weekly cycles — enough to validate all must-haves.

Decision gate: Does it save measurable time? Does the team use it? Is the release health view useful?

Phase 2: Paid Adoption — $100/month

If evaluation succeeds, upgrade to Team plan ($50/app x 2). Unlimited releases, Slack support, 72-hour SLA. Annual cost: $1,200.

Phase 3: Grow Into It (3-6 Months)

If... Then...
Tramline works well as-is Stay on $100/month. Done
We want customizations (Bitdrift, Asana) Fork the Apache 2.0 codebase and add what we need
We want lower cost or full control Self-host the fork (~$30/month infra)
Tramline doesn't meet needs at all Evaluate Runway with real data to justify $24K/year

The open-source angle is key: Tramline isn't just a SaaS we're subscribing to. It's a codebase we can evaluate, fork, customize, and self-host. We start with cloud to validate with zero effort, and we have a path to full ownership if we see the value.

Why This Path

  • $0 to start — zero financial risk during evaluation
  • No vendor lock-in — Apache 2.0 means we own the exit at every stage
  • 2 hours to first release — no infrastructure to set up
  • Customizable — if we outgrow the SaaS, we fork and make it ours
  • Data-driven escalation — if we need Runway, we'll have real usage data to justify $24K
  • 95% cost savings — $1,200/year vs $24,000/year for the same core capabilities

To Start, We Need

  • Tramline Hobby account (free sign-up at tramline.app)
  • GitHub App installation on our repo
  • Google Play Console service account (likely exists for CI already)
  • App Store Connect API key — App Manager role (likely exists already)
  • Slack App for release notifications
  • Firebase BigQuery export enabled (for crash rate monitoring)

No new infrastructure. No procurement. Most credentials likely already exist in our CI setup.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment