Skip to content

Instantly share code, notes, and snippets.

@flrs

flrs/tflite-gcp-setup-2025.md

Last active July 29, 2025 06:05
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save flrs/d62569ef3240fe407c6e4ff7c74dc424 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save flrs/d62569ef3240fe407c6e4ff7c74dc424 to your computer and use it in GitHub Desktop.
Download ZIP
Train TFLite models for Coral using GPU on GCP in 2025
Raw
tflite-gcp-setup-2025.md

🚀 Train TFLite Models for Google Coral on GPU (GCP Guide, 2025 Edition)

Florian Roscheck, July 2025

Want to train TensorFlow Lite models for Google Coral Edge TPU with GPU acceleration? As of 2025, Google Colab no longer supports tflite-model-maker with GPU due to outdated Python dependencies.

This guide walks you through setting up a GPU-powered virtual machine on Google Cloud Platform (GCP) that works seamlessly with TensorFlow 2.8, Python 3.9, and CUDA 11.3 — all preconfigured and ready for model training.

✅ Works with NVIDIA T4 GPU and TensorFlow 2.8
✅ Full dependency list included
✅ Save cost using spot/preemptible instances

📚 Table of Contents

🛠️ Why Not Colab Anymore?

  • Colab Issues: tflite-model-maker depends on Python 3.9, which is no longer supported in Colab.
  • CPU-only Workarounds: Some guides (e.g. this workaround) leverage Conda on Colab but only support CPU training — slow and inefficient.
  • Compatibility Hell: Finding the right combination of Python, CUDA, and TensorFlow versions was frustrating for me and might have been for you – until now.

✅ Working GCP Setup for Coral-Compatible Model Training

Follow these steps to spin up a VM with GPU acceleration that's compatible with tflite-model-maker.

1. Create GCP Project and Enable Compute Engine

2. Install Google Cloud SDK

Install and initialize the SDK:
👉 GCloud SDK Installation Guide

3. Launch a GPU-Backed VM (Optimized for TensorFlow 2.8)

Use the following specifications:

  • Machine Type: n1-highmem-2 (2 vCPUs, 13 GB RAM)
  • GPU: NVIDIA Tesla T4
  • Image: c1-deeplearning-tf-2-6-cu113-v20240417-debian-11-py39
  • Disk: 50 GB minimum
  • Pricing Tip: Use a preemptible VM to reduce cost

🧭 Problem: T4 GPUs are often unavailable in popular zones.

💡 Solution: Use the script below to automatically find an available zone with a T4 GPU and launch your VM.

📍 Find an Available Zone with T4 GPU Automatically and Create VM

Save this as create_gcp_instance.sh and execute it to automatically create a compatible VM:

#!/usr/bin/env bash
set -e

PROJECT="<YOUR PROJECT NAME HERE>"
IMAGE="projects/ml-images/global/images/c1-deeplearning-tf-2-6-cu113-v20240417-debian-11-py39"
ZONE_CANDIDATES=(us-central1-a us-central1-b us-central1-c us-central1-f us-east1-c us-east1-d us-east4-a us-east4-b us-east4-c us-west1-a us-west1-b us-west2-c us-west3-b us-west4-a us-west4-b northamerica-northeast1-c southamerica-east1-a southamerica-east1-c europe-west1-b europe-west1-c europe-west1-d europe-west3-b europe-west4-a europe-west4-b europe-west4-c me-west1-b me-west1-c asia-east1-a asia-east1-c asia-east2-c asia-northeast1-a asia-northeast1-c asia-northeast3-c asia-south1-a asia-south1-b asia-southeast1-a asia-southeast1-b asia-southeast1-c)
MACHINE_TYPE="n1-highmem-2"
GPU_TYPE="nvidia-tesla-t4"
GPU_COUNT=1
INSTANCE_NAME="t4-instance-$(date +%s)"

echo "Attempting to create one T4 GPU VM with cheapest machine..."

for ZONE in "${ZONE_CANDIDATES[@]}"; do
  echo "Trying zone $ZONE..."
  if gcloud compute zones describe "$ZONE" &>/dev/null; then
    if gcloud compute instances create "$INSTANCE_NAME"       --project="$PROJECT"       --zone="$ZONE"       --machine-type="$MACHINE_TYPE"       --accelerator="type=${GPU_TYPE},count=${GPU_COUNT}"       --image="$IMAGE"       --maintenance-policy=TERMINATE       --preemptible       --scopes cloud-platform       --metadata install-nvidia-driver=True       --quiet; then
      echo "✅ Successfully created $INSTANCE_NAME in zone $ZONE."
      exit 0
    else
      echo "❌ Failed in zone $ZONE (likely GPU unavailable). Trying next..."
    fi
  else
    echo "⚠️ Invalid zone $ZONE. Skipping."
  fi
done

echo "🚫 Unable to create instance with T4 GPU in any zone."
exit 1

🧠 Setting Up Your Environment (Python 3.9, TensorFlow 2.8, TFLite)

After SSH-ing into your new instance:

# Deactivate base environment
conda deactivate

# Install venv support and libusb
sudo apt install python3.9-venv libusb-1.0-0

# Create and activate virtualenv
python3.9 -m venv venv
source venv/bin/activate

📦 Install Dependencies

Save the full requirements list (see below) to requirements.txt:

Click to expand full requirements.txt 
absl-py==1.4.0
array_record==0.5.1
astunparse==1.6.3
attrs==25.3.0
audioread==3.0.1
bleach==6.2.0
cachetools==5.5.2
certifi==2025.7.14
cffi==1.17.1
charset-normalizer==3.4.2
click==8.1.8
cycler==0.12.1
Cython==3.1.2
dataclasses==0.6
decorator==5.2.1
dm-tree==0.1.8
etils==1.5.2
fire==0.7.0
flatbuffers==25.2.10
fsspec==2025.7.0
gast==0.6.0
gin-config==0.5.0
google-api-core==2.25.1
google-api-python-client==2.177.0
google-auth==2.40.3
google-auth-httplib2==0.2.0
google-auth-oauthlib==0.4.6
google-cloud-bigquery==3.30.0
google-cloud-core==2.4.3
google-crc32c==1.7.1
google-pasta==0.2.0
google-resumable-media==2.7.2
googleapis-common-protos==1.70.0
grpcio==1.73.1
grpcio-status==1.48.2
h5py==3.14.0
httplib2==0.22.0
idna==3.10
importlib_metadata==8.7.0
importlib_resources==6.5.2
joblib==1.5.1
kaggle==1.7.4.5
keras==2.8.0
Keras-Preprocessing==1.1.2
kiwisolver==1.4.7
libclang==18.1.1
librosa==0.8.1
llvmlite==0.36.0
lxml==6.0.0
Markdown==3.8.2
MarkupSafe==3.0.2
matplotlib==3.4.3
neural-structured-learning==1.4.0
numba==0.53.0
numpy==1.23.4
oauthlib==3.3.1
opencv-python-headless==4.11.0.86
opt_einsum==3.4.0
packaging==20.9
pandas==2.3.1
pillow==11.3.0
platformdirs==4.3.8
pooch==1.8.2
promise==2.3
proto-plus==1.26.1
protobuf==3.20.3
psutil==7.0.0
py-cpuinfo==9.0.0
pyasn1==0.6.1
pyasn1_modules==0.4.2
pybind11==3.0.0
pycocotools==2.0.7
pycparser==2.22
pyparsing==3.2.3
python-dateutil==2.9.0.post0
python-slugify==8.0.4
pytz==2025.2
PyYAML==6.0.2
requests==2.32.4
requests-oauthlib==2.0.0
resampy==0.4.3
rsa==4.9.1
scann==1.2.6
scikit-learn==1.6.1
scipy==1.13.1
sentencepiece==0.2.0
six==1.17.0
sounddevice==0.5.2
soundfile==0.13.1
tensorboard==2.8.0
tensorboard-data-server==0.6.1
tensorboard-plugin-wit==1.8.1
tensorflow==2.8.0
tensorflow-addons==0.23.0
tensorflow-datasets==4.9.3
tensorflow-hub==0.12.0
tensorflow-io-gcs-filesystem==0.37.1
tensorflow-metadata==1.16.1
tensorflow-model-optimization==0.8.0
tensorflowjs==3.18.0
termcolor==3.1.0
text-unidecode==1.3
tf-estimator-nightly==2.8.0.dev2021122109
tf-models-official==2.3.0
tf-slim==1.1.0
tflite-model-maker==0.4.2
tflite-support==0.4.4
threadpoolctl==3.6.0
toml==0.10.2
tqdm==4.67.1
typeguard==2.13.3
typing_extensions==4.14.1
tzdata==2025.2
uritemplate==4.2.0
urllib3==1.25.11
webencodings==0.5.1
Werkzeug==3.1.3
wrapt==1.17.2
zipp==3.23.0

Then install:

pip install -r requirements.txt

🏃‍♀️ Training Your Coral-Compatible Model

Assuming your train_model.py with the desired tflite_model_maker imports is ready:

source venv/bin/activate
python train_model.py

🧩 Extra Tips

✅ Run Training in Background with tmux

tmux
source venv/bin/activate
python train_model.py
# Detach with Ctrl+B then D
# Reattach: tmux attach

✅ Monitor GPU Usage

watch -n 1 nvidia-smi

✅ Monitor CPU/RAM

htop
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment