This is a full account of the steps I ran to get llama.cpp running on the Nvidia Jetson Nano 2GB. It accumulates multiple different fixes and tutorials, whose contributions are referenced at the bottom of this README.
At a high level, the procedure to install llama.cpp on a Jetson Nano consists of 3 steps.
gcc 8.5 compiler from source.
| #python3 | |
| def hook(t): | |
| def inner(bytes_amount): | |
| t.update(bytes_amount) | |
| return inner | |
| BUCKET_NAME = 'your_s3_bucket_name' | |
| FILE_NAME = 'your_s3_file_name' | |
| s3 = boto3.resource('s3') |
| #!/bin/bash | |
| mkdir -p ~/.ssh | |
| # generate new personal ed25519 ssh keys | |
| ssh-keygen -o -a 100 -t ed25519 -f ~/.ssh/id_ed25519 -C "rob thijssen <[email protected]>" | |
| ssh-keygen -o -a 100 -t ed25519 -f ~/.ssh/id_robtn -C "rob thijssen <[email protected]>" | |
| # generate new host cert authority (host_ca) ed25519 ssh key | |
| # used for signing host keys and creating host certs |