Use UTM to Troubleshoot Pyenv on MacOs

If you have upgraded to MacOs Sequoia like I did and found out that pyenv and other Python stuff doesn’t not work ( including compiling ) this is for you. This has been updated to cover general pyenv troubleshooting since its no longer relevant fo Sequoia.

I spent way too much time trying to get a MacOs Beta to install Python properly with pyenv and compiling and installing a binary ( when available ). So I took the logical (?) approach and installed UTM, installed an M1 vm and compiled from scratch to get the binaries and then installed those on my Beta MacOs and its working.

Here are my steps. Feel free to tell me I am wrong about something and I could have done better.

Install UTM:

https://mac.getutm.app/

Install the latest MacOs which should be Sonoma.

Once installed, you can add brew:

xcode-select --install

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)

(echo; echo 'eval "$(/opt/homebrew/bin/brew shellenv)"') >> /Users/leon/.zprofile

eval "$(/opt/homebrew/bin/brew shellenv)"

For Python 3.10.14:

Make a dev directory and get the python source for whichever version you want to install:

mkdir dev cd dev curl -O https://www.python.org/ftp/python/3.10.14/Python-3.10.14.tgz tar -xvf Python-3.10.14.tgz cd Python-3.10.14

Compile time:

./configure --prefix=/usr/local/python3.10 \ --enable-optimizations \ --with-openssl=$(brew --prefix openssl) \ CPPFLAGS="-I$(brew --prefix openssl)/include -I$(brew --prefix readline)/include -I$(brew --prefix sqlite)/include -I$(brew --prefix zlib)/include" \ LDFLAGS="-L$(brew --prefix openssl)/lib -L$(brew --prefix readline)/lib -L$(brew --prefix sqlite)/lib -L$(brew --prefix zlib)/lib"

make -j4

sudo make altinstall

/usr/local/python3.10/bin/python3.10 --version

mkdir -p python_pkg/usr/local cp -r /usr/local/python3.10 python_pkg/usr/local/

pkgbuild --root python_pkg --identifier com.example.python3.10 --version 3.10.14 --install-location python-3.10.14.pkg

pkgutil --check-signature python-3.10.14.pkg

( Probably won’t be valid but it is yours )

scp python-3.10.14.pkg you@your_M1_Mac_Ip:~/Downloads

For Python 3.11.9:

cd ~/dev curl -O https://www.python.org/ftp/python/3.11.9/Python-3.11.9.tgz tar -xvf Python-3.11.9.tgz cd Python-3.11.9

Compile time:

./configure --prefix=/usr/local/python3.11 \ --enable-optimizations \ --with-openssl=$(brew --prefix openssl) \ CPPFLAGS="-I$(brew --prefix openssl)/include -I$(brew --prefix readline)/include -I$(brew --prefix sqlite)/include -I$(brew --prefix zlib)/include" \ LDFLAGS="-L$(brew --prefix openssl)/lib -L$(brew --prefix readline)/lib -L$(brew --prefix sqlite)/lib -L$(brew --prefix zlib)/lib"

make -j4

sudo make altinstall

/usr/local/python3.11/bin/python3.11 --version

mkdir -p python_pkg/usr/local cp -r /usr/local/python3.11 python_pkg/usr/local/

pkgbuild --root python_pkg --identifier com.example.python3.11 --version 3.11.9 --install-location python-3.11.9.pkg

pkgutil --check-signature python-3.11.9.pkg

( Probably won’t be valid but it is yours )

scp python-3.11.9.pkg you@your_M1_Mac_Ip:~/Downloads

Now on your Mac M1:

Setup pyenv:

brew install pyenv

Or

curl https://pyenv.run | bash

Add pyenv to your shell if you installed it manually:

Add the following to your ~/.zshrc:

export PATH="$HOME/.pyenv/bin:$PATH" eval "$(pyenv init --path)" eval "$(pyenv init -)" source ~/.zshrc

If you used brew, you are probably good to go.

Install Python by double clicking on the brand new binaries you created:

Here are the commands to setup pyenv for the two versions I installed.

Create a directory for the custom Python versions

mkdir -p $(pyenv root)/versions/3.10.14 mkdir -p $(pyenv root)/versions/3.11.9

Symlink the custom installations

ln -s /usr/local/python3.10 $(pyenv root)/versions/3.10.14 ln -s /usr/local/python3.11 $(pyenv root)/versions/3.11.9

Rehash pyenv

pyenv rehash

Set the global or local Python version

pyenv global 3.10.14 OR 3.11.9

python --version

You would think this is it but you still need to know that there is a default install of Python so if you have created a new venv and get errors because your script does not use the venv version OR the pyenv version, you need to find the default version and add your packages there. Cloud cli is one of the apps I use which required me to install a bunch of packages there.

Its probably here:
/Library/Frameworks/Python.framework/Versions/3.11

cd /Library/Frameworks/Python.framework/Versions/3.11

Then do your bin/pip3 install --force-reinstall --no-binary :all: your list of packages

Eg. For gcloud cli:

bin/pip3 install --force-reinstall --no-binary :all: certifi cffi crcmod cryptography google-crc32c grpcio pycparser pyOpenSSL setuptools

The python community fixes these all the time but I hope this helps you now.

Please let me know if you find any corrections.

Leon