I wanted to be able to SSH into my Windows laptop directly into Linux. I also wanted to disable password authentication and only allow public key (RSA in my case) authentication.

Scott Hanselman wrote a blog post on how to make your default WSL2 distro your default shell for SSH. Windows OS Hub published an article on using public key authentication. These were both helpful resources.

I'll assume you're already familiar with using SSH keys. If not, this article at DigitalOcean is very informative.

Add your public key to your authorized keys file

First thing you want to do is create the file $HOME\.ssh\authorized_keys. If you run into issues, it could be due to incorrect file ownership.