npx create-react-app@next --scripts-version=@next --template=cra-template@next my-js-app
npx create-react-app@next --scripts-version=@next --template=typescript@next my-ts-app
To update an existing app you can change the react-scripts version in package.json to next and then run yarn or npm install
You can see a list of all the changes in the 4.0 release here

Alrighty, I've got an extremely minimal reproduction. I hope it helps. I'm a bit baffled TBH...
https://github.com/kentcdodds/react-scripts-test-upgrade-bug