After further trial and error, I found the package google-map-react to be the problem. Removing the component that uses this package solved the formik/yup errors.
With regards to google-map-react, I removed this package and used the native google maps api package (@googlemaps/js-api-loader => see documentation for usage).
To get nextjs to transpile both packages from the node_modules, the next-transpile-modules package was added and used within next.config.js.
In the end all I had to do was edit my babel.config.json file, the next.config.js and package.json file.
Below are images of what they look like now.
babel.config.json
next.config.js
add this line to package.json
Keep in mind that @babel/core, @babel/preset-env, core-js need to be installed.
I hope this solution is helpful to anybody experiencing the same problems. I also changed the title since formik wasn't the problem.