It was a simple fix that involved looking at the permissions of the tmp directory on build server in the BuildAgent directory.
Listing the permissions of the files and directories in the BuildAgent directory on the build server
switchbuild@nevcv-build5:~/BuildAgent$ ls -l
total 52
drwxr-xr-x 2 switchbuild switchbuild 4096 Jul 21 2020 bin
-rw-rw-r-- 1 switchbuild switchbuild 0 Jul 21 2020 BUILD_78726
drwxr-xr-x 3 switchbuild switchbuild 4096 Apr 24 09:57 conf
drwxr-xr-x 3 switchbuild switchbuild 4096 Dec 9 2019 contrib
drwxr-xr-x 5 switchbuild switchbuild 4096 Jul 21 2020 launcher
drwxr-xr-x 3 switchbuild switchbuild 4096 Jul 21 2020 lib
drwxrwxr-x 2 switchbuild switchbuild 4096 Apr 24 10:01 logs
drwxr-xr-x 51 switchbuild switchbuild 4096 Jul 21 2020 plugins
-rw-rw-r-- 1 switchbuild switchbuild 105 Mar 31 09:46 README.txt
-rw-r--r-- 1 switchbuild switchbuild 184 Dec 9 2019 service.properties
drwxrwxr-x 14 switchbuild switchbuild 4096 Apr 24 09:57 system
drwxr-xr-x 4 root root 4096 Apr 24 10:12 temp <--- not sure why/how this folder got under root permissions...
drwxrwxr-x 5 switchbuild switchbuild 4096 Jul 21 2020 tools
drwxrwxr-x 3 switchbuild switchbuild 4096 Apr 15 2022 work
Change the folder owner to the build server user using chown
switchbuild@nevcv-build5:~/BuildAgent$ sudo chown switchbuild ./temp
switchbuild@nevcv-build5:~/BuildAgent$ ls -l
total 52
drwxr-xr-x 2 switchbuild switchbuild 4096 Jul 21 2020 bin
-rw-rw-r-- 1 switchbuild switchbuild 0 Jul 21 2020 BUILD_78726
drwxr-xr-x 3 switchbuild switchbuild 4096 Apr 24 09:57 conf
drwxr-xr-x 3 switchbuild switchbuild 4096 Dec 9 2019 contrib
drwxr-xr-x 5 switchbuild switchbuild 4096 Jul 21 2020 launcher
drwxr-xr-x 3 switchbuild switchbuild 4096 Jul 21 2020 lib
drwxrwxr-x 2 switchbuild switchbuild 4096 Apr 24 10:01 logs
drwxr-xr-x 51 switchbuild switchbuild 4096 Jul 21 2020 plugins
-rw-rw-r-- 1 switchbuild switchbuild 105 Mar 31 09:46 README.txt
-rw-r--r-- 1 switchbuild switchbuild 184 Dec 9 2019 service.properties
drwxrwxr-x 14 switchbuild switchbuild 4096 Apr 24 09:57 system
drwxr-xr-x 4 switchbuild root 4096 Apr 24 10:12 temp <--- has the correct owner but not correct group
drwxrwxr-x 5 switchbuild switchbuild 4096 Jul 21 2020 tools
drwxrwxr-x 3 switchbuild switchbuild 4096 Apr 15 2022 work
Change the folder group to the build server user using chgrp
switchbuild@nevcv-build5:~/BuildAgent$ chgrp switchbuild temp
switchbuild@nevcv-build5:~/BuildAgent$ ls -l
total 52
drwxr-xr-x 2 switchbuild switchbuild 4096 Jul 21 2020 bin
-rw-rw-r-- 1 switchbuild switchbuild 0 Jul 21 2020 BUILD_78726
drwxr-xr-x 3 switchbuild switchbuild 4096 Apr 24 09:57 conf
drwxr-xr-x 3 switchbuild switchbuild 4096 Dec 9 2019 contrib
drwxr-xr-x 5 switchbuild switchbuild 4096 Jul 21 2020 launcher
drwxr-xr-x 3 switchbuild switchbuild 4096 Jul 21 2020 lib
drwxrwxr-x 2 switchbuild switchbuild 4096 Apr 24 10:01 logs
drwxr-xr-x 51 switchbuild switchbuild 4096 Jul 21 2020 plugins
-rw-rw-r-- 1 switchbuild switchbuild 105 Mar 31 09:46 README.txt
-rw-r--r-- 1 switchbuild switchbuild 184 Dec 9 2019 service.properties
drwxrwxr-x 14 switchbuild switchbuild 4096 Apr 24 09:57 system
drwxr-xr-x 4 switchbuild switchbuild 4096 Apr 24 10:12 temp <--- has the correct owner and group
drwxrwxr-x 5 switchbuild switchbuild 4096 Jul 21 2020 tools
drwxrwxr-x 3 switchbuild switchbuild 4096 Apr 15 2022 work
After changing everything to the expected owner and group the BuildAgent was able to create the needed temporary directories to build.