If you want to deploy and run the SQL scripts against all your servers. You need to create a deployment group and Register all your servers in it. Then add "deployment group job" in your release pipeline.

Select the deployment group you created in the configuration page. You can then add script tasks(eg. powershell task) to execute your SQL scripts (The SQL scripts should be included in the build artifacts).
When you run the release pipeline, and the tasks of deployment job will run the sql scripts on all your servers registered in the deployment group.
If you just want to run the sql scripts on a single database server in your build pipeline, you can simply create a self-hosted agent on the database server machine. And run your build pipeline on the self-hosted agent. The scripts in your pipeline will be able to access to your local database server, since they are on the same machine.
Hope above helps!