These are different concepts that apply to different scopes...
Security Document
This applies globally to a database. It controls which user names have admin roles and which have access. Provided that the database is not public only users named in this document can access (read or write) the database. Note that authentication is handled elsewhere - this document deals only in authenticated usernames.
Even without any validation functions the security document is important as it controls access at the database level.
validate function
The validation functions of a design document allow the designer to restrict changes to a document. Depending on the result of ALL the validate functions in all the design documents in a database, a PUT/POST will succeed or fail.
The data available to a validation function is limited however - it cannot reference any other document except the one being updated and the security document.
Using the validation function the designer could limit values for fields, control which fields can be changed, and vary permissions based on whether the current user is an admin or not for example. However it is not possible to check a new value is in a lookup list on another document, or that a reference to another document ID is valid.
It is possible to use Validation functions in a public database that has no Security Document - so these two concepts can work together but neither requires the other.