User/group list can't be calculated, but record permission field can be. The logic you described can be implemented if you use Record Permission field and make it Automatic type. This will allow you to add calculation rules and populate it as required. Side affect of this approach is that calculated fields can't accept user input.
If you want to accept user input and automate behavior of the given field at the same time, then you have to write a Custom Object. Code for Custom Object will vary a lot based on the behavior you want, so it is probably a topic for another question.
Good luck!