First of all, your question lacks of a very important information: the objective.
I assume the objective is to assign students to courses in such a way that the total utility the students gain by taking courses is maximized. In this case, all utilities that can be gained by any student by taking any course are known. From you question, i also understand that the joint utilities are known, too. Therefore, we can define a new "parameter" named loss such that loss(k,l) means the amount of "lost utility" when courses k and l are taken at the same time by a student. These values can easily be calculated using the given data. Then, we could solve the problem in the following way:
Let w(i,k) be the parameter determining the amount of utility the student i gains by taking the course k.
Let x(i,k) be the integer binary variable determining whether the student i takes course k or not.
Let u(i,k) be the variable determining the actual amount of utility that the student gains from taking the course.
Objective function:
- maximize sumof(u(i,k)) for all students and courses.
Conditions:
x(i,k) * w(i,k) >= u(i,k) (for all students and courses)
This guarantees that the maximum utility of student i from the course k cannot exceed the assumed (given, maximum possible) utility. This also guarantees that u(i,k) is set to zero when x(i,k) is zero.
x(i,k) w(i,k) + x(i,l) w(i,l)) >= u(i,k) + u(i,l) - loss(k,l) for all "course pairs" and all students
This guarantees that sum of utilities that a student gains from taking two courses at the same time cannot exceed the joint utility.