0

I have a doubt in machine learning..

What is the size of the version space before seeing any training examples. Consider I have 10 attributes and I am using conjunction of constraints as the representation. The attributes have two values each 'yes' or 'no'.

Ravi
  • 477
  • 1
  • 6
  • 13

1 Answers1

0

If you have K classes, than version space is simply the number of possible splitting of all possible attirbute values into K disjoint subsets.

In your case:

  • 10 attributes
  • 2 possible values per attribute
  • There are 2^10=1024 possible points
  • There are K^1024 possible hypotheses (you can think about them, as the labeling of each point with one of K values).

For example, for binary case K=2 we have 2^1024 hypotheses in the version space.

If you consider only subset of possible hypotheses (by putting some constraints on their representation) than this number is smaller, yet analysis of actual number would require diving into details of how "expressive" are your constrained hypotheses.

lejlot
  • 64,777
  • 8
  • 131
  • 164
  • what do you mean by K classes? and isn't the formula different for conjunction of constraints? @lejlot – Ravi Mar 31 '14 at 10:18
  • By K-classes I mean K classes. You can have arbitraty number of classes, labels for your points. In the simplest case K=2. Conjuction of **what type** of constraints? Constraint is a very generic term. – lejlot Mar 31 '14 at 12:03