-1

What does it mean when it is written that- Hypothesis space contains the target concept? If possible with an example.

Shardul Parab
  • 53
  • 8

1 Answers1

3

TLDR: It means you can learn with zero error.

Here is an example what it means: Suppose a concept: f(a,b,c,d) = a & b & (!c | !d) (input are in boolean domain).

This concept is in a ML task usualy represented by the data, so you are given a dataset:

a | b | c | d = f
--+---+---+---+---
T   T   T   T = F
T   T   T   F = T
T   T   F   T = T
   ... etc ...

And your hypothesis space is decision trees. In this case your hypothesis space contains target concept, as you can do (for example, there are more possibilities):

decision tree example

It can be proven, that any binary formula (concept) can be learned as a decision tree. Thus General binary formulas are subset of decision trees. That means, when you know the concept is a binary formula (that you even may not know), you will be able to learn it with a decision tree (given enough examples) with zero error.

On the other hand if you want to learn the example concept by monotone conjunctions, you can't do it, because binary formulas are not subset of monotone conjunctions.

(By subsets, I mean in terms of possible concepts. And from the subset relation, you can make statements about containing target concept in hypothesis space.)

Monotone conjunction is a set of conjunctions in which the variables are not negated. And you have more of those, when any of the conjunctions is true, the output is also true. Is is a subset of DNF where you cannot use negations.

Some concepts can be learned by montone conjunctions, but you cannot learn general binary formula concept by it. That means, you will not be able to learn with zero error, general binary formulas are not subset of monotone conjunctions.

Here is a nice PDF from Princeton on basics of ML: http://www.cs.princeton.edu/courses/archive/spr06/cos511/scribe_notes/0209.pdf

Martin Milichovsky
  • 730
  • 4
  • 9
  • I would say not "with zero error" but with "Bayesian error" or "with smallest error possible", since sometimes a target concept is non deterministic in nature (thus Bayes risk is non zero). – lejlot Sep 04 '16 at 19:46
  • A concept in terms of the ML lectures is a terminus technicus for a situation where you cannot have different output for a single input. Unique input combination can be in the concept, or not be in the concept. It is a crisp set. That is why I wrote zero error. – Martin Milichovsky Sep 04 '16 at 21:47