Suppose there are 100 coins, out of which only 1 coin is unfair. This unfair coin has the probability of less than 0.5 to get a Head. How do I design an experiment to find the unfair coin?
I think one way is for flip each coin for a large number of times (for example, 10000 times). Then the coin that has the smallest number of heads will be picked to be the unfair coin. But is there other smarter way to do this?
That would probably work, but it would be nice if you had some measure of how confident you were that you had selected the right coin, because if the coin is only very slightly unfair you might need a very large number of trials to be able to confidently say which coin was biased, and it doesn't sound like the degree of bias was given to you, so there's not a particular reason to suspect 10000 trials is enough: many more (or many fewer) trials might be necessary. For example, if the coin bias is only P(h) = 0.49999 (with P(h) = 0.5 for all other coins) and the coin which produces the fewest heads after 10000 trials only produced 1 fewer head than the second lowest, then you would probably want to factor this information in and keep going until you had mathematical reason to be confident you had found the right coin.
To achieve this you could take a Bayesian approach and do trials in batches of 100 or 1000, updating after each batch a posterior probability that each coin is fair according to Bayes' rule (giving all coins a prior probability of fairness of 99/100). You could continue this until one coin has a posterior probability of fairness of < 0.05, perhaps taking the additional step of verifying that every other coin has a posterior probability of >= .95. This way your experiment is agnostic about the number of trials necessary to find the biased coin and just proceeds until one is clearly distinguishable from the rest.
