Q:

Logistic regression: one-vs-all method for multi classification

I'm new to Machine learning. When I learn about the Logistic regression, using one-vs-all (one-vs-rest) method for multiclassification:
In logistic regression, the hypothesis function try to estimate the probability of the positive class.
Assume we have 3 classes, then each class, we should predict the hypothesis function h(x)
h1(x)=P(y=1|x)
h2(x)=P(y=2|x)
h3(x)=P(y=3|x)

However,the sum of the three probabilities doesn't equal to 1?
I "feel" that it equal to 1, and I don't understand why it doesn't.
Can someone explain why?

A:

Your results are correct and the sum of h1(x), h2(x) and h3(x) shouldn't be equal to 1.
As you perform one-vs-all classification, then for each class (e.g., class 1) you have two probabilities p(y=1|x) and p(y!=1|x) which sum up to 1: 
p(y=1|x) + p(y!=1|x) = 1.

But, as you one-vs-all classifications are independent, then 
p(y!=1|x) != p(y=2|x) + p(y=3|x) (at least not necessarily).

Maybe, it is easier to understand with an example:

the first classifier says that p(y=1|x) = 0.7 and p(y!=1|x) = 0.3;
the second classifier says that p(y=2|x) = 0.7 and p(y!=2|x) = 0.3;
the third classifier says that p(y=3|x) = 0.7 and p(y!=3|x) = 0.3.

All of them are valid classifiers, but 
p(y=1|x) + p(y=2|x) + p(y=3|x) != 1.

