Dfyjkt

The probability density function of a uniform distribution (continuous) is shown above. The area under the curve is 1 - which makes sense since the sum of all the probabilities in a probability distribution is 1.

Formally, the above probability function (f(x)) can be defined as

1/(b-a) for x in [a,b]

and 0 otherwise

Consider that I have to choose a real number between a (say, 2) and b (say, 6). This makes the uniform probability = 0.25. However, since there are an infinite number of numbers in that interval, shouldn't the sum of all the probabilities sum up to infinity? What am I overlooking?

Is f(x) not the probability of the number x occurring?

$f(x)$ describes the probability density rather than a probability mass in your example. In general, for continuous distributions the events—the things we get probabilities for—are ranges of values, such as for the area under the curve from $a$ to $a+.1$, or from $a$ to $b$ (although such ranges need not be contiguous). For continuous distributions, the probability of any single value occurring is generally 0.

Because each term in the summation is weighted by the infinitesimal d$x$. The importance of this is probably most easily understood by carefully walking through a very basic example.

Consider using Riemann summation to compute the area under the following rectangular region (a rectangle was chosen to remove the approximation aspect of Riemann summation, which is not the focus here):
]
We can compute the area using 2 subregions, or by using 4 subregions. In the case of the 2 subregions (denoted $A_i$), the areas are given by $$A_1=A_2=5times 2 = 10$$ whereas in the case of 4 subregions (denoted $B_i$), the areas are given by $$B_1=B_2=B_3=B_4=5times 1 = 5$$ The total area in both cases correspond to $$sum_i=1^2 A_i = sum_i=1^4B_i = 20$$
Now, this is all fairly obvious, but it raises a subtly important question which is: why do these two answers agree? Intuitively it should be clear that it works because we've reduced the width of the second set of subregions. We could consider doing the same thing with 8 subregions each with a width of $0.5$, and again with 16... and we could continue this process until we have an infinite number of subregions, each with a tiny width of d$x$. As long as everything is always correctly weighted, the answers should always agree. Without the correct weighting, the summation would indeed simply be $infty$.

This is why I always make sure to point out to students that an integral is not simply the symbol $int$, but the pair of symbols $int text dx$.

You're interpreting the probability distribution the wrong way - it is an infinite number of infinitely divided probabilities, so you can't say that "the probability of drawing the value 0.5 from a (0, 1) uniform distribution" because that probability is zero - there are an infinite number of possible values you could get, and all of them are equally likely, so clearly the probability of any individual outcome is $frac1infty = 0$^[1].

Instead, you can look at the probability for a range of outcomes, and measure that using areas (and hence integrals). For example, if you draw from the (0, 1) uniform distribution (with pdf $f(x) = 1$ for $x in left[0, 1right]$ and $f(x) = 0$ otherwise), then the probability that your result lies between $0.2$ and $0.3$ is

$int_0.2^0.3 f(x) dx = int_0.2^0.3 1 dx = left[x right]_0.2^0.3 = 0.3 - 0.2 = 0.1$

i.e. you have a 10% chance of getting a result in that range.

^[1]Sorry for all the people having heart attacks at my over-simplification of the calculation.

In general your reasoning fails in this assumption:

However, since there are an infinite number of numbers in that interval, shouldn't the sum of all the probabilities sum up to infinity?

It's a mathematical problem, known since the Zeno of Elea Paradoxes.

Two of his claims were that

1. An arrow can never reach its target


2. Achilles will never overtake a turtle

Both of them were based on the claim that you can build an infinite sequence of positive numbers (in the former case by saying that an arrow has to fly infinitely times half of the remaining way to the target, in the latter by saying that Achilles has to reach position where the turtle was previously, and in the meantime the turtle moves to a new position that becomes our next reference base point).

Fast forward, this led to a discovery of infinite sums.

So in general sum of infinite many positive numbers does not necessarily have to be infinite; however, it may not be infinite only if (an extreme oversimplification, sorry about that) almost all of the numbers in the sequence are very close to 0, regardless how close to zero you request them to be.

Infinity plays even more tricks. The order in which you add elements of the sequence is important too and might lead to a situation that reordering gives different results!

Explore a bit more about paradoxes of infinity. You might be astonished.

$f(x)$ describes the probability density, and has the unit $fracpx$. Hence for a given x you get $f(x) = frac1b-a$ in $fracpx$ units, and not p, as you are looking for. If you want p, you need the distribution function for a given range, that is the probability p of x being within a and b.

Hope this makes sense.