Probability is a weird child of math and science. One wants to quantify how uncertain one is with given knowledge. But to top it off is the weirder pet in the family - probability density.
Who wants to know how densely uncertain one is? To talk about density in normal science one needs a normalizing factor. Like how heavy is something per m3. Where is that per-ness in a probability density function?
It doesn't even make sense.
Until it does. And once I understood it, it was brilliant.
Well, not really. The last sentence was click-bait. Now you feel betrayed. You may leave. But honestly, even I felt betrayed after realizing the concept. There is no per-ness. There is no external entity (like how volume works with mass to create mass-density). The comparisions in probability density are with its own kind. In a very philosophical sense, that is how universe is. We have mass density as kg-per-m3, but does it have an absolute sense? How relevant is 1kg-per-m3 if there is nothing else to compare? Probabilty density, by itself is meaningless unless compared with another probability density. Let's dig in.