The Higher Dimensions Series — Part One: Distance to the Origin
Welcome to Part One of the Higher Dimensions Series, where we explore some of the strange and delightful curiosities of higher dimensional space. Currently, I have completed Parts One, Two, Three, Four, and Five, with hopefully more to come.
I once took a course in linear algebra and it completely blew my mind. It introduced me to what has become one of my most cherished intellectual pleasures: thinking about higher dimensional spaces. When I tell people how excited I get when I think about higher dimensional space, they tend to think that I’m a weirdo, but they also tend to interpret “higher dimensional spaces” with a science fiction bent, something akin to alternative dimensions or parallel universes (which, as an aside, aren’t necessarily restricted to fiction). However, that’s not what I’m talking about here. I’m talking about taking some simple geometric concepts, things that we are all very comfortable with in two or three dimensions, but applying them to higher and higher dimensions, and seeing some very cool and seemingly very strange results. I owe immense credit to Michael Mahoney and his linear algebra course for introducing me to these extraordinary concepts.
In this first installment of the Higher Dimensions series, I aim to illustrate one of these delightful results. Specifically, I want to show you what happens to points selected randomly from inside a circle as we take that circle from two to three dimensions (where we’d have a 3-dimensional sphere) and onwards to over 100 dimensions (where we’d have a 100-dimensional hypersphere). Don’t worry if that doesn’t make any sense, I’ll try to be very clear later. First, some terminology. A sphere or hypersphere is technically the shell that surrounds a ball in space, so I’ll actually be talking about selecting random points from within an n-ball, where n refers to the dimension of the ball.
The Simplest Case
Let’s start with the simplest possible scenario , selecting random points from within a 2-ball, or what we would typically call a circle. Imagine drawing a circle on a piece of paper and using your pen to dot a bunch of points inside the circle as randomly as possible. I’ll technically be using Python to generate points inside the circle, but the idea is the same.
I want us to think about just one characteristic of these points: how far they are from the center of the circle. Every circle, or n-ball, has a center. The cool thing about balls in space, or rather the spheres that surround the balls, is that every point on the sphere is equidistant from its center. The distance between the center and the outer sphere is the radius of that sphere (or ball). To keep things simple, we are only going to work with balls with radii of one unit — I’m just going to say “unit” and keep it arbitrary, as opposed to talking about a specific unit such as centimeters or inches. So, in the case of the 2-ball, we know that the distance between each point and the center must be somewhere between zero (if the point were exactly on the center) and one (if the point were exactly on the spherical shell surrounding the ball).
What does our intuition say about how far the points will be from the center of the circle, on average? Specifically, what if we used a ruler to measure the distance between the center and each point, then calculated the average distance?Let’s take a look! Here is a 2-ball with its center at the coordinates (0, 0), which we call the origin, a radius of one, and 1000 randomly selected points.
Personally, my immediate instinct would be to guess that the average distance between the points and the origin would be 0.5, half the radius. The points are uniformly distributed throughout the 2-ball, so perhaps it’s reasonable to assume that the distances between the points and the center are also uniformly distributed between zero and one. That seems reasonable, right? Well, it’s not! The average distance between each point and the center is actually about 0.67. If we are a bit more careful in our thinking, it becomes pretty clear why the average distance should be greater than 0.5. Let’s think about the entire area of the 2-ball, which we can calculate using our trusty old geometric equation of Area = π * radius². For the 2-ball with a radius of one, the area is equal to π, or about 3.14. Let’s divide our 2-ball into two different non-overlapping areas: one where all points are less than 0.5 units from the center (blue), and one where all points are greater than 0.5 units from the center (red), like this:
The blue area represents all points where the distance from the center is less than 0.5 and the red area represents all points where the distance from the center is greater than 0.5. We can calculate the area of the blue 2-ball using the same equation as above; the ball has a radius of 0.5, which gives us an area of π/4. Because we know the area of the entire 2-ball encompassing both the red and blue areas, we can calculate the area of the red area by subtracting the area of the blue 2-ball from the original area of π, which gives us a red area of 3π/4. Is your intuition changing, yet? The red area, where all points are greater than 0.5 units from the center, represents 75% of the area of the original 2-ball. So, 75% of randomly selected points are going to be farther than 0.5 units from the center, and 25% are going to have a distance less than 0.5 units from the center. This is why the average distance from the center to each point is greater than 0.5! Let’s take a look at the distribution of the distances from each point to the center of the circle, using a histogram:
The above histogram shows us the percentage of points that fall between 0 and 0.1 units from the center, between 0.1 and 0.2 units from the center, and upward. This clearly shows us that there are more points farther from the center than closer to the center. To be honest, this was an unanticipated tangent from my original aim of exploring higher dimensional space, but I think it highlights a concept that will be important when we soon move to higher dimensions; that is, even in the case of a two-dimensional circle, there is more area farther from the center than closer to the center.
Okay, now that we have belabored the simplest case of the two-dimensional circle, let’s get to the fun stuff! What happens to the distances from the points to the center of an n-ball as we increase n from 2 to 3, 4, 5, and higher?
What is a 4-dimensional ball?
Before we continue our exciting quest to understand what happens to distances between points and the center of higher dimensional n-balls, let’s clarify what it means to points and a ball in greater than three dimensions. We clearly understand the two-dimensional case, it’s just points in a circle. It’s also relatively easy for us to understand the three-dimensional case, it’s just a cloud points in three-dimensional space inside of a three-dimensional ball, like this:
The 3-ball is just like the 2-ball but with one extra dimension; that is, the 3-ball centered at the origin and with a radius of 1 represents every point in three dimensional space that is less than one unit distance from the origin. To calculate the distance between each point and the center, we can imagine using a ruler. We can then calculate the average distance between the points and the center as well as the distribution of the distances, just as we did for the 2-ball.
Unfortunately, we as humans are not able to visualize spaces with more than three dimensions in the way that we can for two- or three-dimensional spaces. Beyond three dimensions, we can no longer visualize the points within the ball and can no longer easily imagine using a ruler to measure the distance between the points and the center of the ball. But worry not, for the fact that these things are difficult or even impossible to visualize is part of the fun! In the absence of visual aids, we must turn to our reliable friend, mathematics. Remember, the 2-ball centered at the origin and with a radius of one represents all points in two-dimensional space within one unit distance of the origin. Similarly, the 3-ball centered at the origin and with a radius of one represents all points in three-dimensional space within one unit distance of the origin.
Now, what about a 4-ball? As you may have guessed, a 4-ball centered at the origin and with a radius of one represents all points in four-dimensional space within one unit distance of the origin! I imagine that some of you may be getting worked up and asking yourselves, “what the hell does he mean four-dimensional space??” Well, just as we can represent points on a two-dimensional piece of paper with two coordinates, say x1 and x2; and we can represent points in three-dimensional space with three coordinates, say x1, x2, and x3; we can represent points in four-dimensional space with four coordinates, say x1, x2, x3, and x4! We don’t have to stop at 4, we can add as many dimensions as we’d like. We may not be able to visualize the cloud of points once we get beyond three dimensions, but we can represent them with coordinates and, importantly, we can still measure their distance from the origin.
How can we measure distances in spaces with more than three dimensions, you ask? Well, for two and three dimensions, we talked about measuring the distance from each point to the center of the relevant 2- or 3-ball using a ruler. Instead of using a ruler, we could just have easily used one of geometry’s most fundamental theorems. Remember the Pythagorean theorem: a² + b² = c²? Well, that is what what we are going to use to measure the distance between two points in space, or specifically between a point in a ball and the center of that ball. You probably recall using the Pythagorean theorem to measure the hypotenuse of a right triangle, as below:
In the case of the 2-ball, the distance between a point and the ball’s center is indeed the hypotenuse of a triangle! Check it out:
Specifically, a is the distance between the center and the point’s coordinate on the x-axis, b is the distance between the center and the point’s coordinate on the y-axis, and c is the distance between the center and the point itself. So, using the Pythagorean Theorem instead of a ruler, we can calculate the distance between a point and the center of the 2-ball as:
Another name for c here is the Euclidean distance between the point and the center of the ball. So, we can use the Pythagorean Theorem to calculate the Euclidean distance between two points in two-dimensional space. What about three-dimensional space? The Euclidean distance between a point in three-dimensional space and the origin is calculated using a generalization of the Pythagorean Theorem. Specifically, if we have a point located at the three-dimensional coordinates (x1, x2, x3), then we can calculate the distance between this point and the origin as follows:
I won’t break down the details, but this is actually equivalent to just applying the plain old two-dimensional Pythagorean Theorem twice — just think about how you could reach a point in three-dimensional space using two two-dimensional right triangles. Woah!
So, if we want to measure the distance between the origin and a point in n-dimensional space, we just need to sum up the squares of each coordinate and take the square root, as follows:
Another name for this is the Euclidean or L2 norm of the vector starting at the origin and ending at the point of interest, which comes up all over the place, including in some super cool statistical methods. But don’t worry too much about that right now.
Okay, we are now equipped with how to understand an n-ball centered at the origin with a one unit radius (all points in n-dimensional space within one unit distance from the origin) and how to measure the distance between points in an n-ball and the origin of that n-ball. Onward ho!
Where Things Get Weird
First, let’s take a look at what happens to the average distance between points and the origin as we increase the dimension n. So, for each n, starting with 2 and going all the way up to 100, let’s calculate the average distance between points and the origin of the relevant n-ball. Specifically, I am going to simulate a whole lot of random points within each n-ball, calculate distance from each point to the origin, then calculate the average of all those distances for each dimension n. As an aside, it turns out to be a nontrivial exercise to generate random points in an n-ball, check out this excellent review of various methods to do so. In case you’re curious, I used the Muller method described there, in which you first randomly generate the direction in which the point is located, then randomly generate the distance between the point and the origin. Below is the plot of the average distance between the points and the origin for each n-ball.
Wow! Let’s unpack this a bit. As we learned earlier, we see that the average distance from randomly selected points to the origin is about 0.67 in a 2-ball, but this average distance increases rapidly as we increase the dimension n. In fact, the average distance from points to the origin is about 0.75 in a 3-ball, 0.91 in a 10-ball, 0.95 in a 20-ball, and 0.99 in a 100-ball! What’s going on here? It seems that as we move into higher and higher dimensions, points seem to be concentrated densely right at boundary of the ball! Because remember, no matter what the dimension, for n-balls with radii of one, the maximum possible distance from the origin to any point inside the ball is one. For fun, let’s check out the distribution of distances from the points to the origin for the 3-, 10-, 20-, and 100-balls, just as we did for the 2-ball.
Here’s the 3-ball:
The 10-ball:
The 20-ball:
And lastly, the 100-ball:
Remember when we started with the circle, the 2-ball? We saw that the majority of points (75% to be exact) were greater than 0.5 units from the origin and 25% were less than 0.5 units from the origin. As we venture up to higher dimensions, the likelihood of finding a point that is less than 0.5 units gets vanishingly small. For instance, even in just 10 dimensions, less than 0.001% of points will be less than 0.5 units from the origin!
Let us pause and reflect on this for a moment. Inside an n-ball of arbitrarily high dimension, the vast majority of the volume is located right up against the outer boundary of the ball. For me, this is where all the fun is! We cannot explicitly visualize these spaces, but they exhibit strange and unexpected characteristics such as this.
What would it be like to move around in one of these spaces? No single point can be farther than one unit away from the origin, so you are never more than a stone’s throw away, yet almost all the volume is somehow concentrated at the boundary. What a world!
Wrapping Up and Looking Forward
This is only the beginning of our strange and wonderful journey into higher dimensional space. Things will only get stranger from here, and I hope you will join me for Parts Two and Three of the journey.
