This Equation Shows You Can't Quantify Everything

Yeah, I used a clickbaity headline. So sue me.

Recently I went on an extensive rant about the fundamental assumptions of science. One of them, I said, was that things subject to scientific analysis have to be measurable. And that's basically true, I think... but there are interesting subtleties which are worth exploring. You might well be familiar with the weirdness of the quantum realms, as in the double-slit experiment where "particles" can apparently be in two places at once. What you might be less aware of is that much, much larger things can be just as hard to measure. You really don't need carefully controlled laboratory conditions to see how bizarre reality can get.

In this post, I'll start with some honest-to-goodness, totally conventional bits of scientific terminology that are genuinely useful in everyday life... yet actually describe something that literally doesn't exist. Then I'll start to look at why this is bloody confusing.


Measuring some things is hard...

In astronomy, if you're hunting for galaxies in a new data set, you have to try and estimate these things called completeness and reliability. They're quite simple concepts but they have very strict meanings - thankfully, for once, quite intuitive ones. Consider a naturalist trying to identify some meerkats at a great distance :

There are ten animals here - nine meerkats and one mere cat. Now the naturalist could, if he really wanted, shoot all of them dead or gas them or something, and count them at leisure. In that case there would be no uncertainty at all.

Real naturalists obviously aren't like that. They're more likely to try and count them from a safe distance, say using a small hand-held telescope. Our naturalist won't be able to hold it perfectly steady, it might be a bit blurry, and the animals are probably going to move around a bit - maybe it's getting a bit dark too. His observations therefore have limited sensitivity, resolution, and various sorts of errors. There are all kinds of reasons he might miss or misidentify some of the animals. Maybe he's also very stupid, blind drunk, or simultaneously fornicating with a rhinoceros. Tonnes of reasons.

Hey, I'm not judging.

If the naturalist correctly catalogues the nine actual meerkats, then we say his catalogue is 100% complete : he's found all the animals he was interested in. It doesn't matter if he also thinks the mere cat is a meerkat or if he goes completely mental and decides that some rocks and blades of grass are also meerkats, the completeness will still be 100%.

If, on the other hand, the naturalist is more diligent and demanding, but too much of a perfectionist, he might only identify one meerkat and nothing else. In this case his catalogue will be 100% reliable. The fact he's missed eight other meerkats doesn't diminish the reliability at all, it just means the completeness isn't as good as it could be.

Ideally of course you want a catalogue which is both 100% complete (finding all the meerkats) and 100% reliable (only finding real meerkats). Of course in reality things are never this good. This terminology matters quite a lot... consider this shark-finding drone, which claims to have a 92% reliability. See the problem ? Reliability is independent of completeness, so - in principle - it could be missing thousands upon thousands of sharks !

And that would be bad.

Naturalists at least have the option of going out and catching their subjects, if they really want to. Astronomers don't have that luxury, making it crucial to understand the difference between sensitivity, reliability, and completeness. Sensitivity is about whether it's even possible to detect something at all, e.g. do you have enough light and/or a sufficiently powerful telescope to see the meerkats ? Completeness and reliability, on the other hand, are about whether you actually do detect them. You might have good enough vision and sufficient light, but all sorts of other errors can lead to misidentifications.


...but measuring other things is impossible

It's possible to rigorously quantify sensitivity. Let's stick with astronomy so we can have some hard numbers. In that case, we can quantify very precisely the smallest mass of a galaxy we can ever possibly detect. This is our theoretical sensitivity limit. With the data that we have, we'll never be able to detect galaxies less massive than this - not ever*. But does that mean we will absolutely definitely actually detect things above this limit ?

* As long as we don't reprocess the data in some fancy way. There are various methods for doing this, but they all have associated penalties.

Of course not. It's just like the meerkats : just because you can spot something doesn't mean you actually will. Except there's an added complication here that makes things fundamentally different and more philosophically interesting. We can never know for sure how many galaxies our data sets contain. It's as though we looked at the African savannah and decided that while we couldn't see any, we couldn't quite rule out the possible existence of a gigantic, fifty tonne super-meerkat.

Dammit, internet ! That meerkat is clearly much heavier than fifty tonnes ! Idiots...

One way to illustrate this is through low surface brightness galaxies. Here's an image of low sensitivity of a fairly boring looking galaxy :

My word, that's dull. We could quite easily work out, though, how much light we'd need in any single pixel to be able to detect it. This would be our sensitivity limit : there'd be no way to detect something fainter than the faintest thing we could see in one pixel. This lower limit would be nice and solid. The problem is that this doesn't tell us anything much at all about features more massive than this that we could detect but just wouldn't. And in fact, a much more sensitive survey of the same region found this :

This is an astronomical fifty tonne super meerkat, otherwise known as the galaxy Malin 1. "Low surface brightness" just means that it doesn't emit much light per unit area, like spreading butter on toast so thin you can barely taste it in any bite. Malin 1 is massive, but so spread out it's difficult to see. This is why completeness is, strictly speaking, impossible to measure in astronomy catalogues - and you have to be extremely careful when you speak of sensitivity limits. Sensitivity limits are not at all the same as completeness limits.

To be fair, the way you calculate sensitivity does matter : if you account for the surface brightness sensitivity, then Malin 1 was indeed undetectable in the first image. But that still means you can't give a mass completeness; you can't ever say, "I've definitely detected all the galaxies more massive than such-and-such", because there could always be something really big but very faint hiding in the noise. And worse, the problem remains that you can never guarantee everything detectable will actually be detected. Here it's helpful to switch to radio astronomy, but the principle applies equally well to optical images.


Let's do some maths (but nothing difficult, I promise)

That's right : maths. Not math. That would be short for mathematic, and that doesn't make any sense at all.

Anyway, in radio astronomy what you often get is not an image (though of course we can get those too) but a spectrum. This plots brightness at different frequencies. Galaxies emit radio waves at different frequencies depending on how fast they're moving towards or away from us. Individual galaxies have stars and gas all moving at slightly different velocities, so each one is typically detected over some small frequency range. They can look like this, for example :

That would be a nice clear detection, very easy to spot. You can see there's quite a lot of random noise - this is due to a whole bunch of different effects and can never be eliminated completely - but the galaxy itself is obvious. A useful way to measure how detectable a galaxy is is through its signal to noise ratio (S/N). An S/N of 1.0 means the galaxy is only as bright as the typical noise values, so it would be impossible to distinguish from the noise and not detectable. That's what gives us our sensitivity limit.

Examples (fake) of a galaxy at lower and lower S/N levels from left to right.

But what about a completeness limit ? That's harder. A S/N of 2 probably wouldn't be detected either, because the noise level does tend to vary a fair bit. Neither would 3, 4, 5 or even higher values... depending on the frequency range the galaxy emits at. If it's very narrow, then we might need really high values - say 10 or 20 - to stand a good chance of detecting it. The reason is that real data sets are often plagued by very narrow spikes in the noise, due to the natural variation in the noise and artificial sources of interference. In contrast, if the range was (a little bit) wider, it might be quite easy to detect at lower S/N levels.

Here's the equation that we need to understand this :

The numerical constants aren't important. What matters is that the S/N level is governed by distance (d), mass (MHI), and velocity (or frequency) width (W). The parameter σ_rms is a measure of how noisy the data is, and not important for us.

So let's imagine we have a galaxy at a fixed distance d and of a fixed mass MHI, but we're magically able to vary its velocity width W. As it happens, real galaxies do in fact have different widths because their rotation speed varies, so this example is very much applicable to real observations. This little animation shows what happens as we make the width greater and greater while keeping everything else constant :

We start off with a narrow spike, reach a happy middle where the galaxy is unambiguous, and then we get the galaxy appearing as little more than a bump in the noise. And remember : the mass is the same at every stage. So again, we can't guarantee that we'd detect every galaxy of a certain mass, just because of the variation in galaxy properties. Mass completeness is impossible to measure. Literally impossible - it's not a matter of using different ways to examine the data, because if the galaxy is wide enough then it becomes absolutely indistinguishable from the noise. Objective algorithms and subjective visual inspection are equally hapless here.

Ironically then, this simple equation has led us to immeasurable properties. There are even more - quite a lot more, actually - subtleties to this, but the point has been made. While we can measure reliability by redoing the observations, we can't know if our survey has missed something. So we can't know what the full properties of the real galaxy population are really like. How wide a frequency range can they really span ? How massive can they get ? We can never know for sure.

Which brings me back to my original point. We have an equation - an actual honest-to-God equation, not some namby-pamby wishy-washy handwaving philosophy discussion - showing to us that there are things we can't measure. And I, for one, think that's rather neat.


You've killed science. Please don't do that.

Does this mean I was wrong to say science assumes things are measurable ? Not exactly, but it does need to be phrased more... delicately. We assume physical things are measurable, but not necessarily with perfect accuracy. The Uncertainty Principle already famously puts fundamental limits on things on ridonculosly teeny-weeny scales, but here we have an example of uncertainty on a much, MUCH BIGGER scale. And just as quantum effects tend to reduce us to probabilistic estimates rather than forbidding measurements completely, so it is here, to some extent.

We can't measure the true completeness limit. But we can at least compare the completeness of different search techniques to each other. Remember, we can verify reliability, by doing repeat observations to see if what we find is really there. So by combining all our different search techniques and follow-up measurements, we can at least estimate completeness if not measure it directly, and we can certainly get a handle on which methods are better.

The point is that completeness, while scientifically of undeniable importance, isn't a physical thing. Some properties are innate, others are relational. Take sheep. If we have two sheep charging across the fields at each other, they have both innate and relational properties. The mass of each sheep (or number of atoms if you want to avoid complications like the nature of mass*) is innate. The velocity of each sheep relative to each other is relational, by definition. While every property arguably does have relations to every other, they aren't all intrinsically relational. The number of atoms in each sheep might be related to what it was doing earlier (e.g. pooping), but at any given moment it doesn't depend on the properties of anything else at all. The relative speed of the sheep, on the other hand, is intrinsically a relational property. It can never be expressed except with reference to the other sheep.

* Let's leave the nature of number for now, mmmkay ?

Completeness isn't a physical thing. Is it a relational thing ? Arguably, in some sense. Completeness can be measured as a relational property, by comparing different measurements. But true completeness can never be measured. It's neither physical nor relational : it's conceptual. And conceptual properties, despite being very useful scientifically, can have disturbingly un-scientific aspects...

At least we can quantifiably estimate completeness, even if we can't know the true numerical value (it's a bit like the difference between countable and uncountable infinities). But consider more subjective concepts like justice, or guilt, or yellow. Can you quantify them ? Can you put a number of how fair an action is ? Guilt's an especially nice one. If someone was discovered to have aided a criminal, the original criminal's guilt clearly isn't diminished, not even as a fraction of the total guilt, because they obviously wouldn't have diminished responsibility because they had assistance. Guilt isn't like mass or energy, which are conserved - you can't even quantify it at all.

In case you thought I'd gone mad by suggesting colour as an immeasurable quality,
 there are no red pixels in this image.

Are conceptual concepts real ? Clearly yes, but they're non-physical. Which means that reality is more than physicality. And if that seems like a very bold statement on such a profound issue, it probably is. I'll make it anyway for the sake of argument. It does seem, though, as though hard science, mathematics, and subjective philosophy all suggest that we simply cannot describe the entirety of the Universe by noting which atoms are currently bashing into each other at any particular moment. Something much more interesting appears to be going on.

What exactly does this mean ?

It depends on how far we can extend this. A pet idea of mine is that notions like these imply that dualism - the old idea that the mind and body are distinct - is true at least in a very limited extent. Descartes had his famous mind-body problem (do read that link), where he couldn't work out how a non-physical mind could apparently control a physical body. Leaving aside the nature of mind and thought for a moment, the basic problem seems to be whether the non-physical can ever affect the physical. Maybe :

• The world is entirely physical, with stuff interacting through direct contact though in ways we clearly don't yet fully understand that gives rise to the mere appearance of non-physicality.
• The world is partially physical and non-physical. Non-physical properties could either interact somehow with physical ones (e.g. E.M. fields, gravity, ideas of justice, etc.), or simply be non-participating, essentially illusory artifacts, like rainbows. 
• The world is entirely non-physical : a shard of the mind of God or a high-tech simulation. Causality may or may not be real.

Philosophy has the liberty to explore all of these possibilities and more, whereas science is constrained by the evidence of the time. While the two have undeniably grown apart, and sometimes estranged, I think this is one issue on which they remain inseparable.

Everyday intuition would probably suggest to most of us that the middle one is correct : conceptual properties are real, non-physical, but interact with the world. If we see something that goes against our idea of justice, we may take action to correct it. If our galaxy-finding algorithm performs badly, we may improve it. And we obviously can't act without having observed these problems. So these conceptual properties do have influence... ah. Oh dear.

Stop and think about that for a moment.

These are non-physical, immeasurable things, apparently having a profound effect on reality ! Does this mean there are some things we'll never be able to understand rationally, or simulate ? Is idealism correct after all ? Is the boundary between physical, objective reality and subjective thought more blurred than we might like ?

Which is something I've previously attempted to depict artistically using radio data.

Well, some of those questions are hard to answer. But don't panic ! We need not fear that the woo-woo merchants are about to disembowel science with ritual chanting and whatnot. Even if we grant that non-physical things affect the world, they do so very much indirectly. They affect our mental states, which only in turn causes us to take direct physical action. They do not cause galaxies to explode or your cactus to sing or anything stupid like that. And your mental actions don't directly cause any crazy things to happen either. Your dream about the giant wombat with the staple gun poses no threat to society or anything else for that matter. It's a bit like the simple "Change" spells of Terry Pratchett's Discworld, i.e. in Wyrd Sisters the young witch Magrat finds her broom has run out of energy mid-flight :

Some kind of Change spell was probably in order. Magrat concentrated.
Well, that seemed to work. 
Nothing in the sight of mortal man had in fact changed. What Magrat had achieved was a mere adjustment of the mental processes, from a bewildered and slightly frightened woman gliding inexorably toward the inhospitable ground to a clearheaded, optimistic and positive thinking woman who had really got it together, was taking full responsibility for her own life and in general knew where she was coming from although, unfortunately, where she was heading had not changed in any way. But she felt a lot better about it. 

So this doesn't appear to be something that can obliterate the scientific method, or even give it a nasty shock. What it does do is say that the scientific method is potentially limited, that there are some things we can't simulate... at least not mathematically. Which is very interesting, but it doesn't suggest that existing simulations or mathematical analyses are wrong.

Let me reinforce that. That some non-physical states exist, in this interpretation, doesn't mean that every conceivable non-physical state is actually possible, much less actually does exist somehow. You're no more compelled to believe in God or ghosts than you are in a Bose-Einstein condensate lurking in your closet or that you'd find a lump of strange matter in your cheese. Just because something can in principle exist in no way means that it's possible that it does exist*, still less that is actually does. Phew, thank goodness for that !

* E.g. in principle the Moon could be made of cheese, in practise this is impossible.

Some people consider even this limited influence of the non-corporeal to be a step too far. They quite rightly point out that it still doesn't solve the problem of how things of such different natures could interact. Most people, I'd say, are quite happy to let this be, accepting that while they can't explain how the physical and non-physical can interact, they quite clearly do, so nah-nah-nah-nah-nah. A more reductionist perspective finds this unsatisfying. They'd probably point out that E.M. fields and the like can be explained by force-carrying particles, so cases of apparent "spooky action at a distance" can be restored to normality.

Of course, there's more to the notion of action at a distance than E.M. fields. There's wave-particle duality, Many Worlds, pilot waves, and all that quantum craziness, not to mention curved spacetime in general relativity. The reductionist view is essentially that either non-physical things just don't exist - they're a sort of illusion but produced entirely by physical things - or that they do exist but have no influence of any kind, not even mentally. Consciousness, for example, has been suggested to be a process that just "observes" what physical processes get up to, whilst being completely unimportant by itself. We might be aware of what we're doing but not actually in control of it.

Here philosophy and science collide head-on, and anyone who thinks they definitely know what the answer is ought to be given a very wide berth indeed. Personally, while admitting that not being able to explain how the physical and non-physical can interact is clearly unsatisfactory, it seems to me at least equally unsatisfactory to suggest the non-physical doesn't really exist. I would even say it's contradicted by not just by advanced contemporary science but also simple relational properties.  Importantly, the reductionist perspective offers no real clue as to where the illusion of non-physical stuff actually comes from (more on that below). And it seems to me that science does seem to allow things of very different natures to interact - e.g. massless photons can excite electrons, neutrinos mostly don't interact but sometimes do, etc. - even if, again, it can't necessarily explain exactly how.


Conclusions

I don't have any, though I do have preferences.

What seems to be reasonably clear is that some things are unmeasurable and unquantifiable. The consequences of that depend very strongly on the true nature of those quantities.

If, as in my preference, they can affect the world, then this means there's a limit to what we can simulate and describe through mathematical analysis. There would be aspects of the world that no amount of improvements to scientific accuracy would ever allow us to measure, because they're fundamentally unmeasurable. This doesn't imply the reality of any kind of Magical Mystical Woo* : the existence of some unmeasurable things doesn't necessitate the existence of all unmeasurable things. It would just mean that we can't know everything scientifically, no matter how careful our examinations.

* Someone should really name their child that. And they should grow up to become a teacher, so that all can benefit from the teachings of Magical Mystical Woo.

Obviously this viewpoint is not without its problems. It wouldn't solve how non-physical and physical things can apparently interact. While we don't necessarily observe the non-physical things, we do conceive them and are thus influenced by them.

The difference is interesting and important. For example if I measure completeness of a survey, or better yet something more mathematically complex that requires extended cognition, I have to write down the number before I can observe it. Doesn't matter how I write the number : I could use ink, bits of pasta, or arrange megalithic stones if I wanted. My brain doesn't care what configuration the number is, it's able to discern the number itself from the infinite different ways it could have been presented. So I'm not observing completeness directly : that's a thing which only arises mentally. It still affects me, but it's very different from, say, a ghost, which would have to interact directly with the observable world to be visible.

Imagining and observing a ghost are clearly different things, despite absurd claims to the contrary.

And this idea wouldn't solve what thoughts are either, or what makes some electrochemcial processes give rise to awareness while others, like those in calculators and possibly in plants, apparently do not. But since this viewpoint holds that some things are unknowable anyway, that ought not to be a major issue... science, so far as I'm aware, anyway isn't yet capable to saying how things of such different natures as photons and atoms can interact - not really, not at the most basic level. It provides a description of what happens, but a description isn't the same as an explanation.

I also favour the view that our awareness allows us control we wouldn't otherwise have. Extended cognition is a great example : here it seems we actually need to be conscious to make calculations, and we can't act on the results until they are consciously observed. Blindsight is interesting, but this seems more like a flawed consciousness than truly lacking one. Anyway, consciousness isn't exactly a binary state : we may be unconscious while dreaming, but it's hardly as if closing our eyes gives us the mental capacity of a rock. We're still thinking, still perceiving our thoughts. It would seem to me a highly contrived scenario if we could do all this unconsciously but somehow, for whatever reason, just didn't. Much more likely we actually do genuinely need awareness for some things.

That's my view then : not everything is measurable, but I discount Mystical Woo; I believe our mental concepts allow us to interact with the world through our own choices. I don't claim to know how it all works. And this view is somewhat dependent on scientific findings, so I'd have to revise it if suitable scientific models came along.

More reductionist approaches aren't uninteresting, however, but I find them unsatisfactory. I rather like the idea of consciousness as a sort of pure observer that isn't able to influence the world, with everything we think we control being a deception. Yet this seems a strange and completely unnecessary process, and doesn't seem to really do away with the unphysical as it might appear to. It doesn't say anything much about how vast, complex, imaginary concepts arise from atoms bashing about, and anyway pure observation is considered by mainstream science to be impossible.

That deserves a little more emphasis, because it's what I've been driving at all along. Completeness is a scientific concept derived from measurements but fundamentally unmeasurable; it has no physical existence within reality. Mental images also don't have physical substance. If I think of a giraffe, there's no spontaneous re-arrangement of neurons that makes a little miniature giraffe within my brain. If I think of justice, I don't even have an image to fall back on. The very notion is completely intangible. So what are these mental constructs ? They're experiences, obviously, but what the hell is an experience ? Even if they are mere observations, that doesn't seem to avoid them having fundamentally non-physical natures. Fifty-tonne meerkats are thankfully imaginary and therefore unable to affect the world, except that I'm writing about them so clearly they do have influence and a non-physical existence.

I'm more intrigued by studies on emergence. Rather than doing away with non-physical things completely, relational, non-physical properties can arise only with sufficient complexity, e.g. two atoms can have relative velocities but enough atoms together can have a sense of social justice and a burning desire for pizza. There are even particularly strange notions wherein complexity is required for emergent behaviour but doesn't directly cause it...

In any case, emergent complexity is intriguing. But it doesn't seem to me to be terribly convincing. I don't think it's going to help with understanding non-physical concepts much at all. Not at their root, at any rate.

Then there's panpsychism, the notion that everything is conscious. This works provided we separate awareness and agency, but that renders it utterly pointless. Sure, a rock could spend billions of years dreaming about turning into hot molten lava, but since it unarguably can't do anything about it deliberately then that doesn't seem to get us anywhere. Like consciousness being purely observational, this feels more like a cheat than an explanation, a way to avoid dealing with the fact that the non-physical, unscientific nature of consciousness is really, really hard.

So I say the common sense view has it right in this case. Imaginary things are imaginary and exist in a different sense than physical things. They can affect things but only mentally, not directly. It's an open question as to whether, as some have suggested, we might need new physics to explain this, or even if we can ever bring it into the scientific fold at all. And as for free will, that's a topic for another post.