Follow the reluctant adventures in the life of a Welsh astrophysicist sent around the world for some reason, wherein I photograph potatoes and destroy galaxies in the name of science. And don't forget about my website,

Tuesday 28 January 2020

Delicious Dark Fudge

"Dark matter," say the good people of the internet, "is just a fudge to make the theory of gravity agree with observations." They usually then go into lengthy monologues about how all of conventional science is wrong, look, I can prove it, here's a bar graph I drew in Microsoft Excel...

Unfortunately they do have a point : there are indeed disagreements between theory and observation. I've been at pains many times to point out that not every disagreement should have us roaming the streets crying, "Bring out yer dead !" and mourning the loss of venerable ideas. If we did that, we'd doubt everything and learn nothing.

Consider first the astonishing success of the dark matter model. Dark matter was first discovered (after a few false starts) when astronomers found that galaxies were rotating much more quickly than expected : extra mass is needed to hold them together. By far the most obvious interpretation of the data was that this extra mass was collisionless, not interacting with normal matter except through gravity. Of course, you can't postulate that the Universe is filled with a hitherto unknown substance and not have it impact a bunch of other things besides the way galaxies spin, and that led to ways of testing the theory which had nothing to do with rotation curves at all.

To do this, astronomers made a leap of terrifying audacity. Based on observations on the scale of individual galaxies, they extrapolated the results by a factor of billions and simulated the evolution of large chunks of the entire Universe.

Guess what ? It worked. Here's the simulations and observations of galaxies on the very largest of scales.

Real galaxies in blue, simulations in red.
A great big sparkly network of filaments and voids in both cases. This achievement is nothing less than stupendous. Few other ideas indeed are able to work so incredibly well across such vast differences of scale. But undeniable and inevitable grandiosities like the large-scale structure of the Universe are rare cases indeed. More usually the implications of a theory are infinitely subtler, more complex - and far more uncertain. Anyone bold enough to challenge dark matter would do well to take care if their finding really threatens the basic idea itself, or only one of those much more nuanced implications.

But when exactly ? When do we say, "this is only a minor disagreement, see, if I just add an extra two in this equation, it all works out nicely...", and when do we say, "holy crap Batman, we're going to have to start over on this one or possibly all just take up fish farming instead" ?

Well, for one thing we have to focus very clearly on which part of the theory is in trouble, and that's not always straightforward. This article expresses things better than I ever could. Theories don't exist in glorious isolation - they come with a baggage train of what the author calls ancillary hypotheses. When you formulate an idea, you're not always aware of its full consequences, especially if, as is often the case, your idea was inspired by just one or two data points. You may have no idea what it means for everything else until you start the long hard graft of crunching numbers. Sometimes it can take years, even decades, before you fully understand your own idea.

What about dark matter ? Undeniable though its success may be, even serious astronomers sometimes feel as though the theory has been modified and patched so many times that it would really be better to finally euthanize the wretched thing*. But is it really like that ? Are astronomers forever moving the goalposts to deal with inconvenient truths, or are they, in fact, still just trying to figure out where the bloody hell the goalposts were to begin with ?

* I'm on a quest to start a twitter outrage without being on twitter. Wish me luck.

With that in mind, let's take another look at seven of the main challenges to the dark matter idea. Maybe some of those "fudges" are actually nothing of the sort.

0) Separation of mass

It may help to start off with a semi-fake example, so this one doesn't count. Hence I start from zero.

What'd we like to test is something absolutely intrinsic to the whole notion of dark matter itself; what we've got is a bunch of galaxies spinning weirdly. And if that's all we have to go on, we could imagine a whole variety of other explanations. Maybe we're measuring rotation incorrectly. Maybe the systems aren't stable. Maybe gravity doesn't work well on scales this large.

Fortunately we do have quite a bit more than just rotation speeds. We know our rotation estimates can't be wildly inaccurate, and we know galaxies are generally not exploding. But the idea of modified gravity, like Bruce Willis or a particularly irritating weed, stubbornly refuses to die.

"Hang on," you might say. "Surely if dark matter exists, we might be able to find it existing independently of normal matter - at least in principle. Modified gravity can't do that*. Hah !"

* Actually there is a possible grey area.

"Hmm," I respond. "That's not unreasonable. But what if dark matter was just, like, spat out by stars and then just goes 'phwoooosh !' into nothingness about two seconds later ?"

"That's just stupid", you respond. And you'd be right. I have absolutely no justification for my totally ad-hoc theory of spontaneous generation of baryon-generated rapidly decaying dark matter. But you made implicit assumptions about the nature of dark matter - perfectly reasonable, sensible assumptions, but assumptions nonetheless. Granted, these are the kind of assumptions that if we don't make, we can basically come up with any stupid theory at all, and the whole farce rapidly tends towards "a wizard did it".

The point, though, is that even this seemingly most basic tenet of the theory comes with a bunch of implicit assumptions. So for testing, we have to resort to considering the full consequences and scope of the idea, even if we don't want to. And there are plenty of cases where it's much less obvious if the underlying assumptions are on firm foundations. As it turns out, there are indeed situations where we think we've observed separation of the dark and normal matter, and I'll return to those at the end.

1) The Missing Satellite Problem

Dark matter is supposed to a) dominate the mass of the Universe and b) only interact with itself and other matter through gravity. This makes it easy to program in a simulation and it doesn't need a lot of computing power. Hooray !

We've seen how blisteringly successful this is on large scales. But when they looked at their simulations on the much smaller scale of an individual galaxy, astronomers found something that made them extremely uncomfortable :

The Via Lactea simulation on the left (which only used dark matter) and the real Milky Way and its satellites on the right, to approximately the same scale.

The number of dark matter "halos" doesn't match the number of observed satellite galaxies very well at all. It's okay(ish) for the largest satellites, but absolutely crap for the smallest ones, predicting about ten times as more small satellites as we've actually found. Granted, the tiniest halos in the simulation are so small that they're not expected to have enough stars (if any) to be detectable, but the problem is very much a thing even at higher masses.

How big of a deal is this ? Well, the authors of one early paper on this problem went so far as to say :
Either the hierarchical model [of how dark matter assembles into galaxies] is fundamentally wrong, or the substructure lumps are present in the galactic halo and contain too few baryons to be observed.
Pretty serious. Right from the start, astronomers realised there were at least three possibilities : 1) the whole dark matter thing was bollocks; 2) dark matter existed, but behaved in radically different ways to the theories; 3) something was wrong with the physics of which halos should become detectable galaxies - maybe most of them never accumulated enough gas to form stars, for example.

Since the disagreement between theory and observation in this case was absolutely hideously obnoxious, initially it wasn't unreasonable to say that maybe it did count as evidence against the whole sorry idea. As for modifying the physics of dark matter, that's been tried a few times with differing results : pull on one thread and the whole tapestry tends to unravel. Best to leave that one aside.

That leaves the physics of the ordinary matter. It wasn't immediately obvious what could be going on here - it seems natural that at the same mass, each halo should gather about the same amount of gas and therefore form about the same number of stars. What on Earth could make some halos detectable but keep most of them invisible ? Why should only a select few blaze through the heavens while all the rest are consigned forever to the silent dark ?

And that's where the ancillary hypotheses and implicit assumptions come in. In order to predict how much normal matter - gas and stars - gets into each dark matter halo, astronomers used "semi-analytic" models. They took the numerical simulations and applied equations to calculate how bright each halo should be. This is not at all easy. The physics of star formation is seriously freakin' hard : way harder than Steven Seagal, harder then Arnie, harder even than Wolverine. It's like, soooper hard. Got that ? Good.

The Expendables 4 will definitely be about a group of hard-as-nails astronomers very carefully working through obscure problems, facing such hazards as slow wi-fi and rejected grant proposals. It's a tough world.
In order for gas to become stars, it has to collapse to the point where nuclear fusion starts. That means it has to cool, and its cooling rate depends on both its density and chemical composition. When stars begin to burn, they inject energy into and change the chemistry of their surrounding gas. And they're not formed in isolation either : most stars are stable, long-lived, low mass little things, but a few are giants that quickly explode, adding even more energy and material back into the interstellar medium in a damnably complex cycle. But wait, it gets worse ! The effect of all this depends on the total mass of the galaxy - in high-mass galaxies, stellar winds and supernovae might only move gas around a bit, whereas in very small ones, they may be able to remove it completely. And that's not accounting for interactions with other galaxies, extragalactic gas, the totally different chemistry of the early Universe, or magnetic fields...

See ! I told you it was hard. Too complex, at the time, to simulate directly, which is why they used the semi-analytic approach.

Unsurprisingly, some early models were completely at odds with how many satellites ought to be detectable. Some said there should be loads of detectable "dark" galaxies containing only gas. Others said there should be hardly any of these at all, with gas almost inevitably leading to star formation.

You can see that the claim that the missing satellite problem is evidence against dark matter is, very credibly, total rubbish. It might be, but to test that, we need those ancially hypotheses, to do the unglamorous work of slogging through the complex physics of star formation. Without that, saying that the missing satellites disprove dark matter is to make a massive set of implicit assumptions about how the gas behaves. In fairness, the knee-jerk "this contradicts dark matter" response was not unreasonable twenty years ago, but knowing what we know now, it's just no longer tenable.

The problem is twofold. Even though normal matter and dark matter interact only through gravity, the highly complex physics of the normal matter means : 1) some dark matter halos might be disrupted (more on that later); 2) gas accumulation and star formation may mean there's a selection effect and we only detect a small fraction of the halos still present.

The complexity is such that we still haven't fully solved this. Don't misunderstand me. It could still turn out to be the case that the missing satellite problem is evidence against dark matter, but that looks increasingly unlikely. We cannot ignore all that horrible physics by simply assuming it's not a major factor. We have no choice but to try and include it whether we want to or not. That we've done so after finding a problem in no way makes it a "fudge" - it's absolutely utterly unavoidable.

But... this cuts both ways. All that complexity means a lot of uncertainty. The latest computer simulations (which can now simulate the gas and stars directly) don't have a missing satellite problem, but there are so many free parameters that it's debatable how much predictive power they really have. We've found a solution to the missing satellite problem, but it's by no means clear if this is the actual answer or just a much, much more complex fudge to save the theory.

2 Planes of satellites

Most of the other discrepancies between theory and observation are just interesting variations of the missing satellite problem. This is good because it saves me a lot of time.

In the above rendering I showed the Milky Way and its attendant satellites from its worst viewing angle. In profile, the satellite cloud isn't nearly as fat as all that - it's actually remarkably skinny.

This galaxy plane is beach body ready ! Twitter, are you listening ?
That's a problem, because clearly the simulations don't show that. They show the satellites in nice spheroidal clouds, not thin planes. Surely this is a direct contradiction of the dark matter model ?

Well, yes. It's perfectly reasonable to say that this observation contradicts the standard model, because it does. Ahh, but which part ? Therein once again lies the problem. It's very difficult to see how planes of satellites have got anything much to do with flat rotation curves or the notion of dark matter itself. But could they relate to the physics of how gas gets into dark matter halos and forms stars ? Hell yes they could.

All the same complexities of the gas physics apply just as they did in the case of missing satellites, and more besides. We could be seeing a selection effect that the gas doesn't inhabit all the halos for whatever reason. Satellites could be brought in along the large-scale filaments (rather than from every direction equally), and interactions could stretch out the satellite clouds to produce flattened pancakes. Worst of all, we only know about one plane with any certainty - all of the other claims are highly questionable at best. Things might be weird if extremely narrow planes were common, but there's no evidence of that.

Overall, claiming that planes of satellites contradict the dark matter paradigm is a bit daft, and makes at least as many implicit assumptions as the missing satellite problem. Until we know more about them, assuming that they're a problem is a massive leap in the dark.

3) Too Big To Fail

Which satellites are missing - just the littlest ones, or the bigger ones too ? This varies depending on the state of the art of both observations and simulations.

The "too big to fail" problem essentially says that there's a problem for the biggest satellites. These galaxies, though still much smaller than the giants, are so big that there doesn't seem to be any way they could possibly avoid accumulating enough stars. So we really ought to find all of them, but we don't - as though they were too big to fail but fail anyway.

This has all the same problems, solutions, and problems with the solutions as the missing satellite problem does - all of which revolve around the baryonic physics, and have little or nothing to do with the dark matter. And there are even more factors at work. See, while ordinary matter is, overall, far less massive than the dark matter, this isn't necessarily the case in every local situation. In the disc of a massive galaxy, ordinary matter can dominate. Any satellite galaxy getting too close to this disc can be torn apart by its gravity. Even those that stay a bit further away can still have their gas ionised by the hot stars in the giant galaxy, preventing further star formation*, or even removed entirely by the "corona" of hot gas surrounding the giant galaxy. None of this is included in the original pure dark matter simulations.

* This is thought to have been particularly strong in the early Universe when the first stars were highly energetic. This so-called "squelching" of the gas is an act of galactic abortion (hello twitter ?), preventing it from ever forming stars by keeping it too hot to condense.

There's one additional factor which has been proposed. It could be that in some cases we're not estimating the total mass of the galaxies correctly. We measure this from the rotation and size of the gas, but if the gas doesn't extend as far (relative to the dark matter) as in other galaxies, we'll underestimate the total mass. So it could be we've already found the most massive satellites, but misidentified them as being smaller than they actually are.

So yet again this is by no means a definitive challenge to the dark matter theory. It could be, but there's still too much we don't know about the ordinary matter to say for sure. Anyway, what grounds do we have to assume this is a dark matter problem rather than one of the more difficult and mundane physics of ordinary matter ? None that I can see.

4) Downsizing

This one's a bit different. There was once a controversy over how galaxies assembled : did they form from huge "monolithic" clouds, or did they grow from the "hierarchical merging" of smaller objects combining to form ever-larger behemoths, like the T1000 from Terminator 2* ?

* And presumably other Terminator sequels, but they don't count.

Eventually, hierarchical merging won the day. Not because it's intrinsically better -  a single spinning collapsing cloud is a much more elegant and simple way to form a galaxy - but because that's what the dark matter model said should happen. That is, given the known physics, no-one could see any reason why the necessary giant monolithic clouds should ever exist (also just like the other Terminator sequels), whereas the merging of smaller halos to form bigger ones happened very naturally. So the smallest halos should form first and the biggest ones last.

The problem is that galaxy star formation histories paint a different picture. The biggest galaxies are dominated by a big burst of star formation early on - a brief life burns brightly - whereas the smaller ones are firmly of the opinion that slow and steady wins the race.

It was never very clear to me why this was ever a big deal. If you smash a bunch of galaxies together, it stands to reason that you'll get a massive orgasm of star formation, whereas if you leave them alone, they're just going to quietly, err, mind their own business. Especially for small galaxies, where gas density may only occasionally and locally become high enough to form any stars at all. It smacks of an implicit bias toward thinking that galaxies = stars, which completely ignores the all-important gas. Galaxies grow in total mass through mergers, but it doesn't follow that their stellar mass only ever increases thanks to gobbling up other galaxies.

5) Flat rotation curves

This one might seem a bit odd. After all, flat rotation curves are the main reason people started believing in dark matter in the first place. What worried people was that the curves always seemed to be flat. Why shouldn't there be a wider variety of shapes ? Admittedly a few were found that were rising, but they seemed to be following the same basic shape as the other curves, just not extending as far.

This does seem intuitively like a problem. The implicit assumption here, though, is that dark matter is not that dissimilar to ordinary matter, which is enormously complex. Left to its own devices, we might well expect ordinary matter to do all kinds of funny things, like explode or play baseball or have a nice cup of tea. But the most popular theory of dark matter - by far - is that it's cold and collisionless. Under those conditions, dark matter halos should have pretty much universal density profiles. And since its mass is so dominant, adding in the smattering of normal matter really can't change the overall shape of the rotation curve very much.

6) The core-cusp problem

For such a romantic pursuit, astronomers are shite at thinking up terms. The core-cusp problem refers to the central dark matter density in galaxies. Simulations say it should be "cusped". The hell does that mean ? "Cusping" sounds like something wantonly depraved to me*, but sadly it just means "spiked". That is, the density keeps on rising until it gets very very high indeed in the centre.

* 100 internet points to whoever comes up with the most NSFW explanation.

By way of contrast, observations show that the real centres tend to be "cored". What, so someone came along and scooped out their innards like an apple ? No, it just means that the density tends to reach an upper value in the central regions where it doesn't vary very much.

As we've seen so often, this is a problem if and only if you make naive assumptions about how the dark matter halos accumulate material. Remember the "too big to fail" problem. We saw that the dark matter isn't dominant everywhere, and the centre of a galaxy is one such place where the ordinary matter rules the roost. So one promising solution is, surprisingly, star formation. All that expulsion of material from hot stellar winds and supernovae is potentially enough to disrupt a dark matter cusp spike, simply by the sheer mass of material being moved around. Sure, dark matter only interacts via gravity, but as anyone falling off a cliff will tell you : sometimes gravity is quite important.

This issue isn't settled yet; although it does seem like a very promising explanation, it might not be able to account for every galaxy.

7) The Tully Fisher relation is too neat !

Last one. It feels appropriate to end on something especially controversial where the end result is still veiled in the mists of uncertainty.

The Tully-Fisher relation is the very tight relationship between how fast a galaxy rotates and the total mass of its stars and gas. Nothing too surprising there : big things are bigger. The more dark matter a galaxy has, the faster it will need to rotate to remain stable, the more ordinary matter it can attract.

What's odd is that it's possible to show that the relationship should be more scattered than we see. The fainter a galaxy is per unit area, the more it should deviate. But they don't. Why not ?

We don't really know. Some alternative theories of gravity do predict a nice tight TFR, and that's a big success for them. It is indeed far from obvious how the matter in galaxies should conspire to produce such a neat relation - less obvious, perhaps, than in all of the other cases of apparent problems.

But progress is being made. The TFR is a manifestation of a much deeper relationship between dark and normal matter. The acceleration of ordinary matter turns out to correspond extremely well to the acceleration of the associated dark matter, which has been shown to be due to a combination of quite subtle selection effects. In principle, this should also be able to explain the TFR, though to my knowledge no-one has done that quite yet.

On the other hand, it looks increasingly as though that we may have hugely underestimated the scatter in the TFR : in effect, we've been assuming that our observations were representative when in fact they were not. Both faint and bright galaxies have now been found that rotate much more slowly than predicted by the TFR, while some of the most massive spirals (and some small isolated gas clouds) apparently rotate too quickly. While in general more accurate measurements seem to decrease the scatter in the TFR for most galaxies, this is by no means necessarily true of all of them.

Faint (left, in orange) and bright (right, in black) galaxies that deviate from the Tully-Fisher relation.
Right now, we genuinely don't know what those deviant galaxies really mean. The ones which are rotating too slowly are consistent with having no dark matter at all (going left => less dark matter; going right => more dark matter). From one perspective this is very bad news for alternative theories of gravity. As long as the mass distribution is the same, gravity should behave the same everywhere - so similar galaxies should always rotate at the same speed (external disturbances notwithstanding). It shouldn't be possible to get differences like this. With dark matter, it's at least possible in principle to separate dark and ordinary matter, so theoretically such objects are entirely possible - galaxies with the same mass of stars and gas could have different amounts of dark matter.

Then again, should we expect dark matter deficient galaxies to actually exist ? Only a very few have been found in simulations, and it seems like they're just the result of an extremely obscure bug in the code. Maybe the simulations don't have the necessary resolution to examine them properly. We just don't know. That awful spectre of our lousy understanding of the physics of star formation becomes all too important for objects without any dark matter at all.

And it feels decidedly odd that for so many galaxies the TFR is so tight yet others have such a wide scatter. How can there simultaneously be a small scatter for some objects yet pretty good evidence that the dark matter content of galaxies is highly variable ? Something's not quite right here, but I'm blowed if I know what it is.


Some of these issues are all but solved. For others, the full implications are not yet known. But in no cases is it justifiable to say that anyone has "fudged" their results to agree with the theory. It's barely even credible that any of them present evidence against dark matter at all, much less that there's some vast unwitting conspiracy by the evil Defenders Of Darkness*.

* Doesn't that sound better than "scientists who think dark matter is the correct explanation" ? I guess we have to call modified gravity theorists Slayers of Einstein or something... 

This notion of "ancillary hypotheses" and "underlying assumptions" is extremely helpful in keeping different aspects of the theory separate. It forces us to take a step back and consider what an unexpected result really means. Does it follow directly and inevitably from the central aspect of the theory ? If so, it could well be evidence that the theory is wrong. If not - if it's due to some other physical process or assumption - then it isn't.

In this case, we know that the physics of star formation is tremendously complicated. We know that gas can do strange things. And we know of mechanisms that can change our more naive expectations. What we've done here is not "fudging" but simple, standard exploration. All of the issues presented here depend heavily and often entirely on those issues, of the complex gas and star formation physics, not on the dark matter itself. 

Now it's true that many of the results really did seem shocking (even, dare I say it, baffling) when they were first discovered, because some of our implicit assumptions were perfectly sensible at the time. But continuing to cling to those early notions in defiance of subsequent, more sophisticated analyses is not science at all : it's faith.

And yet... how dashing ! How noble the acts of a lone maverick genius, bravely standing up to the slings and arrows of his dogmatic opponents, with a simple, clear message that dark matter is disproven - now that's a narrative that sells. The long and grinding process of working the problem, testing every aspect in careful detail, criticising one's way towards self improvement, that's hardly the stuff movie heroes are made of. But it's a damn sight more accurate and a damn sight more scientific than saying we're all just fudging the results.

This is not to say that fudging can't happen, mind you. As our simulations reach unprecedented levels of complexity, there's a dangerous temptation to tweak them to match observations. Sometimes this is unavoidable, since there are physical processes at work we can only observe in deep space and never test under controlled conditions. But too much of this would indeed constitute fudging the results. That, though, is more a warning for the future than a concern of the present.

Given all the uncertainties still around, have we made any actual progress on the issue of dark matter itself ? Yes, absolutely. We have extraordinarily good evidence of the separation of dark and luminous mass in multiple galaxy clusters, something which is damnably hard to do with modified gravity. We've found evidence of the separation of mass on the scale of individual galaxies too. Meanwhile, the prospect of modified gravity looks increasingly desperate. Even the most stalwart enthusiasts openly admit it needs dark matter of its own in clusters, which would seem to defeat its whole purpose. And no-one, after more than three decades, has found a good version compatible with relativity.

Doubt is perfectly natural given extraordinary claims, but this verges on prejudice. It's like dark matter kidnapped their pets and burned their house down, or something. There no longer seems the slightest advantage to modified gravity over dark matter - it replaces one highly successful idea with one that doesn't even fulfill its own raison d'ĂȘtre. While I wouldn't write it off just yet, it increasingly feels to me that support of modified gravity is becoming ever more irrational. That could conceivably change, though I doubt it will.

So here's an idea. Stop thinking about dark matter as a problem to be explained away. Think of it instead as an exciting discovery to be explored. Perhaps, so like so many ideas before it, it will eventually be abandoned, but then again, perhaps it will stand the test of time. Let the investigation carry you where it will, and enjoy it.

No comments:

Post a Comment

Due to a small but consistent influx of spam, comments will now be checked before publishing. Only egregious spam/illegal/racist crap will be disapproved, everything else will be published.