Perfectly Wrong : Or, Necessary But Not Sufficient

Right now the popular science media is all a-flutter with "news" of the "discovery" of "Planet 9". This post isn't about that... at least, not directly. What I want to do here is sound a cautionary note using a surprisingly relevant tale from the normally unrelated field of extragalactic astronomy. There are, however, three things I must say about Planet 9 before we begin :

• It's. Not. A. Discovery. It's an inference. You can't discover something unless you see it. Oh, you can have extremely good evidence for it, to the point where the actual detection becomes almost a formality, but we're nowhere even remotely near to that stage of credibility.
• The attitude of the scientists. Their stridingly-confident approach is the very last thing one should see coming from someone at the forefront of research. Why ? Because science is a process of getting things wrong, and this whole attitude of, "this thing we just thought of is right but all the other claims of more planets were wrong" is just aaaaaargh.
• It would be Planet X anyway because Pluto is a planet, deal with it. Or possibly not because who knows if it's cleared its orbit or not ? No-one, that's who.

But on to our feature presentation. It's the fact that Planety McPlanetface hasn't been discovered that I want to emphasise.

Have a look at this particularly lovely pair... of galaxies, that is. You do your own joke.

NGC 4438 (the big one) and NGC 4435 (the little one) as seen in the Sloan Digital Sky Survey.

The correct first response is "Oooooh !". The correct second response is, "well how did that happen ?".

Being such a big, bright, and downright spectacular object, Markarian's Eyes (as they are sometimes known) have attracted an awful lot of attention from magpie-like astronomers over the years. And rightly so, because aside from observational astronomers naturally liking pretty pictures as much as anyone else (maybe even more so), this system is weird even by the standards of other cosmic train wrecks.

As early as 1988, computer models were being used to try and explain this bizarre-looking duo. Sensibly enough, Francoise Combes et al. carefully combined observations and simulations, rather than doing just one or the other as is often the case. Basically what they did was model a collision between the two galaxies. Which also seems thoroughly sensible because there are quite clearly two galaxies involved in the collision.

A little background might be beneficial here. In some ways, the very broad properties of the system are easy to explain. We know that some galaxies contain gas (which is necessary for star formation), whereas others (for a host of interesting reasons) do not. We also know that massive stars are blue and don't live very long, so we expect regions of gas which is actively forming stars to look blue. So, if one of the two galaxies had gas and the other didn't, we could certainly expect that a collision would cause lots of blue areas to develop in one of them while leaving the other one much more nonchalant about the whole thing.

Which is of course exactly what we see. The problem is one of detail : can we explain those precise structures that are actually observed ? From the Combes et al. simulations it seems the answer is yes, yes we can. Here's their simulation on the left compared to the observations again on the right.

Pretty darn good ! OK, it's not a perfect agreement : the smaller galaxy is a little bit too far to the north in the simulation, the southern tail is straight in the simulation but curves in reality, and the tip of the northern tail is a simple angular point in the simulation but looks distinctly "pinched" in the observations. Yet the broad agreement is very impressive.

One should also remember that this was 1988, when computers had all the raw power more normally associated with a rampaging snail or a high-powered sloth. The simulations were necessarily very primitive compared to what's possible today. They didn't (couldn't) model any of the gas physics or the star formation or anything like that. All they did was to have two gravitational fields for the two galaxies and a bunch of test particles (that is, particles which don't have their own gravity) representing the stars. So they could predict where most of the mass should be, but that's about it.

But don't underestimate the achievement of the result. To explore the enormous parameter space of two colliding galaxies (which could have a huge range of initial positions and relative velocities) and get something that resembles the observations this closely is, frankly, heroic. Which means according to Blackadder we all have to stand like this in celebration : 

Here we get a very pertinent example of Occam's Razor. It seems that we can explain the system very well indeed using nothing more than the simplest of physics. If we were to use the Razor as is popularly supposed (the simplest explanation is usually the correct one) we'd get the absurd result that we should ignore all the other physical processes because they clearly just can't be very important. That's why the Razor needs to be handled as carefully as any other sharp pointy object.

Fortunately real scientists are not movie scientists and don't assume the simplest explanation is the correct one, which isn't what Occam said anyway. They keep investigating. So in 2005, Vollmer et al. (the et al. includes the heroic Francoise Combes, by the way) did more sophisticated modelling of the system - though still with the basic approach of smashing two galaxies together.

Although they were still limited in some ways (no star formation and only simple approximations for the dark matter), they were now able to model the gas and stars separately and include the effects of the the intracluster medium on the gas. Even intergalactic space is not totally empty, and as galaxies move through the particularly dense gas that pervades galaxy clusters, their own gas can get pushed out. Whether or not a model without that intracluster medium works or not is irrelavent : it's there regardless.

Terribly famous image showing the gas in various galaxies (size greatly exaggerated) in the Virgo cluster along with the gas in the cluster itself. 

The result of this more sophisticated modelling does not, on first glance, give a result which is a dramatic improvement over the earlier model. Better, yes, but not massively so. That changes somewhat if you read the details of the gas distribution (not shown here) - there are certain details, the authors found, that just can't be explained without including this intracluster gas.

Unfortunately most of the figures given in the paper are rather small. The only decent one only shows the stars in the larger galaxy, for some reason. However, some rather better movies of the simulations are available here.

Still, the fact that this is an incremental improvement bodes well. The basic structures can be reproduced very well with this model of two colliding galaxies, and even more precise details can be re-created when you include the hot gas outside the galxies. Surely, this means the model must be basically right - two colliding galaxies and some extra gas are all that's needed. Right ?

Wrong.

Recently I described how the scientific method is an incredibly messy process, especially at the forefront of research (though things are rather different if you limit yourself to the least controversial areas). One comment received was, "...it looks as if "Observation" is the start of the scientific method which yields results for peer review."

Yes. That's exactly what I very deliberately wanted to say. Because sometimes, as I shall now show in spectacular fashion, observations tell you something you wouldn't have had a hope in hell of guessing if you limited yourself to the classical theory -> prediction -> observation approach. For when this area of the Virgo cluster was observed at a wavelength sensitive to hot gas of a particular temperature, Kenney et al. 2008 found this :

Oh, crap.

Here we see the new observations overlaid in red and green (depending on the precise velocity of the gas, but that's not important here) on a standard visible-light colour image. Instead of just the two galaxies colliding, it's now virtually certain that they've also interacted with a third, much larger galaxy. All those carefully fine-tuned simulations which reproduced the precise structures very well... nope. The lesson should be powerfully self-evident : even if your model does a fantastic job of reproducing even the very fine details very well, it can still be wrong - or at least woefully incomplete.

Which is, of course, not to say that the Combes and Vollmer models are definitely totally wrong. It's still possible that NGC 4435 and 4438 are interacting, and maybe that is still the main mechanism for the formation of the weird-looking structures. But it's also abundantly clear that that is, at the very least, not the whole story. If the third galaxy was just a tiddler then this likely would be a case of slightly modifying the original suggestion. But, since the third galaxy is enormous, it's also even possible that the major structures formed by a totally different mechanism and perhaps the close proximity of the original two galaxies is just a coincidence. This is commonly known as the "more than one way to skin a cat" principle.

I trust this makes the point about Planety McPlanetface clear. Even if they do have a really good model that explains the observations - and I'm not convinced that's the case - that doesn't mean jack without an observation. You have to be very, very careful to distinguish between, "this model works" and, "this model is correct".

I wouldn't often presume to correct Feynman on the scientific method (though there are other aspects of Feynman that I would have few scruples about attacking) but this is not quite the whole story. It's a minor but important detail, but you've got to be damn sure it really does disagree with experiment (or observation) before you say it's definitively wrong. Given enough research, this is possible. But, as a general rule of thumb, it is foolhardy indeed to reject or accept anything at the forefront of research even if they have seemingly good observational evidence... and positively deranged to accept an idea on the basis of a model alone. 

Now, not for one moment would I dare to suggest that I have any authority to pronounce judgement on Planety McPlanetface* - just that I'm prepared to bet against its existence. I have zero expertise in Solar System dynamics. Someone could potentially find it tomorrow and prove me wrong - but until that happens, I don't find the model-that-fits-the-facts to be anything more than an intriguing possibility, and the barrage of speculation is just hopelessly overblown. 

* Though I insist upon naming rights.

The take-home message from this post is simple. If it really does disagree with observation, it's wrong. But the reverse situation does not follow. If a model really does agree with observation, it isn't necessarily right - no matter how specific its predictions are.