Twenty Years In Astronomy

Twenty years ago this month marks the start of my career in astronomy. So, hooray, I'm old !

Truly, a life where weekly Bingo is the biggest attraction makes a strong case for assisted dying.
Yes, I'll regret that joke in another twenty years, but I'm not there yet.

This gives the perfect opportunity for the blog equivalent of a nostalgia fest a.k.a. clip show. If we as scientists are actually doing our jobs, we ought to have solved at least some of the problems that were plaguing astronomy in the mid-2000s. So how've we done ? How does astronomy today, both in terms of the science itself and the way it's carried out, differ from the halcyon days of 2006, and where are we going next ? 

Let's find out.

Part One : Science Marches On

I hasten to point out that this is going to be a completely arbitrary list of things I personally find most professionally interesting. No attempt is made at collating all the interesting stuff astronomy has done for the last two decades, because that would be a stupid thing to do.

1) Lie-onisation

I want to start with something small that was once treated as common knowledge but now seems very unlikely. At least, the way I was taught this was as a sort of "everyone knows" claim, that being that HI gas (neutral atomic hydrogen) would always be ionised at low densities. Specifically, below about 10¹⁹ atoms cm^-2, it should, so everyone told me, be entirely ionised by the cosmic UV background. Maybe not instantly, but quickly enough that we wouldn't detect anything.

This claim didn't come from nothing. There were in fact observations showing that the edge of the gaseous disc of galaxies was truncated at about this density... but the claim that "everything is always ionised" is so absolute that it can be disproved with a single counter-example. And findings over the last two decades have left this claim as little more than a rather dull piece of astronomy folklore*.

* If you follow Decoherency, you'll know that my idea of entertaining folklore involves a great deal of sex and violence. The precise value at which gas is ionised doesn't really hold a candle to the Polish tale of a psychotic bear with Tardis ears who attempts to kill a random girl by throwing a bunch of keys at her for no good reason, let alone the Bulgarian story in which God embraces cannibalism or the Czech one with the mysterious hot lesbian fairy.

In this rather crude (because I'm recoding my visualisation tool again) figure of M33, the white contour shows the limit of the gas at the 10¹⁹ limit. Everything beyond this should not exist.

Much later, I learned that the star formation community never expected anything like such a simple density limit, knowing (as they do) a great deal more about the microphysics than us extragalactic astronomers. And we now have considerable evidence that the rumoured ionisation simply doesn't happen, or at the very least, it certainly isn't any sort of universal truth. In the profile of M33, shown above, we see gas directly detected at densities almost a hundred times below the supposed threshold. Others have used stacking to combine the results from less sensitive observations on larger samples, and they also find no clear break in the density profiles. It's pretty much inescapable that this ionisation threshold was simply wrong.

Why would anyone care ? I think it's important to acknowledge when things that are taken as mundane facts quietly change without any big announcements. It might very well be the case that some galaxies do indeed have sharp edges to their gas discs*, but we can say categorically that this not true in all (or even most) cases.

* An interesting caveat is that there might still be a much lower threshold below which everything really is ionised. In that case, we ought to be able to detect the ionised gas directly, given sufficiently sensitive observations.

2) Big Disc Energy

A nice corollary to the above concerns the extended UV discs which enjoyed a brief moment of fame and glory exactly when I was starting my PhD. This was the discovery that about a third of spiral galaxies were found to have low-level UV emission from well beyond their main stellar discs. To find that there might be some stars at greater distances wasn't really surprising, but UV specifically traces the hot, massive, young stars that don't live for very long. That's a bit odd.

GALEX UV image of M83. The bright central disc is perfectly normal. The huge faint extensions came as quite the surprise. But if you're wondering about the first point, this UV emission isn't enough to ionise the outer gas.

Well, their short lifespans meant that there simply wasn't time for these stars to have formed in the main disc and migrated outwards, because they should have died before they got there. But if they were formed in situ, where were the older stars ? For every massive, UV-bright star that forms, you expect to find an additional, larger number of smaller, dimmer, much longer-lived stars as well. And they seemed to be weirdly missing. In fact it was even more oddly specific than this : UV emission actually traces the second-tier massive stars, not the most massive of all... and those too were missing.  

So what the hell was going on ? Was this some sort of cosmic Logan's Run, only worse ?

The solution to this terrific puzzle is another example of something that didn't really get a clear, flashy announcement. There was no classic "mystery solved, say scientists" press release. Which is ironic : this is one of the few cases where such an over-used declaration would actually have been valid, and it didn't happen.

The main answer here seems to have been mainly a selection effect. The GALEX UV satellite was disproportionately sensitive to UV in comparison to the optical data of the day, so the older stars were there, they were just harder to spot. 

But there were some other factors at work too. These UV-emitting stars live a lot longer than the brighter H-alpha sources, so those giant stars likely do exist as well... just more intermittently. And XUV discs were discovered at the same time that the received wisdom of a sharp edge to the gas disc was beginning to fade; people were already finding gas at much further distances than they'd previously suspected. True, overall it was lower density than you typically see in star-forming regions, but locally it might well reach the critical density from time to time. 

The outermost regions of a galaxy, then, are just dense enough to keep star formation ticking over, without ever reaching the blazing levels found deeper in its innards. Occasional flickers of star formation are more than enough to maintain the vast ghostly tendrils seen at just the right wavelengths.

3) Dark Galaxies Are Dead, Long Live Dark Galaxies

My very first conference, in 2007, was an extremely lively affair in which there were some fantastic arguments and one occasion where people nearly came to physical blows (not hyperbole) over whose model of dark galaxies was better. You rarely seem to get that any more, which is probably for the best... although sometimes I miss the "tell us how you really feel" atmosphere of those earlier days.

Anyway, back then the idea of galaxies with no stars at all – just gas and dark matter – really was intensely controversial. The problem was, as I've written about ad nauseum, that cosmological models predicted far more dwarf galaxies than we actually see. With hindsight those early models were hopelessly simplified, but the basic premise is now essentially the mainstream consensus : the smallest dark matter clumps, we think, really do form, it's just that most of them never accumulate enough gas to ever form any stars (or at least so few as to be all but undetectable).

The Via Lactea simulation produces a Milky Way-mass central object but which is surrounded by thousands of smaller sub-haloes. We actually see more like sixty or so. Not sixty thousand, just sixty.

This too is a case of a silent shift. What was once seen as an almost silly idea – though it was never quite fringe science – is now the accepted best explanation. And there's some pretty decent evidence for this, both theoretical and observational. On the theory side, modern simulations are now massively more sophisticated than those of 2006, and they don't have a missing satellite problem. Observationally, populations of so-called "Ultra-Compact High Velocity Clouds" have been found which seem to fit the predictions quite well.

But there are two major caveats. First, the kind of much more massive dark galaxy candidates people were proposing back then are now all but dead as idea; I myself have spent most of my career investigating such objects and concluded that our best candidates are something altogether different. Those objects are still sometimes treated with a level of disdain (but for caveats on this caveat, see below). Second, securely identifying any individual object as a dark galaxy* remains extremely challenging. Statistically the evidence is good, but I personally would be wary of declaring "mystery solved" for the missing satellite problem until we can robustly say we've found a dark galaxy beyond reasonable doubt. We've made huge progress, but we're not there yet.

 * Unfortunately the terminology has proliferated. "Dark galaxy" is now often used to refer only to the larger objects, and one group in particular insists on calling everything "almost dark" which I find immensely irritating. Smaller objects are often referred to as "minihalos" and "RELHICS", but functionally these are identical to dark galaxies.

4) Ultra Diffuse Galaxies

Here's where things get fun. From 2015, people started detecting large numbers of very low surface brightness galaxies. Such objects had long been known about as exotica, but now they started turning up in droves. Claims that some of these had in fact been detected years before are technically true (some were known since at least the 1980s) but missing the point : what once seemed like a few extremes were revealed to be part of entire population.

This in itself wasn't terribly surprising. What was surprising was just about everything else about them, and this continues to be the case to the present day. For starters, these weren't boring little tiddly smudges, but great big Milky Way-sized monsters. Nobody expected that*. All our problems seemed to be at the small end, and to find out we might have messed up with the high-mass end as well came as quite the shock.

* Actually Mike Disney did predict a population of low surface brightness galaxies back in 1976, but he subsequently claimed that UDGs weren't what he predicted, so this (rather confusingly) doesn't count.

From one of the very first discovery papers. It's still an emblematic image, and rightly so : this was one of those "whoa" moments rather than the more usual incremental findings that take more effort to get excited about.

My favourite aspect to this is their probable dark matter content. The debate is by no means settled, but the emerging picture looks like the following. Many, perhaps most, UDGs appear to be huge dwarves. That is, their stars are unusually spread out, but their total dark matter content isn't especially high. But a significant minority may well be the "failed giants" that models never predicted, not only as large as the Milky Way but also just as massive. Probably not enough to give us a real cause of concern about the fundamentals, but this is yet to fully play out. And nobody predicted UDGs as a population anyway, regardless of total mass.

Where it gets really fun is at the opposite end. Two headline galaxies have captured attention as being apparently entirely free of dark matter (literally, their missing matter is missing, yo dawg...). Weird though they are, I don't find these two actually that interesting : the explanations that have been put forth are convincing enough that we can explain these two quite nicely, I think. That is, the investigations have involved a lot of careful work and the explanations turn out to be extremely clever and unexpected*. I don't mean to say that these objects are "boring", because they certainly aren't ! Rather, what I mean is that these are special cases : worthy of study, interesting in their own right, but lacking clear implications for the wider population.

* In brief : dark matter can be preferentially removed because dark matter particles have orbits which take them much further out than the stars and gas, where they can be more susceptible to external gravitational forces. 

Not so the UDGs found in isolation. About three hundred of these have been detected with HI, and these as a population show a tendency to have less dark matter than expected : much less, albeit not often to the level of being completely deficient. You can't blame external disturbances for these. How these form remains a genuine mystery, one absolutely nobody in 2006 would ever have predicted.

5) Blue Blobs

Around the same time as UDGs, another class of object started turning up that's had far less attention but might, just might, help to tie some of this together. So far they've only been detected in Virgo, which means we should be extremely cautious about inferring what they mean more generally. But fuck it, this is a blog post, not a paper. To hell with caution !

Anyway, the so-called "Blue Blobs" have the creative naming potential that would make whoever came up with the "Very Large Telescope" blush. Like UDGs, they're of very low surface brightness, but they tend to be more structured and can be extremely gas rich. Like, crazy gassy, as in "seek medical attention immediately". Whereas your typical galaxy might have at most about as much gas as stars, these objects can have (in extreme cases) thousands of times more gas than stars.

A curse of being interested in faint stuff is that it's bloody hard to find any pretty pictures to impress people. This example comes from the Dey et al. 2025 catalogue paper.

The latest development on this front suggests that these are ram pressure dwarves. We've known about tidal dwarves for ages, in which interacting galaxies tear bits off each other and those bits go on to survive and live independently. Here it looks as though ram pressure stripping is removing the gas from the parent galaxies : as they move though the much thinner, much hotter gas in the cluster, the pressure can displace their own gas into long tails and eventually remove it completely. Parts of those tails, it seems, can condense and survive long after their parent has fled the scene.

This explanation works well for the chemistry of the gas, the extraordinary gas fractions of the BBs, and their lack of obvious parents. What's less clear is why parts of the stripped gas survive at all, given that most of the stripped material really does appear, in this case, to be rapidly ionised after its removal.

I find these objects especially interesting for two reasons. First, some of them are associated with the optically "dark" HI clouds I spent so long investigating, and that changes how we understand those objects substantially. Rather than ascribing their peculiar kinematics to being possible dark galaxies, it now looks like turbulent mixing with the cluster gas might be responsible*. It now looks as though these are indeed a weird class of object after all, just nothing like what we were suspecting.

* And more importantly, our dark clouds are actually a diverse population of objects. Explaining one or two as tidal debris was never a problem, it was explaining all of them as these incredibly transient objects that we showed, definitively, doesn't work.

And secondly, they hint – and just hint mind you – at a more unified picture of what's going on. Our classical view of galaxies has long been :

Nice and simple. Stars, gas, dark matter. Bam, done.

But now it looks like we might have more of a sequence :

Almost certainly reality is more complex than this, and really this diagram should have more axes. But again, this isn't a paper.

With the extremely interesting bit being that both UDGs and BBs appear to lack dark matter. The possibility of a direct connection hasn't yet been explored much, but it's awfully tempting to join the dots.

So galaxian parameter space appears to be fully sampled, from objects of almost entirely dark matter to those consisting entirely of gas and/or stars. The issue is whether this really represents an evolutionary sequence. Do Blue Blobs relate directly to UDGs or are they just coincidentally similar ? Could they actually form from UDGs which lose their gas through ram pressure, rather than from the stripped material itself ? Why don't we see BBs elsewhere, and how does their gas survive ? And how the hell do dark matter deficient objects apparently form in isolation ?

If there's one lesson that the last twenty years have taught me, it's that the answers can be found... but they're only rarely what you expect. And that's part of the appeal of the job.

Part Two : A Parade Of Paraphernalia 

I want to end with a brief look at how the practice of astronomy has changed and some thoughts on the future. In many ways, the core process of doing astronomy in 2026 remains very much like it did in 2006, but there have been some interesting changes... and there are certainly far more on the horizon.

The more things change...

One surprising bit of similarity is that dark galaxies are back in vogue. In the last few weeks there have been maybe a half-dozen papers on the subject, making this an ideal time to write this post as I haven't had time to read a single one of them yet. Will the 40th anniversary post in 2046 look back on this as another key development, or will it instead be like last time, in which dark galaxies are trendy for a little while only to become discretely buried ?

One huge change here is in the sheer volume of publication rates. Submissions to arXiv have approximately doubled in the last 20 years, and this isn't entirely a good thing. Journal standards haven't shifted to keep pace, resulting in a deluge of literature which is increasingly difficult to read just due to sheer time constraints. A new approach is needed here with some urgency, as reading excessive papers can certainly constitute negative work. We need clearer standards of what papers are for, the role of authors, reviewers and editors, and new formats to allow readers to get to the crucial points without having to wade through copious amounts of text that is of no use or relevance to them.

Software, too, has progressed enormously. In particular, when I began, there was no widely-accepted source-finding tool for HI work and most work was still visual. I myself still cling to this approach, having quantified at great effort that this is in fact still the gold standard. But I have to admit, the SoFiA algorithm is tremendously powerful, far better – far better – than anything conceivable in 2006. Even as I would still warmly encourage people to look at their own fucking data, I would welcome SoFiA as not just a tool for the larger data sets we now have available, but something genuinely beneficial rather than a necessary evil as some of the early algorithms were.

More broadly, expectations of software are switching, but slowly. It's now much more common to talk about user interface and experience without this being dismissed as somehow lesser than scientific problems; I for one fervently believe that this stuff really matters. Not only should we damn well have software which is actually nice to use (provide a god damn GUI, FFS people !), but it changes the end result. If something is easy to use, you can use it routinely, and instead of only checking special cases, you can apply it everywhere. That kind of change can have direct scientific impacts. The interface shouldn't be seen as an optional extra, and this is somewhere where AI has the very real prospect of bringing about a genuine sea change in scientific practice. Adoption of the principle that "things should be nice to use" remains stubbornly slow, however, and more work is needed to bring about something which really should be absolutely common bloody sense.

The wider impact of AI remains much harder to gauge. I periodically (but irregularly) test ChatGPT on scientific questions and performance and it's shown massive improvements. I wouldn't trust it to do actual research at this stage, but it would be foolish in the extreme not to consider this as entirely plausible even in the short term. In certain domains it's already better than some experts; those who are pretending that no further improvement will occur are almost certainly deluding themselves.

But with AI development being both rapid and unpredictable, I don't want to go any further down that particular rabbit hole. Rather, I want to first note the tremendous technological advances happening in scientific infrastructure, a.k.a. telescopes. FAST and SKA pathfinders are already making the HI studies of 20 years ago obsolete in terms of sheer catalogue size. Euclid and Roman are opening up the low surface brightness universe in a way that would have caused earlier astronomers to have heart palpitations. Not everything is rosy here (the UK's decision to cut funding to Jodrell Bank, though this may be reversed, is utterly baffling), but the overall trajectory is one that is steeply rising.

...the more they stay the same

Which brings me back to the first and final topic : science. I've already mentioned the unanswered questions on the nature of "blue blobs" and the dark matter content of UDGs. We've also now firmly established that we don't know where galaxies are getting their gas from. Star formation rates appear to have been stable for quite some time, which implies that galaxies must be maintaining their gas content. But star formation itself should have caused a significant reduction in gas content... which means they must be being resupplied from somewhere. Perhaps with the next generation of technology, we'll finally be able to see the signatures of gas accretion directly.

One thing we can certainly look forward to is probing the atomic gas out to much, much greater distances. Star formation in the nearby universe is a pale shadow of what it was at so-called "cosmic noon", about 10 billion years ago, but we can only directly detect HI in individual galaxies out to maybe 3 billion years ago. The SKA should allow us to go much further, and measure what the gas was really doing when all the exciting stuff was going on. We can theorise to the nth degree, but there's no substitute for observations.

And of course, we can expect the unexpected. We'll now doubt continue to go down the occasional blind alley which will see half the community periodically obsessed with things that turn out to be totally pointless, but that's the nature of research. Sometimes major results only turn up through painstaking and careful work, sometimes they come at you full in the face and suddenly you have to chuck out half your findings. That is as it should be.

---

My hope for 2046 is that we'll be in much the same situation as we are now : we'll have solved most of the problems that seem difficult today (I'd especially like to know what dark matter really is and if the Hubble "tension" is real) but have whole new ones to tackle instead. I'd like us to have moved beyond a "publish or perish" culture into one where we publish more but much smaller and more accessible papers. I'd like to think that we'll have figured out a good way to use AI as a valuable assistant to enhance our own abilities rather than letting it do all our thinking for us. 

Above all, what I really hope is that we get some bloody good surprises. I'd love to see a true paradigm shift like the one Einstein instigated, preferably to an idea not yet even conceived. To have the rug pulled from under our feet and enter a whole new world akin to the discovery, a century ago now, that galaxies themselves were distant beyond all previous imaginings. I think that would be something quite terrifying and wonderful.

Won't Someone Please Think Of The Astronomers ?

Yet another case of sweeping job losses to AI. This time it's the turn of the astronomers.

Who needs radio astronomers when you can have robo-astronomers ?

What am I talking about ? If you follow my content generally, you'll know I'm actually rather AI-positive. Indeed, this really fun political quiz scores me as more of a techbro than a neo-Luddite. 

Immediate disclaimer : this quiz is quite clearly deliberately a bit silly, so jokes notwithstanding, I have rather strong Views about being identified as a techbro. 

Still, I do maintain that LLMs are generally a net positive, so why am I claiming mass layoffs are just around the corner for us poor hapless astronomers ? Could it be the title was actually just clickbait ?

Yes, yes it could. Today's post is about "my" latest paper, in which "we" use machine learning to quantify HI deficiency slightly more accurately than previous techniques.

Well, that sentence has likely raised several questions. First, why the scare quotes ? That's because I'm last author on this work and wasn't involved with the initial investigation. I got added only after sending admittedly rather extensive comments and consultation on the observational side of things, so readers curious about the whole machine-learning aspect are advised to asking something or someone more knowledgeable about such matters. ChatGPT is probably a suitable place to start, being a machine itself after all.

But you might also be wondering about what HI deficiency is, why it's important, and why a small improvement is worth publishing. There at least I can be of help.

1) What is HI deficiency ?

It's actually quite a simple concept. Galaxies are found in a wide range of environments, but we often simplify this into just two categories : great big clusters, and everywhere else. Rich clusters are crazy, chaotic places, where you might get a thousand or more galaxies crammed together in an orgy of destruction and wanton stellar violence. "Everywhere else" consists of smaller groups, typically a few or maybe a few tens of galaxies, and sometimes just one or two. The violence there tends to be much more sedate and drawn-out. 

We tend to refer to everything-that's-not-a-cluster as the field. This includes individual isolated galaxies and small to medium groups, but when you get to agglomerations of more than, say, a hundred or so, you're into cluster territory. The numbers are very loosey-goosey, but there is a real difference between clusters and groups. In clusters, galaxies are moving much, much faster relative to each other, and the clusters themselves tend to have their own diffuse gas. 

Not shown here are the speeds. In small to medium groups these might be ~200-400 km/s or so. In large clusters they can be in excess of 1,000 km/s.

The diffuse gas means that a cluster isn't just a big group. A galaxy slamming through this hot intracluster medium experiences ram pressure, which is the means by which big clusters can be incredibly effective at stripping even massive galaxies of their gas, sometimes with just a single pass through the cluster. In groups, there might be some level of diffuse gas as well, but it's much lower density and the galaxies there are all moving more slowly. This makes ram pressure much weaker and less important.

Not that groups are entirely safe spaces for galaxies though, as that would be far too woke. Galaxies in groups are more vulnerable to tidal interactions which can sometimes be much more damaging than ram pressure. But broadly, as a zeroth-order approximation, galaxies in groups don't experience gas loss while those in clusters do.

HI deficiency is simply an attempt to quantify how much gas a galaxy has lost. Through large, carefully measured calibration samples of galaxies in the field, people have found relations between the size and shape of galaxies in relative isolation and their total amount of HI gas. This means that when we find a galaxy in the cluster, once we measure its size and shape we can predict how much gas it would have if it was living happily in the field like a sort of cosmic cow. The difference between the amount of gas we expect and the amount it actually has is the HI deficiency. I wrote a short calculator which gives a bit more details on this here.

Not that kind of field ! Though I suppose galaxies in the field do eat gas rather than grass, so maybe the metaphor is more punny than I intended... 

For the enthusiasts, HI deficiency is a logarithmic parameter. A galaxy with a deficiency of 1.0 would only have about 10% of the gas content of a field galaxy, if it had a deficiency of 2.0 it would have just 1%, and if it had a deficiency of 3 it would almost certainly be undetectable because it would have hardly any gas at all.

2) Why do we want to measure this ?

Paycheques, mostly, but also genuine interest. You can't form stars without gas, so the gas content of a galaxy affects potentially everything that happens to it. And knowing how much gas a galaxy has lost means we can understand not just what it's going to do next, but also how it's already been affected by its environment.

I mentioned that galaxy groups weren't safe places to hang out, but their exact role on how galaxies evolve is still rather controversial. Our basic picture is that galaxies are largely born in the field and clusters assemble gradually as galaxies fall in together, whence they're likely to loose all of their gas very quickly. Clusters, be their orgiastic nature, are the sexy bits of galaxy evolution, but just like real orgies, they're highly atypical of normal behaviour*. So studying galaxies in these weird environments is a bit like going to a brothel and asking the participants about their favourite movie to watch with their other half when they're having a lovely night in.

* Only a few percent (low single figures) of galaxies live in clusters.

If you do want to study a galaxy having a lovely night in, that is, a normal, typical galaxy, you then need to go to the field. There it appears that galaxies are far less susceptible to gas loss, but there may be some level of pre-processing even so : they can still lose gas through tidal encounters, just at a much lower rate than through ram pressure. Whether this can be at all significant, i.e. whether it can be enough to affect a galaxy's star formation activity prior to reaching a cluster... that's the bit we don't understand.

In fact it gets a lot more complicated than that, for (at least !) two reasons. First, when it comes to galaxies, size really matters. A massive galaxy can be far more resilient to gas loss than a smaller one, which might even be able to expel its gas through its own stellar winds and supernovae explosions.  

Second, the star formation activity of most field galaxies appears suspiciously stable, as though they were being continuously resupplied with gas to compensate for any losses due to star formation. This is known as accretion, and is widely accepted to be a thing that happens, but proving any specific examples of it – that is, actually spotting gas inflow while it's occurring – is bloody difficult.

And third (I said "at least" two and I meant it literally), the outermost gas of galaxies can be a lot less dense and easier to remove than the stuff within the stellar disc. So this may be vulnerable to ram pressure and other gas removal mechanisms even in relatively low density environments, a prospect known as starvation (or strangulation). Whereas our classical sort of ram pressure shuts down star formation almost immediately by removing all of the gas in a galaxy, starvation just depletes its reservoir, not the gas that's actually currently forming stars. It's a bit like draining the fuel tank versus removing all the fuel that's already actually in the engine.

Potentially, a galaxy might experience all three effects. It might grow initially if it's in a low-density part of the cosmic web, then as it accelerates towards denser regions it might lose its outermost gas. Only as it falls into dense clusters will it experience the full power of ram pressure stripping, though all of this depends on the mass of the galaxy as well as its environment.

Measuring deficiency as accurately as possible is therefore important in helping us understand all of this. If we could measure it precisely enough, we'd have a much better idea of the effects of environment on galaxy evolution.

3) How do we go about measuring it ?

It's not that difficult. The equations that people have come up with typically involve just two parameters : the size and shape of a galaxy. That's it. From these, we can calculate how much gas the equivalent isolated galaxy would have, compare with our target, et voilà, we have ourselves a deficiency.

The real-world problems are where it gets tricky. Until recently, detecting HI wasn't that easy, and even now, the total number of detections is probably no more than 100,000 : statistically large, but vastly smaller than the number of optically-catalogued galaxies. Parametrising the environment, so that we can really determine which galaxies are the helpful hermits we need for calibrating the relation, is also far from straightforward. And measuring size is relatively easy, especially given recent automated advances, but shape remains something that's still largely done by eye.

This means that of the half-a-dozen or so brave attempts to produce the relations we need for measuring deficiency, every single one of the buggers has come up with quite different values. And all agree that we really can't get any more precise or accurate than this. Basically, all this work has meant we can come up with four broad categories :

• Negatively deficient galaxies, which have more gas than expected (these are rare)
• Non-deficient galaxies, which we hope are typical
• Moderately deficient galaxies, which definitely do seem to have lost some gas
• Strongly deficient galaxies, which have lost a lot of gas

So I lied. "It's not that difficult" is only true in the sense that when you've got someone else's relation, it's easy to apply this to your own data. But actually establishing those relations is something that involves a lot of careful work. It can't be rattled off in an afternoon, there's no clear reason to prefer one person's relation to another except for blatant favouritism, and all that work doesn't get you a terribly informative result anyway. 

Perhaps, though, there's a less biased approach that might also increase the precision of the measurements, at least a little bit.

4) Can robots help ?

That's where machine learning comes in. Morphology is hard to measure because you need precision estimates of things like how tightly the spiral arms are wound, the arm/inter-arm contrast, the bulge to disc ratio, etc. etc. You can do all this very easily by eye, but automation is fiendish.

What if we just chucked a bunch of data at it ? You know, the old approach of throwing everything against a wall and see what sticks ?

This is a time-honoured plotting technique when you haven't got a clue what's actually going on.

That's essentially what we've done here. Rather than trying to measure all the complex parameters that define morphology – something that nobody ever seems to have done properly – instead the lead authors collected a bunch of parameters that, being relatively easy to measure, were already available for a large sample. Things like the brightness in different bands, estimate of the concentration, and all combinations of those parameters. Whereas a human would struggle to look for trends in more than a few independent parameters, and collapse in a screaming wreck if they had to try all the different possible combinations, for machine learnings, this is apparently no problem at all.

I'm not going to pretend I understand much about machine learning, though I think the description in the paper (section 3.4) is pretty good. The hope is, though, that there are sufficient measurements here that they effectively encode all the morphology information without having to make it explicit*. To determine their isolation we used someone else's big, popular catalogue, which after some fairly detailed checking does appear to give values which are in good agreement with other definitions of "isolated". And for the HI we had the giant ALFALFA catalogue, which rather surprisingly hasn't been used in his way before.

* I did suggest to include the actual morphology as conventionally determined, which we actually do have for the sample, but this was never implemented as far as I know.

The definition of isolation is actually quite tricky. Basically you want a galaxy which is sufficiently isolated as to have likely experienced no previous interactions, but determining this is not so easy. For this, you need 3D information about not just a target galaxy, but all of its surrounding neighbours : galaxies which look close together on the sky might actually be far apart along our line of sight, and vice-versa. Unfortunately getting this information for every galaxy is just not going to happen (though maybe some next generation surveys might help), so this is certainly one of the main limitations.

One of the other more recent efforts came by a very different approach in 2018. There the authors were much, much more careful to define their isolated control sample very carefully. Even thought they actually had less galaxies than some previous studies, they showed that they were able to decrease the scatter in the final HI deficiency estimates.

Our approach, which is nicely complementary, is more to go in waving our arms and screaming "let's throw MOOOAAAR DAAATTAAAAA at it !!!". Thanks to other people's catalogues, we have something like a order of magnitude larger sample size of nearly 7,000 isolated, gas-detected galaxies. The hope is that this will balance out any imperfections of likely including a few not-truly-isolated galaxies.

5) Does it work ?

Yeah, it seems so. Quite well, actually : the scatter reduces, not by a massive amount to be sure, but it does decrease.

Figure 6 from the paper. This plots the deficiency of isolated galaxies, which should be roughly a Gaussian centred on zero. The "RF" model is the main result of the paper, which does a bit better than the "linear model" (our own, using the more traditional two-parameter approach) and that of Jones. A small improvement is still better than no improvement !

Actually here the referee had a really interesting point, one that had not occurred to me at all : how do you know that the reduced scatter is real and not a statistical artifact ? Specifically, if the model is predicting a narrow range for the expected mass in the galaxies, it will reduce scatter artificially – just because a smaller predicted range means less outliers.

This one took me a good long while to get my head around but it appears to be a sort of "regression to the mean" effect. The essence of it is that it might be reducing scatter by reducing the predicted range, not by actually being any more accurate. Forget the deficiency bit for a moment and let's consider the isolated galaxies : the ones where we expect no gas loss, which we use as our control sample.

That is, suppose our isolated galaxies have HI masses of 3, 4, and 5 in whatever units (simple numbers to keep things simple). The ideal case would be a model that take the optical parameters of each galaxy and predicts corresponding masses of 3, 4, and 5, which would give deficiencies of zero in each case. A model which predicts, say, 17, 69, and 78 would clearly be no good and have a huge scatter.

But a model which happens to give the average mass of 4 for all objects will be much better and reduce scatter... even if it's only doing so by averaging. It might not really have any true individual predictive power, it's just that for the whole data set, the scatter goes down.

Fortunately it seems that this is not a statistical artifact. When we apply the model to isolated galaxies not in the training data, we find much the same range of predicted gas content. Since those objects are independent of the training, it appears that the model is indeed actually predictive.

And when we apply it to the non-isolated galaxies, we find all the classical effects that HI deficiency increases with galaxy density. This is a good sanity check : we haven't uncovered anything new, but we demonstrate that the model gives sensible results. Galaxies in richer environments have generally lost more gas. Hooray !

6) What's next ?

Well, we could try throwing more data and more parameters at it, but it's not clear how much this would help. Reducing the scatter further is probably possible, but how much is very hard to say. We might even be starting to hit a limit.

The problem is that as galaxies lose gas, they evolve in response. If they just lose a little gas, they'll get a bit redder as they stop forming hot, blue, short-lived stars at such a high rate. They won't turn fully red, but they'll redden and their size is also likely to change as their outermost gas gets removed. 

How quickly these changes will proceed is uncertain. And that's a problem. Suppose we measure a galaxy just after it's lost half its gas. If the gas loss has been sudden and recent, it will still have almost exactly the same colour and size as when it had all its original gas content, so we'll be able to measure it as being moderately HI deficient. But if we wait, say, a billion years, then its colour and size will definitely change. At that point our HI deficiency measurement might be quite different, because the model is predicated on the idea that we detect galaxies soon after the stripping occurred. The problem is that the calibration parameters themselves change due to the very effect we're trying to measure.

To be fair, in rich clusters it's likely a safe assumption that gas loss was sudden and recent. In those environments gas loss is so strong that if you detect an object which is deficient, it's quite likely either lost its gas recently or is still in the process of doing so. But even in clusters, galaxies are on orbits which take them well away from the region of gas loss for a time, and in lower density environments, this assumption may well break down completely.

Now to people who work in this area, this may be "intuitively obvious", as the referee stated. But honestly, I work in this field, and it wasn't intuitive to me at all. I'm quite pleased that we managed to include a qualitative estimate of the size of this effect in the paper, because I've never seen it described explicitly anywhere else besides a few (extremely) vague allusions here and there. It definitely seems like this will indeed contribute to the intrinsic scatter, though whether this will put a hard limit on the precision of what we can do is difficult to determine.

---

Unfortunately, what we haven't been able to do is show if we can use this to find anything beyond what was already known. For example, do we find any objects of strong deficiency outside clusters ? Do we see moderate deficiency levels in groups and filaments, i.e. pre-processing ? This was partly due to this being a lot more work – and the lead author has some strict time constraints – and partly because the reduction in scatter is nice but it's not massive.

What we actually do next is not so easy to judge. There are improvements to the modelling techniques which might help, and we might add the morphology parameter directly, or use larger data sets from FAST. All this is more dependent on our own availability (read : willpower) than anything else.

Thankfully, though, we have one clear conclusion. For this particular brand of well-trained robots, the risk of massive job losses and societal collapse appears to be acceptably minimal. We can all rest easy in our beds tonight.

Killer robots yearn to steal the glory from radio astronomers for slightly reducing the scatter in HI deficiency measurements. Look, James Cameron, this would make for a better movie than Avatar, for crying out loud.

Oh Socialist Utopia, Where Art Thou ?

This is a piece I haven't been wanting to write for a very long time. Events, though, have forced my hand.

I've covered Keir Starmer twice on this blog, once in February 2024 and once shortly after the electoral landslide that swept Labour back to power. It's fair to say I've presented the man with a glowing, even gushing, review, and over on Decoherency I've presented similar arguments and defended Labour after they suffered quite the barrage of largely idiotic attacks.

Alas, the happy period in which we could mostly ignore politics lasted about six months. Fourteen years of Tory garbage and we got six months when things were sensible. For fuck's sake.

Now this is not like the time when I reversed my opinion on Jeremy Corbyn. No, rather than ranting against Starmer himself, here I want to largely rant against the feckless stupidity of the Great British public. So let me be absolutely clear about where I stand.

YER ALL A BUNCH OF FUCKING MORONS !

Which is not to say that Starmer himself is blameless, because he certainly isn't. But having seen people on Last Week Tonight describe Starmer as a "prick" and seen his popularity plummet to lower levels than Boris Johnson (!), I have to say that the British are a bunch of contemptibly stupid fuckwits.

Ladies and gentlemen, I give you... the electorate !

Look, I promise I am going to go through the points where I disagree with Starmer, and I do think he's done some incredibly stupid things. I promise I'm coming to that. But first, he is/was HANDS DOWN the best Prime Minister we had in twenty years. Not even close. Yet he's engendered a level of bitterness that would fuel the wet dreams of a true hate figure like Farage, and... why ? 

For not being good enough, apparently.

It's absolutely fucking exhausting to have to deal with this perpetual bullshit. What the absolute flying fuck is wrong with you ? What in God's name will it take to make you people fucking learn something ?

Jesus H. Christ.

As you might guess, I don't intend this to be an especially thoughtful post. If you came here hoping for another repeat of the five-part rebuttal of Epictetus, I'm afraid you're going to be disappointed. I don't much care for polemics, but the hell with it : sometimes people need not careful reasoning, but a damn good verbal slap in the cerebral cortex.

Haters Gonna Hate

My basic question is this. What is it about Keir Starmer, dear proverbial reader, that makes you hate him so ? Do you even know ? I doubt it. 

Not that all of you don't have an answer, mind you : plenty will point to things like (supposedly) transphobic policies, "hard line" immigration stances, or perhaps the Mandelson debacle or any number of various other messaging missteps. But I argue that, in comparison to the previous 14 years of Tory missrule, Starmer's government has done absolutely sod all to deserve anything like the level of contempt with which the majority of the public now seem to view him personally.

Again, LWT features British morons describing their towns as "shitholes" without any explanation as to what it is they think makes them shitholes, let alone what it is they actually want their town to be/do in order for them not to be shitholes. If some of you do have answers, then an awful lot of you certainly don't.

The long-standing appeal of Starmer for me was a lack of magical promises. He never said he would transform Britain overnight, because the recovery the country needs simply can't be done quickly. He never gave the impression of being a miracle worker in the way that Johnson or Truss ever suggested. He made it quite clear that the process would be long and difficult. And lo ! The process is indeed long and difficult, made all the more so by the fascist orange cunt running the White House. Getting disappointed that the recovery isn't going very quickly is just stupid. Nobody was ever promised that it would be easy or fast.

Which leaves me bewildered as to what it is everyone is actually upset by. Could it be the litany of so-called "scandals" that have beset the party since it came to office ? I rather doubt it. Consider each of these :

• Expenses and hospitality. This early "scandal" consisted of Starmer and others claiming expenses for things they were perfectly entitled to, and accepting and declaring donations with full transparency for concerts and other events. It's the most pathetic of stories, but it was at this point – not when the winter fuel story started as the popular narrative has it – that Starmer's approval took a nosedive. It never recovered. I find this absolutely exasperating because nobody ever gave a shit when Boris Johnson was handed vast amounts of money by a still-undisclosed donor or turned Downing Street into a tasteless pile of tat for his idiotic wife.
• Rayner's taxes. Even less of a scandal. The Tories wasted billions – tens of billions, all lost now – on useless PPE in the pandemic while Matt Hancock cavorted with an underling, but the fascist rags couldn't handle a working class woman accidentally misreporting a few grand in taxes. It's disgusting. She never should have resigned.
• Mandelson. I'll grant that this one is different : Mandelson is an evil bastard through and through. But the press acted liked Starmer must have been uniquely stupid not to know about the (very real and incredibly serious) issues that subsequently came to light, despite never having tried to do some, oh, I don't know, investigative journalism of their own. They also downplayed the whole aspect of actually imparting state secrets and concentrated heavily on the Epstein angle. Look, this one isn't a non-story, but the way the press span it was downright dumb. Mandelson, as a duplicitous, evil, manipulative, highly intelligent twat, was a downright inspired choice to deal with the duplicitous, evil, manipulative, thick as shit administration across the pond.

Policy Presentations

So if it's not the "scandals" which are the source of the anti-Starmer nuttery, could it be instead the policies Labour has enacted, or those they've failed to deliver on ? This is more credible, but only slightly :

• Winter fuel allowance. Total non-story. This should always have been means tested but the press refused to consider it could have had any other consequence except millions of pensioners freezing to death in their tiny hovels, dying naked and alone with the name "Staaaarmerrrr !" escaping from their frozen lips as they expired. The idea that this might be good for budgeting was never discussed. Ever.
• Treatment of climate protestors. More of an issue for Guardian readers than anyone else, but some people attempted to paint (pun intended) Starmer as personally responsible for the handful of rather harsh sentences for climate protestors as being emblematic of a police state, or something, while racist rioters were allowed to get off Scott-free and given a big bag of crumpets, or whatever. Utter garbage, pure culture war nonsense. This is the effect of sensationalism gone mad, when instead of legitimately criticising the judiciary you see any minor error as "evidence" that the ruling party is worse than the Nazis.
• Response to Gaza. Now yes, my response would not have been much like the government's response. But I accept that the need to balance morals and diplomacy isn't easy, especially given the Labour Party's disturbing recent indulgence in anti-Semitism under Corbyn. I don't agree with their response to pro-Palestine demonstrations and I'm totally unconvinced by their explanation as to why certain participants in these are somehow terrorists. Do I hate them for it ? No. I think they're being stupid, but not malevolent.
• Farmers. Fuck off the lot of you. You've been saying that farming in the UK is on the verge of collapse for at least the last thirty years and it's wearing a bit thin, no matter how hard you work and how essential you are. The inheritance tax was riddled with exemptions and easy to avoid by gifting away the farms before dying. Not difficult. But no, just like with pensioners, no interpretation other than, "oh no, my poor little hovel will crumble into dirt because now I have to pay three million quid because I own one too many tractors" was ever allowed in the media.
• Online safety act. Yeah, I don't like this one. The government has come up with some mightily stupid ideas, not least of which is age verification and – for some reason, God knows why – banning step-sibling porn (but not actually banning actual step-sibling sex, because <snarkasm> that makes sense </snarkasm>). Does this mean they did a very stupid thing ? Absolutely. Worthy of hatred ? Come off it. Hatred should be reserved for racist buggers like Johnson and deranged loons like Truss, not well-meaning bad decisions like these. And incidentally, not once have I seen the news ever suggesting the bleedin' obvious : encourage parents to monitor and regulate their children's screen time. There are already tools to do this. The need for the government to act and thus treat us all like children was routinely assumed as the only option : the idea that the parents could have anything to do with it was never mentioned. 

It winds me up something mental, it really does. Under the Tories we had austerity and the evisceration of our public institutions, and we let them get away with it because they were apparently competent. Yeah, right, whatever.

I shall say it again. You. Fucking. Morons.

And then I must necessarily rattle through the positive sides of the Starmer government : nationalising energy, nationalising rail, support for Ukraine, stabilising the economy in the face of massive international challenges, playing Trump like a fiddle for a trade deal (albeit that didn't stick, but show me anyone who's done better against that cretinous mass of putrid bile), the employment rights act, the renter's rights act, a national wealth fund, reducing migration (not my preference but it's what most people want for some reason), cutting waiting lists on the NHS, improved public sector pay, vastly better relations with the EU, and free breakfasts for schoolchildren.

Are you seriously going to look me in the eye and say that the Starmer government hasn't been, on balance, the best thing to happen to the UK in decades ?

Come off it. That's stupid, and you know it.

A Solution : Mixed Messages

Could it then be messaging ? This area I think is by far the most promising, but also the one where the implications for the British public are the most damning.

For this, it's worth reading Tony Blair's recent essay in full*. Yes, the man is profoundly amoral, and his stance that we should have let the American's use our bases against Iran is crazy both pragmatically and morally. The man acts as though moral values are something nice to have, not actually of practical benefit, whereas this case is a clear demonstration that they are not. Support America's despot in their pointless and economically wrist-slitting war to distract from the Epstein files ? That's not a sane choice, Tony. Likewise, his belief that we should prioritise cheaper rather than cleaner energy is moronic : the climate crisis isn't something abstract that you can pretend isn't happening. 

*Blair also gives some suggestions regarding actual policy and a way forwards, some of which I agree with, some of which I react strongly against. A much better set of arguments on that front comes in Starmer's own response.

But politically, only the most foolish would disregard a three-times election winner. The root of it seems to be, in his assessment, that Starmer didn't pick a side. Tribalism may have its problems, but it's also a source of strength and a clear heuristic : when people think they know what you're about, you're predictable, and they can use that to judge if they think you're basically decent or not. When they don't, they can't trust you. In Unruly, David Mitchell stresses the importance in ruthlessness for medieval kings in eliminating the fatality of doubt among the populace, and there's certainly more than an echo of that here, at least analogously.

I would add that perhaps worse, though, was that while Starmer could have tried to brand his own approach, he just never did that at all. He just thought that people would essentially do this for themselves, and didn't care how they chose to interpret it. To an extent, it wasn't so much loosing the narrative as never crafting one in the first place... "mission-led government" doesn't strike a chord with people. Wanting this to be the "best country in the world" is a platitude, an abstraction, not something nearly concrete enough that people can tell what you're about.

Blair's second good point is that changing leader won't help without having a coherent strategy. I disagree that Labour doesn't have one, but I violently agree that they haven't marketed it. Its approach of almost pure pragmatism does have a whiff of making-it-up-as-they-go-along more than following a clear path. I agree with Blair when he says that to seize the centre, you must first devise policy and only then employ politics. That is, decide on what your goals are first, and then formulate how you're going to persuade people to go along with them. And these have to be lower-level than the "let's make everything better" stratagems of Starmer. You need to define clearly what it is you want to accomplish, fix, or improve, and only then go about deciding how you're going to do that.

His third point is that there is a real need to be radical. Starmer did have a vision for a radical centrism, ironically by decentralising government*, but that all came to nothing. On other policies he never even really offered that much you could call "radical". Nationalising rail and energy is definitely a good thing, but while the piecemeal approach has many advantages, it's also slow. Blair stresses also the importance of unconventional thinking, noting that while this may indeed result in actual disaster**, it also connects with people at a visceral level. The "stabilise" approach was perfectly sensible for the first year, but after that, incremental approaches were just not going to cut it.

* A federal structure for the UK, greater powers for local councils, replacing the Lords with a Council of Nations... all these were on the cards and promised within the first term. There was little enough sign of any of them actually happening though.
** Likening Trump to a car crashing into a brick wall, he rightly notes that the problem is that "radical people aren't sensible, and sensible people aren't radical".

So Starmer needed something big, bold, and obvious : a four day week, universal basic income, rejoining the EU, a wealth tax for the wealthiest, something of that magnitude. Not necessarily any of those specifically, and certainly not all them, but of that order, and economic rather than in terms of government structure. Reforming or replacing the House of Lords might be popular among some segments, but won't be itself directly affect anyone outside the Westminster bubble.

What Went Wrong

The most disappointing thing about all this was just how well-prepared the Starmer government was before going into office. They had many meetings with business leaders and institutes well before the election; by the time of the event, everyone knew their major policies. They had successfully eliminated the doubt that plagued medieval monarchs, made themselves presentable, serious, and safe enough for Tory voters to ignore rather than cosplaying Reform. Their whole shadow cabinet exuded dull competence after the chaotic farce of the Johnson and other previous administrations.

I don't think I can say too much about why this all went wrong, but I do think I can at least identify what went wrong. Starmer learned many of Blair's key lessons but not how to implement them. He knew how to say no to his own side, but not when or how (like telling a child they can't have a pony versus telling them they can't have a My Little Pony play set). He knew that change must be felt, but not how to make it felt or how to have it associated with his government. He knew that you have to meet people where they are, but not how to change their minds when you need to... opting to appeal to their sentiments rather than ever challenge them very much. And he knew that people judge policies in aggregate, but not how to get them to judge them at all... again, failing to paint himself in any sort of ideological colours.

Perhaps at least one root cause is that Starmer is too experienced at dealing with the dregs of humanity. Having dealt with rapists, murders, terrorists, Boris Johnson and Lizz Truss, he appears to have become thick skinned to the point of being tin eared. He could not acknowledge when messaging had a serious problem, preferring to go to foreign summits with other leaders than engage with the rough-and-tumble of the British electorate. I sympathise, but the Prime Minister of the UK needs to be seen to be dealing with UK issues. Starmer is right when he says that many international issues do affect us directly, but this doesn't acknowledge that this isn't how people feel about them. These issues needed to be left to the foreign secretary (and probably having somebody who wasn't David Lammy would have been a better choice there). Messaging really matters, because if you don't craft a narrative, someone else will do it for you.

And finally, he didn't stick to his guns. There were just too many u-turns and climbdowns, raising that hellish spectre of doubt : since he won't stick to his guns, say dissenters, it appears that if we stick to ours, we'll win. He didn't properly understand when to fight and when not to fight, which resulted in a messy, spitballing approach of trying hopelessly to please everyone. He could be ruthless in dealing with his party-internal opponents, sometimes too much so, but not in offering moral clarity.

Aftermath

So yes, as promised, I've set out what Starmer got wrong. I will say again that there are some policies I also disagree with, particularly transphobic judgements about bathrooms and the bizarrely prudish online safety act. The internet should be for adults; it's children who should bear the difficulties of access, not us, and parents who should be responsible for this, not the government. That one really did feel nanny state.

And yet the government has done an awful lot of good. To hate them for their entirely predictable missteps and utterly minor misdemeanours is profoundly lacking in sense. Frankly, if we can't be grateful for the improvements under Starmer compared to the last decade and a half under the Tories, then we'll never be happy – ever. We've been sold down the river on expectations of absolute perfection, and nothing less, it seems, will every satisfy us : anything less than a miracle is worse than shit. The perfect is indeed the enemy of the good, to the extent that if something is slightly flawed, people seem to want to vote for the exact fucking opposite instead. 

Indeed, it often feels as though the public view someone being slightly incompetent as worse than being deliberately malevolent. Nothing much else can possible explain the appeal of Reform or their brainless cousins in Restore.

For a much better sort of milkshake man, see this.

There is something rotten in the state of Britain. I cannot fully address what it is, but I will venture two major contributing factors. First, nobody seems to have a clue how our economy works. Nobody seems to realise what's really generating the money or where it's going; Blair is convinced AI will affect changes (literally) on the scale of the Industrial Revolution, but as to the basic questions of 1) where is all the money going ? and 2) why was our economy so much better in 2005 ? nobody seems to have a damn clue.

The second is the media. I love this piece on grumpy old men, and its main claim ties in to the British press perfectly. By selecting and sensationalising  all the negative stories, they've become an engine for conditioned responses. The voters are trained to see the bad in everything and the good in nothing, to see politicians as all self-serving hypocrites and worse. They're deft at finding the clouds is every silver lining and assuming that the negative interpretations are the only valid ones, with any positive interpretations being dismissed out of hand as tribal propaganda. We aren't allowed to be happy with anything. We don't appreciate for a moment just how much worse things could be.

Even this, though, I think doesn't fully explain what's happened. There is something rotten also in the psyche of the British people, something that prevents us from seeing the blindingly obvious. When we react with "my town's a shithole" for no good reason, then it makes perfect sense to say, "Starmer is a prick" for no good reason, and by the same token to declare that "Andy Burnham would be much better" even though he's at most 10% more likeable than Starmer.  

I don't claim to know what exactly has come over my fellow countrymen. I'm acutely aware that "they're all just bad people" is an insanely common and idiotic fallacy that explains nothing. But this is the reason why I said that "mixed messages" as a solution for why the Starmer government has failed is such a damning verdict for the public : that they cannot see how minor the problems really are, that they need to be spoon-fed to think for themselves, does not bode well at all.

Let's hope Andy Burnham can figure it out. If he can't, Brexit and austerity will be the least of our problems. And what a cheery thought that is.

The End Of Darkness

My goodness me, another paper ! What is this, a science blog or something ?

This one is by my long-term partner in crime, the inestimable Robert Minchin. After the last few papers – one on a single weird galaxy and two on source extraction – this is a return to my favourite topic : dark galaxies.

And not just any dark galaxies...

Well, the joke works if you're British, at any rate.

What I meant to say is that these are the objects I've spent most of my career on. If you haven't read the link, a dark galaxy is just a galaxy which has some gas, maybe a very few stars (but preferably none at all), all embedded in the classic dark matter halo that we all know and love : the very thing that distinguishes galaxies from all the other crap floating around in the majesty of the cosmos.

Look, I've been through this dozens of times. I'm not gonna do it again. Go and read the link, I'll wait.

Go ! Read up and report back. Off you pop.

Okay, you ought to bloody well know about these things in general by now. But I'll forgive you if you can't quite remember why I like these particular candidates so much. Mainly because I'm extremely biased and narcissistic... that is, they turned up in the data I was analysing during my PhD, and were the main highlight of my very first papers*. I've been working on these for the best part of two decades, which is a scary thought if nothing else.

* It's all "me, me, me" around here.

Yes, yes, but apart from keeping me employed, what's so fascinating about them ? No-one's going to keep paying anyone to study genuinely boring objects, after all.

Well, it's a combination of two factors. The first is that they have a high line width. That is, we can see that parts of them are moving at very different speeds along our line of sight (towards or away from us). This is exactly what we see with normal, rotating galaxies, and it's normally a pretty good signature that they have a lot of dark matter. 

But this, we've learned, isn't decisive. We didn't have good spatial resolution of the clouds, so we couldn't be sure that we were really seeing rotation. Simulations have shown that when galaxies interact with each other, the debris that gets torn off can (briefly) exactly mimic this apparent signature of rotation, so it was possible we were just being fooled. And without rotation, there's no reason to think there's any dark matter.

Really good data would let us map the motion very precisely, like on the left. Here we can see that one side is moving differently to the other, with a smooth gradient across the object : a classic signature of stable spin. But for these objects, all we have is something like the spectrum on the right. The width of the bump looks like rotation, but without an actual map, we can't be sure. (This example was taken from this paper)

That's where the second parameter comes in : these objects are relatively isolated. Our simulations have shown that when fake dark galaxies are formed, they ought to be accompanied by massive streams of material from their parent galaxies. There's no sign of any that with these guys. In fact, their high line width makes this all the more surprising. The simulations show that features with widths this high (180 km/s) ought to be the fastest-moving parts of the stream of all, and therefore the parts that disperse the most rapidly. 

So the lack of a stream, if these are fakes, is paradoxical : how can the fastest-dispersing part of the cloud be the longest-lived ? It all makes a lot more sense if we're seeing genuine rotation due to dark matter, since rotation can be stable on indefinite timescales.

And there's more. We found eight such clouds in a relatively small part of the Virgo Cluster, almost 10% of all the gas detections in that same area. Now if these were transient, unstable objects, they'd have to be forming at a very high rate indeed for us to detect them. To find one or two might be plausible*, but eight ? Nah, that's silly.

* Even then, only just. Actually the simulations found that we could essentially never produce any isolated clouds with line widths this high, because they disperse so bloody quickly.

The heart of the problem is simple. To find an HI stream with some weird velocity structure that briefly looks like a dark galaxy is quite possible. But to find them in isolation, with the rest of the stream having gone but the fastest-dispersing bit somehow still surviving... that flat-out doesn't work. Not for this many objects, at any rate.

---

What we've been lacking for all these years are two things : better statistics and better resolution. For statistics, you'll have to wait for my PhD student's first paper. Today we're dealing with resolution.

Yes, today. Yes, that's nine years after applying for time on the VLA. Stuff kept coming up, mmmkay ?

Seriously, it did. I didn't know much about how to reduce the data and I had a lot more low-hanging fruit to pick. Plus there was that whole global pandemic thing, during which I recoded 20,000 lines of Python code, amongst other things.

Step forward Robert. Having previously been head of astronomy at Arecibo (which collapsed) and a staff scientist at SOFIA (which was cancelled), Robert moved to the VLA a few years ago (and we're all praying nothing happens to that). Actually he managed a preliminary look at the data as far back as 2022, but again, stuff kept coming up, and it's taken until now to look at the thing properly.

The result (drumroll please)... they're weird objects, but they're not dark galaxies.

Betcha didn't see that coming ! Shut up, you didn't, you filthy liar.

How do we know this ? The VLA data is about four times sharper than Arecibo, and that means we can now locate the exact position of the gas much more precisely. When you're looking for especially faint optical counterparts, this matters a great deal. You can find a random starry smudge literally anywhere, so that there's always some ugly bit of stellar faff (maybe real, maybe just noise) that you can't ever be certain isn't associated with the gas. The worse the resolution, the more such faff you have to contend with. At some point you hit a wall, and getting better, sharper data is the only way to make any progress.

The VLA data reveals that in two cases, we can now identify the optical counterpart unambiguously. And they're likely neither dark galaxies nor bits of debris, but something we never expected at all.

1) AGESVC1 231

Figure 2 from the paper. The boxes show where we measured the gas content. The red and blue contours are the original Arecibo data, showing the gas at two different velocities. The inset image is a close-up of the tiny optical counterpart, just visible at the centre of the green contours (VLA) in the main image.

The Arecibo data for this is a bit of mess, but the VLA is clear : it's definitely associated with a pale blue dot. Now if I were Carl Sagan, I'd wax lyrical about how this seemingly insignificant little mote is a rich system of tens of millions of stars, perhaps host to trillions of ancient kings and emperors, a brief candle striving oblivious to the oncoming dark... but I'm not, so I won't. What I will say instead is that it's a pathetic, paltry, stupid little bugger that's bloody hard to spot.

More rigorously, while we always wondered about that tiny blue dot (I mention it right back in the earliest publications), it never seemed at all convincing. The stellar component seemed exceptionally compact, and the gas and stars weren't in a great alignment with each other (normally the positions are very close indeed). It was uniquely extreme : in other cases of small blue smudges, we could see at least some structure. With this one we could see really nothing. It just looked smooth and boring, not like a galaxy at all. And that high line width is something we'd normally expect only from very much bigger objects. It just didn't fit the bill.

But the VLA data explains all this quite nicely. Actually, as shown above, we can see it in the Arecibo data too if we'd only thought to look... but what we see is that HI has a clear tail. The densest gas appears to be associated with the stars, but the majority of the gas is actually in the stream. This is why the overall best fit to the Arecibo data gave such an offset between the gas and the stars.

So why didn't we make such a map with the Arecibo data years ago ? Largely because, as I've covered before, the existing tools to do this were a right pain to use. Remember, this is one source among more than a thousand in our full (still incomplete) catalogue... and the data really didn't suggest we had any chance of finding anything. Looking at the raw data, it's not at all obvious there's any kind of stream. Making such a map felt like a pointless and tedious exercise, surely not capable of showing anything more than we could see in the data. Maps are something you normally make only when you can already see evidence of structure and want to examine it more carefully, not to find structure in the first place.

Making maps is a classic case of something that's not actually difficult but is extremely tedious. "I'll do it this afternoon !", indeed... at least with much of the standard software.

As per another paper, I have of course since learned my lesson on this. Data visualisation and simplicity of tool use should not be considered optional extras : sometimes, they really matter !

Anyway, the new analysis also explains that high line width. Rather than being a signature of rotation, it's the result of gas being stripped out of the unusually tiny galaxy. The VLA data is less sensitive than Arecibo, but this shows us that the compact, dense gas is all associated with that blue blodge. This gives us a very nice, consistent picture of an especially small galaxy being caught in the act of having its gas very rudely shoved out. But then, it should have known better than to try and barge its way into the Virgo Cluster, the jumped-up little upstart*.

* All I'm saying is that there's room in public outreach for people who hate things and don't want to write love sonnets to their magnificent galaxies the whole time. Fuck off Carl ! Suck a lemon, Sagan ! There, that ought to get me some much longed-for hate mail...

This is still an unusual object, mind you. Not only is it especially blue, but small galaxies should lose all of their gas extremely quickly. To catch one right in the moment this happens is pretty neat, but to understand why, let's move on to the other, very different object.

2) AGESVC1 274

Figure 14 from the paper, with the VLA detection as the green contours and a close-up of the optical counterpart in the inset image.

The second object is very different from 231. Of our eight clouds, six have those peculiar high line widths, and this is one of those which doesn't : it's only about 30 km/s, a real tiddler. And its optical counterpart is extremely fuzzy and diffuse. It looks for all the world like a pretty normal dwarf galaxy, just extremely faint. 

To be fair, we actually suggested this same optical counterpart in a 2016 paper based on some deep optical data. As with AGESVC1 231's pale boring blue dot, it didn't seem like a brilliant candidate – again you can find this sort of scrappy starlight all over the place – but the VLA makes it unambiguous. This stupid fuzzy blob really is the optical counterpart after all.

I mean, two candidates I suggested turned out to be right. On the other hand, of course I also said these were shitty candidates and they were more likely to be optically dark, so more fool me.

Interestingly, this same object was re-suggested as a candidate in a paper by Dey et al. last year. They find an optical redshift* of the stars that agrees with that from the HI, which makes this association rock solid. What's more unusual is that they propose this to be a so-called "blue blob", stars which aren't forming in dark matter halos at all. They have a catalogue of 30-odd of these diffuse, blue structures which they interpret as being ram pressure dwarfs : objects which form in the tails of gas when it's stripped out of galaxies. 

* More strictly, they get the redshift from the ionised gas which emits at optical wavelengths. They don't directly measure the redshift of the stars themselves.

This is an exciting idea, essentially introducing a whole new type of object to investigate – as different from dark galaxies as dark galaxies are from tidal debris. We're talking a fundamental rethink here, not a tweak to our pre-existing ideas. And that's really nifty.

Very little indeed is known about these blue blobs, but so far they're unique to Virgo. They appear to be chemically enriched, which strongly suggests they formed within larger galaxies rather than as galaxies in their own right. Not everyone agrees, although personally I think it's extremely credible and some of the most interesting work that's been done in Virgo in years. 

Another key point is that these objects appear to lack dark matter, which seems to be the case for this one. Given that Dey's measurements show that this particular object is chemically enriched, was independently identified as a BB candidate by its images alone, and appears to lack dark matter, this all paints a nice coherent picture. Such objects could also reach large distances from their parent galaxies without the giant streams expected in the tidal debris scenario.

Wait, wait wait.... on, lacks dark matter ? I thought we were talking about dark matter dominated galaxies !

Indeed so. But of our eight clouds, only six had high line widths. Two of them, including this one, actually had widths which are if anything narrower than expected, so much so that it points to a possible deficit of dark matter rather than an excess.

There's just one problem in this case, though it isn't fatal. Unlike with AGESVC1 231, all the gas here appears to be compact, with no evidence of any stripped component. So if its stars really did form within a stripped tail, all of that appears to have dissipated. This makes it very hard to tell if we're really seeing such an exotic object or just a particularly faint but normal dwarf galaxy. 

Fortunately, there's one more object which has an optical counterpart.

3) AGESVC1 266

Figure 12 from the paper. The detection is quite clear in the VLA spectrum but only marginal visible in the map, hence the wobbly contours (dashed green are negative). White labels show possible optical counterparts : BSG is not Battlestar Galactica but the Brightest SDSS Galaxy; T2016 is something I identified previously; D2025 is that found by Dey.

Oi ! You said two objects with optical counterparts, what're you playing at ?

Not quite. I said we could identify two optical counterparts thanks to the VLA data. This one was identified completely independently (by Dey again), and to be honest, the VLA doesn't help here. Dey has that advantage of getting optical redshift data which matches the HI... without this, I would never in a million years believe their optical counterpart. 

I mean, look at it. It's pathetic. As galaxies go, it's utterly shite. It's a total miserable failure.

And that, of course, is what makes it so interesting. In both the previous cases, the mass of gas is a few times more than the stars, maybe approaching a factor of ten. That's pretty extreme... but in this case the ratio is more than a thousand. That's into crazy territory*.

* There are caveats to figures like this. Estimating the stellar mass becomes extremely difficult for things this faint, but nevertheless, it's clearly exceptionally faint – the exact numerical value isn't all that important.

A mass to light ratio of over a thousand, you say ? Might as well stick a handkerchief on your head and two pencils up your nose.

This one seems very much more convincing as a blue blob / ram pressure dwarf. Here the HI and the optical appear offset and the gas is diffuse, not compact. We may be witnessing the birth of a long-lived blue blob (a whole new class of stellar structure, as Jones and Dey and others have been investigating), or possibly just a brief flicker of star formation before the whole thing dissolves into undetectability. So birth or death ? At the moment, we just don't know.

---

What does it all mean ?

It's a bit of a mess. We have eight clouds in total and we observed six. Maybe one day we'll try for the other two, but right now we know nothing more about them. So of those six :

• One with a high line width turns out to have a tiny, compact optical counterpart and a great big stream. This looks convincingly like a stripping galaxy.
• Two have extremely faint, fuzzy optical counterparts. One of these looks like a good candidate for being a ram pressure dwarf, while the other is plausible but uncertain.
• One more was just about detected, but still has no clear optical counterpart. It doesn't look like it's rotating, but the detection is weak so we can't be confident about this.

The other two were not detected. That's expected if they lack any compact gas, which would mean they're similar to the detected-but-dark case of the last bullet point. But we should also bear in mind just how faint some of these optical counterparts can be : just because we haven't found one doesn't mean they don't exist. We would never have spotted the counterpart of AGESVC1 266 ourselves – only Dey's optical redshift allowed that. So these undetected objects could still be optically dark, but we just don't know.

Even so, we now have enough data that we can finally start to say what's going on.

Wait, wait...

I almost forgot about the legitimate clickbait version !

Twenty years ago they found dark clouds in the depths of space. We finally know what's going on.

LISTEN UP JOURNALISTS AND LISTEN GOOD. You get to say, "we finally know" if it takes nearly two decades and the answer is pretty clear. You do not get to say it if somebody comes up with a vague idea a week after a slightly odd discovery. Got that ? Good. I'm glad we had this talk.

Ahem.

Anyway, the dark galaxies hypothesis used to seem tenable because we couldn't really explain their high line widths in any other way. But we don't see any ordered motions in any of these objects, and one shows a clear stream. That means the kinematics likely has nothing much to do with the gravitational field at all, so the dark matter interpretation – which must be said was the most astrophysically exciting one – probably deserves to be chucked out the window.

I know the ejected guy is normally supposed to come up with a sensible idea, but dark galaxies did used to seem like the best explanation. Anyway I don't care, so shut up.

What looks almost certain is that we were being deceived by the apparent similarity of the clouds. It looked as though they all had about the same mass, line width, similar levels of isolation, and lack of optical counterpart. We now know that's not entirely true : the line widths vary (in some cases our new measurements reduce this compared to our original estimates) while some have optical counterparts – which themselves have varied properties. 

That means the number which we can attribute to being some form of dark debris is not eight, but maybe five at the very most... and likely less, as two weren't even observed. Couple that with the variation in properties and the debris scenario begins to look a lot more plausible. Instead of saying "they're all dark galaxies" or "they're all debris", it now looks more like some are extreme galaxies, some are new kinds of stellar structure, and some are debris. No individual explanation has too much of a burden to bear any more.

These objects are still weird though. For one thing, how come compact objects like AGESVC1 231 seem pretty good at retaining their gas even when larger ones like VCC 1964 have it all removed wholesale ? Why does it get fully displaced in one case but elongated in others ? More fundamentally, what governs when the HI gas is destroyed as it's removed from giant galaxies but can apparently survive and prosper in the harsh environment outside its friendly parent galaxy ? That's a bit weird.

And perhaps of more immediate interest are those blue blobs. In a spectacular pivot that would shame a politician, I might well find myself switching more to objects which lack dark matter rather than having too much. The objects in this paper are not, it turns out, the only such blue blobs in our data which show dynamics like this. Given that VCC 1964 shows a similar lack of dark matter, it's tempting to wonder if there's a connection between BBs and Ultra Diffuse Galaxies. Right now, all I can say is that some of our other BBs shown even more extreme gas ratios and dark matter deficits than the objects we examined here... the times they are a-changing, but the future, it seems, remains as dark as ever.