Oh Socialist Utopia, Where Art Thou ?

This is a piece I haven't been wanting to write for a very long time. Events, though, have forced my hand.

I've covered Keir Starmer twice on this blog, once in February 2024 and once shortly after the electoral landslide that swept Labour back to power. It's fair to say I've presented the man with a glowing, even gushing, review, and over on Decoherency I've presented similar arguments and defended Labour after they suffered quite the barrage of largely idiotic attacks.

Alas, the happy period in which we could mostly ignore politics lasted about six months. Fourteen years of Tory garbage and we got six months when things were sensible. For fuck's sake.

Now this is not like the time when I reversed my opinion on Jeremy Corbyn. No, rather than ranting against Starmer himself, here I want to largely rant against the feckless stupidity of the Great British public. So let me be absolutely clear about where I stand.

YER ALL A BUNCH OF FUCKING MORONS !

Which is not to say that Starmer himself is blameless, because he certainly isn't. But having seen people on Last Week Tonight describe Starmer as a "prick" and seen his popularity plummet to lower levels than Boris Johnson (!), I have to say that the British are a bunch of contemptibly stupid fuckwits.

Ladies and gentlemen, I give you... the electorate !

Look, I promise I am going to go through the points where I disagree with Starmer, and I do think he's done some incredibly stupid things. I promise I'm coming to that. But first, he is/was HANDS DOWN the best Prime Minister we had in twenty years. Not even close. Yet he's engendered a level of bitterness that would fuel the wet dreams of a true hate figure like Farage, and... why ? 

For not being good enough, apparently.

It's absolutely fucking exhausting to have to deal with this perpetual bullshit. What the absolute flying fuck is wrong with you ? What in God's name will it take to make you people fucking learn something ?

Jesus H. Christ.

As you might guess, I don't intend this to be an especially thoughtful post. If you came here hoping for another repeat of the five-part rebuttal of Epictetus, I'm afraid you're going to be disappointed. I don't much care for polemics, but the hell with it : sometimes people need not careful reasoning, but a damn good verbal slap in the cerebral cortex.

Haters Gonna Hate

My basic question is this. What is it about Keir Starmer, dear proverbial reader, that makes you hate him so ? Do you even know ? I doubt it. 

Not that all of you don't have an answer, mind you : plenty will point to things like (supposedly) transphobic policies, "hard line" immigration stances, or perhaps the Mandelson debacle or any number of various other messaging missteps. But I argue that, in comparison to the previous 14 years of Tory missrule, Starmer's government has done absolutely sod all to deserve anything like the level of contempt with which the majority of the public now seem to view him personally.

Again, LWT features British morons describing their towns as "shitholes" without any explanation as to what it is they think makes them shitholes, let alone what it is they actually want their town to be/do in order for them not to be shitholes. If some of you do have answers, then an awful lot of you certainly don't.

The long-standing appeal of Starmer for me was a lack of magical promises. He never said he would transform Britain overnight, because the recovery the country needs simply can't be done quickly. He never gave the impression of being a miracle worker in the way that Johnson or Truss ever suggested. He made it quite clear that the process would be long and difficult. And lo ! The process is indeed long and difficult, made all the more so by the fascist orange cunt running the White House. Getting disappointed that the recovery isn't going very quickly is just stupid. Nobody was ever promised that it would be easy or fast.

Which leaves me bewildered as to what it is everyone is actually upset by. Could it be the litany of so-called "scandals" that have beset the party since it came to office ? I rather doubt it. Consider each of these :

• Expenses and hospitality. This early "scandal" consisted of Starmer and others claiming expenses for things they were perfectly entitled to, and accepting and declaring donations with full transparency for concerts and other events. It's the most pathetic of stories, but it was at this point – not when the winter fuel story started as the popular narrative has it – that Starmer's approval took a nosedive. It never recovered. I find this absolutely exasperating because nobody ever gave a shit when Boris Johnson was handed vast amounts of money by a still-undisclosed donor or turned Downing Street into a tasteless pile of tat for his idiotic wife.
• Rayner's taxes. Even less of a scandal. The Tories wasted billions – tens of billions, all lost now – on useless PPE in the pandemic while Matt Hancock cavorted with an underling, but the fascist rags couldn't handle a working class woman accidentally misreporting a few grand in taxes. It's disgusting. She never should have resigned.
• Mandelson. I'll grant that this one is different : Mandelson is an evil bastard through and through. But the press acted liked Starmer must have been uniquely stupid not to know about the (very real and incredibly serious) issues that subsequently came to light, despite never having tried to do some, oh, I don't know, investigative journalism of their own. They also downplayed the whole aspect of actually imparting state secrets and concentrated heavily on the Epstein angle. Look, this one isn't a non-story, but the way the press span it was downright dumb. Mandelson, as a duplicitous, evil, manipulative, highly intelligent twat, was a downright inspired choice to deal with the duplicitous, evil, manipulative, thick as shit administration across the pond.

Policy Presentations

So if it's not the "scandals" which are the source of the anti-Starmer nuttery, could it be instead the policies Labour has enacted, or those they've failed to deliver on ? This is more credible, but only slightly :

• Winter fuel allowance. Total non-story. This should always have been means tested but the press refused to consider it could have had any other consequence except millions of pensioners freezing to death in their tiny hovels, dying naked and alone with the name "Staaaarmerrrr !" escaping from their frozen lips as they expired. The idea that this might be good for budgeting was never discussed. Ever.
• Treatment of climate protestors. More of an issue for Guardian readers than anyone else, but some people attempted to paint (pun intended) Starmer as personally responsible for the handful of rather harsh sentences for climate protestors as being emblematic of a police state, or something, while racist rioters were allowed to get off Scott-free and given a big bag of crumpets, or whatever. Utter garbage, pure culture war nonsense. This is the effect of sensationalism gone mad, when instead of legitimately criticising the judiciary you see any minor error as "evidence" that the ruling party is worse than the Nazis.
• Response to Gaza. Now yes, my response would not have been much like the government's response. But I accept that the need to balance morals and diplomacy isn't easy, especially given the Labour Party's disturbing recent indulgence in anti-Semitism under Corbyn. I don't agree with their response to pro-Palestine demonstrations and I'm totally unconvinced by their explanation as to why certain participants in these are somehow terrorists. Do I hate them for it ? No. I think they're being stupid, but not malevolent.
• Farmers. Fuck off the lot of you. You've been saying that farming in the UK is on the verge of collapse for at least the last thirty years and it's wearing a bit thin, no matter how hard you work and how essential you are. The inheritance tax was riddled with exemptions and easy to avoid by gifting away the farms before dying. Not difficult. But no, just like with pensioners, no interpretation other than, "oh no, my poor little hovel will crumble into dirt because now I have to pay three million quid because I own one too many tractors" was ever allowed in the media.
• Online safety act. Yeah, I don't like this one. The government has come up with some mightily stupid ideas, not least of which is age verification and – for some reason, God knows why – banning step-sibling porn (but not actually banning actual step-sibling sex, because <snarkasm> that makes sense </snarkasm>). Does this mean they did a very stupid thing ? Absolutely. Worthy of hatred ? Come off it. Hatred should be reserved for racist buggers like Johnson and deranged loons like Truss, not well-meaning bad decisions like these. And incidentally, not once have I seen the news ever suggesting the bleedin' obvious : encourage parents to monitor and regulate their children's screen time. There are already tools to do this. The need for the government to act and thus treat us all like children was routinely assumed as the only option : the idea that the parents could have anything to do with it was never mentioned. 

It winds me up something mental, it really does. Under the Tories we had austerity and the evisceration of our public institutions, and we let them get away with it because they were apparently competent. Yeah, right, whatever.

I shall say it again. You. Fucking. Morons.

And then I must necessarily rattle through the positive sides of the Starmer government : nationalising energy, nationalising rail, support for Ukraine, stabilising the economy in the face of massive international challenges, playing Trump like a fiddle for a trade deal (albeit that didn't stick, but show me anyone who's done better against that cretinous mass of putrid bile), the employment rights act, the renter's rights act, a national wealth fund, reducing migration (not my preference but it's what most people want for some reason), cutting waiting lists on the NHS, improved public sector pay, vastly better relations with the EU, and free breakfasts for schoolchildren.

Are you seriously going to look me in the eye and say that the Starmer government hasn't been, on balance, the best thing to happen to the UK in decades ?

Come off it. That's stupid, and you know it.

A Solution : Mixed Messages

Could it then be messaging ? This area I think is by far the most promising, but also the one where the implications for the British public are the most damning.

For this, it's worth reading Tony Blair's recent essay in full*. Yes, the man is profoundly amoral, and his stance that we should have let the American's use our bases against Iran is crazy both pragmatically and morally. The man acts as though moral values are something nice to have, not actually of practical benefit, whereas this case is a clear demonstration that they are not. Support America's despot in their pointless and economically wrist-slitting war to distract from the Epstein files ? That's not a sane choice, Tony. Likewise, his belief that we should prioritise cheaper rather than cleaner energy is moronic : the climate crisis isn't something abstract that you can pretend isn't happening. 

*Blair also gives some suggestions regarding actual policy and a way forwards, some of which I agree with, some of which I react strongly against. A much better set of arguments on that front comes in Starmer's own response.

But politically, only the most foolish would disregard a three-times election winner. The root of it seems to be, in his assessment, that Starmer didn't pick a side. Tribalism may have its problems, but it's also a source of strength and a clear heuristic : when people think they know what you're about, you're predictable, and they can use that to judge if they think you're basically decent or not. When they don't, they can't trust you. In Unruly, David Mitchell stresses the importance in ruthlessness for medieval kings in eliminating the fatality of doubt among the populace, and there's certainly more than an echo of that here, at least analogously.

I would add that perhaps worse, though, was that while Starmer could have tried to brand his own approach, he just never did that at all. He just thought that people would essentially do this for themselves, and didn't care how they chose to interpret it. To an extent, it wasn't so much loosing the narrative as never crafting one in the first place... "mission-led government" doesn't strike a chord with people. Wanting this to be the "best country in the world" is a platitude, an abstraction, not something nearly concrete enough that people can tell what you're about.

Blair's second good point is that changing leader won't help without having a coherent strategy. I disagree that Labour doesn't have one, but I violently agree that they haven't marketed it. Its approach of almost pure pragmatism does have a whiff of making-it-up-as-they-go-along more than following a clear path. I agree with Blair when he says that to seize the centre, you must first devise policy and only then employ politics. That is, decide on what your goals are first, and then formulate how you're going to persuade people to go along with them. And these have to be lower-level than the "let's make everything better" stratagems of Starmer. You need to define clearly what it is you want to accomplish, fix, or improve, and only then go about deciding how you're going to do that.

His third point is that there is a real need to be radical. Starmer did have a vision for a radical centrism, ironically by decentralising government*, but that all came to nothing. On other policies he never even really offered that much you could call "radical". Nationalising rail and energy is definitely a good thing, but while the piecemeal approach has many advantages, it's also slow. Blair stresses also the importance of unconventional thinking, noting that while this may indeed result in actual disaster**, it also connects with people at a visceral level. The "stabilise" approach was perfectly sensible for the first year, but after that, incremental approaches were just not going to cut it.

* A federal structure for the UK, greater powers for local councils, replacing the Lords with a Council of Nations... all these were on the cards and promised within the first term. There was little enough sign of any of them actually happening though.
** Likening Trump to a car crashing into a brick wall, he rightly notes that the problem is that "radical people aren't sensible, and sensible people aren't radical".

So Starmer needed something big, bold, and obvious : a four day week, universal basic income, rejoining the EU, a wealth tax for the wealthiest, something of that magnitude. Not necessarily any of those specifically, and certainly not all them, but of that order, and economic rather than in terms of government structure. Reforming or replacing the House of Lords might be popular among some segments, but won't be itself directly affect anyone outside the Westminster bubble.

What Went Wrong

The most disappointing thing about all this was just how well-prepared the Starmer government was before going into office. They had many meetings with business leaders and institutes well before the election; by the time of the event, everyone knew their major policies. They had successfully eliminated the doubt that plagued medieval monarchs, made themselves presentable, serious, and safe enough for Tory voters to ignore rather than cosplaying Reform. Their whole shadow cabinet exuded dull competence after the chaotic farce of the Johnson and other previous administrations.

I don't think I can say too much about why this all went wrong, but I do think I can at least identify what went wrong. Starmer learned many of Blair's key lessons but not how to implement them. He knew how to say no to his own side, but not when or how (like telling a child they can't have a pony versus telling them they can't have a My Little Pony play set). He knew that change must be felt, but not how to make it felt or how to have it associated with his government. He knew that you have to meet people where they are, but not how to change their minds when you need to... opting to appeal to their sentiments rather than ever challenge them very much. And he knew that people judge policies in aggregate, but not how to get them to judge them at all... again, failing to paint himself in any sort of ideological colours.

Perhaps at least one root cause is that Starmer is too experienced at dealing with the dregs of humanity. Having dealt with rapists, murders, terrorists, Boris Johnson and Lizz Truss, he appears to have become thick skinned to the point of being tin eared. He could not acknowledge when messaging had a serious problem, preferring to go to foreign summits with other leaders than engage with the rough-and-tumble of the British electorate. I sympathise, but the Prime Minister of the UK needs to be seen to be dealing with UK issues. Starmer is right when he says that many international issues do affect us directly, but this doesn't acknowledge that this isn't how people feel about them. These issues needed to be left to the foreign secretary (and probably having somebody who wasn't David Lammy would have been a better choice there). Messaging really matters, because if you don't craft a narrative, someone else will do it for you.

And finally, he didn't stick to his guns. There were just too many u-turns and climbdowns, raising that hellish spectre of doubt : since he won't stick to his guns, say dissenters, it appears that if we stick to ours, we'll win. He didn't properly understand when to fight and when not to fight, which resulted in a messy, spitballing approach of trying hopelessly to please everyone. He could be ruthless in dealing with his party-internal opponents, sometimes too much so, but not in offering moral clarity.

Aftermath

So yes, as promised, I've set out what Starmer got wrong. I will say again that there are some policies I also disagree with, particularly transphobic judgements about bathrooms and the bizarrely prudish online safety act. The internet should be for adults; it's children who should bear the difficulties of access, not us, and parents who should be responsible for this, not the government. That one really did feel nanny state.

And yet the government has done an awful lot of good. To hate them for their entirely predictable missteps and utterly minor misdemeanours is profoundly lacking in sense. Frankly, if we can't be grateful for the improvements under Starmer compared to the last decade and a half under the Tories, then we'll never be happy – ever. We've been sold down the river on expectations of absolute perfection, and nothing less, it seems, will every satisfy us : anything less than a miracle is worse than shit. The perfect is indeed the enemy of the good, to the extent that if something is slightly flawed, people seem to want to vote for the exact fucking opposite instead. 

Indeed, it often feels as though the public view someone being slightly incompetent as worse than being deliberately malevolent. Nothing much else can possible explain the appeal of Reform or their brainless cousins in Restore.

For a much better sort of milkshake man, see this.

There is something rotten in the state of Britain. I cannot fully address what it is, but I will venture two major contributing factors. First, nobody seems to have a clue how our economy works. Nobody seems to realise what's really generating the money or where it's going; Blair is convinced AI will affect changes (literally) on the scale of the Industrial Revolution, but as to the basic questions of 1) where is all the money going ? and 2) why was our economy so much better in 2005 ? nobody seems to have a damn clue.

The second is the media. I love this piece on grumpy old men, and its main claim ties in to the British press perfectly. By selecting and sensationalising  all the negative stories, they've become an engine for conditioned responses. The voters are trained to see the bad in everything and the good in nothing, to see politicians as all self-serving hypocrites and worse. They're deft at finding the clouds is every silver lining and assuming that the negative interpretations are the only valid ones, with any positive interpretations being dismissed out of hand as tribal propaganda. We aren't allowed to be happy with anything. We don't appreciate for a moment just how much worse things could be.

Even this, though, I think doesn't fully explain what's happened. There is something rotten also in the psyche of the British people, something that prevents us from seeing the blindingly obvious. When we react with "my town's a shithole" for no good reason, then it makes perfect sense to say, "Starmer is a prick" for no good reason, and by the same token to declare that "Andy Burnham would be much better" even though he's at most 10% more likeable than Starmer.  

I don't claim to know what exactly has come over my fellow countrymen. I'm acutely aware that "they're all just bad people" is an insanely common and idiotic fallacy that explains nothing. But this is the reason why I said that "mixed messages" as a solution for why the Starmer government has failed is such a damning verdict for the public : that they cannot see how minor the problems really are, that they need to be spoon-fed to think for themselves, does not bode well at all.

Let's hope Andy Burnham can figure it out. If he can't, Brexit and austerity will be the least of our problems. And what a cheery thought that is.

The End Of Darkness

My goodness me, another paper ! What is this, a science blog or something ?

This one is by my long-term partner in crime, the inestimable Robert Minchin. After the last few papers – one on a single weird galaxy and two on source extraction – this is a return to my favourite topic : dark galaxies.

And not just any dark galaxies...

Well, the joke works if you're British, at any rate.

What I meant to say is that these are the objects I've spent most of my career on. If you haven't read the link, a dark galaxy is just a galaxy which has some gas, maybe a very few stars (but preferably none at all), all embedded in the classic dark matter halo that we all know and love : the very thing that distinguishes galaxies from all the other crap floating around in the majesty of the cosmos.

Look, I've been through this dozens of times. I'm not gonna do it again. Go and read the link, I'll wait.

Go ! Read up and report back. Off you pop.

Okay, you ought to bloody well know about these things in general by now. But I'll forgive you if you can't quite remember why I like these particular candidates so much. Mainly because I'm extremely biased and narcissistic... that is, they turned up in the data I was analysing during my PhD, and were the main highlight of my very first papers*. I've been working on these for the best part of two decades, which is a scary thought if nothing else.

* It's all "me, me, me" around here.

Yes, yes, but apart from keeping me employed, what's so fascinating about them ? No-one's going to keep paying anyone to study genuinely boring objects, after all.

Well, it's a combination of two factors. The first is that they have a high line width. That is, we can see that parts of them are moving at very different speeds along our line of sight (towards or away from us). This is exactly what we see with normal, rotating galaxies, and it's normally a pretty good signature that they have a lot of dark matter. 

But this, we've learned, isn't decisive. We didn't have good spatial resolution of the clouds, so we couldn't be sure that we were really seeing rotation. Simulations have shown that when galaxies interact with each other, the debris that gets torn off can (briefly) exactly mimic this apparent signature of rotation, so it was possible we were just being fooled. And without rotation, there's no reason to think there's any dark matter.

Really good data would let us map the motion very precisely, like on the left. Here we can see that one side is moving differently to the other, with a smooth gradient across the object : a classic signature of stable spin. But for these objects, all we have is something like the spectrum on the right. The width of the bump looks like rotation, but without an actual map, we can't be sure. (This example was taken from this paper)

That's where the second parameter comes in : these objects are relatively isolated. Our simulations have shown that when fake dark galaxies are formed, they ought to be accompanied by massive streams of material from their parent galaxies. There's no sign of any that with these guys. In fact, their high line width makes this all the more surprising. The simulations show that features with widths this high (180 km/s) ought to be the fastest-moving parts of the stream of all, and therefore the parts that disperse the most rapidly. 

So the lack of a stream, if these are fakes, is paradoxical : how can the fastest-dispersing part of the cloud be the longest-lived ? It all makes a lot more sense if we're seeing genuine rotation due to dark matter, since rotation can be stable on indefinite timescales.

And there's more. We found eight such clouds in a relatively small part of the Virgo Cluster, almost 10% of all the gas detections in that same area. Now if these were transient, unstable objects, they'd have to be forming at a very high rate indeed for us to detect them. To find one or two might be plausible*, but eight ? Nah, that's silly.

* Even then, only just. Actually the simulations found that we could essentially never produce any isolated clouds with line widths this high, because they disperse so bloody quickly.

The heart of the problem is simple. To find an HI stream with some weird velocity structure that briefly looks like a dark galaxy is quite possible. But to find them in isolation, with the rest of the stream having gone but the fastest-dispersing bit somehow still surviving... that flat-out doesn't work. Not for this many objects, at any rate.

---

What we've been lacking for all these years are two things : better statistics and better resolution. For statistics, you'll have to wait for my PhD student's first paper. Today we're dealing with resolution.

Yes, today. Yes, that's nine years after applying for time on the VLA. Stuff kept coming up, mmmkay ?

Seriously, it did. I didn't know much about how to reduce the data and I had a lot more low-hanging fruit to pick. Plus there was that whole global pandemic thing, during which I recoded 20,000 lines of Python code, amongst other things.

Step forward Robert. Having previously been head of astronomy at Arecibo (which collapsed) and a staff scientist at SOFIA (which was cancelled), Robert moved to the VLA a few years ago (and we're all praying nothing happens to that). Actually he managed a preliminary look at the data as far back as 2022, but again, stuff kept coming up, and it's taken until now to look at the thing properly.

The result (drumroll please)... they're weird objects, but they're not dark galaxies.

Betcha didn't see that coming ! Shut up, you didn't, you filthy liar.

How do we know this ? The VLA data is about four times sharper than Arecibo, and that means we can now locate the exact position of the gas much more precisely. When you're looking for especially faint optical counterparts, this matters a great deal. You can find a random starry smudge literally anywhere, so that there's always some ugly bit of stellar faff (maybe real, maybe just noise) that you can't ever be certain isn't associated with the gas. The worse the resolution, the more such faff you have to contend with. At some point you hit a wall, and getting better, sharper data is the only way to make any progress.

The VLA data reveals that in two cases, we can now identify the optical counterpart unambiguously. And they're likely neither dark galaxies nor bits of debris, but something we never expected at all.

1) AGESVC1 231

Figure 2 from the paper. The boxes show where we measured the gas content. The red and blue contours are the original Arecibo data, showing the gas at two different velocities. The inset image is a close-up of the tiny optical counterpart, just visible at the centre of the green contours (VLA) in the main image.

The Arecibo data for this is a bit of mess, but the VLA is clear : it's definitely associated with a pale blue dot. Now if I were Carl Sagan, I'd wax lyrical about how this seemingly insignificant little mote is a rich system of tens of millions of stars, perhaps host to trillions of ancient kings and emperors, a brief candle striving oblivious to the oncoming dark... but I'm not, so I won't. What I will say instead is that it's a pathetic, paltry, stupid little bugger that's bloody hard to spot.

More rigorously, while we always wondered about that tiny blue dot (I mention it right back in the earliest publications), it never seemed at all convincing. The stellar component seemed exceptionally compact, and the gas and stars weren't in a great alignment with each other (normally the positions are very close indeed). It was uniquely extreme : in other cases of small blue smudges, we could see at least some structure. With this one we could see really nothing. It just looked smooth and boring, not like a galaxy at all. And that high line width is something we'd normally expect only from very much bigger objects. It just didn't fit the bill.

But the VLA data explains all this quite nicely. Actually, as shown above, we can see it in the Arecibo data too if we'd only thought to look... but what we see is that HI has a clear tail. The densest gas appears to be associated with the stars, but the majority of the gas is actually in the stream. This is why the overall best fit to the Arecibo data gave such an offset between the gas and the stars.

So why didn't we make such a map with the Arecibo data years ago ? Largely because, as I've covered before, the existing tools to do this were a right pain to use. Remember, this is one source among more than a thousand in our full (still incomplete) catalogue... and the data really didn't suggest we had any chance of finding anything. Looking at the raw data, it's not at all obvious there's any kind of stream. Making such a map felt like a pointless and tedious exercise, surely not capable of showing anything more than we could see in the data. Maps are something you normally make only when you can already see evidence of structure and want to examine it more carefully, not to find structure in the first place.

Making maps is a classic case of something that's not actually difficult but is extremely tedious. "I'll do it this afternoon !", indeed... at least with much of the standard software.

As per another paper, I have of course since learned my lesson on this. Data visualisation and simplicity of tool use should not be considered optional extras : sometimes, they really matter !

Anyway, the new analysis also explains that high line width. Rather than being a signature of rotation, it's the result of gas being stripped out of the unusually tiny galaxy. The VLA data is less sensitive than Arecibo, but this shows us that the compact, dense gas is all associated with that blue blodge. This gives us a very nice, consistent picture of an especially small galaxy being caught in the act of having its gas very rudely shoved out. But then, it should have known better than to try and barge its way into the Virgo Cluster, the jumped-up little upstart*.

* All I'm saying is that there's room in public outreach for people who hate things and don't want to write love sonnets to their magnificent galaxies the whole time. Fuck off Carl ! Suck a lemon, Sagan ! There, that ought to get me some much longed-for hate mail...

This is still an unusual object, mind you. Not only is it especially blue, but small galaxies should lose all of their gas extremely quickly. To catch one right in the moment this happens is pretty neat, but to understand why, let's move on to the other, very different object.

2) AGESVC1 274

Figure 14 from the paper, with the VLA detection as the green contours and a close-up of the optical counterpart in the inset image.

The second object is very different from 231. Of our eight clouds, six have those peculiar high line widths, and this is one of those which doesn't : it's only about 30 km/s, a real tiddler. And its optical counterpart is extremely fuzzy and diffuse. It looks for all the world like a pretty normal dwarf galaxy, just extremely faint. 

To be fair, we actually suggested this same optical counterpart in a 2016 paper based on some deep optical data. As with AGESVC1 231's pale boring blue dot, it didn't seem like a brilliant candidate – again you can find this sort of scrappy starlight all over the place – but the VLA makes it unambiguous. This stupid fuzzy blob really is the optical counterpart after all.

I mean, two candidates I suggested turned out to be right. On the other hand, of course I also said these were shitty candidates and they were more likely to be optically dark, so more fool me.

Interestingly, this same object was re-suggested as a candidate in a paper by Dey et al. last year. They find an optical redshift* of the stars that agrees with that from the HI, which makes this association rock solid. What's more unusual is that they propose this to be a so-called "blue blob", stars which aren't forming in dark matter halos at all. They have a catalogue of 30-odd of these diffuse, blue structures which they interpret as being ram pressure dwarfs : objects which form in the tails of gas when it's stripped out of galaxies. 

* More strictly, they get the redshift from the ionised gas which emits at optical wavelengths. They don't directly measure the redshift of the stars themselves.

This is an exciting idea, essentially introducing a whole new type of object to investigate – as different from dark galaxies as dark galaxies are from tidal debris. We're talking a fundamental rethink here, not a tweak to our pre-existing ideas. And that's really nifty.

Very little indeed is known about these blue blobs, but so far they're unique to Virgo. They appear to be chemically enriched, which strongly suggests they formed within larger galaxies rather than as galaxies in their own right. Not everyone agrees, although personally I think it's extremely credible and some of the most interesting work that's been done in Virgo in years. 

Another key point is that these objects appear to lack dark matter, which seems to be the case for this one. Given that Dey's measurements show that this particular object is chemically enriched, was independently identified as a BB candidate by its images alone, and appears to lack dark matter, this all paints a nice coherent picture. Such objects could also reach large distances from their parent galaxies without the giant streams expected in the tidal debris scenario.

Wait, wait wait.... on, lacks dark matter ? I thought we were talking about dark matter dominated galaxies !

Indeed so. But of our eight clouds, only six had high line widths. Two of them, including this one, actually had widths which are if anything narrower than expected, so much so that it points to a possible deficit of dark matter rather than an excess.

There's just one problem in this case, though it isn't fatal. Unlike with AGESVC1 231, all the gas here appears to be compact, with no evidence of any stripped component. So if its stars really did form within a stripped tail, all of that appears to have dissipated. This makes it very hard to tell if we're really seeing such an exotic object or just a particularly faint but normal dwarf galaxy. 

Fortunately, there's one more object which has an optical counterpart.

3) AGESVC1 266

Figure 12 from the paper. The detection is quite clear in the VLA spectrum but only marginal visible in the map, hence the wobbly contours (dashed green are negative). White labels show possible optical counterparts : BSG is not Battlestar Galactica but the Brightest SDSS Galaxy; T2016 is something I identified previously; D2025 is that found by Dey.

Oi ! You said two objects with optical counterparts, what're you playing at ?

Not quite. I said we could identify two optical counterparts thanks to the VLA data. This one was identified completely independently (by Dey again), and to be honest, the VLA doesn't help here. Dey has that advantage of getting optical redshift data which matches the HI... without this, I would never in a million years believe their optical counterpart. 

I mean, look at it. It's pathetic. As galaxies go, it's utterly shite. It's a total miserable failure.

And that, of course, is what makes it so interesting. In both the previous cases, the mass of gas is a few times more than the stars, maybe approaching a factor of ten. That's pretty extreme... but in this case the ratio is more than a thousand. That's into crazy territory*.

* There are caveats to figures like this. Estimating the stellar mass becomes extremely difficult for things this faint, but nevertheless, it's clearly exceptionally faint – the exact numerical value isn't all that important.

A mass to light ratio of over a thousand, you say ? Might as well stick a handkerchief on your head and two pencils up your nose.

This one seems very much more convincing as a blue blob / ram pressure dwarf. Here the HI and the optical appear offset and the gas is diffuse, not compact. We may be witnessing the birth of a long-lived blue blob (a whole new class of stellar structure, as Jones and Dey and others have been investigating), or possibly just a brief flicker of star formation before the whole thing dissolves into undetectability. So birth or death ? At the moment, we just don't know.

---

What does it all mean ?

It's a bit of a mess. We have eight clouds in total and we observed six. Maybe one day we'll try for the other two, but right now we know nothing more about them. So of those six :

• One with a high line width turns out to have a tiny, compact optical counterpart and a great big stream. This looks convincingly like a stripping galaxy.
• Two have extremely faint, fuzzy optical counterparts. One of these looks like a good candidate for being a ram pressure dwarf, while the other is plausible but uncertain.
• One more was just about detected, but still has no clear optical counterpart. It doesn't look like it's rotating, but the detection is weak so we can't be confident about this.

The other two were not detected. That's expected if they lack any compact gas, which would mean they're similar to the detected-but-dark case of the last bullet point. But we should also bear in mind just how faint some of these optical counterparts can be : just because we haven't found one doesn't mean they don't exist. We would never have spotted the counterpart of AGESVC1 266 ourselves – only Dey's optical redshift allowed that. So these undetected objects could still be optically dark, but we just don't know.

Even so, we now have enough data that we can finally start to say what's going on.

Wait, wait...

I almost forgot about the legitimate clickbait version !

Twenty years ago they found dark clouds in the depths of space. We finally know what's going on.

LISTEN UP JOURNALISTS AND LISTEN GOOD. You get to say, "we finally know" if it takes nearly two decades and the answer is pretty clear. You do not get to say it if somebody comes up with a vague idea a week after a slightly odd discovery. Got that ? Good. I'm glad we had this talk.

Ahem.

Anyway, the dark galaxies hypothesis used to seem tenable because we couldn't really explain their high line widths in any other way. But we don't see any ordered motions in any of these objects, and one shows a clear stream. That means the kinematics likely has nothing much to do with the gravitational field at all, so the dark matter interpretation – which must be said was the most astrophysically exciting one – probably deserves to be chucked out the window.

I know the ejected guy is normally supposed to come up with a sensible idea, but dark galaxies did used to seem like the best explanation. Anyway I don't care, so shut up.

What looks almost certain is that we were being deceived by the apparent similarity of the clouds. It looked as though they all had about the same mass, line width, similar levels of isolation, and lack of optical counterpart. We now know that's not entirely true : the line widths vary (in some cases our new measurements reduce this compared to our original estimates) while some have optical counterparts – which themselves have varied properties. 

That means the number which we can attribute to being some form of dark debris is not eight, but maybe five at the very most... and likely less, as two weren't even observed. Couple that with the variation in properties and the debris scenario begins to look a lot more plausible. Instead of saying "they're all dark galaxies" or "they're all debris", it now looks more like some are extreme galaxies, some are new kinds of stellar structure, and some are debris. No individual explanation has too much of a burden to bear any more.

These objects are still weird though. For one thing, how come compact objects like AGESVC1 231 seem pretty good at retaining their gas even when larger ones like VCC 1964 have it all removed wholesale ? Why does it get fully displaced in one case but elongated in others ? More fundamentally, what governs when the HI gas is destroyed as it's removed from giant galaxies but can apparently survive and prosper in the harsh environment outside its friendly parent galaxy ? That's a bit weird.

And perhaps of more immediate interest are those blue blobs. In a spectacular pivot that would shame a politician, I might well find myself switching more to objects which lack dark matter rather than having too much. The objects in this paper are not, it turns out, the only such blue blobs in our data which show dynamics like this. Given that VCC 1964 shows a similar lack of dark matter, it's tempting to wonder if there's a connection between BBs and Ultra Diffuse Galaxies. Right now, all I can say is that some of our other BBs shown even more extreme gas ratios and dark matter deficits than the objects we examined here... the times they are a-changing, but the future, it seems, remains as dark as ever.

Got Gassy Galaxies ? Get Gasisgone !

What happens if your galaxy is feeling a little... bloated ? No problem ! Simply pop it into a galaxy cluster and let it dissolve slowly for a few billion years.

Say what you like about AI, it has its uses.

Okay, time for the public-outreach version of my latest paper. I'm gonna do my very best to keep this one to a readable length.

Introduction : How To Do Body Shaming For Galaxies

Back in 2015, astronomers got a nasty shock. Most galaxies, we thought, all had roughly the same surface brightness level, having about the same number of stars per unit area. Oh, sure, they varied a bit, of course, and there were a few oddball hipster galaxies that just insisted on doing things differently. But most of them were basically the same.

Then along came shiny new telescopes which were much better at finding fainter galaxies. Lo and behold, it turned out that there were plenty of such objects to be found. Now if they'd been really small tiddlers, I don't think anyone would be surprised. Dwarf galaxies do all kinds of crazy stuff anyways : we already know we don't fully understand how they work.

Like this guy, caught in the middle of two giants and causing a right mess. In some cases, interacting giants can rip off so much material from each other that whole new dwarf galaxies form. 

But the new galaxies weren't dwarfs : they were, by some measures, as large as giants. The stock phrase – and it's a decent one – is that they're the same size as our own Milky Way but a hundred to a thousand times fainter. And this is really weird, because everyone was quite happy we at least understood the basics of the biggest galaxies. The implication that maybe we didn't made a lot of people quite worried. Well, anyone would be if they suddenly realised they were surrounded by hundreds of invisible giants.

The immediate question was : ah well, yes, they're very big, but are they heavy ? Our models are mainly based on mass, not size. So maybe they're just like Zeppelins : enormous, but they might not weigh very much. And just like Zeppelins, they'd be spectacular but largely useless (or at least irrelevant for our models), and we could write off the whole thing as a Hindenburg-like tragedy except without all the horribly fiery death.

Too soon ?

Then things took a very confusing left turn. More than that, the world of extragalactic astronomy went down blind alleys, into ditches, and got thoroughly lost. We still don't know where exactly we are or where we're going, much less how to get there, but as best as I can tell, the current state of affairs is something like this : 

Most UDGs probably are those over-inflated dwarfs. They don't have much mass, but they spread it around a lot, taking up two seats on a plane not because of poor lifestyle choices or unfortunate physiology, but because they can't stop flailing their arms and legs everywhere. All their stars are very spread out, but their total mass – mainly dark matter, which we measure by seeing how fast their gas and stars are moving – is actually quite modest.

Manspreading is also surely even worse when you're a spindly fellow, and it certainly doesn't help matters if you're a contemptible fuckwit either.

That sounds not too bad, and indeed it isn't. Just like real life, body shaming the UDGs has come back to bite us, because they're (mostly) nowhere near as heavy as we might have guessed. Why, then, are we still confused ? That's because there are two major headaches nobody has yet solved. 

First, at least some UDGs are still plausibly very massive galaxies after all, which we don't know how to explain. Second... some of them seem to have far too little mass. Absolutely nobody expected this, and what's worse is that we see this even in isolation. Some of the first, most widely-reported examples of under-massive UDGs were found in groups, and these we think we can explain quite well as being the result of interactions with other galaxies (though even this isn't 100% proven). But in isolation this simply doesn't work at all, and nobody predicted galaxies lacking dark matter would ever be much of a thing. That's why we're still bloody confused.

So, too massive ? Nothing special ? Not massive enough ? The reality is likely a mixture of all three, and we're still trying to find our feet. 

If you want a more detailed introduction, I did two much longer write-ups on this. This post looked at the initial discovery while this one looked more at the first results of measuring the masses of UDGs. We still don't have much data on that. And there'll be a bunch more links throughout for those interested in more up-to-date results.

One other major issue is that we know about lots of UDGs in clusters but relatively few in the field (a catch-all term that basically means "not clusters"). So it's possible that the situation isn't so bad. Maybe most of the (more numerous) cluster-UDGs are indeed just normal dwarfs after all, but inflated by interactions with the other galaxies so they look bigger. The field objects without much dark matter could just be crazy weirdos, exotica that are extremely interesting but not actually that important in the grand scheme of things. After all, a few lunatics don't tell you much about psychology more generally, thankfully.

Essentially, we know of a few crazy galaxies outside clusters, but just a few. We have no idea of the far more numerous objects within clusters, but if they're as weird as the ones outside... well then things get freaky.

What we need, then, is to find a UDG just entering a cluster which still has its gas so we can measure its dynamics. That would help tell us if cluster UDGs are basically typical or typically weird.

We Need More Data

Which is where the current paper comes in. To be fair, there have been a few cases of estimating the masses of cluster UDGs before*, but very few indeed with data from their gas – quite possibly at the level of low single-figure numbers**. The rest have had to be done almost entirely by less direct methods, making clever inferences from globular cluster numbers and suchlike. It's very clever, but also unsatisfying, a bit like guessing someone's bank balance by the size of their house. It's nobody's preferred option. 

* One of which seems to be overly-massive for its size, but far short of being a true giant. UDGs really do seem to probe the full parameter space of weirdness.
** The paper has more on this in the Discussion section, but basically I only know of two, and even these weren't set out explicitly as UDGs.

This lack of data isn't surprising. Measuring how fast the stars are moving is extremely difficult when there's hardly any stars to measure, and as galaxies enter clusters they can lose their gas very quickly. Even if UDGs have relatively normal masses, it's not that strange that so few have been detected with gas while in clusters. 

So at last, in this paper we present a candidate for a UDG entering a cluster and still retaining most of its gas. And it too seems to have a depleted dark matter content.

Here's the data !

Credit for this discovery goes to my PhD student, who found it at the southern edge of our Widefield Arecibo Virgo Environment Survey (WAVES) data set. This is just north of the Arecibo Galaxy Environment Survey (AGES) data that I studied for my PhD, and there's a little bit of overlap – in fact it's found in both. Now characterising a galaxy as a UDGs is quite difficult since it needs such precise measurements, and while we're working on doing this for ourselves, we began by looking at a pre-existing catalogue of UDGs called SMUDGES. This one, VCC 1964*, was the only one found in our HI (atomic hydrogen) data sets.

* The name is from a catalogue from 1985. Some UDGs were found well before the big kick-off in 2015, which is when people began to recognise them as a distinct class of object.

Well then, here it is :

The contours show the gas, coloured according to how fast it's moving. The galaxy is relatively small, but you can see it more clearly in the inset image on the left. The inset image on the right shows the same galaxy as seen in ultra-violet, which usually corresponds to emission from young stars. Finally, the white arrows show the directions to two of the most massive galaxies in the cluster, and the green circle just shows the resolution of the atomic hydrogen (HI) data.

What we see here is immediately quite interesting. Normally the gas is centred almost exactly on the position of the stars (white X), but here there's a clear offset. Moreover, this same offset is seen in both AGES and WAVES data sets, so it's definitely real. The orientation of the gas and the stars is a bit weird here : we'd expect the gas to be further away from the cluster centre than the stars, but it's actually doing the opposite. That strongly suggests the galaxy has a weird orbit.

Much more fun, though, is the mass of the galaxy. The gas tells us how fast stuff in this galaxy is moving around, and the answer is... not fast enough. For this we use my favourite way to plot galaxies : the Tully-Fisher Relation.

Black circles (filled and open) show normal galaxies. The orange points show VCC 1964 as measured in a couple of different ways, while the grey points show the famous ALFALFA detections. The dashed and dotted lines show the observed scatter, with the uppermost dotted line showing five times the standard deviation – the usual criteria for judging something to be significant.

You might remember my extremely detailed run-through of how this is plotted, but no need to go through all that again. Basically it just shows the total mass of gas and stars (vertical axis) as a function of how fast the galaxy is rotating (horizontal axis). This object, much like some other UDGs with gas measurements, is rotating much more slowly than normal galaxies – suggesting it's got significantly less dark matter.

You might also notice that the mass of this galaxy appears to be much less than the other UDGs plotted. In fact it turns out to have the lowest mass of gas detected in a UDG to date, with the next highest being something like a factor five times more massive (and most much more than this). But can we then trust these measurements ? With so little gas, how can we be sure we're really measuring its rotation speed accurately ? Especially since the gas appears to have been pushed out. As someone said during the analysis phase, you can claim the gas has been removed, or you can claim an offset from the Tully-Fisher, but can you really claim both as important at the same time... ?

That is definitely the most uncertain bit of our analysis. We've found previously that galaxies do deviate from the Tully-Fisher simply when they've lost enough gas, with no implications for their dark matter at all. That can happen because the outermost gas is both the most easily removed and rotating the fastest. Strip this away and the measured rotation speed will shrink, but this doesn't tell you anything about the dark matter content at all.

A simple simulation of ram pressure stripping. Dark matter (not shown) is completely unaffected. But since we infer dark matter by the rotation of the gas, if the fastest-moving gas is removed, it can look as though the galaxy has less dark matter than usual.

It's certainly possible this is the case here, but there are mitigating factors. We found that large deviations only happened when galaxies had lost a lot of gas, but in this case the gas loss only seems to be very modest : actually what we're seeing is more gas displacement than loss. All of the gas appears to have been shifted in bulk away from its happy place; effectively all of the gas should count as the green "disturbed" component in the above diagram. 

This makes the interpretation tricky. We can say quite confidently that the rotation hasn't been reduced just due to sheer loss of the fastest-moving gas. The problem is that we have almost no clue what happens when you get a wholesale bulk movement of the gas like this – it might disturb the gas in other ways we haven't accounted for. There just aren't nearly enough other examples like this for us to make a comparison.

And we also tried another plot of the TFR, this time not using total mass but just optical brightness. This is subject to less corrections that have to be applied to the data, and we found an even stronger deviation : at least six times the scatter in the normal galaxies, and possibly even more than this. So two different plots by two different methods gave us the same result, which is pretty neat. And in a few other cases, it's been shown that the velocity dispersion actually increases when gas is stripped, not decreases. 

What's Going On ?

VCC 1964 has a great deal of collective weirdness :

• Its gas is displaced from its stars, but little is actually missing : it has the lowest mass of gas in any UDG detected to date, but that's consistent with it being a right little tiddler (not because a lot of gas has gone missing).
• The offset between the gas and stars suggest a weird orbit, with most of the motion across the sky rather than along our line of sight.
• The apparent rotation speed is quite a lot lower than expected. This can't be accounted for by gas loss or measurement errors, but we don't know if gas displacement by itself could also cause this effect.
• And what I haven't mentioned is that this galaxy is both smooth and blue. This combination is itself very unusual : more often, blue galaxies have lots of structure and star formation. It might be related to the galaxy just having lost all of its gas, with enough time for the structures to smooth out but not enough for the colour to change. Maybe. It's also strange that removing the gas, which is probably about as massive as the stars, doesn't appear to have altered the stellar structure at all.

What if it's all just a horrible coincidence ? Could the gas actually have nothing to do with the stars at all, with both at different distances ?

This has actually been proposed to be the case for another possible example of a UDG losing gas in the Virgo cluster. Personally I'm a bit skeptical about this, but it's possible in that earlier example. But to find two examples of UDGs in Virgo where the gas is just coincidentally aligned with some unrelated stars... nah, I simply don't believe it. And we looked very carefully to see if there are any other possible sources of the gas nearby and there just aren't. I tend to rule this idea out.

What we know with some confidence is that this is a UDG caught in the act of losing gas. That's already extremely unusual, and we must be seeing it at a very precise moment indeed for it to be both blue and smooth. What's much more uncertain is whether shoving the gas out of the galaxy would cause the illusion of a lower rotation speed : in other words, does it really not have much dark matter, or is this just because it just got kicked in the backside ? Having little dark matter would be consistent with other UDGs, and – maybe – also help explain how come the gas can been removed in bulk, wholesale, rather than dragged into a long tail as is more normal.

And even more speculatively... what does this mean for the screaming hordes of UDGs in clusters more generally ? Are they just giant collections of stars rather than more typical, dark matter-dominated galaxies ? We just don't know, and intuitively, it's hard to see how they could survive very long in a cluster... though it seems they do tend to avoid the densest parts of clusters, so maybe they simply don't. Maybe they just get torn to shreds. 

All we really know is that everything is very confusing.

The final question is whether objects like this are connected to the so-called "blue blobs". In my opinion these are some of the most interesting objects found lately, though I haven't talked about them much. They're... umm, well, they're fuzzy blue things which don't quite resemble ordinary galaxies, and there's been some very interesting suggestions that they might form in the gas stripped out of larger galaxies by ram pressure. They too would lack dark matter. So is there a connection ? Is VCC 1964 actually a very large blue blob ? Or will it even spawn a blue blob from its own stripped gas ? How do all of these relate to the dark galaxies I more usually go on about ?

Well, good news ! Our paper on using the VLA to get higher resolution on those dark clouds is damned close to acceptance. And we have more observations accepted with the VLA for VCC 1964 itself, which should give us a much clearer picture of what its gas looks like than the big blob we see in the Arecibo data. More research is needed... but this time, more research is actually happening.

Masochism For Fun And Science

How can you trust your own senses ? How can you be sure that what you're seeing isn't all just some kind of elaborate illusion set up by a powerful entity with a truly warped sense of humour ?

Don't worry, this isn't going to be much of a philosophical rant. Actually, it's going to be an extended rant about how I hurt myself by looking at over 170,000 pictures of static.

Yes, really. All of which relates, of course, to radio astronomy. It's time for the public-friendly explanation of my latest paper, in which I pit human vision against a set of different algorithms and find that there's life in us old monkeys yet. I also inflict things upon myself that I hope never to repeat ever ever again.

1) How To Hunt For Gassy Galaxies

Regular readers will be only too well aware by now of what galaxies look like in neutral hydrogen (HI - "H one") data cubes. There's a longer explanation here, but in general they look like this :

Here, as in the above GIF, we see slices of the data cube as separate images, though we can also view them all at once as a 3D volume. Now I've waxed lyrical on the joys of volumetric data visualisation many times, but not today. It turns out that the image sequence approach isn't as fun, but is usually better for sensitivity. Which is what I wanted to quantify in my paper.

So how do we actually decide what's a galaxy and what isn't ? Clearly, some bright signals are obviously real, but plenty of others are much fainter, and it isn't always obvious if they're from actual galaxies or just the result of noise. 

We've got two options. The classic method is to do it by eye, visually trawling through the data cube slice-by-slice, image-by-image, recording the coordinates of wherever we find anything that just "looks like" a source by some criteria. The other, increasingly-popular approach, is to rely on algorithms to find the sources for us – maybe using people to check the algorithm's catalogues, but maybe even trusting the algorithms completely.

"Yes, yes," you say, "but how do you ever know if what you've found is real ?"

Indeed. That's a question that plagues both visual and automatic searches. Oh, sure, it's fine if you find a great big blazing beast of a galaxy (like some of those in the above animations), but what about when you've got something more piddly like this ? How could you be sure it wasn't just some slightly brighter-than-usual bit of noise ?

VC1_304, more usually known as NGC 4309. On the left is the source (highlighted by the white outline) as it appears when inspecting the data cube – it's barely visible even with training ! It's only a bit clearer in the spectrum, shown in the middle. Fortunately this object is optically very bright indeed (right), but most objects with this little gas tend to be much dimmer.

Well, there are a few measurements we can make of the gas itself that give us some clues : quantitative values can be a lot more reliable than simply eyeballing it. But first, in the above images you can also see optical data alongside the radio, and that's a very powerful verification check. Gas clouds without associated optical emission aren't non-existent, but they're extremely rare (about 1% of all HI detections by some estimates). You'd be forgiven for missing this one based only on looking at the data cube, but here at least the optical galaxy is unmistakable.

Understandably then, hardly anyone relies exclusively on the original HI data. We almost always have some optical data to act as independent confirmation (except where the Milky Way blocks our view in the aptly-named Zone of Avoidance) and can usually get follow-up radio observations to directly confirm at least the most interesting signals. That's the absolute gold-standard for verification, and how we know that most optically dark sources aren't real : we've checked them many, many times, and indeed still do when we find anything interesting.

But... what if we can't get either of these ? Exactly how good is the eye at distinguishing signal from noise, and what fraction of the faintest signals does it pick up at all ? That's what I wanted to answer with this paper.

2) Yes, But Why ? And How ?

The key aspect of the problem is that, once you get down to the faint stuff, there are no objective criteria, no magical algorithms, that can 100% reliably distinguish between real signals and noise. Some truly pathetic signals turn out to be real while some fairly convincing ones end up being discarded as worthless junk. The only way to be really sure is to do more observations. 

BUT... algorithms are at least objective and repeatable : throw 'em the same data set and search with the same parameters, and you'll get the same objects every time. Different search methods or parameters can give you different catalogues, but at least if you keep everything the same, you'll get the same results again and again. That's a big advantage over using squishy, emotional humans that might get distracted because they haven't had enough tea or they got sick or their pet hamster died or something.

I thought about refining ChatGPT's weird take on this, but decided the bizarreness of putting the hamster in a box labelled NO TEA was just too funny to alter.

How much does this matter though, really ? Exactly how good are humans compared to algorithms ? Can we even quantify it, or are we all such an emotional, whimsical bunch of wet blankets that we just come up with totally different results every time ? Or if you're a Daily MFail reader, HAS WOKENESS KILLED ASTRONOMY ?

The only way I could see to test this was to look for lots and lots and lots of sources, all with different parameters. Throw enough sheer statistics at the problem and it ought to be possible to see if human abilities could be quantified or not. 

Now to do this requires we have full knowledge of what's there for us to find. Ordinarily this isn't the case at all, because the whole point of the problem is that we don't know for sure which sources we've missed. So the only way to do this is by using fake sources – only then can we be absolutely sure if we've found everything. The basic idea is very simple : to try and find as many artificial signals as we can and measure their parameters.

Of course, for this we also need a data set which doesn't have any actual galaxies in it, otherwise we'll confuse the artificial signals with real ones. Fortunately one of our unpublished data sets includes just such a cube, spanning a frequency range in which real signals just can't happen. To find emission here would require galaxies moving towards us at insane velocities, many thousands of kilometres per second – no real galaxy is known which moves at anything even close to this*. Bingo ! We've got a real data cube with all the imperfections of real observational data, but with absolutely no real galaxies in it. Perfect.

* Redshifts of this magnitude are normal, due to the expansion of the Universe. But there the galaxies are moving away from us. Here we'd need galaxies of extreme blueshift, and there's no known mechanism by which this could happen.

Once we've done the detecting, how to parameterise the results ? Well, with any catalogue it's important to understand its completeness and reliability : that is, what fraction of the sources present it detected, and what fraction of its detections are real. With enough sources we could also see if the eye is especially sensitive to particular properties, like the total brightness and velocity width, and maybe also figure out what sort of false signals fool the eye into thinking there's something present. And I also wanted to test the street wisdom that, apart from speed, humans are generally better than algorithms when it comes to sheer detection capabilities.

3) The Experiment

Figuring out the best approach took a lot of trial and error. The simplest method would be to inject lots of signals into a single large data cube, but this wasn't feasible. This would mean I'd have to mask each galaxy as I went along to avoid cataloguing it twice, which is... not a huge amount of work, but it adds up. And for an experiment of the scale this one became, this would have been unbearable.

The problem is that galaxies themselves have two parameters which control their detectability : their width and their brightness. Here's an example spectrum I use in lectures :

What this is showing is a signal of fixed total flux but a varying velocity width. At the very beginning, all that flux is confined to just a few velocity channels, so it's very narrow but bright. Even though it's so bright, because of the way we typically display the data, the narrowness of the signal makes it hard to spot. As the movie advances the velocity width increases, so that it gets wider and wider but appears dimmer and dimmer. At first this makes it much easier to see : it's still bright but it's no longer narrow, so it's really obvious that there's something atypical here. But eventually that flux is spread out over so many channels that it's barely distinguishable from the background noise at all, even though the total amount of flux is the same throughout the animation.

I've long thought it an interesting question as to which one matters most. If a source is wide enough, does this compensate for its dimness ? Or is it brightness alone which determines detectability ? My PhD supervisor took it for granted it was the latter, but I was never quite convinced of this.

The only way I could see to tackle the problem was to inject many galaxies each of a given width and brightness. I'd inject, say, 100 with some combination of values, see how many I could find, and then repeat this ad nauseum. I'd need to have plenty of objects for each combination to get a statistically significant result. Since I had very little clue ahead of time where exactly the detectability threshold would be, this would mean injecting a lot of galaxies.

That made the idea of using a single cube a complete non-starter. Eventually I figured out a working strategy, which goes like this :

1. Pick a width and brightness (signal to noise, S/N) level of the signal.
2. Extract 100 small "cubelets" at random from the main cube.
3. For each cubelet, randomly inject (or not inject) a signal of the specified parameters, at a random location within each one.
4. Modify my source extraction program so I could go through each cube sequentially, just clicking on a source if I thought I could see one, or clicking outside the data set if I thought there wasn't one.
5. Choose new signal parameters and do the whole thing again.

The cubelet is an adorable animal, but, like tribbles, they tend to multiply exponentially if you're not careful.

This was... acceptably fast. Each set of 100 cubelets, containing on average 50 signals, takes about 30 minutes to catalogue. It also made it easy to take breaks, which was absolutely essential. I made it so that every time I clicked to identify a source, I'd be shown if I was correct or not, the result added to a catalogue, and the next cubelet would automatically open. The referee kindly let me get away with language not always typical of an academic paper because here it really does matter :

Without any of these it becomes too easy to be lost in the visual fog - one needs some clue as to what one is looking for or the experience is unendurably frustrating... Being a visual search process, one needs to take much more account of the psychological, emotional experience than in using a pure algorithm.

Finding the initial point was again a matter of trial and error. If I remember correctly, I believe my first guess for the source parameters was too bright so I ended up finding every source (or nearly so), so I kept halving the brightness until the sources became genuinely difficult to spot. From that point I could proceed systematically. The final result is this terrifying table :

Each of those pairs of values represents a search of 100 cubelets. In total I searched 8,500 cubelets, i.e. I looked at 170,000 individual images (slices of data), containing a total of 4,232 sources. But at last, I was done.

This was utterly exhausting. In principle the whole thing could be done in a week; in practice anyone actually trying that would likely claw their own eyes out and hurl them across the room in despair. In terms of calendar time it actually took several months (or... more), and isn't something I ever want to repeat. Fortunately, I'll probably never have to.

170,000 images. I mean, FFS.

4) The Results

The money plot from the paper is this innocuous figure :

Each black circle represents the search of 100 cubelets of a given combination of peak S/N and velocity width. The blue points with error bars show the median at different integrated S/N levels, essentially just a crude fit to the data.

This shows what fraction of sources are found as a function of their integrated S/N (signal to noise). This is a deceptively simple parameter (you can explore it a little more here) that measures how bright the sources are in a more sophisticated way than just the value of the brightest pixel : it accounts for the width of the galaxies as well. The remarkable thing is that the trend is so clear and so tight as a function of S/N_int even thought the experiment was done without any reference to this. Plotting completeness as a function of width or simple peak brightness just gives (more or less) a chart of pure scatter, but this trend emerges like magic. I really wasn't expecting this at all. A parameter designed originally for automated source-finding turns out to describe human vision remarkably well !

You can see that there's a critical threshold. Above 6.5 or so I detected near enough everything, wide or narrow. This value is something we've long known is a good measure of reliability (whether a source is real or not), but now it seems it's also a great way to determine if a sample is complete. If these results are a good reflection of how visual extraction works in the real world, it means we can be confident we've detected every source brighter than 6.5, and about half of all sources at a level of 3.9 or so.

Which by itself is already great news ! Now we can say confidently that above this threshold a) our source is real and b) we've detected all similar sources. In other words we can quantify exactly when our results are unbiased, and also what fraction we're likely to have missed at lower detection criteria.

Lovely. But is this a good approximation to real, in-anger source extraction ? Is the experiment sufficiently realistic ?

5) The Tests

Quite honestly I thought so. The referee, initially, didn't – though quite understandably. So back I went and did a whole bunch more tests just to make sure. Which resulted in this new and improved figure :

This one accounts for several different possible biases (feel free to skip ahead if you already believe the main result) :

• Green points : these test for the fact that I knew ahead of time that the fraction of cubelets containing a source was always about 50%. Here I instead set this fraction to a random number, but lo, the detection statistics were unchanged. In fact even when I went back and searched cubes where I'd previously missed the source, but now with the full knowledge that a source was present, I still couldn't find them. Foreknowledge just doesn't help much.
• Red points : randomised source properties. Here I injected cubelets with three different integrated S/N levels designed to give 25, 50 and 75% completeness levels, but with entirely randomised widths, in random order, without giving any indication of which identifications were correct until the experiment was over. Essentially for each cubelet I had no idea if it contained no source at all or one which would be marginally, modestly, or probably detectable, or what it would look like. This again made no difference to the results.
• Orange points : as above, but now injecting the sources into a single large cube instead of a many cubelets. In the main experiment I knew there was at most one source per cubelet; here I didn't know the number injected (randomised with some sensible upper limit) or their properties. This was about as similar to real-world conditions as it's ever going to get, and it still made no difference.

So yes, after looking at another 918 cubes, I can categorically say that the experiment is realistic. There's one final possible bias, but I'll return to that only at the end.

6) Humans Versus Robots : FIGHT !

"It's a movie about a killer robot radio astronomer who travels back in time for some reason."

Okay, so this robustly quantifies how good visual source extraction can be. The obvious next question is : are we really any better off doing it ourselves, or should we let the algorithms loose instead ?

For this I tried two methods. First, I ran the same experiment again but now using automatic source-finding programs instead. Not quite identical though, because now the 80 GB or so of data was spaced over two machines, and adapting the programs to search many little cubes instead of one big one (as they're designed for) was just yawningly tedious. Instead, I had a script inject sources of a given S/N_int (with randomised widths and peak S/N) throughout the main master empty cube, run the extractors, increase the S/N_int, and iterate this until I'd sampled the whole range of values I'd tested visually.

I'm glossing over a lot of details here, but the end result is another set of unshapely curves :

"Fiducial" is just the main visual experiment. It sounds more sciency than "the one what I did earlier".

Let's start with the thick lines. After much experimentation, these were the settings that gave me the highest completeness rates I could get. With the deservedly popular SoFiA (red), this gives marginally better results than visual, though well within the general scatter. GLADoS, my own obscure code I wrote years ago, does considerably worse – not awful, but it looks a bit pathetic next to SoFiA.

But does this mean that algorithms are actually as good, or even slightly better, than humans ? Actually no, not really. Sort of. Maybe. It's complicated.

The thing is that these curves are all at some given reliability level. It's actually not that easy to show how reliability varies with S/N, but luckily there's an easier way to compare : just count the number of false positives from each method in the same volume. From all the earlier testing, the highest number of spurious signals I found through visual searching of this part of the data was 23. SoFiA, by contrast, found 133, whereas GLADoS found 176.

So that high completeness of SoFiA comes with a huge penalty in reliability. In terms of completeness and reliability, automatic extractors are nowhere near as good as people.

But, they do have a significant compensating advantage : sheer unbridled speed. Even if you have to search a long candidate list from the unreliable algorithms, this can still be faster than visual searching the whole cube. The balance to all this is hard to judge, but there will be a point of diminishing returns – try and dig too deeply into the noise with an algorithm and you'll get so many false sources that the speed advantage will be lost.

Bottom line ? For small cubes use people. Human extraction isn't that slow, it's better than the automated methods (though there's no harm running these as well), and its results are quantifiable. For larger cubes there isn't really much choice but to use algorithms, but at least this experiment gives a rough guide as to how good they can be.

Conclusions and Caveats

Human source extraction is seriously powerful. Over on Little Physicists I've been exploring the wonders of GPT-5, which honestly I've been seriously impressed by for scientific analysis and discussion. But it doesn't hold a candle to billions of years of evolution driving human pattern recognition skills. On that front, for the moment, humans win hands down.

After all, the question "real or fake" ? has caused much debate elsewhere. Methinks I ran the wrong experiment.

 

Interestingly, both visual and automatic techniques tend to pick up on the same sources : it's just that the automatic methods tend to a) miss a lot and b) also find a bunch of crap. Still, that does suggest they're doing something right. Keep dialling up their sensitivity and they'll probably find even more faint sources, it's just that the reliability penalty would be unbearable.

And that's the fun part. How exactly does human vision do so much better at rejecting all the really faint false stuff ? For now we don't know. There are many possibilities, but after looking at this many god damn fecking stupid smudgy feckless bloody false galaxies... I don't intend to investigate this any time soon.

More pragmatically, these results mean we can now say when our results are statistically meaningful, and also give us something to ram down people's throats if they ever dare to suggest that visual extraction isn't any good. But they also show that there's something about human vision, the thing we generally rely on the most for making sense of the world, that we don't fully understand.

And there's one final head-scratcher to end on. That other bias I mentioned ? The referee suggested that maybe in the real world we wouldn't catalogue such faint signals because we wouldn't be able to verify them. "Humph !" I thought, a tad disgusted by such an insinuation. So I ran yet another test, this time injecting fake sources into data cubes also containing real sources. The idea was that if we had missed a lot of real sources in our original search, in this new search we should find the fake sources plus some more real candidates.

We didn't. The fake sources were found at the same rates as before, so the search was just as effective as in the other tests... but hardly any new candidates for real detections. Great ! But... the odd thing is that the real sources are all much brighter than what we should be able to find. Where have all the faintest sources gone ? Not a clue ! Maybe, eventually, we'll get some actual proper science out of all this as well as all these dry statistics. Well, you never know.