The End Of Darkness

My goodness me, another paper ! What is this, a science blog or something ?

This one is by my long-term partner in crime, the inestimable Robert Minchin. After the last few papers – one on a single weird galaxy and two on source extraction – this is a return to my favourite topic : dark galaxies.

And not just any dark galaxies...

Well, the joke works if you're British, at any rate.

What I meant to say is that these are the objects I've spent most of my career on. If you haven't read the link, a dark galaxy is just a galaxy which has some gas, maybe a very few stars (but preferably none at all), all embedded in the classic dark matter halo that we all know and love : the very thing that distinguishes galaxies from all the other crap floating around in the majesty of the cosmos.

Look, I've been through this dozens of times. I'm not gonna do it again. Go and read the link, I'll wait.

Go ! Read up and report back. Off you pop.

Okay, you ought to bloody well know about these things in general by now. But I'll forgive you if you can't quite remember why I like these particular candidates so much. Mainly because I'm extremely biased and narcissistic... that is, they turned up in the data I was analysing during my PhD, and were the main highlight of my very first papers*. I've been working on these for the best part of two decades, which is a scary thought if nothing else.

* It's all "me, me, me" around here.

Yes, yes, but apart from keeping me employed, what's so fascinating about them ? No-one's going to keep paying anyone to study genuinely boring objects, after all.

Well, it's a combination of two factors. The first is that they have a high line width. That is, we can see that parts of them are moving at very different speeds along our line of sight (towards or away from us). This is exactly what we see with normal, rotating galaxies, and it's normally a pretty good signature that they have a lot of dark matter. 

But this, we've learned, isn't decisive. We didn't have good spatial resolution of the clouds, so we couldn't be sure that we were really seeing rotation. Simulations have shown that when galaxies interact with each other, the debris that gets torn off can (briefly) exactly mimic this apparent signature of rotation, so it was possible we were just being fooled. And without rotation, there's no reason to think there's any dark matter.

Really good data would let us map the motion very precisely, like on the left. Here we can see that one side is moving differently to the other, with a smooth gradient across the object : a classic signature of stable spin. But for these objects, all we have is something like the spectrum on the right. The width of the bump looks like rotation, but without an actual map, we can't be sure. (This example was taken from this paper)

That's where the second parameter comes in : these objects are relatively isolated. Our simulations have shown that when fake dark galaxies are formed, they ought to be accompanied by massive streams of material from their parent galaxies. There's no sign of any that with these guys. In fact, their high line width makes this all the more surprising. The simulations show that features with widths this high (180 km/s) ought to be the fastest-moving parts of the stream of all, and therefore the parts that disperse the most rapidly. 

So the lack of a stream, if these are fakes, is paradoxical : how can the fastest-dispersing part of the cloud be the longest-lived ? It all makes a lot more sense if we're seeing genuine rotation due to dark matter, since rotation can be stable on indefinite timescales.

And there's more. We found eight such clouds in a relatively small part of the Virgo Cluster, almost 10% of all the gas detections in that same area. Now if these were transient, unstable objects, they'd have to be forming at a very high rate indeed for us to detect them. To find one or two might be plausible*, but eight ? Nah, that's silly.

* Even then, only just. Actually the simulations found that we could essentially never produce any isolated clouds with line widths this high, because they disperse so bloody quickly.

The heart of the problem is simple. To find an HI stream with some weird velocity structure that briefly looks like a dark galaxy is quite possible. But to find them in isolation, with the rest of the stream having gone but the fastest-dispersing bit somehow still surviving... that flat-out doesn't work. Not for this many objects, at any rate.

---

What we've been lacking for all these years are two things : better statistics and better resolution. For statistics, you'll have to wait for my PhD student's first paper. Today we're dealing with resolution.

Yes, today. Yes, that's nine years after applying for time on the VLA. Stuff kept coming up, mmmkay ?

Seriously, it did. I didn't know much about how to reduce the data and I had a lot more low-hanging fruit to pick. Plus there was that whole global pandemic thing, during which I recoded 20,000 lines of Python code, amongst other things.

Step forward Robert. Having previously been head of astronomy at Arecibo (which collapsed) and a staff scientist at SOFIA (which was cancelled), Robert moved to the VLA a few years ago (and we're all praying nothing happens to that). Actually he managed a preliminary look at the data as far back as 2022, but again, stuff kept coming up, and it's taken until now to look at the thing properly.

The result (drumroll please)... they're weird objects, but they're not dark galaxies.

Betcha didn't see that coming ! Shut up, you didn't, you filthy liar.

How do we know this ? The VLA data is about four times sharper than Arecibo, and that means we can now locate the exact position of the gas much more precisely. When you're looking for especially faint optical counterparts, this matters a great deal. You can find a random starry smudge literally anywhere, so that there's always some ugly bit of stellar faff (maybe real, maybe just noise) that you can't ever be certain isn't associated with the gas. The worse the resolution, the more such faff you have to contend with. At some point you hit a wall, and getting better, sharper data is the only way to make any progress.

The VLA data reveals that in two cases, we can now identify the optical counterpart unambiguously. And they're likely neither dark galaxies nor bits of debris, but something we never expected at all.

1) AGESVC1 231

Figure 2 from the paper. The boxes show where we measured the gas content. The red and blue contours are the original Arecibo data, showing the gas at two different velocities. The inset image is a close-up of the tiny optical counterpart, just visible at the centre of the green contours (VLA) in the main image.

The Arecibo data for this is a bit of mess, but the VLA is clear : it's definitely associated with a pale blue dot. Now if I were Carl Sagan, I'd wax lyrical about how this seemingly insignificant little mote is a rich system of tens of millions of stars, perhaps host to trillions of ancient kings and emperors, a brief candle striving oblivious to the oncoming dark... but I'm not, so I won't. What I will say instead is that it's a pathetic, paltry, stupid little bugger that's bloody hard to spot.

More rigorously, while we always wondered about that tiny blue dot (I mention it right back in the earliest publications), it never seemed at all convincing. The stellar component seemed exceptionally compact, and the gas and stars weren't in a great alignment with each other (normally the positions are very close indeed). It was uniquely extreme : in other cases of small blue smudges, we could see at least some structure. With this one we could see really nothing. It just looked smooth and boring, not like a galaxy at all. And that high line width is something we'd normally expect only from very much bigger objects. It just didn't fit the bill.

But the VLA data explains all this quite nicely. Actually, as shown above, we can see it in the Arecibo data too if we'd only thought to look... but what we see is that HI has a clear tail. The densest gas appears to be associated with the stars, but the majority of the gas is actually in the stream. This is why the overall best fit to the Arecibo data gave such an offset between the gas and the stars.

So why didn't we make such a map with the Arecibo data years ago ? Largely because, as I've covered before, the existing tools to do this were a right pain to use. Remember, this is one source among more than a thousand in our full (still incomplete) catalogue... and the data really didn't suggest we had any chance of finding anything. Looking at the raw data, it's not at all obvious there's any kind of stream. Making such a map felt like a pointless and tedious exercise, surely not capable of showing anything more than we could see in the data. Maps are something you normally make only when you can already see evidence of structure and want to examine it more carefully, not to find structure in the first place.

Making maps is a classic case of something that's not actually difficult but is extremely tedious. "I'll do it this afternoon !", indeed... at least with much of the standard software.

As per another paper, I have of course since learned my lesson on this. Data visualisation and simplicity of tool use should not be considered optional extras : sometimes, they really matter !

Anyway, the new analysis also explains that high line width. Rather than being a signature of rotation, it's the result of gas being stripped out of the unusually tiny galaxy. The VLA data is less sensitive than Arecibo, but this shows us that the compact, dense gas is all associated with that blue blodge. This gives us a very nice, consistent picture of an especially small galaxy being caught in the act of having its gas very rudely shoved out. But then, it should have known better than to try and barge its way into the Virgo Cluster, the jumped-up little upstart*.

* All I'm saying is that there's room in public outreach for people who hate things and don't want to write love sonnets to their magnificent galaxies the whole time. Fuck off Carl ! Suck a lemon, Sagan ! There, that ought to get me some much longed-for hate mail...

This is still an unusual object, mind you. Not only is it especially blue, but small galaxies should lose all of their gas extremely quickly. To catch one right in the moment this happens is pretty neat, but to understand why, let's move on to the other, very different object.

2) AGESVC1 274

Figure 14 from the paper, with the VLA detection as the green contours and a close-up of the optical counterpart in the inset image.

The second object is very different from 231. Of our eight clouds, six have those peculiar high line widths, and this is one of those which doesn't : it's only about 30 km/s, a real tiddler. And its optical counterpart is extremely fuzzy and diffuse. It looks for all the world like a pretty normal dwarf galaxy, just extremely faint. 

To be fair, we actually suggested this same optical counterpart in a 2016 paper based on some deep optical data. As with AGESVC1 231's pale boring blue dot, it didn't seem like a brilliant candidate – again you can find this sort of scrappy starlight all over the place – but the VLA makes it unambiguous. This stupid fuzzy blob really is the optical counterpart after all.

I mean, two candidates I suggested turned out to be right. On the other hand, of course I also said these were shitty candidates and they were more likely to be optically dark, so more fool me.

Interestingly, this same object was re-suggested as a candidate in a paper by Dey et al. last year. They find an optical redshift* of the stars that agrees with that from the HI, which makes this association rock solid. What's more unusual is that they propose this to be a so-called "blue blob", stars which aren't forming in dark matter halos at all. They have a catalogue of 30-odd of these diffuse, blue structures which they interpret as being ram pressure dwarfs : objects which form in the tails of gas when it's stripped out of galaxies. 

* More strictly, they get the redshift from the ionised gas which emits at optical wavelengths. They don't directly measure the redshift of the stars themselves.

This is an exciting idea, essentially introducing a whole new type of object to investigate – as different from dark galaxies as dark galaxies are from tidal debris. We're talking a fundamental rethink here, not a tweak to our pre-existing ideas. And that's really nifty.

Very little indeed is known about these blue blobs, but so far they're unique to Virgo. They appear to be chemically enriched, which strongly suggests they formed within larger galaxies rather than as galaxies in their own right. Not everyone agrees, although personally I think it's extremely credible and some of the most interesting work that's been done in Virgo in years. 

Another key point is that these objects appear to lack dark matter, which seems to be the case for this one. Given that Dey's measurements show that this particular object is chemically enriched, was independently identified as a BB candidate by its images alone, and appears to lack dark matter, this all paints a nice coherent picture. Such objects could also reach large distances from their parent galaxies without the giant streams expected in the tidal debris scenario.

Wait, wait wait.... on, lacks dark matter ? I thought we were talking about dark matter dominated galaxies !

Indeed so. But of our eight clouds, only six had high line widths. Two of them, including this one, actually had widths which are if anything narrower than expected, so much so that it points to a possible deficit of dark matter rather than an excess.

There's just one problem in this case, though it isn't fatal. Unlike with AGESVC1 231, all the gas here appears to be compact, with no evidence of any stripped component. So if its stars really did form within a stripped tail, all of that appears to have dissipated. This makes it very hard to tell if we're really seeing such an exotic object or just a particularly faint but normal dwarf galaxy. 

Fortunately, there's one more object which has an optical counterpart.

3) AGESVC1 266

Figure 12 from the paper. The detection is quite clear in the VLA spectrum but only marginal visible in the map, hence the wobbly contours (dashed green are negative). White labels show possible optical counterparts : BSG is not Battlestar Galactica but the Brightest SDSS Galaxy; T2016 is something I identified previously; D2025 is that found by Dey.

Oi ! You said two objects with optical counterparts, what're you playing at ?

Not quite. I said we could identify two optical counterparts thanks to the VLA data. This one was identified completely independently (by Dey again), and to be honest, the VLA doesn't help here. Dey has that advantage of getting optical redshift data which matches the HI... without this, I would never in a million years believe their optical counterpart. 

I mean, look at it. It's pathetic. As galaxies go, it's utterly shite. It's a total miserable failure.

And that, of course, is what makes it so interesting. In both the previous cases, the mass of gas is a few times more than the stars, maybe approaching a factor of ten. That's pretty extreme... but in this case the ratio is more than a thousand. That's into crazy territory*.

* There are caveats to figures like this. Estimating the stellar mass becomes extremely difficult for things this faint, but nevertheless, it's clearly exceptionally faint – the exact numerical value isn't all that important.

A mass to light ratio of over a thousand, you say ? Might as well stick a handkerchief on your head and two pencils up your nose.

This one seems very much more convincing as a blue blob / ram pressure dwarf. Here the HI and the optical appear offset and the gas is diffuse, not compact. We may be witnessing the birth of a long-lived blue blob (a whole new class of stellar structure, as Jones and Dey and others have been investigating), or possibly just a brief flicker of star formation before the whole thing dissolves into undetectability. So birth or death ? At the moment, we just don't know.

---

What does it all mean ?

It's a bit of a mess. We have eight clouds in total and we observed six. Maybe one day we'll try for the other two, but right now we know nothing more about them. So of those six :

• One with a high line width turns out to have a tiny, compact optical counterpart and a great big stream. This looks convincingly like a stripping galaxy.
• Two have extremely faint, fuzzy optical counterparts. One of these looks like a good candidate for being a ram pressure dwarf, while the other is plausible but uncertain.
• One more was just about detected, but still has no clear optical counterpart. It doesn't look like it's rotating, but the detection is weak so we can't be confident about this.

The other two were not detected. That's expected if they lack any compact gas, which would mean they're similar to the detected-but-dark case of the last bullet point. But we should also bear in mind just how faint some of these optical counterparts can be : just because we haven't found one doesn't mean they don't exist. We would never have spotted the counterpart of AGESVC1 266 ourselves – only Dey's optical redshift allowed that. So these undetected objects could still be optically dark, but we just don't know.

Even so, we now have enough data that we can finally start to say what's going on.

Wait, wait...

I almost forgot about the legitimate clickbait version !

Twenty years ago they found dark clouds in the depths of space. We finally know what's going on.

LISTEN UP JOURNALISTS AND LISTEN GOOD. You get to say, "we finally know" if it takes nearly two decades and the answer is pretty clear. You do not get to say it if somebody comes up with a vague idea a week after a slightly odd discovery. Got that ? Good. I'm glad we had this talk.

Ahem.

Anyway, the dark galaxies hypothesis used to seem tenable because we couldn't really explain their high line widths in any other way. But we don't see any ordered motions in any of these objects, and one shows a clear stream. That means the kinematics likely has nothing much to do with the gravitational field at all, so the dark matter interpretation – which must be said was the most astrophysically exciting one – probably deserves to be chucked out the window.

I know the ejected guy is normally supposed to come up with a sensible idea, but dark galaxies did used to seem like the best explanation. Anyway I don't care, so shut up.

What looks almost certain is that we were being deceived by the apparent similarity of the clouds. It looked as though they all had about the same mass, line width, similar levels of isolation, and lack of optical counterpart. We now know that's not entirely true : the line widths vary (in some cases our new measurements reduce this compared to our original estimates) while some have optical counterparts – which themselves have varied properties. 

That means the number which we can attribute to being some form of dark debris is not eight, but maybe five at the very most... and likely less, as two weren't even observed. Couple that with the variation in properties and the debris scenario begins to look a lot more plausible. Instead of saying "they're all dark galaxies" or "they're all debris", it now looks more like some are extreme galaxies, some are new kinds of stellar structure, and some are debris. No individual explanation has too much of a burden to bear any more.

These objects are still weird though. For one thing, how come compact objects like AGESVC1 231 seem pretty good at retaining their gas even when larger ones like VCC 1964 have it all removed wholesale ? Why does it get fully displaced in one case but elongated in others ? More fundamentally, what governs when the HI gas is destroyed as it's removed from giant galaxies but can apparently survive and prosper in the harsh environment outside its friendly parent galaxy ? That's a bit weird.

And perhaps of more immediate interest are those blue blobs. In a spectacular pivot that would shame a politician, I might well find myself switching more to objects which lack dark matter rather than having too much. The objects in this paper are not, it turns out, the only such blue blobs in our data which show dynamics like this. Given that VCC 1964 shows a similar lack of dark matter, it's tempting to wonder if there's a connection between BBs and Ultra Diffuse Galaxies. Right now, all I can say is that some of our other BBs shown even more extreme gas ratios and dark matter deficits than the objects we examined here... the times they are a-changing, but the future, it seems, remains as dark as ever.

Masochism For Fun And Science

How can you trust your own senses ? How can you be sure that what you're seeing isn't all just some kind of elaborate illusion set up by a powerful entity with a truly warped sense of humour ?

Don't worry, this isn't going to be much of a philosophical rant. Actually, it's going to be an extended rant about how I hurt myself by looking at over 170,000 pictures of static.

Yes, really. All of which relates, of course, to radio astronomy. It's time for the public-friendly explanation of my latest paper, in which I pit human vision against a set of different algorithms and find that there's life in us old monkeys yet. I also inflict things upon myself that I hope never to repeat ever ever again.

1) How To Hunt For Gassy Galaxies

Regular readers will be only too well aware by now of what galaxies look like in neutral hydrogen (HI - "H one") data cubes. There's a longer explanation here, but in general they look like this :

Here, as in the above GIF, we see slices of the data cube as separate images, though we can also view them all at once as a 3D volume. Now I've waxed lyrical on the joys of volumetric data visualisation many times, but not today. It turns out that the image sequence approach isn't as fun, but is usually better for sensitivity. Which is what I wanted to quantify in my paper.

So how do we actually decide what's a galaxy and what isn't ? Clearly, some bright signals are obviously real, but plenty of others are much fainter, and it isn't always obvious if they're from actual galaxies or just the result of noise. 

We've got two options. The classic method is to do it by eye, visually trawling through the data cube slice-by-slice, image-by-image, recording the coordinates of wherever we find anything that just "looks like" a source by some criteria. The other, increasingly-popular approach, is to rely on algorithms to find the sources for us – maybe using people to check the algorithm's catalogues, but maybe even trusting the algorithms completely.

"Yes, yes," you say, "but how do you ever know if what you've found is real ?"

Indeed. That's a question that plagues both visual and automatic searches. Oh, sure, it's fine if you find a great big blazing beast of a galaxy (like some of those in the above animations), but what about when you've got something more piddly like this ? How could you be sure it wasn't just some slightly brighter-than-usual bit of noise ?

VC1_304, more usually known as NGC 4309. On the left is the source (highlighted by the white outline) as it appears when inspecting the data cube – it's barely visible even with training ! It's only a bit clearer in the spectrum, shown in the middle. Fortunately this object is optically very bright indeed (right), but most objects with this little gas tend to be much dimmer.

Well, there are a few measurements we can make of the gas itself that give us some clues : quantitative values can be a lot more reliable than simply eyeballing it. But first, in the above images you can also see optical data alongside the radio, and that's a very powerful verification check. Gas clouds without associated optical emission aren't non-existent, but they're extremely rare (about 1% of all HI detections by some estimates). You'd be forgiven for missing this one based only on looking at the data cube, but here at least the optical galaxy is unmistakable.

Understandably then, hardly anyone relies exclusively on the original HI data. We almost always have some optical data to act as independent confirmation (except where the Milky Way blocks our view in the aptly-named Zone of Avoidance) and can usually get follow-up radio observations to directly confirm at least the most interesting signals. That's the absolute gold-standard for verification, and how we know that most optically dark sources aren't real : we've checked them many, many times, and indeed still do when we find anything interesting.

But... what if we can't get either of these ? Exactly how good is the eye at distinguishing signal from noise, and what fraction of the faintest signals does it pick up at all ? That's what I wanted to answer with this paper.

2) Yes, But Why ? And How ?

The key aspect of the problem is that, once you get down to the faint stuff, there are no objective criteria, no magical algorithms, that can 100% reliably distinguish between real signals and noise. Some truly pathetic signals turn out to be real while some fairly convincing ones end up being discarded as worthless junk. The only way to be really sure is to do more observations. 

BUT... algorithms are at least objective and repeatable : throw 'em the same data set and search with the same parameters, and you'll get the same objects every time. Different search methods or parameters can give you different catalogues, but at least if you keep everything the same, you'll get the same results again and again. That's a big advantage over using squishy, emotional humans that might get distracted because they haven't had enough tea or they got sick or their pet hamster died or something.

I thought about refining ChatGPT's weird take on this, but decided the bizarreness of putting the hamster in a box labelled NO TEA was just too funny to alter.

How much does this matter though, really ? Exactly how good are humans compared to algorithms ? Can we even quantify it, or are we all such an emotional, whimsical bunch of wet blankets that we just come up with totally different results every time ? Or if you're a Daily MFail reader, HAS WOKENESS KILLED ASTRONOMY ?

The only way I could see to test this was to look for lots and lots and lots of sources, all with different parameters. Throw enough sheer statistics at the problem and it ought to be possible to see if human abilities could be quantified or not. 

Now to do this requires we have full knowledge of what's there for us to find. Ordinarily this isn't the case at all, because the whole point of the problem is that we don't know for sure which sources we've missed. So the only way to do this is by using fake sources – only then can we be absolutely sure if we've found everything. The basic idea is very simple : to try and find as many artificial signals as we can and measure their parameters.

Of course, for this we also need a data set which doesn't have any actual galaxies in it, otherwise we'll confuse the artificial signals with real ones. Fortunately one of our unpublished data sets includes just such a cube, spanning a frequency range in which real signals just can't happen. To find emission here would require galaxies moving towards us at insane velocities, many thousands of kilometres per second – no real galaxy is known which moves at anything even close to this*. Bingo ! We've got a real data cube with all the imperfections of real observational data, but with absolutely no real galaxies in it. Perfect.

* Redshifts of this magnitude are normal, due to the expansion of the Universe. But there the galaxies are moving away from us. Here we'd need galaxies of extreme blueshift, and there's no known mechanism by which this could happen.

Once we've done the detecting, how to parameterise the results ? Well, with any catalogue it's important to understand its completeness and reliability : that is, what fraction of the sources present it detected, and what fraction of its detections are real. With enough sources we could also see if the eye is especially sensitive to particular properties, like the total brightness and velocity width, and maybe also figure out what sort of false signals fool the eye into thinking there's something present. And I also wanted to test the street wisdom that, apart from speed, humans are generally better than algorithms when it comes to sheer detection capabilities.

3) The Experiment

Figuring out the best approach took a lot of trial and error. The simplest method would be to inject lots of signals into a single large data cube, but this wasn't feasible. This would mean I'd have to mask each galaxy as I went along to avoid cataloguing it twice, which is... not a huge amount of work, but it adds up. And for an experiment of the scale this one became, this would have been unbearable.

The problem is that galaxies themselves have two parameters which control their detectability : their width and their brightness. Here's an example spectrum I use in lectures :

What this is showing is a signal of fixed total flux but a varying velocity width. At the very beginning, all that flux is confined to just a few velocity channels, so it's very narrow but bright. Even though it's so bright, because of the way we typically display the data, the narrowness of the signal makes it hard to spot. As the movie advances the velocity width increases, so that it gets wider and wider but appears dimmer and dimmer. At first this makes it much easier to see : it's still bright but it's no longer narrow, so it's really obvious that there's something atypical here. But eventually that flux is spread out over so many channels that it's barely distinguishable from the background noise at all, even though the total amount of flux is the same throughout the animation.

I've long thought it an interesting question as to which one matters most. If a source is wide enough, does this compensate for its dimness ? Or is it brightness alone which determines detectability ? My PhD supervisor took it for granted it was the latter, but I was never quite convinced of this.

The only way I could see to tackle the problem was to inject many galaxies each of a given width and brightness. I'd inject, say, 100 with some combination of values, see how many I could find, and then repeat this ad nauseum. I'd need to have plenty of objects for each combination to get a statistically significant result. Since I had very little clue ahead of time where exactly the detectability threshold would be, this would mean injecting a lot of galaxies.

That made the idea of using a single cube a complete non-starter. Eventually I figured out a working strategy, which goes like this :

1. Pick a width and brightness (signal to noise, S/N) level of the signal.
2. Extract 100 small "cubelets" at random from the main cube.
3. For each cubelet, randomly inject (or not inject) a signal of the specified parameters, at a random location within each one.
4. Modify my source extraction program so I could go through each cube sequentially, just clicking on a source if I thought I could see one, or clicking outside the data set if I thought there wasn't one.
5. Choose new signal parameters and do the whole thing again.

The cubelet is an adorable animal, but, like tribbles, they tend to multiply exponentially if you're not careful.

This was... acceptably fast. Each set of 100 cubelets, containing on average 50 signals, takes about 30 minutes to catalogue. It also made it easy to take breaks, which was absolutely essential. I made it so that every time I clicked to identify a source, I'd be shown if I was correct or not, the result added to a catalogue, and the next cubelet would automatically open. The referee kindly let me get away with language not always typical of an academic paper because here it really does matter :

Without any of these it becomes too easy to be lost in the visual fog - one needs some clue as to what one is looking for or the experience is unendurably frustrating... Being a visual search process, one needs to take much more account of the psychological, emotional experience than in using a pure algorithm.

Finding the initial point was again a matter of trial and error. If I remember correctly, I believe my first guess for the source parameters was too bright so I ended up finding every source (or nearly so), so I kept halving the brightness until the sources became genuinely difficult to spot. From that point I could proceed systematically. The final result is this terrifying table :

Each of those pairs of values represents a search of 100 cubelets. In total I searched 8,500 cubelets, i.e. I looked at 170,000 individual images (slices of data), containing a total of 4,232 sources. But at last, I was done.

This was utterly exhausting. In principle the whole thing could be done in a week; in practice anyone actually trying that would likely claw their own eyes out and hurl them across the room in despair. In terms of calendar time it actually took several months (or... more), and isn't something I ever want to repeat. Fortunately, I'll probably never have to.

170,000 images. I mean, FFS.

4) The Results

The money plot from the paper is this innocuous figure :

Each black circle represents the search of 100 cubelets of a given combination of peak S/N and velocity width. The blue points with error bars show the median at different integrated S/N levels, essentially just a crude fit to the data.

This shows what fraction of sources are found as a function of their integrated S/N (signal to noise). This is a deceptively simple parameter (you can explore it a little more here) that measures how bright the sources are in a more sophisticated way than just the value of the brightest pixel : it accounts for the width of the galaxies as well. The remarkable thing is that the trend is so clear and so tight as a function of S/N_int even thought the experiment was done without any reference to this. Plotting completeness as a function of width or simple peak brightness just gives (more or less) a chart of pure scatter, but this trend emerges like magic. I really wasn't expecting this at all. A parameter designed originally for automated source-finding turns out to describe human vision remarkably well !

You can see that there's a critical threshold. Above 6.5 or so I detected near enough everything, wide or narrow. This value is something we've long known is a good measure of reliability (whether a source is real or not), but now it seems it's also a great way to determine if a sample is complete. If these results are a good reflection of how visual extraction works in the real world, it means we can be confident we've detected every source brighter than 6.5, and about half of all sources at a level of 3.9 or so.

Which by itself is already great news ! Now we can say confidently that above this threshold a) our source is real and b) we've detected all similar sources. In other words we can quantify exactly when our results are unbiased, and also what fraction we're likely to have missed at lower detection criteria.

Lovely. But is this a good approximation to real, in-anger source extraction ? Is the experiment sufficiently realistic ?

5) The Tests

Quite honestly I thought so. The referee, initially, didn't – though quite understandably. So back I went and did a whole bunch more tests just to make sure. Which resulted in this new and improved figure :

This one accounts for several different possible biases (feel free to skip ahead if you already believe the main result) :

• Green points : these test for the fact that I knew ahead of time that the fraction of cubelets containing a source was always about 50%. Here I instead set this fraction to a random number, but lo, the detection statistics were unchanged. In fact even when I went back and searched cubes where I'd previously missed the source, but now with the full knowledge that a source was present, I still couldn't find them. Foreknowledge just doesn't help much.
• Red points : randomised source properties. Here I injected cubelets with three different integrated S/N levels designed to give 25, 50 and 75% completeness levels, but with entirely randomised widths, in random order, without giving any indication of which identifications were correct until the experiment was over. Essentially for each cubelet I had no idea if it contained no source at all or one which would be marginally, modestly, or probably detectable, or what it would look like. This again made no difference to the results.
• Orange points : as above, but now injecting the sources into a single large cube instead of a many cubelets. In the main experiment I knew there was at most one source per cubelet; here I didn't know the number injected (randomised with some sensible upper limit) or their properties. This was about as similar to real-world conditions as it's ever going to get, and it still made no difference.

So yes, after looking at another 918 cubes, I can categorically say that the experiment is realistic. There's one final possible bias, but I'll return to that only at the end.

6) Humans Versus Robots : FIGHT !

"It's a movie about a killer robot radio astronomer who travels back in time for some reason."

Okay, so this robustly quantifies how good visual source extraction can be. The obvious next question is : are we really any better off doing it ourselves, or should we let the algorithms loose instead ?

For this I tried two methods. First, I ran the same experiment again but now using automatic source-finding programs instead. Not quite identical though, because now the 80 GB or so of data was spaced over two machines, and adapting the programs to search many little cubes instead of one big one (as they're designed for) was just yawningly tedious. Instead, I had a script inject sources of a given S/N_int (with randomised widths and peak S/N) throughout the main master empty cube, run the extractors, increase the S/N_int, and iterate this until I'd sampled the whole range of values I'd tested visually.

I'm glossing over a lot of details here, but the end result is another set of unshapely curves :

"Fiducial" is just the main visual experiment. It sounds more sciency than "the one what I did earlier".

Let's start with the thick lines. After much experimentation, these were the settings that gave me the highest completeness rates I could get. With the deservedly popular SoFiA (red), this gives marginally better results than visual, though well within the general scatter. GLADoS, my own obscure code I wrote years ago, does considerably worse – not awful, but it looks a bit pathetic next to SoFiA.

But does this mean that algorithms are actually as good, or even slightly better, than humans ? Actually no, not really. Sort of. Maybe. It's complicated.

The thing is that these curves are all at some given reliability level. It's actually not that easy to show how reliability varies with S/N, but luckily there's an easier way to compare : just count the number of false positives from each method in the same volume. From all the earlier testing, the highest number of spurious signals I found through visual searching of this part of the data was 23. SoFiA, by contrast, found 133, whereas GLADoS found 176.

So that high completeness of SoFiA comes with a huge penalty in reliability. In terms of completeness and reliability, automatic extractors are nowhere near as good as people.

But, they do have a significant compensating advantage : sheer unbridled speed. Even if you have to search a long candidate list from the unreliable algorithms, this can still be faster than visual searching the whole cube. The balance to all this is hard to judge, but there will be a point of diminishing returns – try and dig too deeply into the noise with an algorithm and you'll get so many false sources that the speed advantage will be lost.

Bottom line ? For small cubes use people. Human extraction isn't that slow, it's better than the automated methods (though there's no harm running these as well), and its results are quantifiable. For larger cubes there isn't really much choice but to use algorithms, but at least this experiment gives a rough guide as to how good they can be.

Conclusions and Caveats

Human source extraction is seriously powerful. Over on Little Physicists I've been exploring the wonders of GPT-5, which honestly I've been seriously impressed by for scientific analysis and discussion. But it doesn't hold a candle to billions of years of evolution driving human pattern recognition skills. On that front, for the moment, humans win hands down.

After all, the question "real or fake" ? has caused much debate elsewhere. Methinks I ran the wrong experiment.

 

Interestingly, both visual and automatic techniques tend to pick up on the same sources : it's just that the automatic methods tend to a) miss a lot and b) also find a bunch of crap. Still, that does suggest they're doing something right. Keep dialling up their sensitivity and they'll probably find even more faint sources, it's just that the reliability penalty would be unbearable.

And that's the fun part. How exactly does human vision do so much better at rejecting all the really faint false stuff ? For now we don't know. There are many possibilities, but after looking at this many god damn fecking stupid smudgy feckless bloody false galaxies... I don't intend to investigate this any time soon.

More pragmatically, these results mean we can now say when our results are statistically meaningful, and also give us something to ram down people's throats if they ever dare to suggest that visual extraction isn't any good. But they also show that there's something about human vision, the thing we generally rely on the most for making sense of the world, that we don't fully understand.

And there's one final head-scratcher to end on. That other bias I mentioned ? The referee suggested that maybe in the real world we wouldn't catalogue such faint signals because we wouldn't be able to verify them. "Humph !" I thought, a tad disgusted by such an insinuation. So I ran yet another test, this time injecting fake sources into data cubes also containing real sources. The idea was that if we had missed a lot of real sources in our original search, in this new search we should find the fake sources plus some more real candidates.

We didn't. The fake sources were found at the same rates as before, so the search was just as effective as in the other tests... but hardly any new candidates for real detections. Great ! But... the odd thing is that the real sources are all much brighter than what we should be able to find. Where have all the faintest sources gone ? Not a clue ! Maybe, eventually, we'll get some actual proper science out of all this as well as all these dry statistics. Well, you never know.

You Can All Get FRELLED

New Year, New Software

How to start the blogging for the New Year ? Some topics are obligatory, like the current, shockingly abysmal state of global politics. Don't worry, I'll get to that. But I prefer to start on a more optimistic note, and with commendable timing, I had a paper accepted on New Year's Eve*. So let's do that instead.

* Presumably due to a drunken intern at an office party accidentally pressing the "accept all" button.

Presenting the FITS Real-time Explorer of Low Latency in Every Dimension. From the Farscape pseudo-swearword meaning a sort of combination of "hell" and "fucked". Hence the title of this post is extremely insulting.

The physics of galaxies is a jolly interesting thing, but you can't very well study what makes galaxies tick without having a sample of galaxies to start with. Finding the gassy buggers can be a right pain in the backside, so my major side-project is developing better ways to detect them. Sometimes that means writing fully automatic code that turns data directly into catalogues, but I prefer the old-fashioned approach of actually looking at the data we spend some much time and effort collecting. 

My main visualisation effort has been a code called FRELLED, which long-term readers will know is something I've been working on for a very long time. Basically it turns everyone's favourite 3D art software, Blender, into something astronomer-friendly.

... or at least, that was the idea. In practise... eeeehhhh, not so much. When it worked, it did work. You could load a 3D data set and fly around it and catalogue sources and everything ! It was great. I couldn't understand why it hadn't gone viral and made me millions of pounds from a grateful astronomical community. 

Two reasons. The obvious one is that astronomers prefer to spend their millions on telescopes and suchlike, rather than giving random donations to obscure Welshmen who give their codes away for free. The second, more interesting reason is that when FRELLED didn't work it, err, didn't work. It was pretty versatile in some ways but hopelessly limited in others. The learning curve was better than learning Blender from scratch but still pretty steep, it was full of bugs, and calling it astronomer-friendly would be overly-generous. It was astronomer non-hostile at best. 

It's very easy to provoke astronomers, mind you. Especially with clouds.

Lockdown finally provided me with the incentive to undertake the formidable task of rewriting FRELLED's ~11,000+ lines of Python code to bring it into the modern world. And not just because it needed an overhaul, which it did. But also because it had finally become simply unsustainable. With appalling timing, the earlier version had been written for Blender 2.49, which became obsolete almost  immediately as the very next version of Blender totally rewrote all of Blender's Python syntax*. This was hugely irritating : just as I'd got the hang of Python in Blender, they went and rewrote the whole bloody thing. I had the choice of either sticking with the functioning code I had or immediately starting from scratch.

* Python users – this was not like the upgrade from Python 2 to Python 3, which was relatively minor in comparison. No, this was a complete overhaul, so that virtually every internal Blender Python command was replaced with something totally different.

Well ! Naturally I chose the former, since I do have to do actual science as well at some point. I don't have the luxury of spending 100% of my time developing software – I need to actually use it. So FRELLED persisted in an abandoned version of Blender for several years, frequently tinkered with, never fully renovated.

But then lockdown happened, and recoding FRELLED was a much-needed project that could easily be done from home. So off I went and started coding. It helped that I'd already familiarised myself with more recent Blender versions for other projects.

This took quite a while.

Then, it took a while longer.

Things got derailed at various points because of the need for other projects.

But eventually, with the code doubling in size to 20,000+ lines and a bunch of new features, it was done ! Hurrah ! Let joy be unconfined, let there be dancing in the streets.

And so we all got wasted and did tonnes of cocaine or something. Although to give an idea of how long these things take, the paper alone took six months from submission to acceptance. That's after spending a full six months just figuring out the right journal to submit to.

Or at least, it was done enough. A project like this never really ends, there are always bugs to squash and features to add. But it reached a point where it did everything I wanted to do with a satisfyingly low number of bugs that one wouldn't need a flyswatter to turn it on.

I already did a somewhat in-depth look at how FRELLED works more than ten years ago (!) so I'm not going to go through it all again. Here I'll just look at what you can do with it and why you'd want to bother. If you're looking to actually use the code, the best place to get started is the home page, wherein you can find links to the embarrassingly large wiki, a complete set of video walkthroughs, and a 45-minute I-TRAIN video as well. It's got documentation in spades, and of course, you can always get in touch with me for help.

What Does It Do ?

This. It does this :

The main thing it does is look at 3D data cubes inside Blender. If you want the nitty-gritty on this see the earlier blog post, but the basics are pretty simple. Radio observations of galaxies often generate 3D data : as well as position and brightness on the sky, we also get velocity. That's not the same as distance, but using it in place of distance makes it easy to see all of our data all at once. And numerical simulations also create 3D data sets, including ones where the third axis really is distance.

The old version of FRELLED had a lot of manual coding in it because Blender 2.49 simply couldn't access a lot of more modern Python modules. That was hugely inefficient and wasteful, because Python is the equivalent of Ms Universe for astronomical codes : immensely popular. It's increasingly rare to have write your own code for basic astronomical tasks – chances are there's a Python tool for most operations. Not the high-level stuff, but for the low-level everyday processing, you're better off spending a little time searching for someone else's code than writing your own. Which makes it relatively easy to cobble all these snippets together to do the larger task you want them to do.

"Here it is, looking weird !" FRELLED is probably the short one with lots of tentacles.

What this means is that the old FRELLED had a lot of minor but annoying problems when looking at data of anything except 21cm emission. The new version, by using modules which do all the tedious coordinate calculations for me, lets it look at any wavelength with minimal fuss (at least for radio frequencies, at any rate). And cubes now load faster, sometimes very much faster indeed, and it can load data sets up to 27 times larger than the old version. Actually I've got a promising technique to increase this by maybe another order of magnitude in the works... but that's on the backburner for now.

There's more than one way to skin a cat, and there's more than one way to view a cube. Volumetric rendering, as above, is where you see the whole data set all at once. It's by far my favourite method but it's not always the best choice. So FRELLED also supports viewing cubes slice-by-slice, as it did before, but also with a couple of fun new additions. First, it can also render data slices as height maps (you might remember this from a solarigraphy conference) :

This is a nice example because it's a cube I've looked at a whole bunch of times and yet one of the major features escaped me completely. It's the galaxy M33, and in a standard map, you see the colours saturate at high values – in this case shown in red. The problem is that the dynamic range of the data is huge : the brightest features are much, much brighter than the faintest, maybe by a factor of a thousand. Choosing a colour scheme that can show features at all levels is really tough.

Same data set with the same colour scheme, but with different vertical stretching for the height maps (linear on the left, logarithmic on the right).

Height maps provide a nice way around that. As well as using the conventional colour map, each pixel is displaced by an amount according to its value. You can use different ways of mapping the data to colour and displacement (height), so in effect you're showing the data with two colour schemes at once. For example, as here, you could choose the colours to highlight the faintest features, while using the height to examine the structure in the brightest bits of the data. In this case, I'd never noticed the twin peaks in the galaxy, even though they're the dominant features !

Another major new feature is the ability to display the data as isosurfaces. These are like contours but in 3D, showing surfaces of constant data value. I have to say I originally poo-poohed these as a cheap excuse for 3D rendering : sure, it's easy to display a few thousand polygons as a surface, any fool can do that... what about rendering the hundreds of millions of voxels of the full data set, eh ? Honestly I really only put them it because I could. But I began to realise that I'd been far too harsh, because sometimes less is more. By reducing the data to this sort of very carefully, quantifiably selected display, all of the intervening noise is cut out completely. Ironically, the key to finding the most interesting features can be to remove most of the data. That goes well against my training and instincts as someone interested in the faintest objects, but it works.

And finally... virtual reality ! I love VR, I think it's one of the best technological developments of recent years. Yes, even if Mark Zuckerbot is an obsequious twat who's blatantly pandering to far-right despotism... a lot of people seem to confuse the worst sort of techbros (who are awful) with the tech they develop (which can, sometimes, be a genuinely good thing). But I digress.

Anyway, Blender 2.79 doesn't support real-time VR, so FRELLED comes with some export scripts to convert the files into a format that can run in later versions, tested up to 4.0. So you can step around your data either as isosurfaces or volumetrically, as though it were in the living room with you because why the hell not you crazy person for even asking.

Actually, I was a bit skeptical that VR would prove to have any scientific justification. I wanted this feature solely because it's cool, and if more people developed things because they were cool then the world would be a happier place*. Whether it had any real benefit was entirely besides the point. 

* Or perhaps not.

But actually, I have to say there probably is real value to this. It goes like this :

• Viewing data in 2D, slice by slice, takes a long time to build up an intuitive sense of its full 3D structure.
• Viewing the data as a 3D volume you can rotate on a 2D screen helps develop that intuitive sense much faster, because you can see all the data at once.
• Viewing the data in true stereoscopic data helps even more. Now you don't even need to rotate, making it even easier to grasp what's a coherent structure in the data and what's just a chance alignment (so-called projection effects). You instantly see the 3D structures directly.

There's even another program, iDaVIE, which is fully in VR with almost no 2D interface at all. I like it very much (and the performance is oodles and oodles better than FRELLED), though for me the analysis tasks I need aren't quite there yet. Which brings me on to the next section.

Why Should You (Not) Use It ?

You should use it because you're a cool person who loves awesome things ! As the wiki says, it's as much fun as a pile of kittens, and what right-thinking person doesn't love a pile of kittens ? Nobody, that's who.

Well... maybe I exaggerate. A bit. But it should be, I hope, reasonably user-friendly. It certainly won't yell at you or call you rude names in Welsh like the old version did. There is, unavoidably, a learning curve, but it's probably now (almost) as shallow as I can make it. 

Felix Stoehr coined this wonderful term "fastronomy", which is basically something I've mentally labelled as the Facebook Principle for many years. The referee wouldn't let me keep this analogy in the paper, but at its heart, Facebook is little more than glorified email. You could, in principle, set up a vast email list of thousands of strangers and send them pictures of your cats and/or genitalia, or rant at them about how cancer is causing vaccines or why pigeons are all mind-controlling robots. Nobody does any of that by email; lots of people do some of the above on Facebook. It's even considered normal*.

* I dunno why the referee objected. I barely even mentioned genitalia in the paper, let alone robot-pigeons.

Why is that ? Probably because Facebook et al. make otherwise awkward social interactions convenient and casual. And this is something astronomers have been traditionally very reluctant to acknowledge. As discussed at length in Malta, we often focus so much on accuracy that we forget about interface, making mistakes so easy to make that the result is inaccuracy – the very thing we were trying to avoid ! So fastronomy is something I buy into enthusiastically : we should make features as easy and accessible to use as we possibly can.

Surely this is just self-evident ? Yes and no. There is a school of thought that says some things should be challenging and hard to do. And to an extent they've got a point. Some tasks are indeed really very difficult and can't and shouldn't be deliberately over-simplified : that risks fooling people into thinking they immediately understand things when in fact they need years of training. And on occasion this legitimate concern slides into elitist snobbery. Far more common, though, is that we as astronomers just have too much to do to spend their time doing good interface/experience design, though I'm glad to see that there is at least a movement towards encouraging more user-friendly software*.

* One of the stranger objections, which to be honest I couldn't quite get my head around, from the referee was whether or not there was any real evidence that by making things easier to use people would actually use them. I mean, yes, actually yes I do, lots, but the whole question seemed strange to me. Who in their right mind would prefer the more difficult option if the easier one gave results of equal quality ? That bit does seem self-evident to me.

So in that spirit, every operation in FRELLED is accessible via the GUI. You don't need to ever see the code. Every graphical element has an explanatory tooltip. Some things only possible in the previous version by learning some rather hacky tricks from the manual have now been properly and robustly integrated. And every tool panel comes with a simple help menu explaining its individual functions. Once the very basics have been grasped ("here's how to open a file"), most other operations should come naturally. If I can't quite make it so that you can download it and go, I can least make it so that after 5 minutes or less you should have a fair idea of how to do most of the stuff you'll ever need to do.

Which leaves my final point : versatility. There are a lot of good visualisation tools out there, and a lot of good analysis tools too. Much less common are tools that do both. What I've tried to do with FRELLED is seize this unoccupied middle ground and strike a balance. It doesn't do the most advanced visualisation possible but it does have multiple different ways of viewing data : most programs either do 2D or 3D, rarely both, let alone giving a choice of volumetric or isosurface displays. And while it doesn't do every analysis operation possible, it does enough of the basics. At least, for what I need to do. The goal is that users shouldn't have to switch between packages unless they need something which is really quite specialist.

... bugger, I forgot to mention the analysis tools ! I should give a brief rundown, I guess. One of the main things is you can easily click on a source and create an object you can use for cataloguing, masking, and analysing the target. You can give it a human-readable name so you'll remember it and find it again later. This all sounds trivial and honestly it is but nothing else seems to let you catalogue sources quite like this, which to me is still bizarre (though I gather that CARTA has something similar). You can then use those objects for overlaying optical data, adding annotations and axes (both for creating figures but also for training new observers), generating contours plots, and even doing some basic spectral measurements if you also have miriad installed. It does, in short, far from everything, but still quite a bit.

Example figure of some HI contours overlaid on an optical image, created entirely in FRELLED. It sounds minor but the the colour bar on the right was created automatically, which was otherwise a mind-numbingly tedious process that we'd only ever do if we were sure we'd got things just right. Making that automatic allows the user to experiment. The annotation objects and labels were created just by placing the cursor, typing in a toolbox and pressing a button. No need to open PhotoShop ! 

The interface plays a role in versatility too. There's a lot of options for setting things that previous were only visible under the bonnet, i.e. by tinkering with the code itself. Now, for example, you can manually configure the setup to optimise the display for smaller files, or (if you're feeling brave) to go for even larger data sets – all of which include checks to prevent the user from doing something really stupid to the file, or for correcting their mistakes if they insist on being a damn fool about it.

And what I really hope people will do is things I can't predict. I want them to be able to experiment with viewing their data in different ways, rather than sticking with what they already know. I want them to try stuff because it's so easy there's no point in not trying it, without feeling like they're going to muck up they're whole project if something goes wrong. And I want them to be able to load as many different sorts of data sets as they can, not be restricted to just one particular frequency. 

Most of all, I'd like users to feel like they're doing data visualisation (and taking it seriously as a much-needed aspect of analysis) without it feeling like it's a yet another skill set they need to learn. Users should, I hope, be able to learn Blender as astronomers, not become fully-fledged Blender artists who are also interested in science. Ultimately, making fun-looking 3D figures should be normal and for everyone, not a cool-looking bonus option that requires yet more specialist training in an already overloaded discipline.

What's Next ?

Aaaargh ! It happened again ! When I first looked in Blender post-2.79, the testing versions of 2.8 were available that had a version of the EEVEE internal real-time rendering engine that didn't work well. In FRELLED, the cube stays visible at full resolution and a high frame rate. In the pre-release of 2.8 there was a huge, horrible lag in updating the display which would have made it unusable. You can't freely explore a data set like this unless you're a masochist.

Then in the actual releases of 2.8 and later versions this was overcome. And they offer a really powerful extra option : you can adjust the colours of the images after loading them in, which you can't in Blender 2.79. One of the major restrictions of FRELLED is that loading the data sets can be quite slow in comparison to other programs, especially since if you get the colour scaling wrong you can't adjust it. So this alone makes recoding FRELLED, once again, worth doing. It would also give greater performance, allow for larger cubes, have direct viewing in VR, and a bunch of other things.

Recoding FRELLED the next time round won't be the same level of challenge as here, not even close. The Python syntax is more like a modified, incremental change from that of Blender 2.79 with only a few parts which were completely replaced. So this is doable, but it's not happening anytime soon because I did to some actual science again (watch this space). There are, though, a bunch of other features I'd like to add when I eventually get back to this :

• Support for really large cubes (tens of gigabytes) and higher performance for smaller ones
• Automatically overlaying data from online catalogues, to make source identification easier
• Bring back the particle viewer from the old version (I had to drop this just because the project was ballooning into the unsustainable)
• Allowing 2D files, just for the sake of it
• And a whole bunch of under-the-bonnet changes that would make the experience still more natural and faster.

---

So that's this paper. No science here, just a development that's been in progress since 2012. Maybe this, the 13th year, will be the one in which finally people start to use it. Well, you never know.