Into Darkness (I)

My latest paper - sixth as first author - is now online. It's been in draft for about 18 months (as long as two pregnancies, so you decide which was the better choice...), taken over 200 simulations, and the resulting monster is 27 pages long. What great quest could possibly have warranted such fanaticism ? Find out in this and the next blog post. Or if you just don't care that much but are vaguely curious, read the super-short version here. If you don't want the background information, you can do directly to part 2 here.

To start, we have to go much further back than the last 18 months.


Everyone Should Have Their Own Life-Defining GIF

Back in the heady carefree days of 2006, I was a fresh-faced* and fancy free** Masters student about to embark on a PhD course. The direction this took was largely thanks to this, the single most important gif of my life so far :

*Spotty.
** Hapless.

I have, of course, explained such 3D data cubes many times before, so let's keep this simple. What you're looking at is a map of part of the Virgo galaxy cluster as seen through a radio telescope. Instead of looking at the stars, you're seeing the hydrogen gas. It's easier to explain in a 2D image. Let's compare what we'd see in "normal" visible light with these gas-sensitive radio wavelengths  :

The really bright big blob you see in the top gif spinning around is the gas in the spiral galaxy, NGC 4254. You can also see it has a long stream of gas, which apparently ends in a bit of rather denser gas that's the object known as VIRGOHI21. Pronounced, "Virgo H one twenty one". "H_I" is really H Roman numeral 1, and is the symbol for neutral atomic hydrogen (as opposed to H₂, which is molecular hydrogen, and H_II, also pronounced "H two", which is ionised hydrogen). "21" is just the object's catalogue number in the survey.

If this is confusing, don't worry. Forget the molecular and ionised hydrogen, they're a bunch of smelly pants. We're only looking at H_I . And the visible light from stars, but that's not so important either. The important point is that by measuring the gas, we can measure how fast everything's moving. If we're clever, we can use this to work out if things are rotating and the speed of rotation if they are.

The interesting thing about VIRGOHI21 is that it appears to be rotating. And if it's stable, just like ordinary galaxies it would also need a lot of dark matter to hold it together. So when I started my PhD, it was being widely touted as the first detection of a so-called "dark galaxy" - a dark matter cloud (or "halo" as they are usually known) containing a rotating disc of gas, but no stars.

Actually it wasn't the first time such a claim had been made at all, but I'll get back to that.

You might well be thinking, "Hang on a minute, you four-eyed beardy twerp. There's a great big complicated stream of gas coming out of NGC 4254, so how do you know this VIRGOHI21 thingy isn't just a kink in the stream ? Eh ? How do you know it's really a massive rotating disc ?". Well, we don't. To understand why we'd look at this object and say, "dark galaxy" and not "weird kink", we need a bit of backstory.


Galaxies Don't Need Stars, That's Just RACIST !

The theoretical basis for dark galaxies was that they could solve the "missing satellite" problem. Models predict about ten times more dwarf galaxies than we actually observe, so the idea was that maybe some galaxies just never have enough gas to form much (or any) stars. An elegant solution to a complicated problem that I'm grossly over-simplifying. It's one of the biggest problems in contemporary cosmology, so shouldn't be taken lightly. Any proposed solution has to be treated with strong skepticism*.

* And no, fellow Brits, I refuse point blank to spell it "scepticism". That would be pronounced like in septic, which would be unpleasant for all concerned.

The observational measurements just say that VIRGOHI21 could be a massive rotating disc. Or it could be a much less massive kink in the stream, which might have formed when another galaxy flew past and pulled the gas out of NGC 4254. The idea here is that it looks like it's rotating, but isn't really. But there was some pretty intriguing other evidence, at the time :

• Unusually, NGC 4254 has one particularly prominent spiral arm. This could be the result of interacting with a massive object.
• There's no obvious normal galaxy visible nearby that could be responsible for the prominent spiral arm or the gas stream.
• The kink in the gas stream is very sharp compared to the rest of the stream. Why should only this one feature be so strange ? Few (if any) other streams show features like this.

If VIRGOHI21 really is rotating, it would be massive enough to explain the disturbance to NGC 4254. Galaxies this massive and dark would be controversial, but some models said they were possible. The very sharp kink in the velocity would be because VIRGOHI21 is a compact object, which is perfectly consistent with the observations. And based on what was known about galaxy interactions at the time, it was very difficult to see how an interacting galaxy could produce a stream with all these particular features.

My Masters project had been all about simulating the formation of a spiral galaxy without dark matter. Since this hadn't worked, my natural skepticism of dark matter had taken something of a blow. Not a mortal wound by any means, but enough to give pause for thought. In any case, VIRGOHI21 had turned up in a relatively low sensitivity survey of the cluster. Having access to much deeper AGES data (the Arecibo Galaxy Environment Survey, one of the most sensitive surveys of the cluster ever undertaken) as part of my PhD, the prospects for detecting other dark galaxy candidates looked good. Even if they turned out not to be real dark galaxies, it seemed pretty likely that we'd find lots of cool streams and other neat, weird objects.

It didn't really work out like that.

Where Are All These "Dark Galaxies" Then ?

The AGES data I was given charge of for my thesis consisted of two parts of the Virgo cluster. Well, it seemed like such an interesting place that surveying it twice was a sensible enough idea (observing the whole cluster would take much too long).

VC1 is the larger area in red while the smaller area is VC2.

VC2 contains pretty much bugger all. More accurately, it contains precisely bugger all that hadn't already been discovered. To be really accurate, the "bugger all"-ness is confined to new discoveries from the hydrogen data, but that's another story.

Fortunately, VC1 turned out to be a nice happy data cube full of lots and lots of interesting detections. But did it contain lots of hydrogen streams and dark galaxy candidates ? No, no it did not.

Whoa, steady on there, me ! It's true it didn't contain any streams. But the dark galaxy candidates statement is, like, totally way melodramatic. In fact we detected eight hydrogen clouds without any obvious galaxies present. Six of them seemed to be rotating very nearly as fast as VIRGOHI21. So why do I seem reluctant to say that we didn't find any dark galaxies ?

Regular readers will be screaming that this is the most boring visualisation I've ever done. Where are the glass cubes ? Where are the 3D movies ? Unfortunately, while HI is often stunningly beautiful, these particular detections are just too faint  and too small to show any details. 

There's No Way A Bunch Of Poxy Lines Are Going To Convince Me You've Solved The Missing Satellite Problem.

By this point the overall situation had changed. A rather convincing simulation had been published demonstrating that VIRGOHI21 could, in fact, just be the result of an ordinary encounter after all. You may well have been right to doubt my four-eyed beardy twerpiness proclaiming it to be a rotating disc, because this model said that it was indeed just a kink in the stream. It seemed that it's just an illusion of rotation and therefore totally uninteresting.

I'll save the details of that simulation for next time, though most people were rather swept away by the result. Cooler heads noted that the proposed model was far from perfect, but at the time, most people thought these flaws were minor details that needed ironing out - not gaping holes that meant the whole sorry thing should be burned and the ashes spat on and scattered to the four winds.

A perhaps even more serious problem was the extremely low numbers of these candidate dark galaxies. My supervisor's model had predicted that as many as 23% of our survey's detections should be dark. Even with these objects it was at most 1%, so that particular model was decisively ruled out. Which meant that dark galaxies didn't seem likely to explain the missing satellite problem, which was the reason they were proposed to exist in the first place.

There's a third reason. The Tully-Fisher relation is a very well-established correlation between how fast a galaxy is rotating and how much ordinary matter (gas and stars) it contains. In fact it's so well-established that if you wanted to define a galaxy, you could almost say, "anything that obeys the Tully Fisher relation". While even the faintest (but still optically visible) little smudges we'd found in the Virgo cluster with our survey seemed to obey this relation perfectly, these objects which were apparently totally dark do not.

The faster a galaxy appears to be rotating, the more massive it is. The "Tully-Fisher Relation" just refers to the precise numbers : knowing one, you can calculate the other. That's the red line in the picture. It works pretty well for regular galaxies, but doesn't work for the dark galaxy candidates.

And when I say faint smudges, I mean really faint stuff like this :

All of these pathetic little scraps of starlight sit on the TF relation just fine - even VC2_30 (bottom centre), which looks more like someone's sneezed on the lens than a proper galaxy. They have at least a hundred times less mass than giant objects, but they all obey the same relation. The fact that the truly dark objects don't follow this correlation, whereas the merely extremely dim ones do, marks them out as categorically different.

Taken together - the simulation showing VIRGOHI21 was tidal, the insufficient numbers of dark clouds, and the deviation from the Tully Fisher relation - the case against dark galaxies seemed pretty darn damning.

So Galaxies Have Stars After All Then. So Much For Galaxy Justice Warriors.

And yet... these clouds remained unexplained. Yes, there was one model showing how they might be formed - but only one. Public outreach articles will tell you that science values repetition. Well, sometimes this is if anything excessively true, but not always. In this case - for whatever reason - nobody saw fit to go back and re-examine the model and test it using other codes or explore its implications more fully. And while more clouds continued to be discovered from time to time, this one particular model was almost invariably cited to explain them.

Almost invariably. There were a few for which the basic idea of "tidal debris" (i.e. the gas was pulled out of a normal galaxy during an interaction) just didn't make any sense at all. I've previously described a couple of examples - the Smith Cloud and Keenan's Ring - in detail. At least one other example deserves a look in : HI1225+01, sometimes known as the Giovanelli & Haynes cloud.

Gas distribution shown by the white outline. I'd make a better image but I don't have the raw data to play with.

This enormous cloud is at least 650,000 light years long with a mass of around 4 billion Suns (the stupid poxy little clouds I found are at most 55,000 light years across and have masses no more than around 30 million Suns - more than ten times smaller and a hundred times less massive). True, HI1225+01 has a small galaxy at one end - but it's very small indeed. Far too small to be a credible source of all that gas. It only remains to choose which confused animal meme seems most appropriate.

This dog is exactly like the gas in HI1225+01. It has no idea where it is or what's going on. Also it's a bit damp and surrounded by adorable otters, which is probably where the similarities end.

Yes Alright, But The Universe Is A Jolly Big Place, And There Are Bound To Be A Few Weird Things We Can't Explain.

Indeed. But there seems to me to be enough weird objects that we should stop and take a good hard look at what's going on. To try and make sense of all those otters, I decided to catalogue them. That is, I did a literature search for all the really long hydrogen streams and dark galaxy candidates. Then, as best as I was able, I took all the available measurements and made a table of their properties.

The numbers aren't large - around 40 for both the streams and dark galaxy candidates. Even so, finding all of the damn things, combined with the need to re-measure just about everything, took several months. Still, the awkward fact remained that the numbers are thousands of times too low to explain the missing satellite problem.

But the world turned, and turned again. There were hints that there could be much larger populations of some objects in some regions - not as many as models predicted, but enough to make one wonder. A few new objects seemed to have signs of ordered motions - a really strong signature of rotation (the data for the clouds I found wasn't high enough resolution to show this even if present). Much larger cosmological simulations began reviving the idea that maybe not all dark matter halos contain enough gas to form stars. And then there were of course the discoveries of huge numbers of very large but very faint galaxies - not quite dark galaxies, but certainly very dim indeed.

So the mood of the community had apparently done a complete 360. Maybe several. It had gone from originally thinking that dark galaxies should be very common to dismissing them as unlikely when they weren't immediately found by the first large-area H_I surveys. It had turned to outright hostility when they continued to not be found, to the point where alternative explanations for any weird objects became demanded rather than requested. Now it's swung back, at least in some quarters, into a tentative sort of toleration.


Make Up Your Bloody Minds !

If you look hard enough, you'll be able to find a qualified expert with a strong but ridiculous opinion about anything. There were and are certainly some people in this field fervently convinced that dark galaxies do and do not exist. And there's more than a small element of, "my dark galaxy candidate is much better than your dark galaxy candidate" at work. But the more I looked, the more and more it seemed to me that no-one had a really good idea of what was going on. This must, I imagine, be an area that's particularly difficult for journalists - lots of very strong opinions from confident-sounding experts, but no consensus.

Anyway all my cataloguing had left me convinced that my little AGES clouds were indeed particularly weird objects*. None of their nearest galaxies looked disturbed or showed any signs of extended H_I streams. Very few other clouds showed a similar deviation from the Tully-Fisher relation. If their velocity width wasn't due to stable rotation, they ought to have disintegrated so quickly we'd be unlikely to detect them at all. They could be held together by their gas alone... but that would mean they'd have to be incredibly small and dense and form stars like nobody's business. At a more reasonable size, they probably couldn't survive for very long in the chaotic Virgo cluster without dark matter.

* Obviously, I'm not guilty of the my-candidate-is-better-than-your-candidate problem at all, because I'm a special snowflake.

The thing is, none of the original problems with dark galaxies were by themselves fatal. True, not enough had been detected. But we don't really have a great understanding of how gas gets into halos, so maybe most of the dark matter clouds don't even contain detectable amounts of gas. It's a bit like saying, "you told me there were three Easter eggs in this room, but I only found one, so whatever the heck it is I've found, this big chocolatey egg-shaped thing can't possibly be an Easter egg".

THIS IS NOT A MAGRITTE JOKE.

Likewise we don't really understand the Tully-Fisher relation all that well either. Actually the fact that faint galaxies lie on the TFR is surprising ! And these dark little clouds don't, so that becomes very interesting indeed. At least one of the optically faint but not totally dark galaxies also deviates in the same way. So defining galaxies as "anything that obeys the TFR" may be a teensy-weensy bit over-zealous. And here's the really interesting thing : lots of new, extremely faint galaxies have recently been discovered in clusters, and at least one of them (probably more !) seems to be deviating from the TFR just like the clouds do.

Which only leaves that pesky simulation showing that objects like VIRGOHI21 could just be tidal debris. Now here I have to tread a very careful line indeed, because the authors of that study are certainly vastly more experienced than me when it comes to galaxy simulations. And yet the more I examined their findings, the more doubts I had about their conclusions. Their own figures don't show anything resembling VIRGOHI21. Worse, they were so preoccupied with whether or not they could explain VIROGHI21 they never stopped to think if their mechanism would. That is, a possible explanation does not mean it's a very likely one, let alone correct.


Summary

For a topic this controversial, even this long-winded post is a simplification. But it boils down to a few simple points :

• If galaxy formation models are correct, then there must be a lot of dark matter halos which don't contain any stars and gas. Some might contain just enough gas to be detectable but not enough to form stars. 
• But on the other hand, the models aren't that sophisticated. Although all models predict lots of dark matter halos, the numbers predicted to have just enough gas to be detectable but no stars vary wildly depending on the exact values used.

• Gas-rich galaxies with no stars appear to be extremely rare... but, tentatively, a few might exist. That means the models of galaxy formation might be basically correct after all. So even discovering just a few dark galaxies could solve the missing satellite problem.
• But we don't have a great model for the origin of galaxies and alternatives say that what appear to be dark galaxies could just be the relics of old interactions. So using a very few detections to infer huge numbers of truly dark, undetectable halos is rather dodgy, at best.

• Galaxies with very few stars and no gas do exist in large numbers with certainty. Those stars must have formed from gas at some point, so truly dark galaxies should exist.
• But we have absolutely no idea when they converted their gas into stars, or what caused some to remain dark and others to light up. 

• A very few objects are known which are rotating more quickly than expected based on their observed mass - meaning they have extremely high amounts of dark matter. That would explain why they're not on the Tully-Fisher relation.
• Yes, but we don't know if those AGES clouds are rotating because the observations don't have enough resolution. Tidal debris is at least as good an explanation. And even really dim galaxies follow the Tully-Fisher relation (despite predictions that they shouldn't), which suggests that totally dark galaxies should as well.

Let me put it another way :

We don't have a good idea of what's going on here. Anyone who says otherwise is guilty of wishful thinking. The only way to proceed is incrementally, point by point, slowly and very carefully tweaking the nipples of knowledge. Or something. But clearly, understanding these objects relies as much on theory as it does observation. So next time we'll head deeper into darkness and look at one of those key points : the popular idea that the objects could be some form of tidal debris.

Ask An Astronomer Anything At All About Astronomy (XXV)

Time for more questions.

1) Who can explain to me why Einstein was trying to find the Gravitational constant of the universe ?
Me.

2) I thought G was the sum total of all the gravity in the universe. Regardless of the fluctuations of force.
You thought wrong.

3) So the force that holds the planets in orbit and keeps the moons orbiting the planets is the weakest force ?
Yes. Gravity's only strong when there's an immense amount of mass. Like your mum.

4)  What do you think of this meme about a void in space ? Is this the same region that was used for the Hubble Deep Field ?
It's awful.

5) How about these space facts, eh ?
Sure, yeah. "Facts".

6) What's it called when you live in a binary star system and one star occults the other? Is it still an eclipse ?
Yes.

7) I read that there is a planet which orbits two suns. I don't remember the name, but what is the closest two suns can be together without colliding? I imagine it depends on the size too...
Tatooine. And they can touch without coalescing, which is nice.

8) Out of curiosity, wouldn't an advanced civilisation capable of warp speed travel be able to explore the unobservable universe ?
Sure, but the unobservable parts aren't going to be anything special.

9) Why do galaxy-sized turbulent gas clouds look spiky ?
They can if they want to. Stop judging their lifestyle choices.

Just Give Them Some Money Already

If you already think that Arecibo is a nice place and should have some money, you don't need to read this post. Just sign the poll here to show support for the Observatory or even write a letter if you want. A suggested template is at the bottom. If you're not convinced Arecibo deserves a really quite modest amount of money, or you just like reading things because you're bored and even the cat is refusing to play with you, read on.

The Fug

I walked out of the airport and into a damp, warm fug. A fug has the same humidity as a thick, soupy fog, but is invisible. I'd just finished a short hop from the east coast of the US after what was only my third transatlantic trip, which had been preceded by a lengthy drive from Cardiff to London because Cardiff airport is the size of shoebox. All told I'd probably been travelling (after a rather sleepless night) for close to 18 hours at that point, and there was still another 2 hours to go.

The taxi driver, I'd been assured, spoke good English. He did not. He spoke passable English, meaning you could carry on a conversation as long as you were very careful.

"So where are you from ?", he asked.
"Wales." I replied.
Blank look.
"The UK ?" I asked, somewhat more hesitantly.
Still nothing.
"Britain ?" I ventured.
"Ohh...." he said, meekly, with the obvious air of someone who is trying not to sound ignorant.
I gave up. "England". I said, feeling like I'd soiled myself.
"Ahhh, England !" came the enthusiastic response.

Aside from the tremendous heat, the next couple of hours passed uneventfully, and the air conditioning quickly had things down to a pleasantly chilly level. Even just the few brief minutes outside had been enough to make it very clear that this was the Tropics, dammit, and the climate was determined to make sure that everyone knew who was boss. Despite the fact that I'm prone to dozing off on even short car journeys, I remained quite awake for the final leg of the journey to Arecibo Observatory.

The taxi driver had the classical Puerto Rican, "kill 'em all and let God sort 'em out" approach to driving that I quickly learned was characteristic of even the otherwise very intelligent people there. The car should at all times be in the middle of the road, regardless of how many lanes there were or where the potholes were, and should only be moved  to the side when another car made it absolutely impossible to do otherwise. The last few miles were a rollercoaster of a ride through karst terrain, taking blind crests at speed for the sheer hell of it, apparently. It was one of those silly American cars that look enormous on the outside but are much smaller on the inside. The kind that should have the same turning circle as the Eiffel Tower, but somehow the driver managed hairpin bends through sheer bloody-mindedness.

When we arrived at the observatory, I could hardly contain my excitement. This was Arecibo, the world's largest telescope, the place I'd seen in Goldeneye and Contact. The place I'd dreamed of seeing since before I even knew what radio astronomy was. Despite the now nearly 20 hours of travelling, I had to see -

Wait, what ?!? That's not what it was like at all ! That's the manipulative internet sob story, but it's not what really happened.

OK, when we arrived at the observatory, I was too tired to see the telescope, and anyway it was dark. The room was up one of the steepest hills of the whole drive and I didn't feel like walking around and getting lost. Instead I gratefully collapsed into my charming accommodation and let the gentle, soothing sound of the coquis -

No. That's not it either.

The truth is I didn't really want to be there at all. I've never had the slightest inclination toward long-distance travelling. Why would I, when I was happy where I was ? Travelling would only take me away from the place where I most wanted to be. My first trip outside Europe had been just a few months previously, and the return trip (which involved me running around Los Angeles airport for several flustered hours wondering how the hell I was going to get home) hadn't made me eager for another.

Still, I could also make this into a manipulative sob story about how Arecibo helped me to "find myself" or how the whole experience was "character building" or some such other bloody nonsense. No. I liked living with my parents. I had no driving license (nor did any of my friends) nor any desire to get one. I had no laptop (these were still a novelty in 2007 and they certainly weren't considered essential for PhD students). So the idea of being essentially stuck very far away from home with no means of transport and very little to do in the evenings for six weeks was not the slightest bit appealing, and the only reason I was there was because I hadn't been able to think of a good excuse to tell my supervisor.

So what really happened on arrival was that I was exhausted, had really no idea where I was, and now I found myself literally in a state of disbelief that I was apparently going to be living in a small wooden hut. In the dark it didn't look up to much at all. I genuinely thought there must have been some kind of mistake, but there wasn't. The inside turned out to be not that much better. The great Puerto Rican fug permeated everything and the air conditioner was more of an excuse to generate noise than actually cool the totally uninsulated building. I wasn't used to sleeping with loud white noise, so I opted instead to listen to the slightly quieter coquis and other assorted jungle sounds.

The coqui is a tiny frog about 2cm long that whistles. It does this because it is an utter bastard. It is every bit as loud as a person and there are many of them. They're much worse when there's a single one close by, because it's literally like having someone stand three feet away from you whistling loudly and without rhythm. So I was really not particularly happy. I was stuck somewhere I had no desire to be, it was too hot, far too humid, I was living in a hut, had no way of independently escaping, and little to amuse myself on weekends except for a book on the campaigns of Hannibal. That's what really happened.

In fairness, these days the surrounding gravel is quite a pleasant little garden area.

The ageing computer was better than nothing, but it was still the sort of thing that could barely run Minesweeper. No, I don't mean Minecraft.

But because this isn't a sob story but a true account of what it was actually like, it wasn't all bad. My first glimpse of the telescope the next day was indeed quite a sight. I remember standing next to a nice man named Chris Salter who told me about the effects of thermal expansion and the need for great tension in the tiedown cables that keep the platform in place. And I remember being startled when a telescope operator* - I don't remember who exactly - telling me that it was possible to walk underneath the dish, shattering forever the Goldenye idea that the thing was made of concrete.

* They're like telephone operators, only more awesome. They check that the telescope isn't doing anything it isn't supposed to do and sometimes run (but never prepare or analyse) the observations.

Far harder to explain but infinitely more important was the attitude of the staff scientists. I can't say how exactly, but no-one - not in Cardiff nor in Prague - has ever made me feel more welcome. If you can't get along with Chris or Tapasi or Robert, in my book you probably aren't worth knowing and certainly need to go home and rethink your life. Since I can't adequately explain how they did this, I will simply state the blunt fact of it and leave it at that. Except, of course, to post this picture :

Vexation

Yet often their anger was palpable. The Observatory's future was looking decidedly uncertain, and their boss was routinely described as a man who, if given a choice, would choose to tell a lie in preference to the truth. Over the coming weeks and, as my PhD transmuted into a postdoc, years, I'd be privy to and occasionally subject to the gory details as to why they were so angry.

Certainly there were enough day-to-day reasons to drive anyone mad. There were managers who were never there and micromanagers who never shut up; people who were merely vulgar and others who were rather more worrisome... but all these things happen everywhere. What was far worse that the entire permanent staff were chronically over-worked, uncertain about their future, and there were simply too few of them to do all the jobs that needed to be done. It was a toxic combination.

You have to understand that an observatory staff scientist leads a different lifestyle to a university researcher. Scientists in universities do a combination of teaching and their own research. Observers do a combination of other people's research and their own research. It's their job to be "contact scientist" (or in the Arecibo vernacular, friend of the telescope) for any and all non-staff astronomers who use the telescope. That can mean a wide variety of things depending on the project :

1. If the user is very experienced, there might be no contact between them and the staff scientists at all. Or they might just exchange a few emails to check details.
2. New users might decide they want to visit Arecibo, in which case the staff scientist can teach them to use the telescope from scratch.
3. Running the observations themselves. Users rarely visit for long-term projects, but someone still has to be at the telescope to press the buttons to make it point at things.

Much of the observing process is already automated or can be run just fine remotely - you can get all of the displays you'd see in the control room on your desktop, or even a tablet if you're so inclined. For simple projects where everything can be setup beforehand, it's literally a matter of loading a few files, choosing a target, and pressing "Observe". In a very few cases the whole process can be done in less than half a dozen mouse clicks, then there's nothing to do except stay awake. Honestly you could train a monkey to do it, and the only reason no-one's done this (it would win the internet in half a heartbeat) is because the monkeys have herpes and anyway there are telescope operators who need to be there to make sure the telescope doesn't collapse and suchlike.

Looks complicated ? It is and it isn't. If you've got a new project to setup from scratch, there are a huge number of options that must be set. But if you've got a simple, well-tested project... six mouse clicks. Really, that's all.

We couldn't find any monkeys so we used some presumably herpes-free students instead.  

In any case, I wouldn't recommend replacing astronomers with monkeys or automated scripts as a general rule. Often, it's important to monitor the data as they come in and quickly process them to see if all is well, because whether one scan has a detection or not determines what you do next. In my experience, enough weird, unexpected things happen that trying to write an algorithm to do this would be counter-productive*. If the Universe was somewhere so simple that we could guess what the observations would really show, we probably wouldn't need to observe it at all. And then there's the more mundane reality that some projects need someone to physically plug cables into things**.

* I suppose the monkey could be trained to screech and hurl faeces at people until someone came to sort it all out, but this option isn't popular.
** Alas monkeys can't read so there's no way to train a monkey to do the cabling.

What all this means is that when they're literally working through the night, staff scientists usually aren't doing it for their own benefit. They'll never get credited for the resulting research, because they're not even usually interested in it.

Now, if you have a properly-funded observatory with lots of staff scientists, the burden of this support observing isn't too bad. Lots of people share the load and everyone does more science. The fewer staff you have, the more support observing they have to do. At Arecibo the lack of proper funding meant there were so few permanent staff that they - the most experienced people in the world at actually using the telescope for science - were effectively doing their own science as a hobby. Their actual working hours were far in excess of what they were paid for. And hanging over their heads for years was the very real threat of layoffs or even closure (which would essentially mean leaving the island since there aren't a lot of similar other jobs available there). This was one of the world's most powerful and unique scientific instruments, and it was slowing bleeding to death.


God It's A Sexy Telescope

Which brings us to the inevitable part of the post where I must sing (or at least dance) Arecibo's praises, otherwise you could be forgiven for thinking that this doesn't really matter except to a handful of disgruntled scientists - or that Arecibo itself is merely a relic, a technological dinosaur no longer able to compete in the modern world.

First, you must understand that all of the really large telescopes are their own prototypes. There are no "normal" giant telescopes; all of them are special in some way. Of course with Arecibo the main feature is its sheer gargantuan size. This gives two advantages : sensitivity and resolution. Both of these have some surprising subtleties, and they're not completely independent.

Let's say you're building a reflecting dish telescope. You'd probably want to build it as big as technology and funding allow : a bigger dish means greater sensitivity and sharper images. Very simple. But it's not really feasible to build telescopes which are very much larger than Arecibo - it's expensive, and they can't be steered, so their field of view is limited. So then you opt for an interferometer.

The Karl G. Jansky Expanded Very Large Array in New Mexico, which totally wasn't named by committee.

An interferometer is an array of smaller telescopes which are able - through complicated instruments - to act like a single giant telescope. Of course, you don't necessarily have the same collecting area as a giant single dish, so the sensitivity isn't as high, but you can get much better resolution in this way. It's easier to build a few 25m dishes over a few kilometres than it is to build a single dish a few kilometres across - but interferometry isn't a magic bullet.

The problem comes because you've effectively built a single dish with lots and lots of holes in it. The loss of sensitivity is much more subtle and powerful than simply decreasing the collecting area. For complicated mathematical reasons, its turns out that interferometers are way less sensitive to very extended gas. So although they can see fine details far better than single dishes, they're lousy at detecting larger stuff. The VLA, for instance, has a collecting area only about five times less than Arecibo - but it's several hundred (approaching 1,000) times less sensitive to extended gas than Arecibo. You can't really get around this with longer observing times.

In principle, single dishes are inherently better than interferometers - if you had a single dish as large as the VLA, you'd have a telescope that could resolve both large and small structures, whereas the VLA would miss the large stuff (it gets "resolved out" in the technical jargon). It's only for practical reasons that such supergiant telescopes aren't built.

Still, Arecibo's sheer size makes it unique. Its sensitivity to this low-density material is so great that it's not likely to be surpassed until the formidable SKA comes along - and possibly not even then ! Even in the best case that's 10-15 years away. Pretty good for a telescope built in 1963. Without Arecibo we would not be able to detect stuff like this :

The extremely extended gas disc of M33, as revealed with increasingly sensitive Arecibo observations.

... or this....

Keenan's Ring, a gigantic ring of gas behind M33. Despite this region being one of the most studied parts of the nearby Universe, this huge feature was only discovered last year. We're very far indeed from fully exploiting the capabilities of Arecibo.

... or this...

A giant hydrogen stream 1.6 million light years long discovered by the ALFALFA survey. The same survey that measured the hydrogen in tens of thousands of galaxies.

... or this !

An even longer hydrogen stream, because hydrogen streams are awesome.

And those are just some of the less-important recent discoveries from one receiver on Arecibo ! They don't even get a mention in the official list of Arecibo's accomplishments over the last fifty years. The ALFA receiver was installed in 2004 but (even after 12 years) it's hardly as though we've done everything we can with it, because that's just how long this stuff takes. There is so, so much more we could still do with this ! What's more, right now we have the experience and technical capabilities to continue these discoveries - all we need is the willpower from those with the money.

You might be forgiven for thinking that China's astonishing new radio telescope, FAST, being a single dish 500m in diameter compared to Arecibo's 305m, means that Arecibo will at best soon be playing second fiddle to the latest champion. And FAST is indeed a remarkable piece of engineering, so let's all ogle the telescope porn for a moment :

Indeed, in their 2012 portfolio review, the National Science Foundation (they who hold the purse strings) made a similar mistake. They decided that because the Green Bank Telescope is 100m across and the Effelsberg telescope is also 100m across, they must be very similar instruments*. Of course they're not, for all sorts of reasons. For Arecibo there's a very simple reason why FAST is not really like the equivalent of the dreadnoughts, which were such a breakthrough that all existing battleships became obsolete almost overnight. Because it's not the size of your dish that matters, it's the size of your dish that you're actually using.

* Bizarrely that was how they justified cutting funding to the GBT. As a dyed-in-the-wool socialist Brit, it pains me to say this... but that's un-American. The whole point of competition is that you don't surrender, let alone when your opponent isn't even as good as you !

As you can see, FAST's physical dish is indeed much bigger than Arecibo - its area is about 2.7 times greater than Arecibo's. Which is nice, though hardly world-changing. More importantly, because of the design of these giant fixed telescopes, not all of the dish is actually used - that would only be possible when pointing directly overhead, which would be immensely irritating. In practise, the useful collecting area of FAST will only be about 1.8 times that of Arecibo. That's very nice, no doubt about that  - but it's hardly the factor of 1,000 difference in sensitivity between Arecibo and the VLA. FAST will, however, be able to point over a much wider range of angles, so it will be able to see parts of the sky Arecibo will never reach.

Telescopes are complicated pieces of kit. The smoothness of the dish is also important, because that determines the frequency range you can observe. Which, in turn, dictates not only how far into the Universe you can see (more distant objects appear redder), but also what objects you can see at all. The sky looks completely different at different frequencies - so the quality of the dish directly controls how much information the telescope can gather, just as much as the size does.

Arecibo's frequency range is nowhere near as good as the VLA or the GBT - for some objects, you have no choice but to use those telescopes. But it's about three times better than FAST, meaning FAST simply cannot do all the same science that Arecibo can. Arecibo's frequency range could in principle be extended further by replacing the dish. That's no small challenge, but it's do-able. It would be much more difficult to do this for FAST because its dish is actively deformed to point at different parts of the sky, which is harder at higher frequencies because it requires greater precision.

And there are plenty of other reasons why Arecibo is far from being a quaint, grandiose relic from the past. The dish size and quality don't mean much without capable instruments. Arecibo has a whole suite of them. And even though we haven't fully exploited all their capabilities yet, more are in development to increase Arecibo's potential even further. This level of maturity for the facility as a whole takes decades to achieve. Far from being obsolete, with proper investment it could easily be cutting edge for the next twenty years, and plausibly much longer*.

* Two outlandish ideas that were proposed that could genuinely blow Arecibo out of the water : 1) Building a copy of Arecibo on the Moon, where radio frequency interference is much lower - yeah cos that's gonna happen; 2) the Chinese were once planning to build ten Arecibo-size dishes, but this seems to have been a bite more than even the mighty Chinese economy can chew.

The receiver room in the Gregorian dome.

And that's a key point : you have to take a big-picture view of the Observatory. If you focus on individual areas, you can find other telescopes that already or will eventually do better in most areas (with the exception of sensitivity). But thus far I've been concentrating on my own and similar research, which is, of course, a hugely narrow view. In fact Arecibo does far more than just measure distant gas - it does a whole slew of things which are impossible for other facilities. I'm going to skim over these quite quickly, otherwise this post will never end and anyway I'm not qualified to describe them in detail.

Asteroids are perhaps the prime example. Most astronomers just watch; radar astronomers actively zap their targets using a 1 MW transmitter, then measure the signal that bounces back. Only one other facility in the world does this, with (to my knowledge) no others planned. FAST will not have a transmitter - possibly (but someone please correct me if I'm wrong) because it would weigh too much for its lightweight platform design, so it will never have one.

Radar observations tell you a heck of a lot about the target. It's easy to get sky positions using a conventional optical telescope, but radar observations give you a precise measure of the distance. That gives a very strong constraint on the object's orbit, so they can be tracked for further observations later (and yes, it's also important in determining if they're likely to hit us, though that's hardly the main scientific goal). They can also be used to measure the true 3D shapes of the asteroids to a resolution down to a few metres. That's a big deal. Other astronomers would make injured puppies listen to the complete works of Justin Beiber for the chance to know the true 3D structure of their sources.

Radar images look a lot like optical images, but they contain very different information.

After the Chelyabinsk near-miss, the dangers of asteroids suddenly don't look so far-fetched. 

Pulsars are completely different again. They use the same instruments as everyone else, but their data have much higher time resolution since pulsars can vary on the scale of milliseconds. They're rare, weak, and hard to find - so sensitivity is important. Data processing and source finding is completely different from that of other types of radio astronomy. Of course, they're interesting objects in their own right - they're the dead hearts of massive stars and also giant floating bombs, in a sense - but they can also be used to find gravitational waves. For a short while, there was a real chance that Arecibo could have scored a detection before the LIGO laser interferometer, but alas it was not to be.

YOU SHUT YOUR UGLY FACE, KIP THORNE !

While general astronomy projects use up the bulk of observing time and funding from NASA via the radar team brings in the cash, it's pulsars that meant Arecibo played an essential role in the 1993 Nobel Prize. The orbital decay of a pulsar discovered and measured at Arecibo was found to be in perfect agreement with the predictions of general relativity, which was the first really good evidence that gravitational waves exist. The direct detection if of gravitational waves through pulsar timing arrays might not win Arecibo a second Nobel, but it will be extremely important both for gravitational science and more interesting things like galaxy evolution.

Finally we come to aeronomy - the study of the upper atmosphere. This isn't what most people first think of these days when Arecibo is mentioned - they probably think of James Bond or aliens - but it is in fact the reason Arecibo was built. It's still an ongoing area of research (about which I know absolutely nothing, hence I won't try and blag my way about how cool the science is) with a new heating facility being installed to try and do something awesome, probably. All I will say is that conspiracy theories about similar facilities are just utter codswallop. I've met most of the people involved and they couldn't sustain a conspiracy as far as I could throw them. At least one of them was infamous for sending bat-shit crazy emails after 5pm when the Scotch came out. Conspiracies ? Not bloody likely.

And yes, it also searches for aliens as a somewhat minor side-project.

Modifications to the dish to allow more advanced studies of the ionosphere.

So that's what Arecibo does and what makes it special. It is a unique, versatile, and cutting edge tool with no other equivalent planned to ever replace it. But it's fully capable of maintaining that status : telescopes aren't just built and then that's it. Arecibo's original dish wasn't the nearly-opaque series of panels you see today - it was a much sparser affair, since it wasn't originally designed to observe the higher frequencies it does these days. Nor did it have the distinctive Gregorian dome with its incredibly powerful ALFA receiver essential for so many modern Arecibo projects - all this came later. There's no reason similar upgrades can't be made to keep Arecibo at the forefront of research for years and decades to come.

Help !

The price for all this is about $12 million per year. At the current rate, a single Falcon 9 rocket launch could fund the Observatory for the next five years. Each season of Star Trek cost something like $30 million, while Jonathan Ross was paid around $7 million just to be a bit of a pillock. A single typical Hollywood blockbuster could fund the telescope for the next 15-20 years, while CERN costs over a $1 billion per year - enough to keep Arecibo going strong for the next 80 years. I always used to naively imagine that the UK was too poor to afford something as grand and wonderful as Arecibo and only the Americans had that much cash; actually a single one of our shiny new aircraft carriers could keep Arecibo going until around the middle of this millennium. It's not a matter of money, it's a matter of choices.

But it's the cynics who know the price of everything and the value of nothing. The price of maintaining Arecibo is not that high and if offers unique returns in a huge variety of scientific disciplines - a greater breadth, perhaps, than at any other telescope. All telescopes are unique but some are more unique than others. 

Arecibo's funding situation has made a mockery of the word, "crisis", just as "breaking news" stories that go on for twelve hours are linguistically offensive. It can't still be breaking twelve hours later, it's broken news. You can't really have an emergency or a crisis that lasts for ten years, you've just got a bloody awful situation. Nevertheless, currently the NSF are preparing an "environmental impact statement", which is fancy talk for, "let's at least pretend we're trying to sort this mess out." The easiest way you can help - yeah, YOU ! - is to sign this poll. It'll take all of 30 seconds of your time, so if you read this post through to the end and didn't sign it, what the hell is wrong with you ? You don't have to be an American either, or at least I can't find anything saying, "you must be an American".

If you've got slightly more time on your hands, you can submit a "formal comment" to the NSF. You can either do so online here or send an email with the subject, "Arecibo Observatory". In any case your message will be public and visible online (after review). The options the NSF are considering are :

• Continued NSF investment for science-focused operations (No-Action Alternative) [Status quo. This would be fine if  a) funding was slightly increased and b) guaranteed for some sensible timescale and c) the amount of required report-writing to the NSF was significantly cut.]
• Collaboration with interested parties for continued science-focused operations [Have the NSF work with unspecified others to find alternative sources of money (e.g. Breakthrough Imitatives). This is also a very good option, possibly the best option provided it allows the scientists to get on with doing their jobs and at least some observing time should be open to anyone who succesfully applies for it.]
• Collaboration with interested parties for transition to education-focused operations [Basically make the Observatory into a purely teaching facility. I can see where they're coming from - the Observatory does quite a bit of teaching and outreach already - but it would be a strange choice. In order to make the telescope a selling point of the teaching, it has to be used for doing science as well - otherwise you may as well close it down and have the staff move to other institutes.]
• Mothballing of facilities (suspension of operations in a manner such that operations could resume efficiently at some future date) [It would cost several million per year to keep the facility mothballed without the telescope collapsing or becoming irreversibly damaged. This option only make senses if there was a sudden but short-lived financial gap to plug, which is not the case.]
• Deconstruction and site restoration [No no no no no - if you want to see the telescope collapse, you can watch Goldeneye.]

If you strongly prefer one of those options, go ahead and write in. I've been advised that it should be short (~200 words - a long paragraph) and use minimal jargon. If you simply wish to express support for Arecibo, you could just sign the poll, which won't commit you to any specific option but will send a message that Arecibo needs funding of some form or other. Or, if you like, here's a generic "I love Arecibo and I would marry it and have its babies if I could" message. Feel free to use it or modify however you like.

The Arecibo Observatory provides capabilities to US and international radio astronomy which are unmatched by any other facility. It continues to offer unique contributions to a highly diverse range of studies including galaxy evolution, pulsars, asteroids, and atmospheric physics. No other comparable facility exists or is planned, and without Arecibo several avenues with the potential for major scientific discoveries would be closed – as well as losing decades of accumulated experience. Discoveries from Arecibo routinely generate spin-off projects involving new lines of theoretical inquiry, numerical simulations, and further observations, which directly and indirectly support the research activities of scientists around the world. In addition, the Observatory operates highly successful outreach and teaching programmes, using its iconic status and the expertise of its staff to inspire young scientists and inform the general public. The Angel Ramos visitor centre attracts tens of thousands of visitors per year, and undergraduate research programs have helped many Puerto Ricans to choose a career in science, advancing the local knowledge economy. Sustained investment to maintain and further develop the telescope and its instrumentation will ensure it continues to play a leading role in US and international science for the next decade and far beyond.

Au Revoir

Think this post is long ? It isn't. I've barely scratched the surface as to what Arecibo is capable of, that would take an entire book if not an encyclopedia. If I was to stick with the manipulative sob story, I'd try and bring it back to the personal element right about now. I'd saying something to the effect of, "though I never wanted to go to Arecibo, now I found the thought of leaving it impossible." And that would be true, because honestly when I got a job offer in Prague I couldn't believe my luck.

I do not like the tropics. In fact I despise them. The constant heat and humidity are like wearing a badly-fitting pair of shoes or having mild but increasing toothache : uncomfortable at first, painful after a while, and then eventually you want to scream in agony though you know full well it won't do any good. Then there was my mortal terror I experienced thanks to the *!*@ing godforsaken need to drive everywhere. I hate it. Being literally too terrified to go anywhere is no way to live. Not to mention that having lived in Cardiff for 27 years, all the people that matter to me are there - being away leads to a nagging awareness that I'm not where I'm supposed to be, with the effect being very similar to that of the constant, oppressive fug.

I could have coped with any one of these, but not all three. So for me staying in Puerto Rico was never a long-term option. That's not to say there weren't some good times...

... some goat times...

... some damn surreal times...

... and let's not forget the "stuck in a hurricane" time, which is in a class all of its own...

... the cute and cuddly times...

... and of course the epic times.

My point is that the staff there, unlike me, have built lives for themselves over the last twenty years and more. Just as I left for personal reasons, so those who stay do so for exactly the same. Closing Arecibo would waste decades of experience accumulated on one of the world's most uniquely capable scientific facilities when its full potential remains unknown. That's part of the thrill of observational astronomy : you don't know what you'll find next. Without Arecibo, it could be decades before we find out what we missed. It would not only harm the lives of people who have dedicated themselves with the fiery passion of a thousand flaming suns to helping other people do science, but it would also prevent innumerable spin-off projects enabled by Arecibo discoveries.

Thus far my entire scientific career has resolved around Arecibo data. With the accumulated data, desired follow-up observations, and ongoing numerical simulations to try and understand all this, there's no reason that even what we have so far couldn't keep me extremely busy for the next five years or longer. But we shouldn't even be thinking of Arecibo's legacy for at least the next ten, twenty, thirty, maybe even forty years. We should instead be thinking of it as what it is : a first-rate scientific instrument that is an asset, not a millstone or a plaything, to anyone who funds it.

So sign the poll, write a letter. It'll take a few moments of your time and might just help secure the future of one of the world's greatest observatories for years to come.

Ask An Astronomer Anything At All About Astronomy (XXIV)

Overcoming laziness, the Q&A page has finally been updated once again.


1) How many gas clouds are there around M33 ?
All of them.

2) Is Keenan's giant gassy Ring really all that giant ?
Yeah, it's pretty big.

3) Are we sure the Tired Light Theory was wrong, then?
It's tough to see it getting any wronger.

4) What do you think of this press release about dust ?
I hate dust.

5) What will happen if the earth suddenly vanishes from its orbit ?
We will have an interesting day.

6) Why do meteors always land in craters ?
That's where the exogorths live.

7) Will Arecibo's radar attract unwanted attention from passing aliens ?
Good God no.

8) Will the new FAST radio telescope be better at signalling/detecting aliens ?
Yes, but only a little.