Fifty Shades of Garching

A career in astronomy almost inevitably carries with it the curse and boon of travel. From the tropical jungles of Puerto Rico to the glaciers of Alaska, from the burning sands of Socorro to the Alpine range of Switzerland... and now to the grey wastes of Garching. This may well be the least exotic place I've ever been to for any reason ever. And that includes Milton Keynes, for goodness' sake.

Cue the least exciting travel blog ever.

Garching is a small, pointless town in southern Germany which for some utterly unknowable reason is a centre of high technology. Astrophysical institutes, cyber technologies, plasma research, General Electric... it's got them all. With a population of just 16,000 you might think this makes it some sort of scientific Mecca, a researcher's paradise without equal, where scientists frolic freely in gay meadows with sunshine and rainbows and lollipops in abundance.

Well, it's got this thing, at any rate.

What it actually is is a bunch of large, grey buildings under a large, grey sky in the middle of a large, grey field. The key point for the designers was that everything should apparently be large, grey, and above all soulless. Why this particular luckless field was chosen to be a centre of European scientific excellence I've no idea. I'd like to say that a cold, grey wind moans across a barren icy waste, but that's too exciting; the poor place can't even be described as a "blasted heath". It's just a big, boring field with some big, boring buildings stuck in it. Even the tea is a sort of grey colour, and I have to reluctantly concede that it's even worse than American tea - which I've long regarded as (in the worlds of Douglas Adams) a liquid almost, but not quite, entirely unlike tea.

I mean now this is just being silly.

The European Southern Observatory building isn't grey. It's brown, which is not much better. Inside it feels uncannily like an airport terminal, except that it's harder to navigate. The building design is a weird triple combination of circular buildings linked by a covered walkway very much like the ramp to an aircraft. Even after a week I still had to consciously think which way I wanted to go whenever I left the office. As for the wider campus, that reminded me of nothing so much as the grounds of a hospital. Not exactly inspirational.

The under-construction "Supernova" planetarium sticks with the brown theme but somehow contrives to make a weird-looking building look reeeeally boring.

My visit was for QA2 training for ALMA. The Atacama Large Millimetre Array is a billion-dollar radio telescope in Chile that's searching the skies at hitherto unexplored frequencies with exquisite sensitivity and resolution. That's what they tell me anyway; while ALMA might pay my salary I can't honestly claim to really know anything about it. Arecibo works at much lower frequencies, and while there's certainly some broad overlap between the two research areas the technical differences are enormous.

At least the ALMA facility itself is more photogenic.

Arecibo is a single-dish telescope. Point it at the sky and it will measure how bright the sky is at that point at whatever particular frequency it's tuned to. Move it around and record the brightness at different points and you can construct an image. The measurements are real - you directly determine the brightness just as with an optical telescope. Oh, you can process the data afterwards in some fancy ways and get different results, but at the most basic level you can (with proper care) actually know how bright the sky is at any point you look at. Easy peasy, relatively speaking.

Not so with an interferometer like ALMA. Interferometers are arrays of telescopes that are combined to produce images with extremely high resolution, equivalent to building a single telescope as big as the largest distance between the individual telescopes. Well, sort of. Not exactly.

The thing is you can't just wave a magic wand and shout, "ALACKAZAM !" and hope to improve your resolution. No, your incantations have to be much more mathematical. You've got to combine the signals from each antenna somehow, and therein lies the difficulty. It would be an understatement to say that the maths behind this is fiendishly difficult, but the bottom line is that interferometers do not measure the brightness of the sky. The images they produce - though marvellous - are reconstructions based on their raw data. They are not direct measurements; you cannot obtain "the right answer" with an interferometer because other ways of processing the data give different results which are equally valid.

As usual, don't go nuts with this. The images interferometers produce are wrong but only for a given value of wrong - they are certainly not fictitious. The point is that with the proper care an interferometer can give you wonderful results if you know what you're doing. I can and have taught students to observe with Arecibo in about an hour or less, whereas with an interferometer it might be optimistic to say "in about a year or less".

Students remotely operating Arecibo from Green Bank.

Which is where QA2 training comes in. ALMA has the incredible and laudable goal of producing a fully automatic pipeline for its observing, so that eventually the scientists will get their data delivered to them in a "science ready" state - that is, corrected so that the images are the best representations of the sky that they can possibly be. What they'll be looking at may not be the whole story and it won't be 100% accurate, but it will be good enough. And this will be done entirely by algorithms with no need for the experienced users to step in and check anything.

Such is the goal. Remarkable progress is being made toward that extremely complicated end, but for now, it's still much easier for a human to gain the necessary experience. So ALMA data undergoes several levels of "quality assessment". First there's "phase 2"*, which consists of coordinating the observing scripts between the scientists and the telescope operators. Then when the observing starts there's QA0** at the telescope itself, making sure all the equipment is working and the weather is nice and so on. No-one seems to have any idea what QA1 is at all - it's presumed to have been lost in the blizzard of acronyms (that list doesn't cover half of them) the telescope generates with a profligacy equalled only by its raw data. After that comes QA2 - checking that the data has reached the requested sensitivity and resolution, followed very occasionally by QA3 when the scientist finds a problem with the data that the QA2 staff didn't.

* Yes, I realise that it doesn't make sense to start with phase 2. I have no idea what phase 1 is or if it even exists.
** Which presumably means there's also a phase 0, further adding to the things I have no idea about. Yay.

As you'll have gathered, they do love their corporate jargon, do ALMA. They routinely say things like, "The DRM at the JAO will open a JIRA ticket and you can download the AOT file and inspect it with the OT to determine if it's TDM or FDM data in TP mode and you can find out lots of other information on SCOPS with communications to the P2G but obviously it's easier if you've already looked at the project during Phase 2". Honestly, they're one step away from, "By streamlining the paradigm we can ensure increased productivity through synergy maximisation but let's talk offline !"

Well, alright, it's not that bad. They're still scientists after all. Nice people that it would be hard to dislike. But while other scientists love to moan about minor bureaucratic procedures, here they don't seem to realise just how incredibly corporate things have become. Maybe this is the future of astronomy, but I hope not.

Learning QA2 procedures if you're already a dab hand at interferometry is probably not that bad. You can certainly learn it inside a week, though I'd hazard you might still think there's some scope for improvement. But what if, like muggins here, you've had barely any experience with interferometry at all ?

Imagine, if you will, an enormous online technical manual from which you need to extract a few key points. Some of these are easily visible but most are hidden in obscure hyperlinks. You can control+F but not search the entire site in any way. Some points are labelled in an obvious way but others require reading and understanding to extract the relevant detail. You'll need at least a dozen different pages from which to extract all the information, but maybe not all at the same time and you can't minimise windows or have multiple desktops (because frak you, that's why). Some information can't be extracted directly from the manual but only by running a series of long, complicated tasks which then require you to check yet more extremely long web pages that are incredibly poorly-labelled and use a program which just plain doesn't work for no reason. The result of all this is access to data which doesn't interest you and you're not allowed to use in any case, and the absolute best you can hope for is that someone will eventually write you a thank-you email.

It's not much fun at all really. In fact it's a lot like completing a difficult mission in a computer game and having it crash at a critical moment - you've got to do the whole bloody thing again. Except instead of monsters and dragons, the most interesting thing you'll see is a fuzzy blob that will eventually be tremendously important to someone else. Not you though.

It got better throughout the week, in that the "long, complicated tasks" became short and relatively easy. Unfortunately the rest didn't improve much. I simply cannot conceive of why there's a version of Linux where you can't minimise windows or even see which ones are active in the taskbar (let alone why it's running at ESO). That alone made trivial tasks very difficult. It doesn't help when those tasks - like retrieving values from files - are made needlessly more difficult by those files being given meaningless names like "textfile.txt" (yes, literally that). Or when there's no clear list of what parameters you're eventually going to need from those files (you often need several so you may as well get them all at once), or which directories they're in (you need at least a dozen different windows open) or a linear set of instructions to follow.

The really, viscerally frustrating thing about the process was not how tedious it was, but how unnecessarily tedious it felt. So much of the really difficult process - the hard mathematics - has been taken care of by the computer, but these final, tiny baby steps feel like they've been deliberately designed to make things as gut-wrenchingly obnoxious as possible. All a human really needs to do is some relatively simple data processing and check the resulting plots for problems. If it's difficult to automatically extract parameters from a file and record them (for whatever reason) then OK, fine, but, come on - there's no need to obfuscate the process to the extent that many times I felt my brain simply shut down and refuse to co-operate any further.

No, I don't want to look for the file again. I already closed it because the instructions didn't say I'd need it again and it took me ten minutes to find it ! AAAAAARRRGHHH !

At that point I realised that no amount of willpower was going to get me anywhere so I gave up and drew on the whiteboard instead.

It would be so, so simple to create a "QA2" directory holding copies of the data files needed to extract the vital parameters from in a single location. I guess maybe if you've done this a hundred times you don't see the need for it, but lordy that's a steep learning curve. The group of us from Prague decided to collaborate and eventually (when we understand the process fully enough) produce our own, linear set of instructions, which ought to help a great deal. Hopefully at some point this learning curve is going to become a plateau...