You Can All Get FRELLED

New Year, New Software

How to start the blogging for the New Year ? Some topics are obligatory, like the current, shockingly abysmal state of global politics. Don't worry, I'll get to that. But I prefer to start on a more optimistic note, and with commendable timing, I had a paper accepted on New Year's Eve*. So let's do that instead.

* Presumably due to a drunken intern at an office party accidentally pressing the "accept all" button.

Presenting the FITS Real-time Explorer of Low Latency in Every Dimension. From the Farscape pseudo-swearword meaning a sort of combination of "hell" and "fucked". Hence the title of this post is extremely insulting.

The physics of galaxies is a jolly interesting thing, but you can't very well study what makes galaxies tick without having a sample of galaxies to start with. Finding the gassy buggers can be a right pain in the backside, so my major side-project is developing better ways to detect them. Sometimes that means writing fully automatic code that turns data directly into catalogues, but I prefer the old-fashioned approach of actually looking at the data we spend some much time and effort collecting. 

My main visualisation effort has been a code called FRELLED, which long-term readers will know is something I've been working on for a very long time. Basically it turns everyone's favourite 3D art software, Blender, into something astronomer-friendly.

... or at least, that was the idea. In practise... eeeehhhh, not so much. When it worked, it did work. You could load a 3D data set and fly around it and catalogue sources and everything ! It was great. I couldn't understand why it hadn't gone viral and made me millions of pounds from a grateful astronomical community. 

Two reasons. The obvious one is that astronomers prefer to spend their millions on telescopes and suchlike, rather than giving random donations to obscure Welshmen who give their codes away for free. The second, more interesting reason is that when FRELLED didn't work it, err, didn't work. It was pretty versatile in some ways but hopelessly limited in others. The learning curve was better than learning Blender from scratch but still pretty steep, it was full of bugs, and calling it astronomer-friendly would be overly-generous. It was astronomer non-hostile at best. 

It's very easy to provoke astronomers, mind you. Especially with clouds.

Lockdown finally provided me with the incentive to undertake the formidable task of rewriting FRELLED's ~11,000+ lines of Python code to bring it into the modern world. And not just because it needed an overhaul, which it did. But also because it had finally become simply unsustainable. With appalling timing, the earlier version had been written for Blender 2.49, which became obsolete almost  immediately as the very next version of Blender totally rewrote all of Blender's Python syntax*. This was hugely irritating : just as I'd got the hang of Python in Blender, they went and rewrote the whole bloody thing. I had the choice of either sticking with the functioning code I had or immediately starting from scratch.

* Python users – this was not like the upgrade from Python 2 to Python 3, which was relatively minor in comparison. No, this was a complete overhaul, so that virtually every internal Blender Python command was replaced with something totally different.

Well ! Naturally I chose the former, since I do have to do actual science as well at some point. I don't have the luxury of spending 100% of my time developing software – I need to actually use it. So FRELLED persisted in an abandoned version of Blender for several years, frequently tinkered with, never fully renovated.

But then lockdown happened, and recoding FRELLED was a much-needed project that could easily be done from home. So off I went and started coding. It helped that I'd already familiarised myself with more recent Blender versions for other projects.

This took quite a while.

Then, it took a while longer.

Things got derailed at various points because of the need for other projects.

But eventually, with the code doubling in size to 20,000+ lines and a bunch of new features, it was done ! Hurrah ! Let joy be unconfined, let there be dancing in the streets.

And so we all got wasted and did tonnes of cocaine or something. Although to give an idea of how long these things take, the paper alone took six months from submission to acceptance. That's after spending a full six months just figuring out the right journal to submit to.

Or at least, it was done enough. A project like this never really ends, there are always bugs to squash and features to add. But it reached a point where it did everything I wanted to do with a satisfyingly low number of bugs that one wouldn't need a flyswatter to turn it on.

I already did a somewhat in-depth look at how FRELLED works more than ten years ago (!) so I'm not going to go through it all again. Here I'll just look at what you can do with it and why you'd want to bother. If you're looking to actually use the code, the best place to get started is the home page, wherein you can find links to the embarrassingly large wiki, a complete set of video walkthroughs, and a 45-minute I-TRAIN video as well. It's got documentation in spades, and of course, you can always get in touch with me for help.

What Does It Do ?

This. It does this :

The main thing it does is look at 3D data cubes inside Blender. If you want the nitty-gritty on this see the earlier blog post, but the basics are pretty simple. Radio observations of galaxies often generate 3D data : as well as position and brightness on the sky, we also get velocity. That's not the same as distance, but using it in place of distance makes it easy to see all of our data all at once. And numerical simulations also create 3D data sets, including ones where the third axis really is distance.

The old version of FRELLED had a lot of manual coding in it because Blender 2.49 simply couldn't access a lot of more modern Python modules. That was hugely inefficient and wasteful, because Python is the equivalent of Ms Universe for astronomical codes : immensely popular. It's increasingly rare to have write your own code for basic astronomical tasks – chances are there's a Python tool for most operations. Not the high-level stuff, but for the low-level everyday processing, you're better off spending a little time searching for someone else's code than writing your own. Which makes it relatively easy to cobble all these snippets together to do the larger task you want them to do.

"Here it is, looking weird !" FRELLED is probably the short one with lots of tentacles.

What this means is that the old FRELLED had a lot of minor but annoying problems when looking at data of anything except 21cm emission. The new version, by using modules which do all the tedious coordinate calculations for me, lets it look at any wavelength with minimal fuss (at least for radio frequencies, at any rate). And cubes now load faster, sometimes very much faster indeed, and it can load data sets up to 27 times larger than the old version. Actually I've got a promising technique to increase this by maybe another order of magnitude in the works... but that's on the backburner for now.

There's more than one way to skin a cat, and there's more than one way to view a cube. Volumetric rendering, as above, is where you see the whole data set all at once. It's by far my favourite method but it's not always the best choice. So FRELLED also supports viewing cubes slice-by-slice, as it did before, but also with a couple of fun new additions. First, it can also render data slices as height maps (you might remember this from a solarigraphy conference) :

This is a nice example because it's a cube I've looked at a whole bunch of times and yet one of the major features escaped me completely. It's the galaxy M33, and in a standard map, you see the colours saturate at high values – in this case shown in red. The problem is that the dynamic range of the data is huge : the brightest features are much, much brighter than the faintest, maybe by a factor of a thousand. Choosing a colour scheme that can show features at all levels is really tough.

Same data set with the same colour scheme, but with different vertical stretching for the height maps (linear on the left, logarithmic on the right).

Height maps provide a nice way around that. As well as using the conventional colour map, each pixel is displaced by an amount according to its value. You can use different ways of mapping the data to colour and displacement (height), so in effect you're showing the data with two colour schemes at once. For example, as here, you could choose the colours to highlight the faintest features, while using the height to examine the structure in the brightest bits of the data. In this case, I'd never noticed the twin peaks in the galaxy, even though they're the dominant features !

Another major new feature is the ability to display the data as isosurfaces. These are like contours but in 3D, showing surfaces of constant data value. I have to say I originally poo-poohed these as a cheap excuse for 3D rendering : sure, it's easy to display a few thousand polygons as a surface, any fool can do that... what about rendering the hundreds of millions of voxels of the full data set, eh ? Honestly I really only put them it because I could. But I began to realise that I'd been far too harsh, because sometimes less is more. By reducing the data to this sort of very carefully, quantifiably selected display, all of the intervening noise is cut out completely. Ironically, the key to finding the most interesting features can be to remove most of the data. That goes well against my training and instincts as someone interested in the faintest objects, but it works.

And finally... virtual reality ! I love VR, I think it's one of the best technological developments of recent years. Yes, even if Mark Zuckerbot is an obsequious twat who's blatantly pandering to far-right despotism... a lot of people seem to confuse the worst sort of techbros (who are awful) with the tech they develop (which can, sometimes, be a genuinely good thing). But I digress.

Anyway, Blender 2.79 doesn't support real-time VR, so FRELLED comes with some export scripts to convert the files into a format that can run in later versions, tested up to 4.0. So you can step around your data either as isosurfaces or volumetrically, as though it were in the living room with you because why the hell not you crazy person for even asking.

Actually, I was a bit skeptical that VR would prove to have any scientific justification. I wanted this feature solely because it's cool, and if more people developed things because they were cool then the world would be a happier place*. Whether it had any real benefit was entirely besides the point. 

* Or perhaps not.

But actually, I have to say there probably is real value to this. It goes like this :

• Viewing data in 2D, slice by slice, takes a long time to build up an intuitive sense of its full 3D structure.
• Viewing the data as a 3D volume you can rotate on a 2D screen helps develop that intuitive sense much faster, because you can see all the data at once.
• Viewing the data in true stereoscopic data helps even more. Now you don't even need to rotate, making it even easier to grasp what's a coherent structure in the data and what's just a chance alignment (so-called projection effects). You instantly see the 3D structures directly.

There's even another program, iDaVIE, which is fully in VR with almost no 2D interface at all. I like it very much (and the performance is oodles and oodles better than FRELLED), though for me the analysis tasks I need aren't quite there yet. Which brings me on to the next section.

Why Should You (Not) Use It ?

You should use it because you're a cool person who loves awesome things ! As the wiki says, it's as much fun as a pile of kittens, and what right-thinking person doesn't love a pile of kittens ? Nobody, that's who.

Well... maybe I exaggerate. A bit. But it should be, I hope, reasonably user-friendly. It certainly won't yell at you or call you rude names in Welsh like the old version did. There is, unavoidably, a learning curve, but it's probably now (almost) as shallow as I can make it. 

Felix Stoehr coined this wonderful term "fastronomy", which is basically something I've mentally labelled as the Facebook Principle for many years. The referee wouldn't let me keep this analogy in the paper, but at its heart, Facebook is little more than glorified email. You could, in principle, set up a vast email list of thousands of strangers and send them pictures of your cats and/or genitalia, or rant at them about how cancer is causing vaccines or why pigeons are all mind-controlling robots. Nobody does any of that by email; lots of people do some of the above on Facebook. It's even considered normal*.

* I dunno why the referee objected. I barely even mentioned genitalia in the paper, let alone robot-pigeons.

Why is that ? Probably because Facebook et al. make otherwise awkward social interactions convenient and casual. And this is something astronomers have been traditionally very reluctant to acknowledge. As discussed at length in Malta, we often focus so much on accuracy that we forget about interface, making mistakes so easy to make that the result is inaccuracy – the very thing we were trying to avoid ! So fastronomy is something I buy into enthusiastically : we should make features as easy and accessible to use as we possibly can.

Surely this is just self-evident ? Yes and no. There is a school of thought that says some things should be challenging and hard to do. And to an extent they've got a point. Some tasks are indeed really very difficult and can't and shouldn't be deliberately over-simplified : that risks fooling people into thinking they immediately understand things when in fact they need years of training. And on occasion this legitimate concern slides into elitist snobbery. Far more common, though, is that we as astronomers just have too much to do to spend their time doing good interface/experience design, though I'm glad to see that there is at least a movement towards encouraging more user-friendly software*.

* One of the stranger objections, which to be honest I couldn't quite get my head around, from the referee was whether or not there was any real evidence that by making things easier to use people would actually use them. I mean, yes, actually yes I do, lots, but the whole question seemed strange to me. Who in their right mind would prefer the more difficult option if the easier one gave results of equal quality ? That bit does seem self-evident to me.

So in that spirit, every operation in FRELLED is accessible via the GUI. You don't need to ever see the code. Every graphical element has an explanatory tooltip. Some things only possible in the previous version by learning some rather hacky tricks from the manual have now been properly and robustly integrated. And every tool panel comes with a simple help menu explaining its individual functions. Once the very basics have been grasped ("here's how to open a file"), most other operations should come naturally. If I can't quite make it so that you can download it and go, I can least make it so that after 5 minutes or less you should have a fair idea of how to do most of the stuff you'll ever need to do.

Which leaves my final point : versatility. There are a lot of good visualisation tools out there, and a lot of good analysis tools too. Much less common are tools that do both. What I've tried to do with FRELLED is seize this unoccupied middle ground and strike a balance. It doesn't do the most advanced visualisation possible but it does have multiple different ways of viewing data : most programs either do 2D or 3D, rarely both, let alone giving a choice of volumetric or isosurface displays. And while it doesn't do every analysis operation possible, it does enough of the basics. At least, for what I need to do. The goal is that users shouldn't have to switch between packages unless they need something which is really quite specialist.

... bugger, I forgot to mention the analysis tools ! I should give a brief rundown, I guess. One of the main things is you can easily click on a source and create an object you can use for cataloguing, masking, and analysing the target. You can give it a human-readable name so you'll remember it and find it again later. This all sounds trivial and honestly it is but nothing else seems to let you catalogue sources quite like this, which to me is still bizarre (though I gather that CARTA has something similar). You can then use those objects for overlaying optical data, adding annotations and axes (both for creating figures but also for training new observers), generating contours plots, and even doing some basic spectral measurements if you also have miriad installed. It does, in short, far from everything, but still quite a bit.

Example figure of some HI contours overlaid on an optical image, created entirely in FRELLED. It sounds minor but the the colour bar on the right was created automatically, which was otherwise a mind-numbingly tedious process that we'd only ever do if we were sure we'd got things just right. Making that automatic allows the user to experiment. The annotation objects and labels were created just by placing the cursor, typing in a toolbox and pressing a button. No need to open PhotoShop ! 

The interface plays a role in versatility too. There's a lot of options for setting things that previous were only visible under the bonnet, i.e. by tinkering with the code itself. Now, for example, you can manually configure the setup to optimise the display for smaller files, or (if you're feeling brave) to go for even larger data sets – all of which include checks to prevent the user from doing something really stupid to the file, or for correcting their mistakes if they insist on being a damn fool about it.

And what I really hope people will do is things I can't predict. I want them to be able to experiment with viewing their data in different ways, rather than sticking with what they already know. I want them to try stuff because it's so easy there's no point in not trying it, without feeling like they're going to muck up they're whole project if something goes wrong. And I want them to be able to load as many different sorts of data sets as they can, not be restricted to just one particular frequency. 

Most of all, I'd like users to feel like they're doing data visualisation (and taking it seriously as a much-needed aspect of analysis) without it feeling like it's a yet another skill set they need to learn. Users should, I hope, be able to learn Blender as astronomers, not become fully-fledged Blender artists who are also interested in science. Ultimately, making fun-looking 3D figures should be normal and for everyone, not a cool-looking bonus option that requires yet more specialist training in an already overloaded discipline.

What's Next ?

Aaaargh ! It happened again ! When I first looked in Blender post-2.79, the testing versions of 2.8 were available that had a version of the EEVEE internal real-time rendering engine that didn't work well. In FRELLED, the cube stays visible at full resolution and a high frame rate. In the pre-release of 2.8 there was a huge, horrible lag in updating the display which would have made it unusable. You can't freely explore a data set like this unless you're a masochist.

Then in the actual releases of 2.8 and later versions this was overcome. And they offer a really powerful extra option : you can adjust the colours of the images after loading them in, which you can't in Blender 2.79. One of the major restrictions of FRELLED is that loading the data sets can be quite slow in comparison to other programs, especially since if you get the colour scaling wrong you can't adjust it. So this alone makes recoding FRELLED, once again, worth doing. It would also give greater performance, allow for larger cubes, have direct viewing in VR, and a bunch of other things.

Recoding FRELLED the next time round won't be the same level of challenge as here, not even close. The Python syntax is more like a modified, incremental change from that of Blender 2.79 with only a few parts which were completely replaced. So this is doable, but it's not happening anytime soon because I did to some actual science again (watch this space). There are, though, a bunch of other features I'd like to add when I eventually get back to this :

• Support for really large cubes (tens of gigabytes) and higher performance for smaller ones
• Automatically overlaying data from online catalogues, to make source identification easier
• Bring back the particle viewer from the old version (I had to drop this just because the project was ballooning into the unsustainable)
• Allowing 2D files, just for the sake of it
• And a whole bunch of under-the-bonnet changes that would make the experience still more natural and faster.

---

So that's this paper. No science here, just a development that's been in progress since 2012. Maybe this, the 13th year, will be the one in which finally people start to use it. Well, you never know.