Nope

Sometime late last summer I saw a job advert I was morally obligated to apply for. Astronomy Visualisation Specialist ? I am one already ! Experience with visualisation software and generic programming languages, e.g. Python ? The 11,000 lines of Python code for Blender that I wrote ought to tick that box and then some. New ways of visualising data ? Assisting astronomers cre... look, just visit my website. It's all there. All of it. Never have I seen a job description that felt so precisely tailored to me.

The application deadline was 1st October, though I submitted mine well before that. It's normal in astronomy for responses to take at least one month, sometimes two or even three. A few places - which are downright rude - never bother to respond. Still, by early December I was beginning to suspect that despite being objectively very, very qualified for the job, they must have given it to someone else. Well, it did say, "as soon as possible" on the job description.

They hadn't. I'm not sure if you'd call it an early Christmas present or not but I had a Skype interview on 18th December. Which was the day after I moved out of my flat in Prague (roomate left, couldn't possibly afford the place on my own) which had involved a week of hauling heavy suitcases back and forth to move my stuff to the institute. And it was the day after I got back to Cardiff, just to make things as frantic as possible.

Anyway it went well, but unfortunately it went well for everyone else as well. After a rather nervous Christmas, in early January I got en email saying that they'd go to a second stage round where they'd send us all a data set to visualise. Which they duly did a couple of weeks later.

It was actually quite a fun little project to work on, because the data set wasn't in a format I was familiar with. 3D data sets generally describe the density or temperature or whatever at different locations in space. The location is specified by 3 positions : x, y, z. Nothing very complicated about that.

And that's fine if, as is usually the case, your data set describes something that's roughly box-shaped. And by roughly I mean very roughly indeed, like this :

Simulation of a star-forming filamentary cloud, or something.

But this data set didn't use ordinary "Cartesian" coordinates, it used spherical polar coordinates. These aren't difficult either, but they may be unfamiliar. Instead of specifying 3 linear distances from the origin, they specify one distance and two angles :

Why in the world would you want to use such things ? Surely, it's more intuitive to think in terms of distances, not angles ! No, not always. There's a very simple everyday example that should help you understand : maps. With a street map, you could easily specify a position in Cartesian coordinates. You could say, for example, that Cardiff Castle is about 150m north and 25m west of the Revolution bar, if you thought that breaking into the castle on a Saturday night was somehow a good idea.

You could also specify how high the castle keep is, if it was vitally important to reach a precise level for some reason.

On a scale this small, the fact that the Earth is curved doesn't matter. You could hold out your arm and say, "go 50 metres in that direction" and no-one would have any difficulty. But if you said, "go 5,000 miles in that direction", anyone taking you literally would have ended up in space. Of course, they intuitively understand that you mean "along the surface of the Earth", not really, "in the path followed by a perfectly straight laser beam going in that direction". Unless they're a cat, of course.

This is why you don't give cats directions using laser pointers, because if it involves going into space then damn it that's what they'll do.

North, south, east and west are really just angles. Nothing very complicated about that : if you want to go to Australia you can say it's around 140 degrees east of Great Britain and 20 degrees south of the equator. Or you can give this in miles, it's the same thing.

We don't normally specify the distance from the centre r unless you're a mountaineer, pilot, miner, or deep sea diver. OK, you'd normally give distance from sea level rather than the centre of the Earth, but it's the same thing.

Or is it ? Well, not quite. 50 miles north or south is the same everywhere, unless you're so close to a pole you can't actually go that far. E.g. if you head in a northerly direction when you're 25 miles south of the north pole, after 25 miles you'll find yourself heading south. Much worse is the case of walking east or west if you're near the pole. You can't actually go east or west if you're standing on the pole itself, and if you're just a few steps away from the pole, then walking 25 miles east or west is going to involve walking around in a lot of circles until you get dizzy.

And at the north pole it will also involve discovering Santa's secret hideout or being eaten by a polar bear.

Using angles makes things a lot easier for cartographers. Line of longitude (east or west position) have constant angular separation, even though the physical distance between them varies (i.e. 1 degree involves walking a much larger distance at the equator than near the poles). And the mathematics to convert between the two is easy and precise, so if we have two lat-long positions we can easily compute how far we have to travel to get from one to the other - even if we're at weird positions like the poles.

In numerical simulations, polar coordinates have some other advantages, which are a bit more complicated. Imagine if you will a cat on a record player*. If you wanted to specify any point on the cat, you could give its x,y position. Or you could state its r,φ (pronounced "phi") coordinates instead. There's not really any advantage to either... unless the record player is turned on and it stars spinning.

*Conjecture : there is no scientific concept which cannot be explained with the right cat gif.

If that happens it's very much easier to specify how fast each point is moving in the φ direction. If you wanted to specify its velocity using x,y coordinates, you'd have to give two velocities - which is much less intuitive than saying, "it's spinning at such-and-such a speed". And anyway the velocities in the x,y directions are constantly changing and depend on distance from the centre of the record player, whereas the angular speed in the φ direction remains constant everywhere.

"The cat is spinning at 20 rpm (or 120 degrees per second)", vs, "the cat's x velocity is 1 m/s and its y velocity is 0.5 m/s, no wait now it's 0.6 m/s and 0.2 m/s, no wait it's changing again, aaaaaargh !"

Or to illustrate this slightly more scientifically :

When the green point is at the top or bottom of the circle, it has no velocity in the y-direction at all. Similarly, when it's at the extreme left or right, it has no velocity in the x-direction. But it always has a constant angular velocity.

So polar coordinates are much more useful for describing rotating discs. The problem is that of course for rendering images, pixels generally aren't in polar coordinates : we have to convert back to Cartesian. That's a problem when visualising simulations : just like trying to map the spherical Earth with a flat surface, you can get horrible distortions or lose detail if you're not careful.

Converting between polar and Cartesian coordinates is literally like trying to square the circle.

My first approach was to convert the data into the regular Cartesian system : knowing the r,θ,φ coordinates directly from the data, it was easy to convert to x,y,z. Which gave me this :

It's simulation of a protoplanetary disc.

... at which I make a brief interjection because at about that moment I received the following email, which I will keep anonymous because I'm not a total douchebag :

...and I do my first steps in scientific visualization. I am very interested in astronomy, though my main job till now was connected only with graphic design in university sector. I have some 3D modelling experience (several years ago me and my colleague made a fulldome video). Now I learn Blender and try to write my first scripts in Python. Slowly I become the idea, how everything works. Although the more I read, the more question I get...But it is normal I guess :)

Hopefully soon the quantity of my knowledges will transfer to their quality. About a month ago I got a chance to apply for a position as a specialist for astronomical visualization. My interview was quite successful and now we got a test task, which will probably define the proper candidate. I have already an idea, how to solve it. But it would be nice to find someone, who understand the materia, could evaluate my job and give me some practical advices. So I would like to ask you, if you had time and wish to answer some of my questions.

OK... one of the other candidates is asking me for help without even realising that I'm applying for the same job.

I decided the only safe course of action was to make absolutely no response whatsoever, so that's what I did.

Anyway the first result was not bad, but not great. The problem is that when you convert between coordinate systems there's no guarantee the Cartesian data set will be completely filled - especially at large radii. The polar coordinates tell you the centres of each data cell (pixel) and the density (or temperature or whatever) in that entire cell. But the centre of that cell only corresponds to the position of one particular pixel in Cartesian coordinates - it's not the same as checking every pixel in Cartesian coordinates and finding which polar cell they're in. The upshot is that you end up losing detail in the centre (where the polar cells are closer together than the Cartesian cells) and large blank areas at the edges (where the polar cells are further apart than the Cartesian cells).

To illustrate that, let's take another look at the comparison between polar and Cartesian grids. First in the very centre :

Every large square of the Cartesian grid contains multiple points from the polar grid. So multiple polar cells get reduced to a few Cartesian cells - detail is lost.

And now in the outskirts, shading every Cartesian cell that's intersected by at least one polar cell corner point :

Oh noes ! Not every Cartesian cell is filled ! And this only gets worse at larger radii.

That's not a hopeless problem. One nice feature when converting is that you're free to choose how many Cartesian pixels you want very easily, so you can optimise for a balance of detail in the centre vs. empty regions on the outskirts. In principle, you could then fill in the blanks based on the nearest pixel, or accurately determine the value for each Cartesian cell by working out which polar cell it corresponds to. Doable, but not easy - and certainly not doable in the space of an afternoon, which was the stated scope of the exercise.

There's a more fundamental problem : to you show all the detail in the central regions, you'll need a lot more cells in Cartesian coordinates than if you used polar. Large data sets can easily run into hundreds of millions of cells, which means hundreds of millions of pixels : ouch ! Wouldn't it be better if we could somehow have non-square pixels ? Then we could show the data in its original polar form, with no loss of detail and no need to have a single pixel more than we really needed.

It turns out that we can do just that in Blender. Consider a slice right through the centre of the protoplanetary discs. If we pretend that r and φ are really y and x, we get this :

Of course they're not really x and y at all, which means we're looking at something that's weirdly distorted. But we can correct for this. The method I came up with was to assign each pixel to a face in Blender (UV mapping). Then we can move each vertex (i.e. distort each face) to put it back where it would have been in polar coordinates.

Bingo - we can have our cake (original spherical polar coordinates) and eat it too (no need to convert to square pixels). Of course, the real data isn't just one slice - it's lots of slices, each of a constant angle θ. So what we have is a series of cones :

And if we show all the cones, we get this - which is a pretty convincing way to fake a volumetric render, with the gaps between the cones only becoming apparent at certain angles :

In the above, density controls bother temperature and opacity (transparency). But it doesn't have to be density. It could be, for example, this mysterious Q parameter which is apparently heat transport that I know nothing about except that it looks nice :

In principle, we could fill those gaps by switching to spheres when the viewing angle is through the cones. Actually, I started with spheres because I'd already tried this* - I only had the idea to use cones during a long and boring meeting. They look nice enough on their own, though to really get things perfect we'd need to combine the two.

* Spheres are much easier to do because there's no need for UV mapping - Blender can calculate the distortion to a sphere itself just fine, but not cones.

So having figured out this somewhat complicated process, some considerable time passed before I heard anything back. It felt like forever.

Eventually at the beginning of March - a full five months after the application deadline, I got the news that... I was invited to an on-site interview ! Which led me to an odd mix of glee and frustration.

Many wisecracks did ensue, of course. Maybe, said colleagues, they just wanted more data visualised as a sort of way of getting cheap labour. Maybe they hadn't rejected anyone from the first two rounds. Maybe the number of candidates was actually increasing at each stage.

Nonetheless, I want along to said interview about three weeks later and went at it hell for leather. I brought along both glass cubes, 3D glasses, a copy of the Discover magazine in which I nuked a potato, and I even organized most of my Blender files and scripts from the last 14 years. It would be a five year position with the possibility of a permanent contract at the end, with a salary far more... European than those of the Czech Republic. Worth fighting for, even if the "as soon as possible" phrase had long since rung hollow.

Getting to Heidelberg involved a 7 hour bus trip. It was a very nice bus and I had the whole lower compartment to myself, which was nice. With wi-fi. Heck, it was better than most British trains by a considerable margin. There wasn't much to look at on the trip, but I've always thought that it's far easier to make my own entertainment than it is to make my own legroom.

The only real event that happened was that there was a perilously short connection between the bus to Mannheim and the train to Heidelberg, so I ended up jumping on a plausible-looking train (German trains do not indicate the destination and route very clearly) more out of hope than expectation. Which resulted in a fairly tense 20 minutes until the train stopped at the correct destination. After hauling my really quite surprisingly heavy bags to my hotel, I had little enough time in the evening to do anything except a short walk. I didn't get to see any of the pretty parts of Heidelberg.

Of course I did see the Haus der Astronomie the next morning. My interview went as well as it could have gone. With hindsight I wouldn't have changed a damn thing. There wasn't time to show everything, but I left fully satisfied that I'd done as much as I could possibly have done. I answered all their questions. I showed them the extremely heavy data cubes and 3D movies. I was a little peeved that - bizarrely - they really did want someone to start extremely soon, but I'd have been more than prepared to take the job anyway (even though I'd really, really, really like an extended period back in the UK). So off I went back to Prague while they interviewed the other candidate.

Alas the return trip was not without incident. About one hour in to the seven hour trip, the bus ground to a halt due to an accident up ahead. It didn't move an inch for the next four hours. The bus driver let everyone off to walk around (and even some people on to use the toilet). I talked to some otherwise politically like-minded Germans who were, it must be said, none too fond of the Czechs, but even the nicest bus becomes wearying after 11 hours. I eventually crawled back into my room - still dragging my laptop and heavy glass cubes, of course, at about 2:30am.

Then I proceeded to play the waiting game. Again. For another three weeks. Until finally :

Dear Rhys,

thank you for your patience. It was a difficult decision, but in the end we offered the job to the other remaining candidate. This is in no way a reflection on the skills you demonstrated - we were very much impressed by your visualizations, found that you communicated well with the astronomers who would have been your colleagues here, and you would have been a great addition to our team. In the end, it came down to experience - the candidate to whom we offered, and who has now accepted, the job, is older than you and had put those additional years to good use in the visualization field.

We wish you all the best for your future career - you have an unusual and interesting mix of skills, and I hope you will find a good place to put them to the best possible use, either in astronomy or beyond!

It's a very nice rejection letter, but a rejection all the same. All of that effort had been for absolutely nothing except a free but incredibly exhausting 24 hours in Heidelberg. Was it worth it ? I'll let John Cleese answer that. Skip to 46:28 if it doesn't automatically.

On the one hand, the successful applicant is about 10 years older than me, done four-dimensional relativistic raytracing calculations, and has written a freakin' book. Fair enough, I'd have hired him instead of me. On the other hand, it doesn't take six months to decide who's more experienced. You can do that from the start. Especially if you use the words, "as soon as possible" in the advertisement.

Oh well. As Captain Picard once eloquently put it, "shit happens". In another reality, alternate me is taking a short break in Cardiff before preparing to move to yet another country despite never wanting to leave home in the first place. Actual me is now in the more mundane process of searching for a flat before he gets kicked out of the institute's accommodation. Aaaargh.