The week beginning February 8th was a big one for the LIGO and Virgo Collaborations. You might remember something about a few papers on the merger of a couple of black holes; however, those weren’t the only papers we published that week. In fact, they are not even (currently) the most cited…

Prospects for Observing and Localizing Gravitational-Wave Transients with Advanced LIGO and Advanced Virgo is known within the Collaboration as the Observing Scenarios Document. It has a couple of interesting aspects

• Its content is a mix of a schedule for detector commissioning and an explanation of data analysis. It is a rare paper that spans both the instrumental and data-analysis sides of the Collaboration.
• It is a living review: it is intended to be periodically updated as we get new information.

There is also one further point of interest for me: I was heavily involved in producing this latest version.

In this post I’m going to give an outline of the paper’s content, but delve a little deeper into the story of how this paper made it to print.

The Observing Scenarios

The paper is divided up into four sections.

1. It opens, as is traditional, with the introduction. This has no mentions of windows, which is a good start.
2. Section 2 is the instrumental bit. Here we give a possible timeline for the commissioning of the LIGO and Virgo detectors and a plausible schedule for our observing runs.
3. Next we talk about data analysis for transient (short) gravitational waves. We discuss detection and then sky localization.
4. Finally, we bring everything together to give an estimate of how well we expect to be able to locate the sources of gravitational-wave signals as time goes on.

Packaged up, the paper is useful if you want to know when LIGO and Virgo might be observing or if you want to know how we locate the source of a signal on the sky. The aim was to provide a guide for those interested in multimessenger astronomy—astronomy where you rely on multiple types of signals like electromagnetic radiation (light, radio, X-rays, etc.), gravitational waves, neutrinos or cosmic rays.

The development of the detectors’ sensitivity is shown below. It takes many years of tweaking and optimising to reach design sensitivity, but we don’t wait until then to do some science. It’s just as important to practise running the instruments and analysing the data as it is to improve the sensitivity. Therefore, we have a series of observing runs at progressively higher sensitivity. Our first observing run (O1), featured just the two LIGO detectors, which were towards the better end of the expected sensitivity.

It’s difficult to predict exactly how the detectors will progress (we’re doing many things for the first time ever), but the plot above shows our current best plan.

I’ll not go into any more details about the science in the paper as I’ve already used up my best ideas writing the LIGO science summary.

If you’re particularly interested in sky localization, you might like to check out the data releases for studies using (simulated) binary neutron star and burst signals. The binary neutron star analysis is similar to that we do for any compact binary coalescence (the merger of a binary containing neutron stars or black holes), and the burst analysis works more generally as it doesn’t require a template for the expected signal.

The path to publication

Now, this is the story of how a Collaboration paper got published. I’d like to take a minute to tell you how I became responsible for updating the Observing Scenarios…

In the beginning

The Observing Scenarios has its origins long before I joined the Collaboration. The first version of the document I can find is from July 2012. Amongst the labyrinth of internal wiki pages we have, the earliest reference I’ve uncovered was from August 2012 (the plan was to have a mature draft by September). The aim was to give a road map for the advanced-detector era, so the wider astronomical community would know what to expect.

I imagine it took a huge effort to bring together all the necessary experts from across the Collaboration to sit down and write the document.

Any document detailing our plans would need to be updated regularly as we get a better understanding of our progress on commissioning the detectors (and perhaps understanding what signals we will see). Fortunately, there is a journal that can cope with just that: Living Reviews in Relativity. Living Reviews is designed so that authors can update their articles so that they never become (too) out-of-date.

A version was submitted to Living Reviews early in 2013, around the same time as a version was posted to the arXiv. We had referee reports (from two referees), and were preparing to resubmit. Unfortunately, Living Reviews suspended operations before we could. However, work continued.

Updating sky localization

I joined the LIGO Scientific Collaboration when I started at the University of Birmingham in October 2013. I soon became involved in a variety of activities of the Parameter Estimation group (my boss, Alberto Vecchio, is the chair of the group).

Sky localization was a particularly active area as we prepared for the first runs of Advanced LIGO. The original version of the Observing Scenarios Document used a simple approximate means of estimating sky localization, using just timing triangulation (it didn’t even give numbers for when we only had two detectors running). We knew we could do better.

We had all the code developed, but we needed numbers for a realistic population of signals. I was one of the people who helped running the analyses to get these. We had the results by the summer of 2014; we now needed someone to write up the results. I have a distinct recollection of there being silence on our weekly teleconference. Then Alberto asked me if I would do it? I said yes: it would probably only take me a week or two to write a short technical note.

Saying yes is a slippery slope.

That note became Parameter estimation for binary neutron-star coalescences with realistic noise during the Advanced LIGO era, a 24-page paper (it considers more than just sky localization).

Numbers in hand, it was time to update the Observing Scenarios. Even if things were currently on hold with Living Reviews, we could still update the arXiv version. I thought it would be easiest if I put them in, with a little explanation, myself. I compiled a draft and circulated in the Parameter Estimation group. Then it was time to present to the Data Analysis Council.

The Data Analysis Council either sounds like a shadowy organisation orchestrating things from behind the scene, or a place where people bicker over trivial technical issues. In reality it is a little of both. This is the body that should coordinate all the various bits of analysis done by the Collaboration, and they have responsibility for the Observing Scenarios Document. I presented my update on the last call before Christmas 2014. They were generally happy, but said that the sky localization on the burst side needed updating too! There was once again a silence on the call when it came to the question of who would finish off the document. The Observing Scenarios became my responsibility.

(I had though that if I helped out with this Collaboration paper, I could take the next 900 off. This hasn’t worked out.)

The review

With some help from the Burst group (in particular Reed Essick, who had lead their sky localization study), I soon had a new version with fully up-to-date sky localization. This was ready for our March Collaboration meeting. I didn’t go (I was saving my travel budget for the summer), so Alberto presented on my behalf. It was now agreed that the document should go through internal review.

It’s this which I really want to write about. Peer review is central to modern science. New results are always discussed by experts in the community, to try to understand the value of the work; however, peer review is formalised in the refereeing of journal articles, when one or more (usually anonymous) experts examine work before it can be published. There are many ups and down with this… For Collaboration papers, we want to be sure that things are right before we share them publicly. We go through internal peer review. In my opinion this is much more thorough than journal review, and this shows how seriously the Collaboration take their science.

Unfortunately, setting up the review was also where we hit a hurdle—it took until July. I’m not entirely sure why there was a delay: I suspect it was partly because everyone was busy assembling things ahead of O1 and partly because there were various discussions amongst the high-level management about what exactly we should be aiming for. Working as part of a large collaboration can mean that you get to be involved in wonderful science, but it can means lots of bureaucracy and politics. However, in the intervening time, Living Reviews was back in operation.

The review team consisted of five senior people, each of whom had easily five times as much experience as I do, with expertise in each of the areas covered in the document. The chair of the review was Alan Weinstein, head of the Caltech LIGO Laboratory Astrophysics Group, who has an excellent eye for detail. Our aim was to produce the update for the start of O1 in September. (Spolier: We didn’t make it)

The review team discussed things amongst themselves and I got the first comments at the end of August. The consensus was that we should not just update the sky localization, but update everything too (including the structure of the document). This precipitated a flurry of conversations with the people who organise the schedules for the detectors, those who liaise with our partner astronomers on electromagnetic follow-up, and everyone who does sky localization. I was initially depressed that we wouldn’t make our start of O1 deadline; however, then something happened that altered my perspective.

On September 14, four days before the official start of O1, we made a detection. GW150914 would change everything.

First, we could no longer claim that binary neutron stars were expected to be our most common source—instead they became the source we expect would most commonly have an electromagnetic counterpart.

Second, we needed to be careful how we described engineering runs. GW150914 occurred in our final engineering run (ER8). Practically, there was difference between the state of the detector then and in O1. The point of the final engineering run was to get everything running smoothly so all we needed to do at the official start of O1 was open the champagne. However, we couldn’t make any claims about being able to make detections during engineering runs without being krass and letting the cat out of the bag. I’m rather pleased with the sentence

Engineering runs in the commissioning phase allow us to understand our detectors and analyses in an observational mode; these are not intended to produce astrophysical results, but that does not preclude the possibility of this happening.

I don’t know if anyone noticed the implication. (Checking my notes, this was in the September 18 draft, which shows how quickly we realised the possible significance of The Event).

Finally, since the start of observations proved to be interesting, and because the detectors were running so smoothly, it was decided to extend O1 from three months to four so that it would finish in January. No commissioning was going to be done over the holidays, so it wouldn’t affect the schedule. I’m not sure how happy the people who run the detectors were about working over this period, but they agreed to the plan. (No-one asked if we would be happy to run parameter estimation over the holidays).

After half-a-dozen drafts, the review team were finally happy with the document. It was now October 20, and time to proceed to the next step of review: circulation to the Collaboration.

Collaboration papers go through a sequence of stages. First they are circulated to the everyone for comments. This can be pointing out typos, suggesting references or asking questions about the analysis. This lasts two weeks. During this time, the results must also be presented on a Collaboration-wide teleconference. After comments are addressed, the paper is sent for examination Executive Committees of the LIGO and Virgo Collaborations. After approval from them (and the review team check any changes), the paper is circulated to the Collaboration again for any last comments and checking of the author list. At the same time it is sent to the Gravitational Wave International Committee, a group of all the collaborations interested in gravitational waves. This final stage is a week. Then you can you can submit the paper.

Peer review for the journal doesn’t seem to arduous in comparison does it?

Since things were rather busy with all the analysis of GW150914, the Observing Scenario took a little longer than usual to clear all these hoops. I presented to the Collaboration on Friday 13 November. (This was rather unlucky as I was at a workshop in Italy and I had to miss the tour of the underground Laboratori Nazionali del Gran Sasso). After addressing comments from everyone (the Executive Committees do read things carefully), I got the final sign-off to submit December 21. At least we made it before the end of O1.

Good things come…

This may sound like a tale of frustration and delay. However, I hope that it is more than that, and it shows how careful the Collaboration is. The Observing Scenarios is really a review: it doesn’t contain new science. The updated sky localization results are from studies which have appeared in peer-reviewed journals, and are based upon codes that have been separately reviewed. Despite this, every statement was examined and every number checked and rechecked, and every member of the Collaboration had opportunity to examine the results and comment on the document.

I guess this attention to detail isn’t surprising given that our work is based on measuring a change in length of one part in 1,000,000,000,000,000,000,000.

Since this is how we treat review articles, can you imagine how much scrutiny the Discovery Paper had? Everything had at least one extra layer of review, every number had to be signed-off individually by the appropriate review team, and there were so many comments on the paper that the editors had to switch to using a ticketing system we normally use for tracking bugs in our software. This level of oversight helped me to sleep a little more easily: there are six numbers in the abstract alone I could have potentially messed up.

Of course, all this doesn’t mean we can’t make mistakes…

Looking forward

The Living Reviews version was accepted January 22, just after the end of O1. We made had to make a couple of tweaks to correct tenses. The final version appeared February 8, in time to be the last paper of the pre-discovery era.

It is now time to be thinking about the next update! There are certainly a few things on the to-do list (perhaps even some news on LIGO-India). We are having a Collaboration meeting in a couple of weeks’ time, so hopefully I can start talking to people about it then. Perhaps it’ll be done by the start of O2? [update]

 

arXiv: 1304.0670 [gr-qc]
Journal: Living Reviews In Relativity; 19:1(39); 2016
Science summary: Planning for a Bright Tomorrow: Prospects for Gravitational-wave Astronomy with Advanced LIGO and Advanced Virgo
Bonus fact: This is the only paper whose arXiv ID I know by heart [update].

arXiv IDs

Papers whose arXiv numbers I know by heart are: 1304.0670, 1602.03840 (I count to other GW150914 companion papers from here), 1606.04856 and 1706.01812. These might tell you something about my reading habits.

The next version

Despite aiming for the start of O2, the next version wasn’t ready for submission until just after the end of O2, in September 2017. It was finally published (after an excpetionally long time in type-setting) in April 2018.