Month: October 2014

Tips for scientific writing

/ CPLB / Leave a comment

Second year physics undergraduates at the University of Birmingham have to write an essay as part of their course. As a tutor, it’s my job to give them advice on how to write in a scientific style (and then mark the results). I have assembled these tips to try to aid them (and make my marking less painful). Much of this advice also translates to paper writing, and I try to follow these tips myself.

Writing well is difficult. It requires practice. It is an important skill, yet it is something that I do not believe is frequently formally taught (at least in the sciences). Scientific and other technical writing can be especially hard, as it has its own rules that can be at odds with what we learn at school (when studying literature or creative writing). Reading the work of others is a good way for figuring out what works well and what does not.

In this post, I include some tips that I hope are useful (not everyone will agree). I begin by considering how to plan and structure a piece of writing (section 1), from the largest scale (section 1.2) progressing down to the smallest (section 1.4); then I discuss various aspects of technical writing (section 2), both in terms of content and style, including referencing (section 2.5), which is often problematic, and I conclude with some general editing advice (section 3) before summarising (section 4). If you have anything extra to add, please do so in the comments.

1 Structure and planning

The structure of your writing is important as it reflects the logical flow of your arguments. It is worth spending some time before you start writing considering what you want to say, and what is the best order for your ideas. (This is also true in exams: I have found when trying to answer essay questions it is worth the time to spend a couple of minutes planning, otherwise I am liable to miss out an important point). I frequently get frustrated that I must write linearly, one idea after another, and cannot introduce multiple strands at a time, with arguments intertwining with each other. However, putting in the effort to construct a clear progression does help your reader.

1.1 Title and audience

The first thing to consider is what you want to write and who is going to read it. Always write for your audience, and remember that professional scientists and the general public look for different things (this blog may be a poor example of this, as different posts are targeted towards different audiences).

Having thought about what you want to say, pick a title that reflects this. Don’t have a title “The life and works of Albert Einstein” if you are only going to cover special relativity, and don’t have a title “Equilibrium thermodynamics of non-oxide perovskite superconductors” if you are writing for a general audience. If your title is a question, make sure you answer it. It might be a good idea to write your title after you have finished your main text so that you can match it to what you have actually written.

1.2 Beginning, middle and end

To help your audience understand what you are telling them, begin with an introduction, and end with a summary. This is also true when giving a talk. Start by explaining what you will tell them, then tell them, then tell them what you told them. Repetition of key ideas makes them more memorable and help to emphasise what your audience should take away.

At the beginning, introduce the key ideas you will talk about. If you are writing an essay titled “The Solar Neutrino Problem“, you should explain what a solar neutrino is and why there is a problem. You might also like to explain why the reader should care. Sketching out the contents of the rest of the work is useful as it prepares the reader for what will follow: it’s like warm-up stretches for the mind. The introduction sets the scene for the arguments to follow.

The main body of your text contains most of the information, this is where you introduce your ideas and explain them. It is the burger between the buns of the introduction and conclusion. For longer documents, or subjects with many aspects, you might consider breaking this up into sections (and subsections). Using headings (perhaps numbered for reference) is good: skimming section headings should give an outline of the contents. Some sections within the main body might be sufficiently involved to merit their own introduction and summary. There should be a clear progression of ideas: if you find there is a big jump, try writing some text to cover the transition (“Having explained how neutrinos are produced in the Sun, we now consider how they are detected on the Earth”).

After presenting your arguments, it is good to summarise. As an example, a summary on the solar neutrino problem could be:

“Experiments measuring neutrinos from the Sun only detected about a third as many as expected. This could indicate either a problem with our understanding of solar physics or of particle physics. It is not possible to modify solar models to match both the measured neutrino flux and observations of luminosity and composition; however, the reduced flux could be explained by introducing neutrino oscillations. These were subsequently observed in several experiments. The solar neutrino problem has therefore been resolved by introducing new particle physics.”

Don’t introduce new arguments at this stage, this is just as unsatisfying as reading a murder mystery and discovering the murderer was someone never mentioned before. In my solar neutrino example, both the solar models and neutrino oscillations should have been discussed. Distilling your argument down to few lines also helps you to double-check your logic.

Either as part of your summary, of following on from it, end your writing with a conclusion. This is what you want your audience to have learnt (it should be the answer to your question). It is OK if you cannot produce a concrete answer, there are many cases where there is no clear-cut solution, perhaps more data is needed: in these cases, your conclusion is that there is no simple answer. To check that you have successfully wrapped things up, try reading just your introduction and conclusion; these should pair up to form a delicious (but bite-sized) sandwich.

1.3 Paragraphs

On a smaller scale, your writing is organised using paragraphing. Paragraphs are the building blocks of your arguments; each paragraph should address a single point or idea. Big blocks of text are hard to read (and look intimidating), so it is good to break them up. You can think of each paragraph as a micro-essay: the first sentence (usually) introduces the subject, you then go on to elaborate, before reaching a conclusion at the end (see section 1.2). To check that your paragraph sticks to a single point (and doesn’t need to be broken up), try reading the first and last sentences, usually they should make sense together.

1.4 Sentences

Paragraphs are constructed from sentences. Ensure your sentences make sense, that they are grammatically correct and that their subject is clear.

Vary your sentence length. In technical writing there is often the temptation, even amongst the best writers, to include long, convoluted sentences in order to fully describe a complicated idea and include all the relevant details, but these can be hard to read, both because of the complexity of their structure, which may require significant mental effort to unpack, and because by the time they finally conclude, the reader has forgotten the initial topic of the over-long, rambling sentence. Brevity gives impact. Shorter sentences are easier to understand. Breaking up your ideas helps the reader. Short sentences also get boring. They seem repetitive. They are tiring to read. They can send your reader to sleep. It is, therefore, better to have a range of sentence lengths. Include some short. In addition to these, have some longer sentences, as these allow you to join up your ideas. If you are unsure where to break up long sentences, look for commas (or semi-colons, etc.); if you are unsure where to put commas, read the sentence and see where you would pause.

2 Writing style and referencing

Having discussed how to structure your writing, we now move on to what to write. Technical writing has some specific requirements with regards to content, these might seem peculiar when first encountered. I’ll try to explain why we do certain things in technical writing, and give some ideas on how to incorporate these ideas to improve your own writing.

2.1 Be specific

The most common mistake I come across in my students’ work is the failure to be specific. The following two points (sections 2.2 and 2.3) are closely related to this. As an example, consider making a comparison:

  • Poor — “Nuclear power provides more energy than fossil fuel.”
  • Better — “Per unit mass of fuel, nuclear fission releases more energy than the burning of fossil fuel.”
  • Even better — “Nuclear fission can produce ~8000 times as much energy per unit mass of fuel as burning fossil fuels: the same amount of energy is produced from 16 kg of fossil fuels as by using 2 g of uranium in a standard reactor (MacKay, 2008).”

Here, we have specified exactly what we are comparing, given figures to allow a quantitative comparison, and provided references for those figures (see section 2.5). If possible, give numbers; don’t say “many ” or “lots” or “some”, but say “70%”, “9 billion” or “six Olympic swimming pools”.

Weak modifiers like “very”, “quite”, “somewhat” or “highly” are another example where it is better to be specific. What is the difference between being “hot” and being “very hot”? I might say that my bowl of soup is very hot, but does that tell you any more than if I just said it was hot? It is tempting to use these words for emphasis, surely if I were talking about the surface of the Sun we can agree that’s very hot? Not if you were to compare it to the centre of the Sun! Often, what is hot or cold, big or small, fast or slow depends upon the context. What is hot for soup is cold for the Sun, and what is cold for soup is hot for superconductors. It is much better to make distinctions by using figures: “The surface of the Sun is about 6000 K”.

It is OK to use “very” if you define the range where this is applicable, for example “High frequency radio waves are between 3 MHz and 30 MHz, very high frequency radio waves are between 30 MHz and 300 MHz, and ultra high frequency radio waves are between 300 MHz and 3 GHz.”

2.2 Provide justification

When putting forward an argument, it is necessary to include some evidence or justification to back it up. It is not sufficient merely to assert your opinion because you need the reader to follow your reasoning. If you are using someone else’s argument, you should provide a citation (section 2.5); the reader can then check there to find the reasoning. However, if it is an important point you might like to add some exposition. If you are being good about providing quantitative statements (section 2.1), you are already part way there as you can use those figures a back-up. For example, if discussing global warming, it is easy to argue it is important if you have already included figures on how many people would lose their homes to rising sea-levels, or if comparing materials, it is straightforward to argue that aluminium is better for making aeroplanes than steel if you have already included their densities. Sometimes, all that is required is an explanation of your reasoning, for example, “It is a good idea to build nuclear power plants because this reduces reliance on fossil fuels” or “It is not advisable to lick the surface of the Sun because it doesn’t taste of golden syrup.” Here, the reader might disagree that it is a good idea to build nuclear power plants, but they understand that you are using dependence on fossil fuels as an argument instead of, say, environmental issues, or the reader might agree that it is a bad idea to lick the Sun, but might have been thinking more about its temperature than its flavour. Even if the reader does not agree with your conclusions, they should understand how you reached them.

A similar idea is to show rather than tell. Don’t tell me that something is a fascinating topic or an exciting concept, get on with explaining it! Similarly, don’t just say something is important, but explain why it is important. This allows the reader to decide upon things themselves, if you have justified your arguments then they should follow your logic.

2.3 Use the correct word

In technical writing there is often a specific word that should be used in a particular context. In common usage we might use weight and mass interchangeably, in physics they have different meanings. This sometimes trips people up as they naturally try to find synonyms to reduce the monotony of their work. Always use the correct term.

Technical language can be full of jargon. This makes things difficult to understand for an outsider. It is important to define unfamiliar terms to help the reader. In particular, acronyms must be defined the first time they are used. As an example, “When talking about online materials, the uniform resource locator (URL), otherwise known as the web address, is a string of characters that identifies a resource.” Avoid jargon as much as possible; try to always use the simplest word for the job. It will be necessary to use technical terms to describe things accurately, but if they are introduced carefully, these need not confuse the reader.

A particular pet-peeve of mine is the use of scare quotes, which I always read as if the author is making air quotes. If quoting someone else’s choice of phrase then quotation marks are appropriate, and a reference must be provided (section 2.5). Most of the time, these quotation marks are used to indicate that the author thinks the terminology isn’t quite right. If the terminology is incorrect, use a different word (the correct one); if the terminology is correct (if that is what is used in the field), then the quotation marks aren’t needed!

2.4 Use equations and diagrams

Most physics problems involve solving an equation or two. For these mathematical questions, I am always encouraging my students to explain their work, to use words. When writing essays, I find they have the opposite problem: they only use prose and don’t include equations (or diagrams). Equations are useful for concisely and precisely explaining relationships, it is good to include them in writing.

Equations may put off general readers, but they improve the readability of technical work. Consider describing the kinetic energy of a (non-relativistic) particle:

  • With only words — “The kinetic energy of a particle depends upon its mass and speed: it is directly proportional to the mass and increases with the square of the speed.”
  • Using an equation — “The kinetic energy of a particle E is given by E = (1/2)mv^2, where m is its mass and v is its velocity.”

The second method is more straightforward, there is no ambiguity in our description, and we also get the factor of a half so the reader can go away at calculate things for themselves. This was just a simple equation; if we were considering something more complicated, such as the kinetic energy of a relativistic particle

\displaystyle E = \left(\frac{1}{\sqrt{1- v^2/c^2}} - 1\right)mc^2,

where c is the speed of light, it is much harder to produce a comprehensive description using only words. In this case, it is tempting to miss out reference to the equation. Sometimes this is justified: if the equation is too complicated a reader will not understand its meaning, but, in many cases, an equation allows you to show exactly how a system changes, and this is extremely valuable.

When including an equation, always define the symbols that you are using. Some common constants, such as \pi, might be understood, but it is better safe than sorry.

Equations should be correctly punctuated. They are read as part of the surrounding text, with the equals sign read as the verb “equals”, etc.

Using diagrams is another way of providing information in a clear, concise format. Like equations, diagrams can replace long and potentially confusing sections of text. Diagrams can be pictures of experimental set-up, schematics of the system under discussion, or show more abstract information, such as illustrating processes (perhaps as a flow chart). The cliché is that a picture is worth a thousand words; as diagrams are so awesome for conveying information, I’m not even going to attempt to give an example where I try to use only words. Below is as example figure, which I have chosen as it also includes equations.

“Figure 1 shows the proton–proton (pp) chain, the series of thermonuclear reactions that provides most (~99%) of Sun’s energy (Bahcall, Serenelli & Basu, 2005). There are several neutrino-producing reactions.”

The pp chain

Figure 1: The thermonuclear reactions of the pp chain. The traditional names of the produced neutrinos are given in bold and the branch names are given in parentheses. Percentages indicate branching fractions. Adapted from Giunti & Kin (2007).

Graphs can be used to show relationships between quantities, or collections of data. They can be used for theoretical models or experimental results. In the example below I show both. Graphs might be useful for plotting especially complicated functions, where the equation isn’t easy to understand. There are many types of graph (scatter plots, histograms, pie charts), and picking the best way to show your data can be as challenging as obtaining it in the first place!

“In figure 2 we plot the orbital decay of the Hulse–Taylor binary pulsar, indicated by the shift in periastron time (the point in the orbit where the stars are closest together). The data are in excellent agreement with the prediction assuming that the orbit evolves because of the emission of gravitational waves.”

Periastron shift of binary pulsar

Figure 2: The cumulative shift of periastron time as a function of time of the Hulse–Taylor binary pulsar (PSR B1913+16). The points are measured values, while the curve is the theoretical prediction assuming gravitational-wave emission. Taken from Weisberg & Taylor (2005).

All diagrams should have a descriptive caption. It is usually good to number these for ease of reference. If you are using someone else’s figure, make sure to explicitly cite them in the caption (see section 2.5)—you need to unambiguously acknowledge that you have taken someone else’s work, and have not just used their data or ideas (which would also warrant a citation) to make your own.

Tables can also be used to present data. Tables might be better than plots for when there are only a few numbers to present. Like figures, tables should have a caption (which includes relevant references if the data is taken from another source), they should be numbered, and they should be referred to explicitly in the text.

When writing, it is useful to remember that different people learn better through different means: some prefer words, some love equations, and other like visual representations. Including equations and figures can help you communicate effectively with a wider audience.

There are conventions for how to present equations, graphs and tables. I shall return to this in future posts. The rules may seem arcane, but they are designed to make communication clear.

2.5 Referencing

At the end of any good piece of technical writing there should be a list of references, hence I have tackled referencing last in the section. (Sometimes this is done in footnotes rather than the end, but I’m ignoring that). However, referencing should not be considered something that is just done at the end, or something that is tacked on at the end as an after-thought; it is one of the most important components of academic writing.

We include references for several reasons:

  1. To show the source of facts, figures and ideas. This allows readers to verify things that we quote, to double-check we’ve not made an error or misinterpreted things. It also shows distinguishes what is our own from what we have taken from elsewhere. This is important in avoiding plagiarism, as we acknowledge when we use someone else’s work.
  2. To provide the reader with a further source of information. It is not possible to explain everything, and a reader might be interested in finding out more about a topic, how a particular quantity was measured or how a particular calculation was done. By providing a reference we give the reader something further they can read if they want to (that doesn’t mean our work shouldn’t make sense on it’s own: you should be able to watch The Avengers without having seen Iron Man, but it’s still useful to know what to watch to find out the back-story). By following references readers can see how ideas have developed and changed, and gain a fuller understanding of a topic.
  3. To give credit for useful work. This is linked to the idea of not claiming the ideas as your own (avoiding plagiarism), but in addition to that, by referencing something you are publicising it, by using it you are claiming that it is of good-enough quality to be trusted. If you are to look at an academic article you will often see a link to citing articles. The number of citations is used as a crude measure of the value of that paper. Furthermore, this linking can allow a reader to work forwards, finding new ideas built upon those in that paper, just as they can work backwards by following references.
  4. To show you know your stuff. This might sound rather cynical, but it is important to do your research. To understand a topic you need to know what work has been done in that area (you can’t always derive everything from first principles yourself), and you demonstrate your familiarity with a field by include references.

You must always include citations in the text at the relevant point: if you use an idea include your source, if you introduce a concept say where it came from. It is not acceptable just to have a list of references at the end: does the reader have to go through all of these to figure out what came from where?

There are multiple styles for putting citations in text. The two most common are the following:

  • Numeric (or Vancouver) — using a number, e.g., [1], where the references at the end form an ordered list. This has the advantage of not taking up much space, especially when including citations to multiple papers, e.g. [1–5].
  • Author–year (or Harvard) — using the authors and year of publication to identify the paper, e.g., (Einstein, 1905). This has the advantage of making it easier to identify a paper: I’ve no idea what [13] is until I flick to the end, but I know what (Hulse & Taylor, 1975) is about.

Which style you use might be specified for you or it might be a free choice. Whichever style you use, the important thing is to include relevant references at the appropriate place in the text.

Having figured out why we should reference, where we should put references and how to include citations in the text, the last piece is how to assemble the bibliographic information to include at the end (or in footnotes). Exactly what information is included and how it is formatted depends on the particular style: there are endless combinations. Again, this might be specified for you or might be a free choice, just make sure you are consistent. Basic information that is always included are an author (this may be an organisation rather than a person), so we know who to attribute the work to, and a date so we know how up-to-date it is. Other information that is included depends upon the source we are referencing: a journal article will need the name of the journal, the volume and page number; a book will need a title, edition and publisher; a website will need a title and URL, etc. We need to include all the necessary information for the reader to find the exact source we used (hence we need to include the edition of a book, the date updated or written for a website, and so on).

There are numerous guides online for how to format references correctly. Some software does it automatically (I use Mendeley to produce BibTeX, but that’s not for everyone). The University of Birmingham has a guide to using Havard-style referencing that is comprehensive.

A final issue remains of which sources to reference: how do you know that a source is reliable? This is an in-depth question, so I shall return to it is a dedicated post.

3 Editing

Writing isn’t finished as soon as you have all your ideas on the page, things often take some polishing up. Some people like to perfect things as they go along, others prefer to get everything down in whatever form and go back through after. Here, I conclude with some tips for editing.

3.1 Be merciless

Keep your writing short. Don’t waste your readers’ time or overcomplicate things. Cut unnecessary words.

There are some phrases that are typically superfluous:

  • “Obviously…” — If it is obvious, then the reader will realise it; if it’s not, you are patronising them.
  • “It should be noted that…” — That would be why it’s written down! (I hope you are not writing things that shouldn’t be noted).
  • “Remember that…” — You’re reminding the reader by writing it.
  • Any of the modifiers like “very”, “quite” or “extremely” mentioned in the section 2.3.

3.2 Proof-read

The single best method to improve a piece of writing is to proof-read it. Reread what you have written to check that it says what you think it should. I find I have to wait for a while after writing something to read it properly, otherwise I read what I intended to write rather than what I actually did. Having others read it is an excellent way to check it makes sense (especially if you are not a native English speaker); this is best if they are representative of your target audience.

I hate it when others find a mistake in my writing. It’s like rubbing a cat the wrong way. However, each mistake you find and correct makes your writing a little better, and that’s really the important thing.

4 Summary

In conclusion, my main tips for good scientific writing are:

  • Plan what you want to tell your audience and how they will take your message away.
  • Say what you’re going to say (introduction), then say it (main text), then say what you said (conclusion).
  • Have a clear, logical flow, with one point per paragraph.
  • Be specific and back up with your points with quantitative data and references.
  • Use equations and diagrams to help explain.
  • Be concise.
  • Proof-read (and get a second opinion).

If you have any further ideas for improving essay writing, please leave a comment.

An introduction to probability: Inference and learning from data

/ CPLB / Leave a comment

Probabilities are a way of quantifying your degree of belief. The more confident you are that something is true, the larger the probability assigned to it, with 1 used for absolute certainty and 0 used for complete impossibility. When you get new information that updates your knowledge, you should revise your probabilities. This is what we do all the time in science: we perform an experiment and use our results to update what we believe is true. In this post, I’ll explain how to update your probabilities, just as Sherlock Holmes updates his suspicions after uncovering new evidence.

Taking an umbrella

Imagine that you are a hard-working PhD student and you have been working late in your windowless office. Having finally finished analysing your data, you decide it’s about time to go home. You’ve been trapped inside so long that you no idea what the weather is like outside: should you take your umbrella with you? What is the probability that it is raining? This will depend upon where you are, what time of year it is, and so on. I did my PhD in Cambridge, which is one of the driest places in England, so I’d be confident that I wouldn’t need one. We’ll assume that you’re somewhere it doesn’t rain most of the time too, so at any random time you probably wouldn’t need an umbrella. Just as you are about to leave, your office-mate Iris comes in dripping wet. Do you reconsider taking that umbrella? We’re still not certain that it’s raining outside (it could have stopped, or Iris could’ve just been in a massive water-balloon fight), but it’s now more probable that it is raining. I’d take the umbrella. When we get outside, we can finally check the weather, and be pretty certain if it’s raining or not (maybe not entirely certain as, after plotting that many graphs, we could be hallucinating).

In this story we get two new pieces of information: that newly-arrived Iris is soaked, and what we experience when we get outside. Both of these cause us to update our probability that it is raining. What we learn doesn’t influence whether it is raining or not, just what we believe regarding if it is raining. Some people worry that probabilities should be some statement of absolute truth, and so because we changed our probability of it raining after seeing that our office-mate is wet, there should be some causal link between office-mates and the weather. We’re not saying that (you can’t control the weather by tipping a bucket of water over your office-mate), our probabilities just reflect what we believe. Hopefully you can imagine how your own belief that it is raining would change throughout the story, we’ll now discuss how to put this on a mathematical footing.

Bayes’ theorem

We’re going to venture into using some maths now, but it’s not too serious. You might like to skip to the example below if you prefer to see demonstrations first. I’ll use P(A) to mean the probability of A. A joint probability describes the probability of two (or more things), so we have P(A, B) as the probability that both A and B happen. The probability that A happens given that B happens is the conditional probability P(A|B). Consider the the joint probability of A and B: we want both to happen. We could construct this in a couple of ways. First we could imagine that A happens, and then B. In this case we build up the joint probability of both by working out the probability that A happens and then the probability B happens given A. Putting that in equation form

P(A,B) = P(A)P(B|A).

Alternatively, we could have B first and then A. This gives us a similar result of

P(A,B) = P(B)P(A|B).

Both of our equations give the same result. (We’ve checked this before). If we put the two together then

P(B|A)P(A) = P(A|B)P(B).

Now we divide both sides by P(A) and bam:

\displaystyle P(B|A) = \frac{P(A|B)P(B)}{P(A)},

this is Bayes’ theorem. I think the Reverend Bayes did rather well to get a theorem named after him for noting something that is true and then rearranging! We use Bayes’ theorem to update our probabilities.

Usually, when doing inference (when trying to learn from some evidence), we have some data (that our office-mate is damp) and we want to work out the probability of our hypothesis (that it’s raining). We want to calculate P(\mathrm{hypothesis}|\mathrm{data}). We normally have a model that can predict how likely it would be to observe that data if our hypothesis is true, so we know P(\mathrm{data}|\mathrm{hypothesis}), so we just need to convert between the two. This is known as the inverse problem.

We can do this using Bayes’ theorem

\displaystyle P(\mathrm{hypothesis}|\mathrm{data}) = \frac{P(\mathrm{data}|\mathrm{hypothesis})P(\mathrm{hypothesis})}{P(\mathrm{data})}.

In this context, we give names to each of the probabilities (to make things sound extra fancy): P(\mathrm{hypothesis}|\mathrm{data}) is the posterior, because it’s what we get at the end; P(\mathrm{data}|\mathrm{hypothesis}) is the likelihood, it’s what you may remember calculating in statistics classes; P(\mathrm{hypothesis}) is the prior, because it’s what we believed about our hypothesis before we got the data, and P(\mathrm{data}) is the evidence. If ever you hear of someone doing something in a Bayesian way, it just means they are using the formula above. I think it’s rather silly to point this out, as it’s really the only logical way to do science, but people like to put “Bayesian” in the title of their papers as it sounds cool.

Whenever you get some new information, some new data, you should update your belief in your hypothesis using the above. The prior is what you believed about hypothesis before, and the posterior is what you believe after (you’ll use this posterior as your prior next time you learn something new). There are a couple of examples below, but before we get there I will take a moment to discuss priors.

About priors: what we already know

There have been many philosophical arguments about the use of priors in science. People worry that what you believe affects the results of science. Surely science should be above such things: it should be about truth, and should not be subjective! Sadly, this is not the case. Using Bayes’ theorem is the only logical thing to do. You can’t calculate a probability of what you believe after you get some data unless you know what you believed beforehand. If this makes you unhappy, just remember that when we changed our probability for it being raining outside, it didn’t change whether it was raining or not. If two different people use two different priors they can get two different results, but that’s OK, because they know different things, and so they should expect different things to happen.

To try to convince yourself that priors are necessary, consider the case that you are Sherlock Holmes (one of the modern versions), and you are trying to solve a bank robbery. There is a witness who saw the getaway, and they can remember what they saw with 99% accuracy (this gives the likelihood). If they say the getaway vehicle was a white transit van, do you believe them? What if they say it was a blue unicorn? In both cases the witness is the same, the likelihood is the same, but one is much more believable than the other. My prior that the getaway vehicle is a transit van is much greater than my prior for a blue unicorn: the latter can’t carry nearly as many bags of loot, and so is a silly choice.

If you find that changing your prior (to something else sensible) significantly changes your results, this just means that your data don’t tell you much. Imagine that you checked the weather forecast before leaving the office and it said “cloudy with 0–100% chance of precipitation”. You basically believe the same thing before and after. This really means that you need more (or better) data. I’ll talk about some good ways of calculating priors in the future.

Solving the inverse problem

Example 1: Doughnut allergy

We shall now attempt to use Bayes’ theorem to calculate some posterior probabilities. First, let’s consider a worrying situation. Imagine there is a rare genetic disease that makes you allergic to doughnuts. One in a million people have this disease, that only manifests later in life. You have tested positive. The test is 99% successful at detecting the disease if it is present, and returns a false positive (when you don’t have the disease) 1% of the time. How worried should you be? Let’s work out the probability of having the disease given that you tested positive

\displaystyle P(\mathrm{allergy}|\mathrm{positive}) = \frac{P(\mathrm{positive}|\mathrm{allergy})P(\mathrm{allergy})}{P(\mathrm{positive})}.

Our prior for having the disease is given by how common it is, P(\mathrm{allergy}) = 10^{-6}. The prior probability of not having the disease is P(\mathrm{no\: allergy}) = 1 - P(\mathrm{allergy}). The likelihood of our positive result is P(\mathrm{positive}|\mathrm{allergy}) = 0.99, which seems worrying. The evidence, the total probability of testing positive P(\mathrm{positive}) is found by adding the probability of a true positive and a false positive

 P(\mathrm{positive}) = P(\mathrm{positive}|\mathrm{allergy})P(\mathrm{allergy}) + P(\mathrm{positive}|\mathrm{no\: allergy})P(\mathrm{no\: allergy}).

The probability of a false positive is P(\mathrm{positive}|\mathrm{no\: allergy}) = 0.01. We thus have everything we need. Substituting everything in, gives

\displaystyle P(\mathrm{allergy}|\mathrm{positive}) = \frac{0.99 \times 10^{-6}}{0.99 \times 10^{-6} + 0.01 \times (1 - 10^{-6})} = 9.899 \times 10^{-5}.

Even after testing positive, you still only have about a one in ten thousand chance of having the allergy. While it is more likely that you have the allergy than a random member of the public, it’s still overwhelmingly probable that you’ll be fine continuing to eat doughnuts. Hurray!

Doughnut love

Doughnut love: probably fine.

Example 2: Boys, girls and water balloons

Second, imagine that Iris has three children. You know she has a boy and a girl, but you don’t know if she has two boys or two girls. You pop around for doughnuts one afternoon, and a girl opens the door. She is holding a large water balloon. What’s the probability that Iris has two girls? We want to calculate the posterior

\displaystyle P(\mathrm{two\: girls}|\mathrm{girl\:at\:door}) = \frac{P(\mathrm{girl\:at\:door}|\mathrm{two\: girls})P(\mathrm{two\: girls})}{P(\mathrm{girl\:at\:door})}.

As a prior, we’d expect boys and girls to be equally common, so P(\mathrm{two\: girls}) = P(\mathrm{two\: boys}) = 1/2. If we assume that it is equally likely that any one of the children opened the door, then the likelihood that one of the girls did so when their are two of them is P(\mathrm{girl\:at\:door}|\mathrm{two\: girls}) = 2/3. Similarly, if there were two boys, the probability of a girl answering the door is P(\mathrm{girl\:at\:door}|\mathrm{two\: boys}) = 1/3. The evidence, the total probability of a girl being at the door is

P(\mathrm{girl\:at\:door}) =P(\mathrm{girl\:at\:door}|\mathrm{two\: girls})P(\mathrm{two\: girls}) +P(\mathrm{girl\:at\:door}|\mathrm{two\: boys}) P(\mathrm{two\: boys}).

Using all of these,

\displaystyle P(\mathrm{two\: girls}|\mathrm{girl\:at\:door}) = \frac{(2/3)(1/2)}{(2/3)(1/2) + (1/3)(1/2)} = \frac{2}{3}.

Even though we already knew there was at least one girl, seeing a girl first makes it much more likely that the Iris has two daughters. Whether or not you end up soaked is a different question.

Example 3: Fudge!

Finally, we shall return to the case of Ted and his overconsumption of fudge. Ted claims to have eaten a lethal dose of fudge. Given that he is alive to tell the anecdote, what is the probability that he actually ate the fudge? Here, our data is that Ted is alive, and our hypothesis is that he did eat the fudge. We have

\displaystyle P(\mathrm{fudge}|\mathrm{survive}) = \frac{P(\mathrm{survive}|\mathrm{fudge})P(\mathrm{fudge})}{P(\mathrm{survive})}.

This is a case where our prior, the probability that he would eat a lethal dose of fudge P(\mathrm{fudge}), makes a difference. We know the probability of surviving the fatal dose is P(\mathrm{survive}|\mathrm{fudge}) = 0.5. The evidence, the total probability of surviving P(\mathrm{survive}),  is calculated by considering the two possible sequence of events: either Ted ate the fudge and survived or he didn’t eat the fudge and survived

P(\mathrm{survive}) = P(\mathrm{survive}|\mathrm{fudge})P(\mathrm{fudge}) + P(\mathrm{survive}|\mathrm{no\: fudge})P(\mathrm{no\: fudge}).

We’ll assume if he didn’t eat the fudge he is guaranteed to be alive, P(\mathrm{survive}| \mathrm{no\: fudge}) = 1. Since Ted either ate the fudge or he didn’t P(\mathrm{fudge}) + P(\mathrm{no\: fudge}) = 1. Therefore,

P(\mathrm{survive}) = 0.5 P(\mathrm{fudge}) + [1 - P(\mathrm{fudge})] = 1 - 0.5 P(\mathrm{fudge}).

This gives us a posterior

\displaystyle P(\mathrm{fudge}|\mathrm{survive}) = \frac{0.5 P(\mathrm{fudge})}{1 - 0.5 P(\mathrm{fudge})}.

We just need to decide on a suitable prior.

If we believe Ted could never possibly lie, then he must have eaten that fudge and P(\mathrm{fudge}) = 1. In this case,

\displaystyle P(\mathrm{fudge}|\mathrm{survive}) = \frac{0.5}{1 - 0.5} = 1.

Since we started being absolutely sure, we end up being absolutely sure: nothing could have changed our mind! This is a poor prior: it is too strong, we are being closed-minded. If you are closed-minded you can never learn anything new.

If we don’t know who Ted is, what fudge is, or the ease of consuming a lethal dose, then we might assume an equal prior on eating the fudge and not eating the fudge, P(\mathrm{fudge}) = 0.5. In this case we are in a state of ignorance. Our posterior is

\displaystyle P(\mathrm{fudge}|\mathrm{survive}) = \frac{0.5 \times 0.5}{1 - 0.5 \times 0.5} = \frac{1}{3}.

 Even though we know nothing, we conclude that it’s more probable that the Ted did not eat the fudge. In fact, it’s twice as probable that he didn’t eat the fudge than he did as P(\mathrm{no\: fudge}|\mathrm{survive}) = 1 -P(\mathrm{fudge}|\mathrm{survive}) = 2/3.

In reality, I think that it’s extremely improbable anyone could consume a lethal dose of fudge. I’m fairly certain that your body tries to protect you from such stupidity by expelling the fudge from your system before such a point. However, I will concede that it is not impossible. I want to assign a small probability to P(\mathrm{fudge}). I don’t know if this should be one in a thousand, one in a million or one in a billion, but let’s just say it is some small value p. Then

\displaystyle P(\mathrm{fudge}|\mathrm{survive}) = \frac{0.5 p}{1 - 0.5 p} \approx 0.5 p.

as the denominator is approximately one. Whatever small probability I pick, it is half as probable that Ted ate the fudge.

Mr. Impossible

I would assign a much higher probability to Mr. Impossible being able to eat that much fudge than Ted.

While it might not be too satisfying that we can’t come up with incontrovertible proof that Ted didn’t eat the fudge, we might be able to shut him up by telling him that even someone who knows nothing would think his story is unlikely, and that we will need much stronger evidence before we can overcome our prior.

Homework example: Monty Hall

You now have all the tools necessary to tackle the Monty Hall problem, one of the most famous probability puzzles:

You are on a game show and are given the choice of three doors. Behind one is a car (a Lincoln Continental), but behind the others are goats (which you don’t want). You pick a door. The host, who knows what is behind the doors, opens another door to reveal goat. They then offer you the chance to switch doors. Should you stick with your current door or not? — Monty Hall problem

You should be able to work out the probability of winning the prize by switching and sticking. You can’t guarantee you win, but you can maximise your chances.

Summary

Whenever you encounter new evidence, you should revise how probable you think things are. This is true in science, where we perform experiments to test hypotheses; it is true when trying to solve a mystery using evidence, or trying to avoid getting a goat on a game show. Bayes’ theorem is used to update probabilities. Although Bayes’ theorem itself is quite simple, calculating likelihoods, priors and evidences for use in it can be difficult. I hope to return to all these topics in the future.