How to Do Content Analysis for Your Dissertation: Codebook, Reliability and PRISMA-Style Reporting

If you are working out how to do content analysis for a health, media or journalism dissertation, these are probably the questions on your mind. Which of my sources count as the content I will code? How do I turn a research question into a codebook my supervisor will trust? Which reliability score should I report, and what is good enough? Do I need a PRISMA flow diagram if I am analysing health news instead of trials? Can I use an AI tool to help with coding without losing marks?
Most examples below come from health communication and journalism. The codebook and reliability steps work for any text, image or video corpus, though. The PRISMA section matters most when your corpus is a set of published studies.
What Is Content Analysis and When Should You Use It?
Content analysis is a systematic way of turning texts, images or recordings into coded categories, so you can describe patterns and draw careful inferences about messages, their producers or their audiences. Early definitions, such as Berelson’s from 1952, stress objective and quantitative description of manifest content. Manifest content is what a text visibly says. Latent content is the meaning underneath, such as tone or framing. Graneheim and Lundman (2004) note that both involve interpretation, only at different depths. Our guide on how to operationalise variables goes into more detail.
Hsieh and Shannon (2005) describe three approaches, and your choice shapes your whole codebook:
- Conventional: categories come straight from the data. It suits topics with little existing theory.
- Directed: you start from a theory or earlier findings and use them to build the initial codes.
- Summative: you count keywords or content first, then interpret the context around them.
Elo and Kyngäs (2008) make a similar split. Inductive analysis fits cases with no earlier studies or fragmented knowledge, while deductive analysis fits cases where the structure comes from previous knowledge. If you are still weighing designs, read about choosing your research methodology first. If you are unsure how to approach this, read our guide to doing a case study dissertation.
Content analysis works well when the material already exists: news coverage, policy documents, campaign materials, social media posts or open-ended survey answers. It is also unobtrusive, so your presence cannot change what people say. It works less well when your question is about lived experience. In that case, semi-structured interviews usually fit better. It also differs from thematic analysis. Content analysis often reports counts, while thematic analysis teaches you not to treat frequency as importance, so check the thematic analysis guide before you decide.
Content Analysis Steps: Seven Stages From Research Question to Write-Up
Competitor guides usually give five or seven steps. Scribbr’s five are: select the content, define units and categories, set coding rules, code the text, and analyse the results. The seven stages below add a pilot and a reliability check, because examiners look for both. Elo and Kyngäs group the process into preparation, organising and reporting, so stages 1 to 4 are preparation, 5 and 6 are organising, and 7 is reporting. Our guide to how to write a theoretical framework walks through it step by step.
- Write a codable research question.
- Define and sample your corpus.
- Choose your unit of analysis.
- Write and freeze your codebook.
- Pilot the codebook and train coders.
- Code the corpus and test intercoder reliability.
- Analyse the results and report them.
Step 1: Turn Your Question Into Something You Can Code
A codable question names the content, the thing you will measure and, ideally, a comparison. For example: how do UK online news reports describe the risks and benefits of a vaccine, and does the tone differ between national and regional outlets? That example is illustrative, not taken from a study. If your wording is still loose, practise how to write a focused research question before building anything else. If you have not settled a direction, browse journalism dissertation topics for starting points.
Step 2: Define and Sample Your Corpus
Scribbr’s first step is to select the content you will analyse. Decide the medium, the date range and your inclusion criteria. If few texts qualify, analyse them all. If thousands qualify, sample them. Existing material like this is often treated as secondary data, but your ethics office has the final say. Write the inclusion and exclusion criteria down before you search, and record the database, search terms and search date so someone else could repeat your selection.
Step 3: Choose Your Unit of Analysis
Columbia’s Mailman School of Public Health lists the levels you can code at: word, word sense, phrase, sentence or theme. Decide early whether you code whole articles, paragraphs, sentences or individual claims. Changing the unit halfway forces you to recode everything.
How to Write a Codebook for Content Analysis (With an Example)
A codebook is the written manual that lets another person code your material the way you do. Scribbr points out that recording coding rules makes your method more transparent even when you are the only coder. Decide how much implication you allow, for instance whether a word that only hints at danger counts as the concept. A usable codebook contains:
- a variable name and a one-sentence definition;
- categories that do not overlap and cover every case, including “other” and “cannot tell”;
- coding rules, including what counts as explicit and what counts as implied;
- decision rules for hard cases, with short examples and non-examples;
- the level of measurement, which later decides your reliability statistic.
Getting the definitions right is the same skill as operationalising your variables, so read that guide if the wording feels slippery. Here is an illustrative extract for a health news study. It is invented for teaching and not copied from any published codebook.
| Variable | Categories | Rule for hard cases |
|---|---|---|
| Source of main health claim | 1 Named clinician or researcher; 2 Public health body or regulator; 3 Patient or member of the public; 4 No source given | Code the first source attributed to the main claim in the headline or opening paragraphs. If two sources appear together, code the one the article treats as more authoritative. |
| Risk framing | 1 Mostly benefits; 2 Mostly risks; 3 Balanced; 4 Neither mentioned | Code “balanced” only when both appear in similar amounts or the article weighs them explicitly. |
| Names a retracted study | 0 No; 1 Yes | Code 1 only if the article says the study was retracted or withdrawn. |
If you work inductively, you build and refine categories while reading the data. If you work deductively, you derive them from theory or earlier research. Either way, freeze a version before the main coding, keep a dated copy, and report any later changes. Software can speed up counting, but a count can be accurate while the meaning is wrong, so check context.
Pilot Testing and Coder Training: Fix the Codebook Before It Costs You Marks
Test the codebook on texts that are not in your main sample. The SAGE Encyclopedia of Communication Research Methods summarises the guidance of Lombard, Snyder-Duch and Bracken: use at least 30 randomly selected units for a formal pilot, with coders working independently. For the main reliability check, it suggests no fewer than 50 units or 10 percent of the full sample, and rarely more than 300. These are guidelines, so confirm what your module handbook or supervisor expects.
Training means coding a practice set, comparing decisions and then improving the definitions, instead of simply overruling one coder. If you are working alone, ask your supervisor whether a second coder can double-code a subsample. Another option is to recode a subsample yourself after a gap. Columbia calls this stability: the same coder classifying the same data the same way over time. That tests you against yourself, which is weaker than independent agreement, so describe it honestly.
Intercoder Reliability: Krippendorff’s Alpha, Cohen’s Kappa and Percent Agreement
Intercoder reliability shows whether independent coders applying the same codebook reach the same decisions. Report a chance-corrected coefficient for each variable, not one overall number. A 2010 systematic review in Health Communication looked at 441 articles using quantitative content analysis of health messages in mass media, from 1985 to 2005. It found that communication journals were more likely than other journals to report intercoder reliability. Reporting it per variable is good practice, and examiners tend to expect it.

Which Reliability Statistic Should You Choose?
| Statistic | Best used for | Watch out for |
|---|---|---|
| Percent agreement | A quick pilot diagnostic | Ignores agreement that happens by chance (McHugh, 2012) |
| Cohen’s kappa | Two coders, categorical codes | Corrects for chance, but the usual interpretation bands may be too lenient for health research (McHugh, 2012) |
| Fleiss’ kappa | Three or more coders | An adaptation of Cohen’s kappa (McHugh, 2012) |
| Scott’s pi | Nominal categories only | Not suitable for ordinal or interval codes |
| Krippendorff’s alpha | Any number of coders, any level of measurement, missing data | Needs software; Hayes and Krippendorff (2007) propose it as the standard |
Do not use correlation or Cronbach’s alpha as a stand-in. The SAGE entry explains why: if one coder always scores one point higher than another on a five-point scale, the two are perfectly correlated, yet they never agree.
What Score Counts as Reliable?
Krippendorff (2018) proposes alpha of 0.80 or above as the usual requirement. He treats values between 0.667 and 0.80 as good enough only for tentative conclusions, and treats lower values as unreliable. He also argues that the minimum should reflect the cost of drawing a wrong conclusion. Columbia’s older rule of thumb says 80 percent is an acceptable margin, but that describes raw agreement, which is the weakest measure in the table above.
A Worked Example: High Agreement, Low Kappa
These numbers are invented for illustration. Two coders judge 100 articles on whether each names a retracted study. Both say yes for 2 articles and both say no for 90, and they disagree on 8. Raw agreement is 92 percent. But each coder says yes only 6 percent of the time, so agreement expected by chance is already about 88.7 percent. Cohen’s kappa is therefore (0.92 − 0.887) ÷ (1 − 0.887), which is roughly 0.29. The score looks poor because the category is rare. If this happens to you, explain it openly, and ask your supervisor whether a different reliability sample or a merged category is acceptable.
Free Tools and What to Do If the Score Is Low
Kimberly Neuendorf’s online resources for The Content Analysis Guidebook list ReCal, which computes coefficients for nominal data, PRAM, and the KALPHA macro by Hayes and Krippendorff for SPSS and SAS. Check which tool your university licenses. If a variable falls short, tighten its definition, retrain and test again, or merge or drop the variable and say so. Neuendorf’s guidebook covers how to treat variables that do not reach acceptable levels. For a wider view, see the checks in the guide to reliability and validity.
Reliability Is Not Validity: Making Your Content Analysis Trustworthy
High intercoder reliability does not guarantee validity. Two coders can agree on a poor category. Columbia’s page also sets out validity criteria, so ask whether your categories capture the concept and whether your conclusions follow from the data. Graneheim and Lundman (2004) frame the same issue as credibility, dependability and transferability. If you are unsure how to approach this, read our guide to checking reliability and validity.
- Note your own assumptions before coding. Paperpile calls this bracketing.
- Keep an audit trail: codebook versions, a decision log and the pilot results.
- Show short extracts from the texts so a reader can judge your categories.
- Limit your claims. What a text says does not prove how audiences respond.
PRISMA-Style Steps for a Health Content Analysis: Is PRISMA 2020 Still Current?
PRISMA 2020 is still the current main PRISMA guideline. The official PRISMA website, checked in November 2026, lists it with a checklist, an expanded checklist, flow diagram templates, the statement paper and an explanation and elaboration paper. The statement (Page et al., 2021) contains a 27-item checklist and a separate 12-item abstract checklist. Some websites claim a “PRISMA 2026”. The official site does not list one. Position correct as of November 2026; check the PRISMA website for the current status.
PRISMA is a guideline for reporting systematic reviews, and the official site describes the main version as primarily for reviews of the effects of interventions. A newspaper or social media corpus is not a systematic review, so PRISMA is not a formal requirement for it. What you can borrow is the habit of showing exactly how your material was found and filtered. That is my recommendation, not a rule, so confirm it with your supervisor.
If Your Corpus Is Published Studies
Here PRISMA 2020 applies. Health content analysis has a track record of this design. Suarez-Lledo and Alvarez-Galvez (2021) followed PRISMA for a review of 69 studies on health misinformation on social media, in which content or text analysis was one of the main methods. Catalán-Matamoros and Peñafiel-Saiz (2019) reviewed vaccine coverage studies with a registered PROSPERO protocol. Follow the full workflow in the systematic review dissertation guide.
If your aim is to map a topic instead of answering a narrow question, a scoping review may fit better. PRISMA-ScR is the extension for that design (Tricco et al., 2018).
If Your Corpus Is News, Posts or Documents
Keep a selection log and draw a flow diagram in the same spirit. Record:
- the databases or platforms searched, the search strings and the search date;
- the number of items identified and the number of duplicates removed;
- the number screened and excluded, with reasons;
- the final corpus size and how you sampled from it, if you did.
PRISMA 2020 also asks authors to report any automation used in selection, so say so if a tool helped you screen. The diagram goes straight into your methodology chapter.
Ethics, Data Protection and Social Media Data
Content that is easy to reach is not automatically fair game. The British Psychological Society’s guidelines for internet-mediated research (2021) discuss the public and private distinction, confidentiality and consent online. Published guidance at several UK universities, including Sussex, Bath and Sheffield, indicates that projects using social media data usually need institutional ethics review. Rules differ between universities, and news articles and published reports usually carry less risk, but ask your ethics office before you collect anything. Our guide to getting ethics approval for a dissertation explains the usual route.
Can You Use ChatGPT or Another AI Tool to Help With Coding?
Check your university’s AI policy and ask your supervisor first, because rules differ and I cannot tell what yours says. The research is mixed and mostly preprints, so treat it as early evidence:
- Reiss (2023) found that ChatGPT’s zero-shot annotation reliability can fall short. He advises validating outputs against human-coded data and does not recommend unsupervised use.
- Chew and colleagues (2023) propose a framework for LLM-assisted deductive coding, in which agreement between the model and human coders is checked with the same reliability statistics.
- Goyanes (2026) found that ChatGPT miscomputes intercoder reliability statistics and tends to overestimate agreement, so calculate your coefficients in statistics software.
If your university allows it, keep human coding for a subsample, record the model name, version, prompt and date, and report agreement between human and model. Do not paste personal data into public tools.
How to Write Up Content Analysis in Your Dissertation
Stage 7 is analysis and reporting. Because coded data are usually categories, your test depends on your variables, and the statistical test selector can narrow the options. In your methodology chapter, cover:
- your approach (conventional, directed or summative, or conceptual or relational) and why;
- the corpus, sampling and your selection log or flow diagram;
- the unit of analysis and the codebook, with the full version in an appendix;
- coder training, the pilot, and the reliability coefficient for each variable;
- validity steps, ethics approval and limitations.
The methodology chapter guide shows how to structure those points. In your results chapter, give counts or percentages by category and add short illustrative extracts, as the results chapter guide explains. Cite your method sources in Harvard style, and check your module handbook for word limits and appendix rules.
Common Content Analysis Mistakes to Avoid
- Reporting only percent agreement, or no reliability at all. Lombard, Snyder-Duch and Bracken (2002) reviewed 200 content analyses published from 1994 to 1998, and 69 percent reported reliability. Examiners expect you to do better.
- Skipping the pilot, then finding categories overlap after coding 300 articles.
- Treating frequency as importance in a qualitative study.
- Trusting keyword counts without reading the context.
- Claiming audience effects from content alone.
- Changing the codebook mid-coding without recording the change.
Content Analysis FAQs
What is an example of content analysis?
A health example would code how news stories describe a treatment’s risks and benefits, such as the vaccine question in step 1. The method is the same: define units, code against written rules, then analyse the pattern.
How many coders do I need?
Two is the usual minimum for an intercoder check. Krippendorff’s alpha can handle more, and it copes with missing data. If you are the only coder, discuss a second-coder subsample or a recoding check with your supervisor, and describe the limits.
Is content analysis qualitative or quantitative?
It can be either, or both. Quantitative designs count predefined categories, while qualitative designs interpret meaning and often still report some counts. State which one you chose and why.
Next Steps for Your Content Analysis
Coding takes longer than most students expect, so build time for the pilot, recoding and the reliability check into your dissertation timeline planner. If you would like a second pair of eyes on your codebook or methodology wording, dissertation methodology support is there for reference and learning. Use any guidance according to your institution’s policy.
Key Sources for Your Harvard Reference List
- Berelson, B. (1952) Content analysis in communication research. New York: Free Press.
- British Psychological Society (2021) Ethics guidelines for internet-mediated research. Leicester: BPS.
- Catalán-Matamoros, D. and Peñafiel-Saiz, C. (2019) ‘How is communication of vaccines in traditional media: a systematic review’, Perspectives in Public Health, 139(1), pp. 34-43.
- Chew, R. et al. (2023) ‘LLM-assisted content analysis: using large language models to support deductive coding’, arXiv:2306.14924 (preprint).
- Elo, S. and Kyngäs, H. (2008) ‘The qualitative content analysis process’, Journal of Advanced Nursing, 62(1), pp. 107-115.
- Goyanes, M. (2026) [article title to be confirmed], Infonomy, 4(2), e26006.
- Graneheim, U.H. and Lundman, B. (2004) ‘Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness’, Nurse Education Today, 24(2), pp. 105-112.
- Hayes, A.F. and Krippendorff, K. (2007) ‘Answering the call for a standard reliability measure for coding data’, Communication Methods and Measures, 1(1), pp. 77-89.
- Hsieh, H.-F. and Shannon, S.E. (2005) ‘Three approaches to qualitative content analysis’, Qualitative Health Research, 15(9), pp. 1277-1288.
- Krippendorff, K. (2018) Content analysis: an introduction to its methodology. 4th edn. Thousand Oaks, CA: Sage.
- Lombard, M., Snyder-Duch, J. and Bracken, C.C. (2002) ‘Content analysis in mass communication: assessment and reporting of intercoder reliability’, Human Communication Research, 28(4), pp. 587-604.
- McHugh, M.L. (2012) ‘Interrater reliability: the kappa statistic’, Biochemia Medica, 22(3), pp. 276-282.
- Neuendorf, K.A. (2017) The content analysis guidebook. 2nd edn. Los Angeles: SAGE.
- Page, M.J. et al. (2021) ‘The PRISMA 2020 statement: an updated guideline for reporting systematic reviews’, BMJ, 372, n71.
- Reiss, M.V. (2023) ‘Testing the reliability of ChatGPT for text annotation and classification: a cautionary remark’, arXiv:2304.11085 (preprint).
- Suarez-Lledo, V. and Alvarez-Galvez, J. (2021) ‘Prevalence of health misinformation on social media: systematic review’, Journal of Medical Internet Research, 23(1), e17187.
- Tricco, A.C. et al. (2018) ‘PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation’, Annals of Internal Medicine, 169(7), pp. 467-473.