N and n: population and sample

Most reporting charts carry a small caption in the bottom-right corner, for example N: 1240, n: 587.¹ This page explains what those two figures mean, what is eligible to be counted in each, and the rules applied at each stage of the pipeline that produces them.

Definitions

N is the population — the total role count. It is the number of roles on the productions included in the chart: every credited engagement, whether or not the person holding it took part in the survey. N answers the question “how many roles could, in principle, have been surveyed?”

n is the sample — the respondent role count. It is the number of those roles whose holder completed the survey. n answers “how many roles do we actually have survey data for?”

Three things to keep in mind:

• Both figures count roles, not people. A person credited in three roles on a production is counted three times, in both N and n.
• Both are exact counts, but n is published with privacy noise. N is derived purely from cast and crew lists, so it carries no information about anyone’s survey answers and can be published unchanged. n reveals survey participation, so — like every published count — a small amount of random noise is added to it before publication, as required by differential privacy. The n on a chart is the true count plus or minus that noise — not an approximation made some other way.
• Completing the survey is what puts a role in n — not answering the specific question a chart shows. Respondents who skipped an individual question generally remain in n (see people who did not complete the survey).

How the figures flow through the system

All counting of raw data happens up front, in a scheduled pre-calculation step. The parts of the system that users interact with never read individual survey answers — they only combine pre-calculated, privacy-protected totals.

1. Pre-calculation — the Difference Engine. A scheduled processing run reads the production, episode, cast/crew, and survey data, computes all the counts, applies differential-privacy noise where required, and spends the privacy budget doing so.
2. Storage — the reporting database. The run writes its results as finished count documents: one document per production (or episode), per reporting table, per reporting period. Each document holds the noised per-answer counts along with that production’s N and n.
3. Assembly — the reporting API. When a user builds a chart, the API selects the right count documents for the chosen productions and period, adds them together, and derives a single N and a single n for the chart. It applies its own rules for role filters and for multi-select questions.
4. Display — the charts. The chart code decides whether N is trustworthy enough to show, converts counts to percentages, computes confidence intervals, and classifies each category into a reliability band.

The stages are described in detail below, after the eligibility rules.

What is eligible to be counted

Productions

A production’s data is included in reporting only when it is approved and delivered — its delivery date must be in the past.² Two refinements:

• Deployments can configure a lock period: a number of days after delivery before the production becomes reportable.
• Where a production has no delivery date recorded, its verification date can be used as a fallback (this behaviour is enabled per deployment).

Episodes

An episode is eligible when it belongs to an approved production and its own delivery date is in the past. Eligible episodes get their own count documents in addition to the production-level ones.

Roles — what N counts

A role is a single credited engagement on a production: one job entry, on one person, naming a recognised role from the role taxonomy. The identity of a role is the combination of the role itself and the character played:³

• The same role listed twice for the same person (a duplicate data entry) counts once.
• The same person playing the same role as two different characters counts twice.
• A job entry with no recognised role attached is not counted at all — it cannot produce survey rows, so it is excluded from every figure.

There is no survey requirement: every role on an eligible production counts toward N.

Respondent roles — what n counts

A role counts toward n when the person holding it has a completed survey linked to their record.⁴ Whether they answered any particular question does not matter at this stage. Roles whose survey link is broken or missing are treated as not having responded.

How each stage calculates

Stage 1: pre-calculation in the Difference Engine

For every eligible production, the engine computes three families of numbers:

The per-answer counts — for each reporting table (a demographic question, a question restricted to a role group such as Crew, or a pair of questions cross-tabulated), the number of respondent roles giving each answer. These are the bars you see on charts. Each count has differential-privacy noise added, and each role’s influence on a single table is bounded. The noised values are stored signed — a small true count can come out slightly negative — because keeping the sign lets the noise cancel out when counts are added together later, rather than biasing every small number upward.⁵

n (the respondent role count) — the number of distinct respondent roles, counted with differential-privacy noise and never allowed below zero.⁶ It is computed once per production (and separately for each eligible episode and for each role group), and then that same figure is stamped onto every table that shares that scope.

N (the total role count) — the number of distinct roles, counted exactly, with no noise.⁷ Because it is based only on cast and crew lists and never touches survey data, publishing it exactly poses no privacy risk. N is computed once per production — the same figure is stamped onto every one of that production’s tables and onto its episode-level documents as well.

Role-group tables (for example Gender identity — Crew) are each measured as their own direct, independently noised count. A parent group’s total is not assembled by adding up the noised counts of its children, which would compound the noise; membership of the group is worked out first, then a single noise draw is applied to the group’s true total.⁸

Stage 2: what is written to the database

Each run writes one count document per production (or episode), per table, per reporting period (year and quarter). A document records:

• which production and, if applicable, which episode it describes;
• which table it belongs to, and whether that question is multi-select;
• the reporting period and a generation version (later runs supersede earlier ones);
• the noised per-answer counts;
• a data-quality block holding that scope’s N and n;
• the privacy budget spent producing it.

Nothing downstream ever recomputes these figures from raw data — the count documents are the only source the API and charts read.

Stage 3: assembling a chart in the reporting API

When a user requests a chart, the API:⁹

1. Resolves the reporting period. With no period specified, it uses the latest available data for each production individually. If a specific period is requested, it uses the most recent data at or before that period. Different productions can resolve to different periods; the chart’s context notes when this happens.
2. Adds up the answer counts across every selected production’s matching documents. The addition keeps the signs of the noisy values, so the total is unbiased — noise cancels on average, though each document contributes its own independent draw and the random spread therefore grows as more productions are included (see charts that combine many productions). Only the final total is floored at zero just before it is returned — a chart never displays a negative count.
3. Derives N by adding up the stored N of every document it matched. When a role or department filter is active, N is instead recounted live from the current cast and crew lists, restricted to the filtered roles (see filtering by role or department).
4. Derives n. For ordinary single-select questions, n is simply the sum of the answer counts being returned — each respondent lands in exactly one category, so the categories add up to the sample. For multi-select questions that logic would count selections rather than people, so n is instead taken from the stored respondent role counts.
5. Never caps n at N. Because n carries privacy noise — and both figures are assembled by adding up separately protected parts — n can legitimately come out above N; the API passes both through unchanged and leaves the handling to the chart (see when n is larger than N).

Stage 4: display rules on the charts

The chart receives the combined counts plus the assembled N and n, and applies the final rules:¹⁰

• Caption. n is always shown. N is shown only when it is a trustworthy population figure: it must exceed n by a safety margin of at least 10 roles or 5% of n, whichever is larger. If it does not clear that margin, N is left off the caption entirely — no fallback number is shown.
• Percentages. For single-select questions, each category is divided by the total of all the categories on the chart — which by construction is the same figure as n. For multi-select questions each category is divided by n directly, so the bars can legitimately total more than 100%.
• Confidence intervals. Each category gets a 95% confidence interval using a hypergeometric score method — a finite-population version of the Wilson score interval (see the confidence-interval discussion). The finite-population correction, which narrows intervals when the sample covers most of the population, is applied only when N cleared the trust margin above. Otherwise the population is treated as effectively unlimited, giving honest, wider intervals.
• Reliability bands. Each category is classified using its confidence-interval width and its raw count — see Data Quality. Categories at or below the small-count threshold K are shown as Insufficient data regardless of interval width (see small categories).

The caption is part of the chart image, so it appears in PNG/SVG exports and prints, but it is not included in CSV or XLSX data exports.

Edge cases

When N is not shown on a chart

N is omitted from the caption whenever it fails to clear n by the safety margin (at least 10 roles, or 5% of n). This is deliberate: n is noisy while N is exact, so an N at or near n usually signals a data artifact rather than a genuine near-census. The common causes:

• differential-privacy noise pushed n up close to, or above, the true role count;
• the chart aggregates respondents across multiple periods, tables, or episode documents, beyond what a point-in-time role count describes;
• no N was recorded for the data in question, which is treated as zero and therefore never clears the margin.

Showing such an N — or worse, using it for the finite-population correction — would collapse confidence intervals toward zero width and falsely present noisy data as certain. Omitting N and skipping the correction gives honest, wider intervals instead.

When n is larger than N

Because n carries privacy noise, and because some requests combine documents in ways that scale n and N differently, the assembled n can exceed the assembled N. This is expected, and nothing clamps it. The consequences are handled at display time: N fails the trust margin, so it is not shown and the finite-population correction is skipped. Percentages are unaffected — they are computed from the answer counts, not from N.

People with more than one role

Both N and n count roles, so a person holding several distinct roles (or the same role as several characters) is counted once per role. This is intentional: reporting is about the composition of a production’s workforce as credited, and one person can occupy several seats in that workforce. There is no cross-role cap on one person’s influence — someone with three roles affects the counts three times.¹¹

People who did not complete the survey

A role whose holder never completed the survey is included in N but not in n — that is the entire difference between the two figures.

A person who completed the survey but skipped an individual question still counts toward n in the stored respondent role count. Skipped questions are counted and then either displayed as another category or merged into another configurable category (for example “no answer” or “prefer not to say”).

Episodic productions

For an episodic production, counting happens at two levels. Each role is counted once at the production level, and once for every eligible episode it worked on (a role not tagged to specific episodes is treated as having worked on all of them). A production’s N includes both its production-level roles and these per-episode engagements, and that single figure is stamped on the production-level document and on every episode-level document. n, by contrast, is measured separately for the production level and for each episode.

The practical consequence: figures for episodic productions are best read as counts of engagements rather than headcounts, and N will often sit well above n even with strong survey participation.

Charts that combine many productions

Every count document carries its own independent draw of privacy noise, so when a chart combines documents — one per production, per table, per reporting period, plus one per eligible episode — it adds up their noise along with their counts. Because the signed values are kept, the draws partly cancel rather than piling up one-for-one: the combined noise grows with the square root of the number of documents added. A chart built from 100 productions carries roughly 10× the noise of a single-production chart, not 100×.¹²

Two practical consequences:

• Absolute noise grows, but relative noise usually shrinks. Widening a selection adds true respondents faster than it adds noise, so percentages on broad selections generally land closer to the real proportions, not further away. The exception is small categories: an answer that is rare everywhere gains little true count from each extra production but still collects a full noise contribution from every one. On a wide selection, a small bar’s displayed count can therefore sit noticeably further from the real count than the same bar on a narrow selection — it can even be pushed to zero, or conjured from a true zero, by accumulated noise alone.
• The confidence intervals do not include this noise. The intervals on a chart measure sampling uncertainty — how well the respondents represent the population. They are computed from the noised counts, but the noise scale itself is not known to the chart code, so the extra spread from combining many documents is not drawn. For small categories on wide selections, true uncertainty is somewhat larger than the interval shown.

Charts that draw on more than one table

Some selections make a chart combine several stored tables in one request. The assembly stage adds N and n up once per matched document, so both figures can read several times higher than the true number of roles. Because they inflate by the same factor, the proportions and confidence intervals remain correct — but the absolute N and n printed in the caption can overstate the underlying role count.¹³ Requests that would inflate n and N by different factors (mixing tables from different role groups without a role filter) are rejected outright rather than answered misleadingly.

Filtering by role or department

When a chart is filtered to a role group (for example Crew or Directors), the stored production-wide N would be the wrong population, so the API recounts N live from the current cast and crew lists, counting only roles in the filtered group and using the same role-identity rules as the engine.¹⁴ n is likewise de-duplicated so each production and role group is counted exactly once, however many tables the chart touches.

The live recount only exists for current data — cast lists have no historical snapshots. For a report on a past period with a role filter active, N falls back to the stored production-wide figure for that period. That fallback N is always at least as large as the role group’s n, so the confidence-interval logic stays safe, but it describes the whole production rather than the filtered group.

”Select all that apply” (multi-select) questions

For multi-select questions the answer counts record selections, and one respondent can make several. Summing the categories would therefore overstate the sample, so n is taken from the stored respondent role count — a distinct-role measurement — instead. Percentages are computed against n, so they can genuinely add to more than 100%. Benchmark comparisons are not shown for multi-select questions, because a benchmark percentage normalised against selections and an organisation percentage normalised against respondents would not be comparable.

Small categories and the K threshold

Neither the engine nor the API suppresses small counts — the privacy protection is the noise itself, not hiding numbers. The only small-count rule is applied at display time: any category where the organisation’s own count is at or below the threshold K (default 5, configurable per deployment) is classified Insufficient data — its bar is not drawn and its tooltip shows no percentage.¹⁵ Benchmark series are never suppressed by this rule, since they describe populations far larger than K.

Slightly negative counts

A category whose true count is very small can come out slightly negative after noise is added. These values are stored as-is (so the noise cancels rather than biases when productions are added together) and floored at zero only once, at the end of assembly. A category that ends at zero renders as an empty bar; very small positive results are described in the tooltip as “1% or less” rather than as a precise figure.

Benchmarks

Benchmark series (population, prior year, industry) carry no N or n of their own — they are delivered purely as percentage distributions, each normalised against its own totals. The N and n in the caption always describe the organisation’s own data.

Footnotes

1. Caption construction and placement: getChartOptions.ts in tep-client/packages/reporting/src/components/helper/ (responseRateText). The same function serves both essential and advanced reports. ↩

2. Production and episode eligibility filters: _project_eligibility_filter in difference-engine/difference-engine/main.py, and the candidate-selection pipeline in src/dp/dp_count_table_generator.py (_resolve_projects_and_population). The lock period is the --project-lock-days option; the verification-date fallback is --handle-verified-at. ↩

3. Role identity and de-duplication — distinct (role, character) per contributor: src/dp/dp_count_table_generator.py (TEP-673, TEP-678). Role-less jobs are filtered out in the same pipeline and in src/extract/raw_role_extractor.py. ↩

4. Respondent resolution, including the handling of missing and dangling survey links: _resolve_survey_answer_set_id and fetch_survey_respondents in difference-engine/difference-engine/src/extract/raw_role_extractor.py. ↩

5. Signed (unclamped) storage of noised answer cells: _validate_count in difference-engine/difference-engine/src/export/dp_role_exporter.py (TEP-628). ↩

6. n as a noised distinct-role count, clamped at zero: dp_count_survey_respondents_by_project and the role-group variant in difference-engine/difference-engine/src/dp/tumult_service.py; computed once per scope and reused across tables via lookups in src/export/dp_role_exporter.py. ↩

7. N computed exactly, outside the privacy mechanism, keyed only by production: the role_count expression in difference-engine/difference-engine/src/dp/dp_count_table_generator.py and _project_population in src/export/dp_role_exporter.py. ↩

8. Direct per-node measurement of role-group tables (TEP-654): map_role_to_group in difference-engine/difference-engine/src/dp/count_helpers.py and transform_by_role_group in src/dp/tumult_service.py. ↩

9. API assembly: MetricsFetcher.ts in tep-api/src/schema/reporting/resolvers/serviceLogic/, with period resolution in src/modules/reporting/temporalUtils.ts, aggregation in aggregationUtils.ts, and the N/n derivation rules in resolveQualityMetrics.ts and sumReturnedCounts.ts. ↩

10. Display rules: resolvePopulationSize and computeConfidenceInterval in tep-client/packages/reporting/src/helpers/confidenceInterval.ts; percentages in transformReportingData.ts; bands in getDataQuality in getChartOptions.ts. ↩

11. The differential-privacy contribution bound is applied per role, not per person: each role is its own protected unit, with its rows per table bounded (MaxRowsPerID in difference-engine/difference-engine/src/dp/tumult_service.py). ↩

12. Each document’s counts receive an independent noise draw at a fixed scale set by the privacy budget (tumult_service.py in difference-engine/difference-engine/src/dp/); the sign-preserving addition is the group stage in tep-api/src/modules/reporting/aggregationUtils.ts. Independent draws add in quadrature, so k documents carry √k times the per-document noise. The chart code deliberately does not model this extra spread (confidenceInterval.ts in tep-client/packages/reporting/src/helpers/ — the noise scale is not exposed to the client), which is also part of why the N trust margin exists. ↩

13. The equal-factor inflation of N and n on multi-document requests is an accepted, documented limitation of the base aggregation: see the group-stage comment in tep-api/src/modules/reporting/aggregationUtils.ts; the differing-factor case is rejected by validation in src/schema/reporting/validation/reportingSchemas.ts. ↩

14. Role-filtered recounts (TEP-627): countRoleInstancesForAnchors in tep-api/src/modules/reporting/roleGroupRoleCount.ts (N, live, current data only) and sumRoleGroupRespondentCount in roleGroupRespondentCount.ts (n, de-duplicated per production and role group). ↩

15. The K threshold: getDataQuality in tep-client/packages/reporting/src/components/helper/getChartOptions.ts, default DEFAULT_K_VALUE = 5, overridable via the NEXT_PUBLIC_K_VALUE environment variable. Setting K to 0 disables the insufficient-data band. ↩