Assessment Centre Design: A Practical Guide

Assessment centre design is the work of deciding what a centre will measure, which exercises will draw that behaviour out, who will observe it and how their evidence becomes a decision. This guide follows that sequence from job analysis to the exercise matrix, assessor training, the wash-up, piloting, validation, virtual delivery and the faults a design review should catch.

What does good assessment centre design involve?

Good assessment centre design links every score back to a documented job requirement through a chain of deliberate decisions. The International Taskforce on Assessment Center Guidelines states that all assessment centre programmes must contain 10 essential elements, from systematic job analysis through to standardised administration [Source: International Taskforce on Assessment Center Guidelines 2015].

Those elements map onto the design stages most practitioners follow.

Where should the design process start?

It starts with an analysis of the target role, not with a favourite exercise. The Taskforce requires behavioural constructs to be derived “via a rigorous and systematic process (e.g., job analysis, competency modeling)” involving subject matter experts who know the job [Source: International Taskforce on Assessment Center Guidelines 2015]. South Africa’s fifth-edition best practice guidelines agree: “The starting point of an AC is an analysis of the job and/or managerial context” [Source: Meiring and Buckett 2016].

The output is a short list of competencies, each written as observable behaviour. “Strategic thinking” is a label. “Identifies longer-term risks in the data and adjusts priorities to address them” is something an assessor can see and record.

How do you build an exercise-by-competency matrix?

List the competencies down one side and the exercises across the top, then mark which exercise measures which competency. The Taskforce says this matrix “must be constructed” and that each construct should be measured in multiple ways [Source: International Taskforce on Assessment Center Guidelines 2015]. The South African guidelines recommend that “ideally between four and six competencies should be measured for each behavioural simulation exercise” [Source: Meiring and Buckett 2016].

An illustrative matrix for a middle-management centre:

Read it both ways. Every row needs at least two ticks, so no competency rests on one observation. Every column must stay within what an assessor can actually observe.

Which exercises should an assessment centre include?

Choose exercises that recreate situations the role really faces, at a fidelity that suits the purpose. The South African guidelines state that “a minimum of two behavioural simulation exercises are required” [Source: Meiring and Buckett 2016].

The Taskforce draws a firm line on what counts as a simulation: the participant must produce a constructed response. Multiple-choice tests and situational judgement tests, however valid, “would not be classified as a behavioral simulation exercise” [Source: International Taskforce on Assessment Center Guidelines 2015]. It also suggests higher fidelity when diagnosing experienced managers and executives, role-player cues written down in advance, and steps to ensure exercise content “does not unfairly favor certain assessees”.

Psychometrics and interviews add evidence alongside simulations; the guide to psychometric assessment in hiring covers that layer.

How many assessors do you need, and how should they be trained?

Use multiple assessors for every participant, keep each assessor’s load low, and train them to a demonstrated standard. The Taskforce notes that one day of training may suffice for a well-structured centre, but inexperienced assessors may need considerably more, for example “2 days of assessor training for each day of assessment center exercises” [Source: International Taskforce on Assessment Center Guidelines 2015].

The guidelines set no fixed ratio. They ask designers to minimise the participant-to-assessor ratio, seek diversity among assessors, and keep a participant’s current supervisor out of any assessment used for selection or promotion.

Training covers observing, recording, classifying and evaluating behaviour, common rating errors, and separating behaviour from inference. The CIPD adds that selectors “must be trained to observe, actively listen, record, classify and rate behaviour” [Source: CIPD].

How should assessors run the wash-up?

The wash-up, or integration meeting, should pool documented evidence only, after each assessor has finished their own ratings. The Taskforce requires a record of observations “before the integration discussion or before statistical integration takes place”, and lists four possible outputs: exercise-specific dimension scores, exercise scores, across-exercise dimension scores or an overall assessment rating [Source: International Taskforce on Assessment Center Guidelines 2015].

Where ratings diverge, the panel returns to recorded behaviour; a participant’s reputation outside the centre has no place.

The Taskforce treats a “split” rating, strong in one exercise and weak in another, as potentially meaningful rather than noise. In a development centre, a split rating can be a useful starting point for feedback and for an individual development plan.

How do you pilot and validate an assessment centre design?

Pilot every exercise with people similar to the target group, then gather evidence that the scores mean what the design claims. The South African guidelines describe pretesting that “may entail conducting a pilot AC with participants similar to the target participant group” and set a minimum of content, face and construct validity [Source: Meiring and Buckett 2016].

Criterion validity needs a realistic expectation. In a 2022 re-analysis of selection research, Paul Sackett and colleagues wrote: “The assessment center validity estimate is reduced by .08, from .37 to .29” [Source: Sackett, Zhang, Berry and Lievens 2022]. That figure is an average across many centres and studies, so it says nothing about any single centre; local validation evidence is what shows how well a particular assessment centre design predicts performance.

Fairness belongs in validation too: monitor outcomes by group, review content for cultural bias, and document every accommodation.

Can a virtual assessment centre still meet the standard?

Yes, provided participants still display overt behaviour and trained assessors still observe it. Otherwise, the Taskforce says, the programme “could no longer be referred to as an assessment center”. It points to International Test Commission guidance on four delivery modes (open access, controlled, supervised and managed), noting selection and promotion centres “require the most control and security” [Source: International Taskforce on Assessment Center Guidelines 2015].

The South African guidelines agree that the design “can be extended to virtual applications”, but a technology-enabled centre “must still comply with the essential elements” [Source: Meiring and Buckett 2016]. Plan for identity checks, secure materials, cross-border data protection, and participants whose computer literacy is not part of the job.

What design faults should a pre-launch review catch?

Most faults are visible on paper before anyone is assessed.

For how a centre runs on the day and how to brief participants, see the primer on assessment and development centres.

How does Phi Management support centre design and assessor training?

Phi Management runs a 2 to 3 day, in-person Assessment and Development Centre Design certification for HR professionals and talent managers. It covers building a competency model, the ORCE model of observation, recording, classification and evaluation, creating centre matrices, and implementation steps including calibration, structured feedback and reports for decision-making.

The two-day Assessor Skills Certification gives assessors practice with role plays, case studies and competency-based interviews, plus calibration and reporting. Both sit within Phi’s certification programmes, alongside competency-based interview certification.

Phi also delivers assessment and development centres using six instruments, including role play, an e-tray and psychometrics, with individual and group reports and one-to-one feedback. Read more about Phi Management.

Frequently asked questions

How long does it take to design an assessment centre? There is no fixed timetable, because the work depends on how much job analysis already exists, whether exercises are built from scratch or adapted, and how many assessors need training. The steps most often squeezed are the pilot and assessor calibration, and both are hard to recover later. Plan backwards from the first live centre date, allowing time to revise exercises after the pilot and to retrain assessors on any changes.

Should organisations buy off-the-shelf exercises or build bespoke ones? Either can work if the exercises fit the job analysis and the matrix. Off-the-shelf exercises usually come with tested materials and scoring guides, which saves time. Bespoke exercises can mirror the organisation’s real situations more closely, which often improves fidelity and participant acceptance. The Taskforce allows designers to rely on past research or existing exercises only when they can show the jobs are comparable. Whichever route is chosen, every exercise should still be pretested with people similar to the participants and mapped to the competencies it is meant to measure.

Can line managers act as assessors? Yes, and trained managers often make credible assessors because they know the work. The main condition is training to a demonstrated standard before they rate real participants, including practice in separating observed behaviour from inference. The Taskforce adds one firm limit: a participant’s current supervisor should not assess a direct report when the results will be used for selection or promotion. Mixing managers with occupational psychologists or HR specialists on a panel also helps, because the guidelines encourage diversity in both background and experience among assessors.

The bottom line

Assessment centre design is a chain: job analysis defines the competencies, the matrix links them to exercises, trained assessors record behaviour, and a disciplined wash-up turns evidence into a decision. Each link can be checked against published guidance before the first participant arrives. To build that capability in an HR and talent team, or to discuss a centre for your organisation, talk to Phi Management.