Getting requirements right the first time Part 2

1 Getting requirements right the first time: using job analysis (Part 2)

1.1 Introduction

In Part 1 of this article, I outlined how the requirements methods used by developers are flawed for two reasons. The first is that they do not have the requisite skills to analyse and understand human performance and the second is that the methods used are insufficient and often lead to poor requirements. Critically, the way things are done leads to the fact that requirements continually change, making development very hard.

In Part 2, I’m going to cover the job analysis method from industrial and organisational psychology used by us to research, analyse and redesign jobs to improve efficiency, effectiveness and satisfaction. This is a requirements gathering technique to ensure that we have accurately mapped people’s jobs and that it is known to be the right way for people to perform their job.

It is critical to get this right so we know what people’s jobs should be like and we can then design the user interface to support their jobs. Therefore, if people know how to do their job, then they should know how to use the application.

1.2 About job analysis

1.2.1 What is job analysis

Traditional job analysis refers to the systematic process of collecting information on the important aspects of jobs in order to develop criteria necessary for all Human Resource Management (HRM) functions.

This information usually includes the work activities (or tasks) carried out, the Key Performance Indicators which set out the necessary standard of performance expected when performing these tasks, as well as the Knowledge, Skills, Abilities, and Other attributes (KSAOs), required by a person in order to successfully perform the job. It can also refer to the tools and equipment used while carrying out the tasks within the job.

The final product of the job analysis is a Job Description that can be used for the purposes of recruitment/selection, remuneration/job classification, performance appraisal, career development/succession planning, or training and development. In our case, we are most interested in how the job analysis influences user interface decisions.

1.2.2 Using job analysis to manage human resources and performance

Job analysis is a technique used by industrial / Organisational (I/O) psychologists to develop business focussed, accurate job descriptions to:

Select candidates for vacant jobs,
Evaluate people’s performance,
Remunerate people based on performance in the job,
Establish training needs,
Develop succession plans,
Exit people who are not performing by providing objective job performance data.

You can see from the activities that the job descriptions are used to make important business and human resource decisions. Depending on your working context, the job description needs to be particularly accurate and well constructed to avoid litigation from events like unfair dismissal. The job descriptions are usually done by human resources where the people in that department have specialist skills in creating job descriptions.

I made the point earlier and in Part 1, that the same jobs, across organisations, are mostly the same. Now I extend that point by stating that most jobs do not change significantly over time. The jobs in the sales, marketing, accounting, customer service and manufacturing departments do not change much over time. Although the specific product mix may change, the basic job activities do not.

More importantly, the key performance indicators that are used to measure how well people are performing are even less likely to change over time. Take a sales job for example - can you ever think of a time where a sales target was not part of the KPIs for that type of job? Or in accounting — an accurate and up-to-date set of books is a primary measure for that job.

1.2.3 Using job analysis to define user requirements for an application

While the primary purpose of a job description is the effective management of human resources, it is also the primary source of information of what people should be doing in their job.

If you accept that the applications people use are supposed to help them do their job, then it follows that the application’s functionality and behaviour should directly align with the associated job description.

Analysing people’s tasks helps us understand the things people need to do, the words they use, and how they go about doing them.This helps in designing interface elements including:

Menus (number of, and names),
Dialog boxes / window panels,
Sequences of dialog boxes, panels, screens (i.e. work flow)
Required information on any given screen to make a decision
Differentiating between activities that the application is better suited to do (e.g. calculations), and activities that people are better able to do (complex, ambiguous decision making), and ensuring the application is supporting people in the right way.

1.2.4 The structure of a job description

In general, the job description should have two components to it:

The Job: Key result areas, key tasks and key performance indicators,
The Competencies: Knowledge, skills, abilities and other attributes.

1.2.4.1 The Job

Key result areas organise the tasks into logical groups. For example, the KRAs of a manager might comprise: staff management, client management, general management, and administration.

KRAs signify the most important areas of the job — broad areas of responsibility, the major sphere of work activity, functional area, or area of accountability,
KRAs describe the outcomes or results — why the job exists,
KRAs encompass the key tasks and performance indicators for the job (that have a common purpose),
There are usually 4 — 8 KRAs per job, with most jobs having a minimum of 3, and generally 5 — 7,
KRAs for a job are consistent and stable over time unless the job is redefined.

Key tasks are the set of activities (usually sequential) within a KRA that lead to the achievement of that KRA. KTs:

Can be performed over a short period of time,
Have an identifiable beginning and end (cues),
Can be performed independent of other work,
Consist of two or more steps,
Can be observed and measured,
Involve a person’s interaction with people, machines, or media,
Result in a product, service or decision (not necessarily tangible).

Key performance indicators are critical to measure the outcomes of the job, however performed, and accounting for individual differences in style and approach. They are measures of effectiveness and efficiency.

They specify what to measure to determine a person’s success in meeting requirements of the Key Result Areas (KRAs). Many jobs and people can share the same KPIs, however the Performance Standard can change across people and jobs, reflecting a unique (or shared) set of specific accountabilities. They usually fall into one of four categories:

Quality,
Quantity,
Cost,
Timeliness.

1.2.4.2 The Competencies

Knowledge refers to a body of information that a person needs in order to perform a particular job. It can include the completion of a university or TAFE qualification, knowledge of how to perform particular job tasks (i.e., procedural knowledge), or knowledge of rules and regulations.

Skills refer to the observable behaviours a person needs to be able to do in order to carry out work tasks. Skills are usually gained through previous job experience or vocational training.

Abilities refer to the physical or mental capabilities, or aptitudes, required by the person in the job. Abilities are an individual's potential to develop the necessary skills for the job, and most skills require usually one or more associated or underlying abilities.

Other Attributes include values, interests, attitudes, or personal style needed by the person in order to cope with the conditions of the job.It may also include the tools and equipment used while performing the job, or special certifications that the worker should have.

(McIntyre, Bucklan, & Scott, 1995; Spector, 1996)

1.3 The Job analysis method

In this section I’ll provide a high level summary of the job analysis method as a way of capturing user requirements.

Job and competency analysis relies on structured workshops with selected job incumbents and other stakeholders, all of whom should possess a range of experiences in the job under analysis. Ideally, however, it is best to involve ‘expert’ and ‘high performing’ people in the workshops to ensure the best practice is being documented.

What we do not want to do is document and implement they way low performers do their job. This will have the effect of lowering the overall standard of performance, and provide little scope to actual achieve any real performance improvements as a result of designing appropriate technology. Our role is to help people do their jobs better, not at a lower standard.

The workshops are led by a skilled facilitator and focus on identifying and analysing the key result areas, key tasks, performance measures, competencies and workplace behaviours for each job role. The facilitator guides group discussion and gains information from the participants on the tasks and competencies required of the job.

Competency analysis draws upon established human resources techniques of strategic planning, occupational, job and task analysis, skills auditing, psychological assessment, performance appraisal and training needs analysis.

The result of this approach is to provide a cohesive and integrated human resources management model. This enables organisations to improve the way they manager their people to improve organisational performance.

Once the job data is captured, it requires further analysis to compare it to best practice, identify bottlenecks in processes. This ensures that we design an interface that reflects efficient and effective processes. It cannot be assumed that because the organisation performs activities in a certain way, that this is the best way. There may be objective performance data that clearly differentiates high and low performers, however, we must attempt to extract even more performance from people.

1.3.1 Define the organisational context

The most important aspect of defining the organisational context is defining the organisation’s major activities and key performance indicators. This ensures that when determining what people should be doing it directly aligns with business requirements.

The most effective tool to use is the Kaplan and Norton’s Balanced Scorecard which defines KPIs in the areas of Customers, Learning and Staff, Business process and Financials.

We use this context to continuously ask ‘how does that KRA and task help achieve this specific business KRA and KPI?’ It also ensures that when defining a job’s KPIs, there is a clear line of sight to the business KPI. For example, if you have defined a job’s KPI to be sales, and given it a target, and then you add the targets from all the people who hold that job and find that it is short, you have a real problem.

1.3.2 Collect job data

The two main data collection methods are workshops and on-job observations. Although in Part 1, I pointed out that collecting data from people in workshops doesn’t work well, there are two differences here. Firstly, the job data is being collected by an expert in job analysis and, secondly, we also use on-job observation to capture information while people are actually performing the job.

When selecting job experts to participate, you should include job incumbents, subordinates, managers, trainers, and customers (if possible). In general, the manager should not be present, as s/he can unduly influence people’s responses. The manager can review the resulting job profile after the event to provide input and critique.

A general guideline is to plan for three to five job incumbents for a job with few employees working, and at least 10-15 job incumbent for a job with many employees (in this case it would be more appropriate to have ‘group’ interviews).

Needless to say, do not include people who have a political interest in the swaying the results in some way. Further, it is important during the onsite observations to include people who are not expert at the job so you can see the differences in knowledge, skills and abilities between high and low performers. We need to confirm what constitutes high and low performance to ensure we do not mistakenly document low performance data.

1.3.2.1 Conducting on-job observation

This method involves observing employees performing their job activities, and then recording these activities on a standardised form. The observers generally do not speak to the employees or interfere in any way with them performing their job. They will often ask questions afterwards to check their understanding of what they saw and determine why a task is performed in that way and not another.

1.3.2.2 Conducting workshops

The facilitator does not need to be an expert in the domain, but a broad overview of job types is recommended. This knowledge can be used to ensure core components that do not change significantly between jobs is covered (e.g. people management / project management / financial management). This knowledge is also useful to ensure that important components of a job are not missed out, and to recognise the level of importance of specific job components with a job.

The facilitator should have good interpersonal skills in terms of managing groups and ensuring all people contribute to the discussion.

1. Define the job

The mission statement for the job should be defined.This should consist of a brief statement (5-15 words) that describes the primary function and deliverable of the job.This will be refined as the job is documented throughout the workshop.

2. Identify the key result areas (KRAs)

From the on job observation and interviews, you have a good idea of the kinds of activities (tasks) people perform. This will help you guide the group towards the four to eight major areas of responsibility in the job. These KRAs organise the tasks into logical groups.

Generally, any KRA that cannot be divided into six or more specific tasks should merge with another. There will, of course, be exceptions. Similarly, if there are too many tasks in one KRA, it should be split.

The KRA should be expressed using four or less words to describe the outputs or outcomes.

Do not get bogged down in review at this stage. The KRAs can change as the analysis process continues and the panel should be able to make the changes later.

3. Identify the key tasks (KTs)

A task statement should begin with an action verb (e.g., Analyse), followed by the object of the verb (e.g., data), followed by the purpose (e.g., in order to answer the research question), and any additional information such as equipment or tools used, the location, etc., (e.g., using statistical packages and a computer).

For example: ‘Analyse data in order to answer the research question, using statistical packages and a computer.’

The facilitator will often help with the wording to ensure it complies with the required grammatical structure. S/he will also ensure that as much data is captured as quickly as possible and ensures the group does not become bogged down in the detail.

The tasks should generally be sequenced within the KRA and the most common way is time. That is, perform the first task, then the second and so on, until the KRA is achieved. If there is no time-based flow, then sequence them by the most important ones first.

4. Identify the key performance indicators (KPIs)

Once KRAs have been clearly defined, Key Performance Indicators can then be identified. This process involves going back to the Key Performance Indicators listed by the job experts within the interviews, and matching these to their corresponding KRAs.

The KTs can often be used to generate the KPIs, however, care must be taken to not tie KPIs too closely with KTs, since there are many ways to perform a job. The KPI is critical to measure the outcomes of the job, however performed, and accounting for individual differences in style and approach.

When establishing current or developing new performance indicators, the facilitator needs to encourage participants to come up with `hard’ measures - indicators that fall into one of the four categories — rather than `soft’ measures, such as feedback. In some instances, feedback is the only acceptable measure or indicator of a person’s performance. It should only be used as the indicator in such instances.

Performance indicators should be real, i.e. if an indicator is specified then it should in fact be used on-the-job. (It is useful to remember here that this information should feed straight into a performance review system - so the performance indicators need to represent the performance objectives set for the job being analysed).

The KPIs must be:

Measurable
Time bound,
Within influence,
Realistic,
Understandable.

Aim for 2 — 4 KPIs per KRA. If you have too many things to measure, you spend more time measuring than analysing and doing.

5. Identify the knowledge, skills, abilities and other attributes (KSAOs)

Competencies are the knowledge, skills, abilities and other attributes required to perform the Key tasks. Identifying competencies starts with defining the behaviours exhibited by high performers.They usually begin with:

ability to...
understanding/knowledge of...
demonstrated appreciation/desire to...
willingness to...

Competencies will be hard for people to articulate because they are usually automated. Therefore, the facilitator will use his/her skill in reviewing the task statements and deciding what KSAOs are likely to be important.

For each competency, a four or five point scale must be developed where the first point describes ‘beginner’ levels and the fourth or fifth point describes ‘expert’ levels.

Competencies are often generally across jobs where some jobs require lower levels of a certain KSAO and other jobs require higher levels.

1.3.3 Analyse the job data

Once the job data is captured, it requires further analysis to compare it to best practice, identify bottlenecks in processes, and seek agreement from other job incumbents. This ensures that when we design a corresponding user interface it reflects efficient and effective processes. It cannot be assumed that because the organisation performs activities in a certain way, that this is the best way.

There are several methods to use when analysing the data:

Job surveys
Comparison with best practice

1.3.3.1 Job Surveys

Job surveys are used to collect quantitative data about the ‘importance’ and ‘frequency’ of each task in performing the job, and the competencies are rated for the required level necessary to perform in that job.

The surveys are collected and the scores averaged. The job analyst then reviews the aggregated data for appropriateness.It can often happen that people over-state the level of competence required for certain jobs.

1.3.3.2 Comparison with best practice

There are numerous references available, such as books on high performing sales, that can be used for further information on how a particular job should be performed. The changes to make to the job data can be conveyed to managers and incumbents for comment and integration as appropriate.

1.3.4 Document the job description

A job description format outlines the job details, the job summary, job responsibilities and tasks, working conditions, job requirements, and Key Performance Indicators (as an optional).

Job details
Title of job, department/division, location, grade level, and salary/wages.

Job summary
Two or three sentences outlining the main purpose of the job.

Job responsibilities
Includes the most important Key Result Areas, with 3 or 4 tasks that were most highly rated in the job analysis under each KRA. Key Performance Indicators can also be listed here.

Working conditions
Summary of particular conditions involved that are worthy of mention. For example, adverse weather conditions, frequent customer contact, difficult environmental factors, etc.

Competencies
List here the most important representative KSAOs as rated in the job analysis.

Other job requirements
Outline educational and experience requirements, and certifications, (e.g., training as a nurse, previous nursing experience, and registration as a nurse).

McIntire, S.A., Bucklan, M.A., & Scott, D.R. (1995). The Job Analysis Kit. Psychological Assessment Resources, Inc.

Spector, P.E. (1996). Industrial and organisational psychology. Research and practice. New York: John Wiley & Sons, Inc.

1.4 Conclusion: Job analysis is critical in application / user interface design

You can see from the description of the method that there is quite a lot of work in capturing job data from people and documenting it for use in application / user interface design.

However, using experts in job analysis means the entire process for a given job can be done within a week. As other jobs are analysed and documented, you end up developing a library of KRAs and competencies that can be re-used, with some modification for other jobs. For example, people management in one job is the same as in another. A junior sales person needs to perform the same tasks as a senior sales person. The only difference is the level of competence they have as a result of their experience.

It’s a small investment to really understand the nature of people’s work and their activities as inputs into functional and non-functional requirements. The job description provides an evaluation framework during design to determine how closely it meets requirements.

In future articles, I’ll cover the transition between job descriptions and user interface design.

Job details	Title of job, department/division, location, grade level, and salary/wages.
Job summary	Two or three sentences outlining the main purpose of the job.
Job responsibilities	Includes the most important Key Result Areas, with 3 or 4 tasks that were most highly rated in the job analysis under each KRA. Key Performance Indicators can also be listed here.
Working conditions	Summary of particular conditions involved that are worthy of mention. For example, adverse weather conditions, frequent customer contact, difficult environmental factors, etc.
Competencies	List here the most important representative KSAOs as rated in the job analysis.
Other job requirements	Outline educational and experience requirements, and certifications, (e.g., training as a nurse, previous nursing experience, and registration as a nurse).