Assessors and use of assessment centre dimensions: a fresh look at a troubling issue*

Previous studies on the construct validity of assessment centres have generally produced puzzling results. The premise of this study is that these prior studies were relatively one‐sided. Actually, most previous studies were field studies, which typically used the multitrait–multimethod approach to distinguish between two sources of variance (i.e., exercises and dimensions). Therefore, this study aims to shed light on the issue of assessment centre construct validity by addressing substantive and methodological concerns inherent in previous research. In this study, 85 industrial and organizational psychology students and 39 managers rated videotaped assessment centre candidates in three exercises on six dimensions. Results from generalizability analyses showed that assessors' ratings were relatively veridical. In addition, when assessors rated candidates whose performances varied across dimensions and whose performances were relatively consistent across exercises, they were reasonably able to differentiate among the various dimensions. They also rated such candidate profiles similarly on the various dimensions across exercises. When assessors rated a candidate profile without clear performance fluctuations across dimensions, distinctions about dimensions were more blurred. Results from student and managerial assessors were similar, although managers distinguished somewhat less between the various dimensions. The research and practical implications of these findings are discussed. Copyright © 2001 John Wiley & Sons, Ltd.     

The validity of situational interviews for administrative jobs

This study was conducted to determine the validity of situational interviews for making predictions of ratings of job performance and potential. The validation sample consisted of 63 candidates. The correlations of potential and performance with situational interview scores were statistically significant and both predictor and criterion measures showed acceptable levels of reliability. The study provides additional evidence on the validity of situational interviews in a new job family, i.e. administrative jobs in a large U.K. financial services organization. The discussion questions the extent to which situational interviews provide unique or overlapping predictive validity when compared with mental ability tests.     

The impact of multiple source feedback on management development: findings from a longitudinal study

Despite the rapid uptake of multi‐source multi‐rater (MSMR) feedback systems by UK organizations, comparatively little research exists describing the actual impact on participant managers, in terms of changes in management competence. Ratings of 104 target managers (by self assessments, bosses‐, first‐ and second‐level subordinates) were investigated within the context of a developmental feedback programme in operation within an organization. The study compared ratings over two administrations (with two years between administrations) to determine: (1) changes in co‐workers' perceptions of their target manager's competence, (2) changes in target managers' development needs over time, (3) factors influencing a target manager's revised self‐assessment and co‐workers ratings, (4) changes in congruence between self and co‐workers ratings and (5) the relationship of feedback to the organization's formal performance appraisal process. Significant increases in managers' competence were perceived by the managers' themselves and by their subordinates, development needs were seen to reduce and self and co‐workers ratings were largely seen to become more congruent. However, polynomial regression analyses revealed co‐workers feedback at Time One was not predictive of targets' self‐assessments at Time Two. The implications of these findings with regard to the utility of MSMR feedback as a tool for management development are discussed. Copyright © 2002 John Wiley & Sons, Ltd.     

Perceptions of observed and model values of male and female managers

To provide evidence of the relationship between expectations for the values that should be exhibited for effective performance by female and male managers and their actual values, data were used that had been collected as part of a leadership program conducted for managers of business organizations in the United States. From a pool of approximately 700 managers, samples were drawn of 130 male and 130 female managers. As part of leadership program, the managers had completed 26-item SYMLOG value questionnaires, rating themselves and the values they judged to be effective for a manager. Prior to their participation in the program, sets of ratings were made by the coworkers of each manager, rating the actual values of the manager and the coworkers' ideal profile for the manager. The questionnaire covers three dimensions of values: dominant versus submissive, friendly versus unfriendly, and accepting the task-orientation of established authority versus opposing it. Comparisons were made of self and coworkers' ratings on the observed (self/actual) and model (effective/ideal) values of female and male managers with ‘matched’ samples that include only female and male managers with similar ratings of actual values, as judged by their coworkers, by removing the ratings of ‘outliers’. The only significant difference in gender ratings that remains is that female managers rate themselves as more positive. There are more differences between observed and model ratings. Managers and their coworkers believe that model managers should be more dominant and friendly than they are rated to be. However the managers also believe that they should be more task-oriented while their coworkers believe that they should be less task-oriented. The majority of the female managers dropped from the matched sample because no matching male could be found were more dominant and positive and less task-oriented, thus closer to the stereotype role for females. © 1997 John Wiley & Sons, Ltd.     

Relation between senior managers’ safety leadership and safety behavior in the Chinese petrochemical industry

Senior managers in organizations are authorized and obliged to maintain organizational safety. However, to date, little research has considered the relation of senior managers' safety leadership to safety behavior. This study addresses this gap by using path analysis to confirm the validity of a hypothetical model that relates six dimensions of senior managers' safety leadership to two safety behaviors through the safety climate in the petrochemical industry. A questionnaire survey was sent randomly to workers (other than senior managers) in two petrochemical companies in China, and data from 155 usable responses were compiled for the path analysis. Results indicate that in the petrochemical industry, senior managers' safety leadership has a positive impact on safety behavior, and the safety climate plays an intermediary role between them. From the perspective of the dimensions of senior managers' safety leadership and safety behavior, safety concern has the greatest positive effect on safety compliance. Moreover, safety vision has the greatest positive impact on safety participation, whereas safety inspiration and safety awards and punishment have negative effects on safety compliance. Personal character does not directly influence any dimension of safety behavior but indirectly does so by influencing the safety climate. On the basis of these results, measures of improving senior managers' safety leadership in the petrochemical industry are presented to help improve the overall safety performance of the industry. A new view is provided for the petrochemical industry in China to suggest that senior managers’ safety leadership can be treated earnestly.     

Personality and job performance: the importance of narrow traits

In a recent discussion of the bandwidth–fidelity dilemma in personality measurement for personnel selection, Ones and Viswesvaran (1996) concluded that ‘broader and richer personality traits will have higher predictive validity than narrower traits’. In this paper, the arguments made by Ones and Viswesvaran in favor of the exclusive use of broad personality dimensions are discussed. New data are presented that contradict Ones and Viswesvaran's claim of the existence of a general, integrity-related personality factor, and that show two narrow measures—the Responsibility and Risk Taking scales of the Jackson Personality Inventory—to have higher validities than the Big Five dimensions with respect to job performance criteria based on self-reported workplace delinquency in a sample of 127 entry-level employees. © 1998 John Wiley & Sons, Ltd.     

How does coaching interviewees affect the validity of a structured interview?

Coaching interviewees to perform in employment interviews can influence the interview scores. We describe different types of coaching, emphasizing interventions designed to focus interviewees on core, interview‐relevant content and to help them convey the content accurately (as opposed to interventions designed to teach interviewees to manipulate their scores using peripheral means). We then study the effects of the former type of intervention in relation to the criterion‐related validity of a structured employment interview. In a combined sample of 146 public safety job incumbents, a predictive validation study was conducted using scores from a situational panel interview. Predictive validity and reliability was observed to be higher in a sample of coached interviewees compared to a sample of uncoached interviewees. Implications for future research and practice in organizational behavior are discussed. Copyright © 2007 John Wiley & Sons, Ltd.     

Differentiating work autonomy facets in a non‐Western context

Work autonomy is one important component of job design theory which in recent decades has been elaborated upon by a number of researchers who have argued that it may be disaggregated into separate work method, work schedule and work criterion autonomy facets. Breaugh (1985) developed the Work Autonomy Scales as measures of each of these. This article reports the results of two studies carried out in Egypt that explored the validity of Breaugh's scales in relation to job design theory. In Study 1, in which Breaugh's scales were administered to 534 employees in two large Egyptian organizations, the Work Autonomy Scales' three‐factor structure was verified using exploratory and confirmatory factor analyses. In Study 2, using a sample of 120 managers from four organizations, the associations between the three facets of work autonomy and other variables with which they would be expected to correlate, along with their relationships with a number of outcome variables, were explored. Statistically significant correlations were observed between certain of the work autonomy scales and task interdependence, Hackman and Oldham's autonomy scale and job complexity. In terms of outcomes, work schedule autonomy was associated with job commitment, while work criterion autonomy was associated with job satisfaction. The results are discussed in the light of previous findings and some suggestions for future research are offered. Copyright © 2003 John Wiley & Sons, Ltd.     

Affect and favorable work outcomes: two longitudinal tests of the happy–productive worker thesis

This research examined relationships between alternative measures of affect and supervisory performance ratings. The first study showed that dispositional rather than state affect significantly predicted supervisory ratings of performance over time. Since the measures of affect differed on both content and temporal dimensions, a follow‐up study was conducted to explicate the results. The second study found that a pleasantness‐based measure of dispositional affect (Berkman, 1971a) again predicted rated performance over time, but activation‐based measures of both dispositional and state affect (using PANAS scales) were not predictive of supervisory evaluations of performance. The implications of these findings in terms of research on affect and the longstanding pursuit of the happy–productive worker are discussed. Copyright © 1999 John Wiley & Sons, Ltd.     

Actual safety performance of the Malaysian offshore oil platforms: Correlations between the leading and lagging indicators

IntroductionThis study establishes the correlations between performance of a set of key safety factors and the actual lagging performance of oil platforms in Malaysia, hence the relevance of the key safety factors in evaluating and predicting the safety performance of oil and gas platforms. The key factors are crucial components of a safety performance evaluation framework and each key safety factor corresponds to a list of underlying safety indicators.MethodIn this study, participating industrial practitioners rated the compliance status of each indicator using a numbering system adapted from the traffic light system, based on the actual performance of 10 oil platforms in Malaysia. Safety scores of the platforms were calculated based on the ratings and compared with the actual lagging performance of the platforms. Safety scores of two platforms were compared with the facility status reports' findings of the respective platforms.ResultsThe platforms studied generally had good performance. Total recordable incident rates of the platforms were found to show significant negative correlations with management and work engagement on safety, compliance score for number of incident and near misses, personal safety, and management of change. Lost time injury rates, however, correlated negatively with hazard identification and risk assessment. The safety scores generally agreed with findings of the facility status reports with substandard process containment found as a contributor of hydrocarbon leaks.ConclusionsThis study proves the criterion validity of the safety performance evaluation framework and demonstrates its usability for benchmarking and continuous improvement of safety practices on the Malaysian offshore oil and gas platforms.Practical applicationsThis study reveals the applicability of the framework and the potential of extending safety reporting beyond the few conventional lagging safety performance indicators used. The study also highlights the synergy between correlating safety factors to streamline safety management on offshore platforms.     

Charismatic leadership viewed from above: the impact of proactive personality

We tested hypotheses regarding the relationship between proactive personality and perceptions of charismatic leadership. A sample of 156 managers completed measures of proactive personality along with measures of the five‐factor model of personality and other individual differences. The managers' immediate supervisors rated their charismatic leadership and in‐role behavior. Results suggest that self‐reported proactive personality is positively associated with supervisors' independent ratings of charismatic leadership. Hierarchical regression analyses revealed that proactive personality accounts for variance in a manager's charismatic leadership above and beyond that accounted for by an array of control variables (the Big Five personality factors, in‐role behavior, and social desirability). Copyright © 2000 John Wiley & Sons, Ltd.     

Developing leading indicators from OHS management audit data: Determining the measurement properties of audit data from the field

IntroductionOHS management audits are one means of obtaining data that may serve as leading indicators. The measurement properties of such data are therefore important. This study used data from Workwell audit program in Ontario, a Canadian province. The audit instrument consisted of 122 items related to 17 OHS management elements. The study sought answers regarding (a) the ability of audit-based scores to predict workers' compensation claims outcomes, (b) structural characteristics of the data in relation to the organization of the audit instrument, and (c) internal consistency of items within audit elements.MethodThe sample consisted of audit and claims data from 1240 unique firms that had completed one or two OHS management audits during 2007–2010. Predictors derived from the audit results were used in multivariable negative binomial regression modeling of workers' compensation claims outcomes. Confirmatory factor analyses were used to examine the instrument's structural characteristics. Kuder–Richardson coefficients of internal consistency were calculated for each audit element.ResultsThe ability of audit scores to predict subsequent claims data could not be established. Factor analysis supported the audit instrument's element-based structure. KR-20 values were high (≥ 0.83).ConclusionsThe Workwell audit data display structural validity and high internal consistency, but not, to date, construct validity, since the audit scores are generally not predictive of subsequent firm claim experience. Audit scores should not be treated as leading indicators of workplace OHS performance without supporting empirical data.Practical applicationsAnalyses of the measurement properties of audit data can inform decisionmakers about the operation of an audit program, possible future directions in audit instrument development, and the appropriate use of audit data. In particular, decision-makers should be cautious in their use of audit scores as leading indicators, in the absence of supporting empirical data.     

Person- and context-oriented perceptual age measures: additional evidence of distinctiveness and usefulness

Cleveland and Shore (1992) suggested that four perceptual age measures could be grouped into person-oriented and context-oriented factors. This study examined longitudinal data from their same sample, and tested three propositions related to the distinctiveness and usefulness of the age measures. Confirmatory factor analyses showed that the factor structure proposed by Cleveland and Shore was invariant over time, and that new multi-item scales measuring two types of age loaded on appropriate factors. As hypothesized, context-oriented measures showed less temporal stability than person-oriented measures, and the temporal relationships among person-oriented measures were more easily explained in terms of a strict simplex structure than was the case for context-oriented measures. Perceptual age measures accounted for variance in self-ratings and managers' ratings of employee health, self-ratings of retirement intentions, and managers' ratings of promotability not accounted for by chronological age. © 1997 John Wiley & Sons, Ltd.     

Configuration of a functional department: A study of contextual and structural variables

This study investigates whether departments performing the same functions in different firms incur similar changes in structural configuration in response to industry-wide stimuli. Based on the proposition that the context of functional departments is similar across firms, changes in differentiation, integration, and perspective are hypothesized. Managers' perceptions of the validity of a series of statements to conditions five to 10 years ago and to present conditions are contrasted and show statistically significant shifts in these structural configurations. This change is confirmed by the analysis of responses of other managers in different departments within the same firms. Correlation analysis shows significant agreement among alternative measures of the same aspects of structure, and significant comovement among different structural dimensions. The later result supports the quantum theory of organizations.     

Broad and narrow measures on both sides of the personality–job performance relationship

Two studies compared specific versus broad measures in linking personality with work behavior. In Study 1, 100 university students completed the 20 subscales of the Personality Research Form and an in‐basket exercise scored on 16 distinct managerial behaviors. In Study 2, 335 market research field representatives completed the Hogan Personality Inventory, containing 41 specific trait scales organized into seven primary scales, and were rated by their supervisors on seven performance dimensions. In both studies, significant linkages between broad personality and criterion variables (e.g., factors) were explained by stronger relations among relatively few specific variables. Moreover, consistent with prior research, weak relations among broad measures obscured important linkages at the specific level, including several cases of cancellation (i.e., specific traits loading the same factor in the same direction correlated with criteria in opposite directions). Canonical correlations with appropriate shrinkage correction revealed notable improvements in criterion validity over inter‐factor correlations and helped summarize the data while retaining the diagnostic advantages of specificity. Our findings are unique by demonstrating the value of specificity on both sides of the prediction equation. Implications for personnel selection are discussed. Copyright © 2003 John Wiley & Sons, Ltd.     

Safety compliance climate concerning risk assessment and preventive measures in EU legislation: A Finnish survey

The aim of the present study is to explore the process dynamics of the safety compliance climate and work relationships, including their antecedents and consequences. After investigating the many dimensions of safety compliance climate and of work relationships, the study concludes that a multi-component measure is needed to fully assess all of their dimensions. With respect to the impacting positively on safety compliance legislation, the present study found the following factors: “contribution measures”, “concrete preventive measures” and “risk assessment process”. For risk assessment factors on safety compliance the study identified “planning, guidelines, policy, management”, “prioritization of proactive measures”, “results in cooperation and information” and “active use of risks assessment document”. In a regression model for the workers, both risk assessment process and contribution measures had a statistically highly significant (p < 0.001) effect on concrete preventive measures. In the regression analysis of the employers risk assessment process had a statistically significant (p < 0.01) effect and contribution measures had a statistically highly significant (p < 0.001) effect on concrete preventive measures.     

Measuring safety climate: identifying the common features

In UK industry, particularly in the energy sector, there has been a movement away from ‘lagging’ measures of safety based on retrospective data, such as lost time accidents and incidents, towards ‘leading’ or predictive assessments of the safety climate of the organisation or worksite. A number of different instruments have been developed by industrial psychologists for this purpose, resulting in a proliferation of scales with distinct developmental histories. Reviewing the methods and results from a sample of industrial surveys, the thematic basis of 18 scales used to assess safety climate is examined. This suggests that the most typically assessed dimensions relate to management (72% of studies), the safety system (67%), and risk (67%), in addition themes relating to work pressure and competence appear in a third of the studies.     

New recruit safety expectations: Relationships with trust and perceived job risk

Very little research has examined the safety expectations of new recruits, particularly those of individuals about to enter full-time work for the first time. There is evidence that new recruits have proportionally more accidents in the first period of their employment. One possible explanation for this is that the safety expectations of new recruits do not match the reality of the workplace they are about to enter. In Study 1 data on workplace safety expectations were collected from 142 final year high school students from six schools. Study 2 collected data from 40 organizations on the safety expectations of a new recruit and compared it with safety expectation data from a manager of the job they were entering. Both studies found that new recruit safety expectations were significantly correlated with ratings of safety specific trust in co-workers and management. Study 2 found that new recruits safety expectation scores were significantly higher than those given by managers. The results suggest that organizations need to develop a clear safety-specific psychological contract with new recruits.