Measuring Political Perceptions in Organizations: Assessing Reliability and Validity of POPS across Cultures

This study addresses the reliability and validity across cultures of the perceptions of organizational politics scale (POPS) developed by Kacmar and Carlson (1997). First, POPS is discussed generally by tracing the development of previous scales and how POPS fits into that stream of research. Subsequently, reliability and validity are addressed from across cultural settings. Reliability of POPS is assessed using Cronbach's alpha; validity is discussed by considering discriminate and concurrent validities. The general finding of this study is that POPS is consistently reliable across cultural settings and consistently demonstrates strong correlations with criterion variables related to POPS from within the nomological network. However, concerns about the discriminate validity of POPS in relation to perceptions of organizational support and organizational justice are shown to exist across cultural settings. In conclusion, suggestions are made for ways to build a stronger case for the reliability and validity of POPS within future research.


Introduction
Political perceptions have been described as the silent enemy within organizations (Vigoda, 2000) as well as the primary means by which employees make sense of organizational life (Parker, Dipboye, & Jackson, 1995). The phenomenon of perceptions of organizational politics is defined as the "subjective evaluation about the extent to which the work environment is characterized by co-workers or supervisors who demonstrate… selfserving behavior" (Ferris, Harrell-Cook, & Dulebohn, 2000). Based on the conceptual work of Ferris, Russ, and Fandt (1989), Kacmar and Ferris (1991) developed an initial scale to measure political perceptions and expand research of the phenomenon. Since that time, three additional scales have been developed and utilized significantly within perceptions of organizational politics research (Ferris and Kacmar, 1992;Kacmar & Carlson, 1997;Hochwarter, Kacmar, Treadway, & Watson, 2003). The purpose of the current paper is to trace the development of the perceptions of organizational politics scale (POPS) provided by Kacmar and Carlson and discuss its reliability and validity across cultural settings. Accordingly, this assessment accounts for two unique and significant contributions to the POPS literature: (a) it provides a comprehensive assessment of the reliability and validity of Kacmar and Carlson's scale; and (b) it provides a comprehensive account of the scale's use within a global setting.

Measurement of Perceptions of Organizational Politics
Kacmar and Carlson's perceptions of organizational politics scale (POPS) are arguably the most accepted instrument for measuring political perceptions (Hsiung, Lin, & Lin, 2012). However, it is important to understand the scale in the light of the stream of research predating its development. Contemporary studies of political perceptions are almost entirely built on the theoretical model proposed by Ferris, et al. (1989). This model is traced back to the prior work of theoreticians such as Gandz and Murray (1980), Madison, Allen, Porter, Renwick, andMayes (1980), Jablin (1981), Cheng (1983), and Drory and Romm (1988). For example, Gandz and Murray describe organizational politics as "a subjective state in which organizational members perceive themselves or others as intentionally seeking selfish ends in an organizational context when such ends are opposed to those of others". The model developed by Ferris, et al. expanded on the earlier theoretical foundation by suggesting antecedents and outcomes of political perceptions as well as the moderating effects of perceived control and understanding on the relationship between political perceptions and outcomes. In an effort to respond to the need for a validated and reliable instrument to measure political perceptions, Kacmar and Ferris (1991) and Ferris and Kacmar (1992) provided two scales with varying dimensionalities along with a subsequent call for further research. Although both of these scales are still used in contemporary research (e.g. Othman, 2008;Karatepe, Babakus, & Yavas, 2011), Kacmar and Carlson's scale was developed in response to the need for further validation and refinement of these instruments. This section describes the two instruments that predate Kacmar and Carlson's scale in order to demonstrate its dependence on and expansion of previous empirical studies. Kacmar and Ferris (1991): Kacmar and Ferris (1991) developed a 12-item, three-dimension scale to measure perceptions of organizational politics. The scale was developed in two phases. During the first phase, a 31-item scale was used to survey 387 participants. Items were generated from a review of the related literature. Using principle components factor analysis and reliability measures for the resulting dimensions, the researchers removed items, resulting in a 15-item, five-dimension scale: (a) general political behavior; (b) go along to get ahead; (c) coworkers; (d) supervisor; and (d) pay and promotion. In the second phase, data was analyzed from 822 new respondents using the same 31-item scale along with nine additional items (intended to extend factors identified in the first phase that had with too few items). In addition to these 40 items, a job satisfaction scale was included in the survey in order to remove items from POPS that were found to be redundant with job satisfaction, thereby contributing to the discriminate validity of the resulting scale. Again, factor analysis and reliability measures were used to study the data. Although 20 of the original 31 items loaded the same as in the first phase, factor analysis with the job satisfaction scale resulted in the complete removal of two dimensions. Accordingly, items were reduced, resulting in a 12-item, three-factor scale: (a) general political behavior; (b) go along to get ahead; and (c) pay and promotion. Overall reliability of the final scale was reported to be .87. Ferris and Kacmar (1992) performed two different studies to further explore the model of perceptions of organizational politics suggested by Ferris, et al. (1989). In their first study, they utilized a five-item instrument to simply explore the relationships between perceived political behavior and various outcomes. In their second study, their concern was to develop "a broader and more comprehensive multidimensional measure" (Ferris & Kacmar, 1991). The researchers utilized a 31-item measure (the same scale tested by Kacmar & Ferris, 1991) among a sample size of 95 participants. Principal components factor analysis led to the removal of items, resulting a 22-item, three-factor scale: (a) supervisor behavior; (b) coworker and clique behavior; and (c) organizational policies and practices. Ferris and Kacmar's results confirm Kacmar and Ferris are finding that POPS is a multi-dimensional construct. However, Ferris and Kacmar's dimensions lend themselves to a multi-level analysis of political perceptions (peers, supervisors, and organizations), whereas Kacmar and Ferris's dimensions are differentiated based on behaviors and experiences indiscriminant of organizational level. This distinction has subsequently led to further calls for reflection on the multiple levels of analysis of POPS (Dipboye & Foster, 2002) as well as the development of a six-item, three-factor measure used to assess perceptions of politics at multiple levels of respondents' organizations (Hochwarter, Kacmar, Treadway, & Watson, 2003).

Challenges to POPS' Dimensionality and Discriminate Validity
In response to Kacmar and Ferris's (1991) call for further validation of the scale, Nye and Witt (1993) tested the dimensionality and construct validity of POPS. Using a sample of 1,297 government employees, the researchers collected data using Kacmar and Ferris's 12-item, three dimension scale, as well as scales for measuring perceptions of organizational support (POS), equal opportunity environment, job satisfaction, and job commitment. Nye and Witt (1993) used principal components factor analysis (following Kacmar and Ferris) to study the data, finding that a single factor structure fit the data best. When they forced the data into a three-factor structure, one factor accounted for 81% of the variance. Accordingly, they suggest that POPS is one-dimensional. Reliability of the single-factor model was reported to be .93. Additionally, the other constructs included in the survey were strongly and negatively correlated with POPS, the strongest of which was POS (r = -.85). According to these findings, Nye and Witt suggest caution in utilizing Kacmar and Ferris's three-dimensional model and in assuming that POPS measures a distinct construct. Kacmar and Carlson (1997) continued the process of validating the perceptions of organizational politics scale by performing three studies to respond to the concerns of Nye and Witt (1993). First, the researchers used data from 749 responses to Kacmar and Ferris's (1991) perceptions of politics scale to establish the most acceptable model for the scale. They found that the three-factor model was superior to one factor and reported reliability of .87. In their second study, Kacmar and Carlson (1997) assessed the adequacy of each of the 12 items to measure the three dimensions by asking 102 respondents to rate the degree to which each of the items represented each of the three factors. Additionally, the 12-item scale was distributed to four samples along with other measures of faith in people, alienation, cynicism, altruism, trust, social attitude, and self-activity inventory in order to further assess the convergent and discriminate validity of POPS. Using factor analyses, the scale was reduced to six items, as it was found that six items were not functioning as intended. Comparison with the other scales incorporated into the survey showed that although POPS is theoretically similar, it demonstrates discriminate validity. In their third study, Kacmar and Carlson (1997) sought to strengthen the remaining six-item, three-factor scale by generating new items to measure these factors. Three tactics were used to add items: (a) the researchers studied items that had been previously removed to see if they could be altered to more appropriately fit the factors they were intended to measure; (b) a review of organizational politics theory was used to develop new items; and (c) anecdotal evidence and personal experiences were used to develop additional items. The same 102 respondents from the second study were asked to rate the degree to which the new set of 20 items (14 new) represented each of the three factors. Additionally, 15 external evaluators were asked to pair the 20 items with the three factors. Finally, the 20-item scale was distributed to two samples. Factor analyses were utilized to generate a final 15-item, three-factor scale with reliability of .87. Kacmar and Carlson's (1997) POPS scale has been used within empirical research in the United States as well as eight other countries (China, South Korea, Pakistan, Iran, Israel, Germany, Taiwan, and Nigeria). The purpose of this section is to describe reliability and validity of the scale across cultural settings. The assessment of reliability rests upon the assumption that the reliability of a scale can be demonstrated through consistent results across multiple uses of the measurement (Hair, Black, Babin, Anderson, & Tatham, 2005). The assessment of validity across cultural settings is based on the assumption that a scale should demonstrate consistency in its relationship with other criteria variables across measurements (Cozby, 2009). Although there are multiple ways to empirically assess reliability and validity, the current study is limited by what is available in extant research. Reliability is discussed first, followed by validity.

Reliability of POPS across Cultural Settings:
Reliability is primarily concerned with repeatability and consistency of an instrument and can be assessed using test-retest, internal consistency, and interrater reliability (Girden, 2001). However, this study uses Cronbach's alpha to assess reliability. Cronbach's alpha is a measure that assesses reliability of a scale by calculating the correlation of each item with other items (Cozby, 2009). It is assumed that a reliable measure will show consistent Cronbach's alpha coefficients across studies and settings. Hair, et al. (2005) explains that Cronbach's alpha measures internal consistency on a scale of 0 to 1; values of .60 to .70 are considered the lower limit of acceptable reliability. Table 1 provides the internal consistency alphas from studies using Kacmar and Carlson's (1997) scale as a complete 15-item onedimensional instrument. All of the studies report alpha reliabilities over .70. The average alpha value for studies in the United States is .85; the average among the two non-American studies that utilized a full onedimensional scale is .84. Other studies have utilized adjusted versions of Kacmar and Carlson's (1997) scale. Because Cronbach's alpha is sensitive to the number of items on a scale (Pallant, 2010) as well as the items that comprise a scale (DeVellis, 2012), the studies that have used alternative versions of POPS are moderately helpful in establishing reliability. However, as is demonstrated in Table 2, alpha coefficients continue to maintain sufficient reliability across cultures, as all alpha values are above .70. The average among American samples is .91; the average among non-American samples is .81. In addition to studies that have utilized a one-dimensional version, there are a small number of studies (see Table 3) that have used a three-factor structure or a single subscale from among Kacmar and Carlson's (1997) three-factor model. Although these findings are not extensive, they provide additional support for the reliability of POPS across cultural settings. Girden (2001) identifies validity as "synonymous with appropriateness, meaningfulness, and usefulness". A scale's validity is based on the extent to which it measures what it is designed to measure (Pallant, 2010). As is described elsewhere in this study, during the development of POPS, Kacmar and Carlson (1997) included in their research a number of other scales that were considered to be conceptually similar to perceptions of organizational politics. Factor analyses of each of these variables confirmed convergent validity of POPS. The following sections address the discriminate and concurrent validity of POPS across cultural settings. As is shown below, the discriminate validity of POPS has been widely addressed, whereas demonstrating concurrent validity based on anticipated relationships within the nomological network is unique to this study.   Go along to get ahead (7 items) Pay and promotion (6 items) Kiewitz, Hochwarter, Ferris, & Castro (2002) .79 Hochwarter, Kiewitz, et al. (2003)  Discriminate validity: Cozby (2009) explains that discriminate validity is concerned with the extent to which an instrument discriminates between the construct in question and other related constructs. The primary threats to the discriminate validity of POPS have come from its conceptually close (but negative) relationship with perceptions of organizational support (POS) and organizational justice. All of the studies (see Table 4) that have included these instruments demonstrate this concern, showing strong correlations between the scales -including one study from a non-American context.  (2001) USA r = -.48* r = .43* r = -.72 * Valle, et al. (2004) USA r = -.72* Harris, Andrews, et al. (2007) USA r = -.48*** r = -.43***  USA For example, using Kacmar and Carlson's (1997) POPS instrument, Andrews and Kacmar (2001) explored the construct validities of POS, perceptions of justice (procedural and distributive), and POPS, finding that there is significant correlation between each of these constructs. Their study demonstrates a strong but negative relationship between POS & POPS (r = -.72); a strong positive relationship between POS and justice (procedural justice, r = .61; distributive justice, r = .62); and a slightly less strong relationship between POPS and justice (procedural justice, r = -.48; distributive justice, r = -.43). Andrews and Kacmar also studied the predictive role of six antecedent variables, finding that one variable (centralization) did not differentially predict POPS and POS, whereas the others (LMX, formalization, role conflict, locus of control, and coworker cooperation) demonstrated different strengths of relationship with POS, POPS, and justice. Consistent with the findings of Nye and Witt (1993), Andrews and Kacmarsuggest that there is questionable discriminate validity between POPS and POS. Additionally, Andrews and Kacmar propose that the four constructs (POPS, POS, distributive justice, and procedural justice) fit best into a single four-factor model that shares the underlying construct of organizational fairness. In a non-American study, Lee and Peccei (2011) examined the discriminate validity of POS and POPS in a two-phase panel study in South Korea. Using confirmatory factor analysis, they found sufficient distinction between the two constructs. Their research design also included three outcome variables (affective commitment, intention to stay, and organizational citizenship behaviors). According to Lee and Peccei, analysis of POS and POPS in relation to these variables shows that POS and POPS are separate constructs that differentially relate to organizational outcomes.

Concurrent validity:
Concurrent validity is concerned with the relationship between the construct being studied and other criterion variables being measured at the same time (Cozby, 2009). Concurrent validity is based on what is theoretically expected within the nomological network of variables (Neider & Schriesheim, 2011). Kacmar and Carlson's (1997) scale is built upon the theoretical conceptualization offered by Ferris, et al. (1989). Within Ferris, et al.'s original model, it is proposed that perceptions of organizational politics will have a negative relationship with job anxiety and job satisfaction and a positive relationship with job stress and turnover intentions. These outcome variables have been studied across cultural settings using Kacmar and Carlson's scale and consistently demonstrate the relationships proposed by Ferris, et al.'s model. Table 5 provides a summary of correlations among these variables from multiple studies and contexts.

Conclusion and Discussion
The current paper addresses the reliability and validity across cultures of the perceptions of organizational politics scale (POPS) developed by Kacmar and Carlson (1997). In particular, this paper provides information regarding the development of POPS and its use in American and non-American contexts. Reliability is addressed by considering the scale's internal consistency using Cronbach's alpha across studies; validity is addressed by focusing on the scale's relationship with theoretically similar constructs (discriminate validity) and the consistency of the relationship between POPS and theoretically relevant outcome variables across cultures. According to the review of reliability coefficients found across studies in various cultures, there seems to be no reason to doubt the internal consistency of POPS as well as its reliability in non-American contexts. As is demonstrated in Tables 1, 2, and 3, all reliability coefficients are above .70. However, there remains a paucity of research that has utilized Kacmar and Carlson's (1997) scale as a three-factor model. Additionally, analysis of the correlation of POPS with variables that are likely to be related with the nomological network suggests that the scale retains very consistent concurrent validity across cultures. Ferris, et al. (1989) suggest that job satisfaction and affective commitment have significant negative relationships with political perceptions, whereas work stress and turnover intentions have significant positive relationships with political perceptions. These relationships are consistent across studies and cultures. As has been shown in previous studies, the most significant concern with respect to POPS is discriminate validity. Although there has been very little empirical research that has explored discriminate validity outside of the American context, the one non-American study that exists (Lee & Peccei, 2011) confirms the close relationship between POPS and POS. Other studies in the United States have demonstrated the close relationship between POPS and organizational justice. To date, no study has used Kacmar and Carlson's (1997) scale to study the relationship between POPS and justice in a non-American research context. This study concludes with suggestions for further establishing the reliability and validity of POPS across cultural settings. First, researchers should conscientiously report reliability coefficients for the full scale, even though through the process of factor analysis they may decide to drop certain items for their studies. Cronbach's alpha is sensitive to both the number and nature of items on a scale (DeVellis, 2012). Even though removing items may prove beneficial for individual studies, the ongoing discussion about the reliability of POPS is dependent on researchers reporting adequate and consistent information. Additionally, the multidimensionality of POPS in American and non-American contexts has been insufficiently explored. Future studies should take advantage of the three-factor model proposed by Kacmar and Carlson (1997), thereby providing additional data for understanding the dimensionality of POPS across cultures. Lastly, researchers who utilize POPS in non-American contexts should be encouraged to incorporate additional related constructs into their research design in order to add to the body of research contributing to the validation of the scale. This study makes two important contributions to the study of perceptions of organizational politics. As is noted earlier, there are various scales that are being used to measure political perceptions. The variations in scales make comparisons between studies difficult, not to mention the unique challenges presented by comparing empirical findings across cultures. Thus, first, this study provides a comprehensive picture of the use of one scale within empirical research. In addition, second, given that the study of political perceptions in organizations has become a global interest, this study provides empirical data to assess the reliability and validity of Kacmar and Carlson's (1997) version of POPS for continued use across cultures.