Kappa as a Measure of Concordance in Categorical Sorting
Cohen's Unweighted Kappa
Kappa with Linear Weighting
Kappa with Quadratic Weighting
Frequencies and Proportions of Agreement
B Total
1 2 3
A 1 44 5 1 50
2 7 20 3 30
3 9 5 6 20
 Total  60 30 10 100
k = 3
N = 100
Kappa provides a measure of the degree to which two judges, A and B, concur in their respective sortings of N items into k mutually exclusive categories. A 'judge' in this context can be an individual human being, a set of individuals who sort the N items collectively, or some non-human agency, such as a computer program or diagnostic test, that performs a sorting on the basis of specified criteria. [Click here for an explanation of the conceptual and computational details of kappa.]
To begin, select the number of categories by clicking the appropriate button below; then enter your data into the appropriate cells of the data-entry matrix. After all data have been entered, click the «Calculate» button. To perform a new analysis, click the «Reset» button and start over. The analysis assumes that each entered value is an integer equal to or greater than zero.T
Note that measures of weighted kappa are meaningful only if the categories are ordinal and if the weightings ascribed to the categories faithfully reflect the reality of the situation. The weightings in this case are determined by the imputed relative distances between successive ordinal categories. By default, each of these distances is set at '1'. You are free to change any or all of these distances, though I recommend you do so only if you have good reason for it.
The author is grateful to CÚsar Roberto de Souza for detecting an error in the original programming for this module and suggesting the appropriate correction.

Select the number of categories:    2     3     4     5     6     7     8  
Number selected =   


Basis for weighting: imputed relative
distances between ordinal categoriesT
 1~2   2~3   3~4   4~5   5~6   6~7   7~8    successive ordinal categories 

  imputed relative distances 

Data Entry
B
Totals
1 2 3 4 5 6 7 8
A
Totals 
The designation "nc" appearing in any of the following
cells means "this quantity cannot be calculated." This
will typically occur only when your data entries in the
above table include a substantial proportion of zeros.

Unweighted Kappa
Observed
Kappa
Standard
Error
 .95 Confidence Interval 
Lower
Limit
Upper
Limit
Method 1
Method 2
maximum possible unweighted kappa, given
the observed marginal frequencies
observed as proportion of maximum possible
Kappa with Linear Weighting
Observed
Kappa
Standard
Error
 .95 Confidence Interval 
Lower
Limit
Upper
Limit
maximum possible linear-weighted kappa,
given the observed marginal frequencies
observed as proportion of maximum possible
Kappa with Quadratic Weighting
Observed
Kappa
Standard
Error
 .95 Confidence Interval 
Lower
Limit
Upper
Limit
maximum possible quadratic-weighted kappa,
given the observed marginal frequencies
observed as proportion of maximum possible
Frequencies of Agreement
  Category   Maximum
Possible
Chance
Expected
Observed
 1 
2
3
4
5
6
7
8
Total
Proportions of Agreement .95 CI
of Observed
  Category   Maximum
Possible
Chance
Expected
Observed Lower
Limit
Upper
Limit
 1 
2
3
4
5
6
7
8
Composite
Confidence intervals for proportions are calculated according
to the Wilson efficient-score method, corrected for continuity.

Home Click this link only if you did not arrive here via the VassarStats main page.


©Richard Lowry 2001-
All rights reserved.