brain with red checkmark



cognicheck.com online memory screenings

red and blue line red and blue line red and blue line
Terms & Conditions Privacy Copyright Contact Us
home
memory screening overview
sample screening results
prior participant login
company information
professional credentials
frequently asked questions
memory loss and alzheimers
psychometric properties

PSYCHOMETRIC PROPERTIES


The CogniCheck Online Memory Screening tests known collectively as the Computer Reliant Evaluative and Web Screening (CREWS) Tests of Neuropsychological Functioning have been standardized, normed, and scientifically evaluated in samples of individuals between 45 and 86 years of age who resided in the United States. This page provides an overview of the psychometric properties of the CREWS Tests of Neuropsychological Functioning, that includes the results of the normative study, as well as a summary of the reliability and validity of the memory screening measures. Please note that the information provided herewithin is intended primarily for licensed health care professionals and individuals with extensive/advanced training in psychometrics and neuropsychological assessment. While other individuals may find this information enlightening, it is intended to be utilized only for informational purposes and is not to be used to make definitive diagnoses, recommendations, and/or treatment decisions, nor is it intended for use in any litigation cases or proceedings. Furthermore, utilization of this information by persons without adequate psychometric training may result in inaccurate/erroneous interpretations of the data, as well as individuals' memory screening report scores.

NORMATIVE STUDY

To determine the "norms" (i.e., means and standard deviations) for each portion of the CREWS Tests of Neuropsychological Functioning and age group, a scientifically rigorous normative study was conducted where CogniCheck Online Memory Screenings were completed by a large sample of cognitively intact individuals, with generally unremarkable medical and psychiatric histories, who resided in geographically diverse regions of the United States. Table 1 below provides the demographic characteristics of the normative sample who ranged in age from 45 to 69 years, while Table 2 contains the demographic characteristics of the normative study participants who were 70 to 86 years of age. Table 2 also includes summary data for the entire normative study sample of 297 participants.


Table 1. Demographic Characteristics of the Normative Study Sample: Ages 45 - 69

Variable Age 45-49
(n = 45)
Age 50-54
(n = 45)
Age 55-59
(n = 51)
Age 60-64
(n = 40)
Age 65-69
(n = 39)
Gender
Male 16 (35.6) 15 (33.3) 21 (41.2) 14 (35.0) 16 (41.0)
Female 29 (64.4) 30 (66.7) 30 (58.8) 26 (65.0) 23 (59.0)
Race/Ethnic Origin
Caucasian 41 (91.1) 32 (71.1) 43 (84.3) 33 (82.5) 37 (94.9)
African-American 1 (2.2) 9 (20.0) 5 (9.8) 3 (7.5) 0 (0.0)
Hispanic 2 (4.4) 3 (6.7) 2 (3.9) 3 (7.5) 1 (2.6)
Asian/Other Origin 1 (2.2) 1 (2.2) 1 (2.0) 1 (2.5) 1 (2.6)
Educational Level
<= 12 Years 11 (24.4) 10 (22.2) 12 (23.5) 7 (17.5) 10 (25.6)
13 - 15 Years 8 (17.8) 14 (31.1) 14 (27.5) 10 (25.0) 14 (35.9)
>= 16 Years 26 (57.8) 21 (46.7) 25 (49.0) 23 (57.5) 15 (38.5)
Mean Educ. Level* 15.04 (2.26) 15.11 (2.57) 15.02 (2.35) 15.55 (2.51) 14.56 (2.48)
Notes: n = Number of normative study participants per age group.
Except where noted, numerical values represent the total number of participants in each category.
( ) = Except where noted, values within parentheses denote the percentage of participants within each category.
* = Mean Educational Level values represent the mean (and standard deviation) for each age group.


Table 2. Demographic Characteristics of the Normative Study Sample: Ages 70 - 86

Variable Age 70-74
(n = 29)
Age 75-79
(n = 35)
Age 80-86
(n = 13)
Total
(n = 297)
Gender
Male 13 (44.8) 13 (37.1) 7 (53.8) 115 (38.7)
Female 16 (55.2) 22 (62.9) 6 (46.2) 182 (61.3)
Race/Ethnic Origin
Caucasian 26 (89.7) 32 (91.4) 13 (100.0) 257 (86.5)
African-American 3 (10.3) 3 (8.6) 0 (0.0) 24 (8.1)
Hispanic 0 (0.0) 0 (0.0) 0 (0.0) 11 (3.7)
Asian/Other Origin 0 (0.0) 0 (0.0) 0 (0.0) 5 (1.7)
Educational Level
<= 12 Years 5 (17.2) 12 (34.3) 4 (30.8) 71 (23.9)
13 - 15 Years 4 (13.8) 11 (31.4) 2 (15.4) 77 (25.9)
>= 16 Years 20 (69.0) 12 (34.3) 7 (53.8) 149 (50.2)
Mean Educ. Level* 15.48 (2.73) 14.43 (2.57) 14.85 (2.44) 15.02 (2.48)
Notes: n = Number of normative study participants per age group.
Except where noted, numerical values represent the total number of participants in each category.
( ) = Except where noted, values within parentheses denote the percentage of participants within each category.
* = Mean Educational Level values represent the mean (and standard deviation) for each age group and
the total normative sample.




Age-related normative data (i.e., "norms"), cited in terms of the mean and standard deviation for each portion of the CREWS Tests of Neuropsychological Functioning and age group, are provided in Tables 3 and 4 below. Specifically, Table 3 provides the memory screening test norms for persons between 45 and 64 years of age, while Table 4 contains the norms for individuals 65 to 86 years of age. The maximum possible scores for each portion of the CREWS Tests of Neuropsychological Functioning are also included in these tables.


Table 3. Age-related Norms (i.e., Means and Standard Deviations) for the CREWS Tests of Neuropsychological Functioning: Ages 45 - 64

Age 45-49
(n = 45)
Age 50-54
(n = 45)
Age 55-59
(n = 51)
Age 60-64
(n = 40)
CREWS Test [Max] M (SD) M (SD) M (SD) M (SD)
Test of Verbal Learning*
Immediate Recall (Trials 1-5) [80] 49.62 (8.27) 48.78 (7.93) 45.63 (9.07) 45.18 (10.54)
Short-delay Recall [16] 10.64 (3.30) 10.67 (3.04) 9.47 (3.44) 9.75 (3.69)
Delayed Recall [16] 11.49 (2.88) 11.26 (2.68) 9.96 (3.34) 10.14 (3.33)
Delayed Recognition [32] 30.76 (3.43) 31.05 (1.62) 30.60 (1.84) 31.06 (1.39)
Test of Facial Memory**
Immediate Recognition [50] 41.18 (4.17) 39.64 (3.91) 40.08 (4.07) 39.63 (4.20)
Delayed Recognition [50] 40.73 (4.43) 39.67 (3.92) 39.12 (3.93) 38.46 (3.86)
Test of Digit Recall***
Total Score [32] 20.11 (4.65) 19.27 (4.68) 17.67 (5.50) 17.82 (6.30)
Key: [Max] = Maximum possible raw score M = Mean score (SD) = Standard Deviation
Notes: n = Number of normative study participants per age group.
* = Mean scores represent total number of words recalled/recognized correctly.
** = Mean scores represent total number of faces recognized/selected correctly.
*** = Mean scores represent total number of digit series (i.e., trials) recalled correctly.


Table 4. Age-related Norms (i.e., Means and Standard Deviations) for the CREWS Tests of Neuropsychological Functioning: Ages 65 - 86

Age 65-69
(n = 39)
Age 70-74
(n = 29)
Age 75-79
(n = 35)
Age 80-86
(n = 13)
CREWS Test [Max] M (SD) M (SD) M (SD) M (SD)
Test of Verbal Learning*
Immediate Recall (Trials 1-5) [80] 42.67 (6.92) 38.57 (8.67) 35.79 (10.98) 35.54 (8.58)
Short-delay Recall [16] 8.85 (3.19) 7.29 (3.02) 6.79 (3.92) 7.85 (3.24)
Delayed Recall [16] 9.11 (3.15) 7.12 (3.15) 7.60 (3.64) 7.38 (3.89)
Delayed Recognition [32] 30.38 (1.76) 28.50 (3.58) 28.27 (3.82) 28.69 (3.43)
Test of Facial Memory**
Immediate Recognition [50] 39.74 (4.64) 36.00 (4.36) 36.71 (4.50) 35.54 (2.70)
Delayed Recognition [50] 38.45 (4.74) 36.11 (4.26) 36.29 (4.83) 34.31 (2.78)
Test of Digit Recall***
Total Score [32] 17.92 (5.28) 15.61 (6.26) 15.23 (5.50) 16.18 (6.19)
Key: [Max] = Maximum possible raw score M = Mean score (SD) = Standard Deviation
Notes: n = Number of normative study participants per age group.
* = Mean scores represent total number of words recalled/recognized correctly.
** = Mean scores represent total number of faces recognized/selected correctly.
*** = Mean scores represent total number of digit series (i.e., trials) recalled correctly.




DETERMINATION of DESCRIPTIVE CATEGORIES

The distances, in terms of standard deviations (SDs) that particular memory screening raw scores fall from their respective age-related means, are utilized in the determination of raw scores' descriptive categories. Such standard deviations from the age-related means may be calculated via use of the following formula:

Raw Score of Test A - Age-related Mean of Test A
---------------------------------------------------------------------- = SDs from the Age-related Mean of Test A
Age-related Standard Deviation of Test A

For example, the distance (in standard deviations) that a 76-year-old individual's CREWS Test of Facial Memory Delayed Recognition task raw score of 29 falls from the respective age-related mean may be calculated as follows:
  1. Enter the memory screening test portion's raw score (e.g., 29) in the formula provided above.
     
  2. Obtain the test portion's age-related (e.g., Age 75-79 group) mean (e.g., 36.29) from the tables provided above (e.g., Table 4) and subtract this age-related mean from the test portion's raw score (e.g., 29) as noted in the formula (e.g., 29 - 36.29 = -7.29).
     
  3. Obtain the test portion's age-related (e.g., Age 75-79 group) standard deviation (e.g., 4.83) from the tables provided above (e.g., Table 4) and divide the value obtained in Step 2 (e.g., -7.29) by this age-related standard deviation to calculate the distance, in standard deviations, that a particular raw score falls from the age-related mean (e.g., -7.29 / 4.83 = -1.509 standard deviations from the age-related mean).
The value obtained in Step 3 can then be compared to the standard deviation ranges provided in Table 5 below to determine the descriptive category in which the raw score falls. Specifically, in the example above, a 76-year-old individual's raw score of 29 on the CREWS Test of Facial Memory Delayed Recognition task is -1.509 standard deviations below the age-related mean of 36.29. Raw scores that are more than -1.500 standard deviations below the age-related mean to -2.000 standard deviations below the age-related mean fall within the "moderately impaired" descriptive category. It should be noted that positive (+) values indicate standard deviations above the age-related mean(s), while negative (-) values denote standard deviations below the age-related mean(s). Additionally, all calculations in Steps 1 through 3 above should be carried out at least three decimal places.

Please also note that, for your convenience, all CogniCheck Online Memory Screening reports provide interpretation of the CREWS Tests of Neuropsychological Functioning raw scores in terms of both percentiles (%) and descriptive categories.


Table 5. Descriptive Categories and Standard Deviation Ranges

Descriptive Category Standard Deviation (SD) Range
Superior More than +2.000 SDs above the mean
Well Above Average More than +1.500 SDs to +2.000 SDs above the mean
Above Average More than +1.000 SD to +1.500 SDs above the mean
High Average More than +.500 SD to +1.000 SD above the mean
Average Between +.500 SD above and -.500 SD below the mean
Low Average More than -.500 SD to -1.000 SD below the mean
Mildly Impaired More than -1.000 SD to -1.500 SDs below the mean
Moderately Impaired More than -1.500 SDs to -2.000 SDs below the mean
Significantly Impaired More than -2.000 SDs below the mean




RELIABILITY and INTERCORRELATIONS

The reliability of neuropsychological tests typically refers to the amount/level of consistency, dependability, and stability in the tests' measurements and scores. To examine the reliability/internal consistency of the CREWS Tests of Neuropsychological Functioning, Chronbach's alpha reliability coefficients were calculated for each test portion utilizing the memory screening data obtained from the 297 normative study participants. Evidence of the reliability of the CREWS Tests of Neuropsychological Functioning is found in Table 6, which provides the Chronbach's alpha reliability coefficients for each memory screening test portion when the entire sample of participants was utilized in the analyses. As noted in this table, reliability coefficients range from .64 for portions of the CREWS Test of Facial Memory to a high of .90 for the CREWS Test of Digit Recall Total Score.

The intercorrelations between the various portions of the CREWS Tests of Neuropsychological Functioning, that are also cited in Table 6, provide evidence of the validity of the memory screening tests. In particular, the correlations between portions of the CREWS Test of Verbal Learning are notably higher than those between portions of this test and portions of the CREWS Tests of Facial Memory and Digit Recall. Similarly, the correlation between the Immediate and Delayed Recognition portions of the CREWS Test of Facial Memory is greater than the correlations between these test portions and portions of the CREWS Tests of Verbal Learning and Digit Recall.


Table 6. Reliability Coefficients and Intercorrelations for the CREWS Tests of Neuropsychological Functioning

CREWS
Test of Verbal Learning
CREWS
Test of Facial Memory
CREWS Test
of Digit Recall
Immed.   S-delay   Delayed   Delayed Immed. Delayed Total
CREWS Test Recall  Recall Recall  Recogn. Recogn. Recogn. Score
Test of Verbal Learning
Immediate Recall
Trls. 1-5*
(.85)
Short-delay Recall .76 (.76)
Delayed Recall .80 .81 (.75)
Delayed Recognition .46 .47 .55 (.81)
Test of Facial Memory
Immediate Recognition .39 .28 .36 .27 (.64)
Delayed Recognition .29 .18 .27 .16 .69 (.64)
Test of Digit Recall
Total Score .27 .19 .17 .13 .15 .14 (.90)
Notes: * = Chronbach's alpha reliability coefficient utilizing all word items on Trials 1 through 5.
(.00) = Chronbach's alpha reliability coefficients utilizing the entire sample of normative study participants.




VALIDATION STUDY

In addition to the intercorrelations cited in Table 6, the construct validity, and in particular, the convergent and discriminant validity, of the CREWS Tests of Neuropsychological Functioning was examined via a scientifically rigorous "validation" study of 25 individuals who ranged in age from 45 to 74 years (Mean = 57.08, SD = 8.40), with educational levels ranging from 10 to 24 years (Mean = 14.90, SD = 2.87). Among this sample of participants, two reported medical histories of strokes, three indicated histories of transient ischemic attacks (i.e., TIAs), while one person's history was remarkable for meningitis and another individual's history included migraine headaches. Additionally, at the time of this study, four participants acknowledged depressive symptomatology and nine individuals were taking at least one psychotropic medication (e.g., an antidepressant).

To evaluate convergent and discriminant validity, study participants were administered (in a counterbalanced order) the CREWS Tests of Neuropsychological Functioning and a series of standardized neuropsychological tests and measures that are commonly utilized by licensed clinical neuropsychologists in clinical practice, including measures of hand grip strength (i.e., Grip Strength task; Reitan & Wolfson, 1993), verbal fluency (i.e., FAS Test; Spreen & Strauss, 1998), visual-organization skills (i.e., Hooper Visual Organization Test; Hooper, 1983), overall cognitive status (i.e., Mini Mental State Examination; Folstein, Folstein & McHugh, 1975), auditory-verbal learning/memory (i.e., Rey Auditory Verbal Learning Test; Rey, 1964), cognitive processing speed and response inhibition (i.e., Stroop Color & Word Test; Stroop, 1935, Golden, 1978), visuo-motor scanning abilities, sequencing skills, and cognitive flexibility (i.e., Trail Making Test, Parts A & B; Reitan, 1979, Reitan & Wolfson, 1993), attention/concentration, rote memory, and aspects of working memory (Wechsler Memory Scale-III, Digit Span subtest; Wechsler, 1997), and visual memory/facial recognition (i.e., Wechsler Memory Scale-III, Faces I & II subtests; Wechsler, 1997).

Evidence of the convergent and discriminant validity of the CREWS Tests of Neuropsychological Functioning may be found in Tables 7 and 8 below. Specifically, these tables provide the correlations/validity coefficients between the CREWS Tests of Neuropsychological Functioning and the standardized neuropsychological tests and measures commonly utilized in clinical practice. For example, the generally higher correlations between portions of the CREWS Test of Verbal Learning and portions of another verbal memory test (i.e., Rey Auditory Verbal Learning Test), as compared to the lower correlations between portions of the CREWS Test of Verbal Learning and measurements of hand grip strength (i.e., Grip Strength task) and visual-organization (i.e., Hooper Visual Organization Test) provide evidence of the convergent and discriminant validity, respectively, of the CREWS Test of Verbal Learning.

Similarly, examples of the convergent validity of the CREWS Tests of Neuropsychological Functioning are found in the relatively higher correlations between portions of the CREWS Test of Facial Memory and other tests that assess visual memory/facial recognition (i.e., Wechsler Memory Scale-III, Faces I & II), while discriminant validity is demonstrated in the lower correlations between the two portions of the CREWS Test of Facial Memory and measures of cognitive processing speed (i.e., Stroop Color & Word Test) and cognitive flexibility (i.e., Trail Making Test, Part B). Furthermore, the generally higher correlations between the CREWS Test of Digit Recall Total Score and alternate measures of verbal memory (i.e., Rey Auditory Verbal Learning Test, Wechsler Memory Scale-III, Digit Span subtest), as compared to the lower correlations between the CREWS Test of Digit Recall Total Score and measures of visual-organization abilities (i.e., Hooper Visual Organization Test) and cognitive processing speed (i.e., Stroop Color & Word Test) provide examples of the convergent and discriminant validity of the CREWS Test of Digit Recall. It should be noted that numerous other examples of convergent and discriminant validity may be found upon detailed analysis of the correlations between the CREWS Tests of Neuropsychological Functioning and the other commonly utilized neuropsychological tests and measures that are cited in Tables 7 and 8.


Table 7. Correlations between the CREWS Tests of Neuropsychological Functioning and other Neuropsychological Tests and Measures

Grip Strength Rey Auditory Verbal Learning Test
CREWS Test DH NDH  FAS   HVOT   MMSE  IMRL SDR DLRL DRCG
Test of Verbal Learning
Immediate Recall (Trials 1-5) .22 .12 .21 .02 .00 .55 .56 .50 .04
Short-delay Recall -.07 -.13 .21 .33 .26 .59 .69 .70 .27
Delayed Recall -.14 -.17 .23 .19 .05 .58 .54 .55 .28
Delayed Recognition .14 .02 -.02 .01 .08 .55 .52 .45 .10
Test of Facial Memory
Immediate Recognition .38 .22 .51 .41 .12 .53 .39 .35 .32
Delayed Recognition .45 .27 .29 .20 .16 .35 .02 .03 .16
Test of Digit Recall
Total Score .49 .36 .35 .23 -.03 .54 .59 .46 .19
Key: DH = Dominant Hand IMRL = Immediate Recall (Trials 1-5)
NDH = Non-dominant Hand SDR = Short-delay Recall
FAS = FAS Test DLRL = Delayed Recall
HVOT = Hooper Visual Organization Test DRCG = Delayed Recognition
MMSE = Mini Mental State Examination


Table 8. Correlations between the CREWS Tests of Neuropsychological Functioning and other Neuropsychological Tests and Measures

 Stroop C & W Test   Trail Making Test   Wechsler Memory Scale-III 
CREWS Test Word Color C-W Part A Part B DS, Total Faces I Faces II
Test of Verbal Learning
Immediate Recall (Trials 1-5) .34 .34 .24 -.39 -.55 .28 .27 .24
Short-delay Recall .48 .40 .25 -.30 -.46 .39 .33 .22
Delayed Recall .52 .48 .36 -.47 -.54 .17 .41 .45
Delayed Recognition .39 .35 .24 -.19 .-.45 .16 .35 .25
Test of Facial Memory
Immediate Recognition .30 .18 .33 -.44 -.33 -.03 .81 .68
Delayed Recognition .29 .21 .53 -.48 -.19 -.20 .77 .74
Test of Digit Recall
Total Score .26 .15 .11 -.21 -.48 .53 .36 .16
Key: Word = Word Naming task C-W = Color-Word task
Color = Color Naming task DS = Digit Span subtest




References


Folstein, M. F., Folstein, S. E., & McHugh, P. R. (1975). "Mini-Mental State:" A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatry Research, 12, 189-198.

Golden, J. C. (1978). Stroop Color and Word Test. Stoelting Co., Chicago, IL.

Hooper, H. E. (1983). Hooper Visual Organization Test (VOT). Los Angeles: Western Psychological Services.

Reitan, R. M. (1979). Manual of Administration of Neuropsychological Test Batteries for Adults and Children. Tucson, AZ: Reitan Neuropsychological Laboratories.

Reitan, R. M. & Wolfson, D. (1993). Halstead-Reitan Neuropsychological Test Battery: Theory and Clinical Interpretation (Second Edition). Tucson, AZ: Neuropsychology Press.

Rey, A. (1964). L'examen clinique en psychologie. Paris: Presses Universitaires de France.

Spreen, O. & Strauss, E. (1998). A Compendium of Neuropsychological Tests: Administration, Norms, and Commentary (Second Edition). New York: Oxford University Press.

Stroop, J. R. (1935). Studies of interference in serial verbal reaction. Journal of Experimental Psychology, 18, 643-662.

Wechsler, D. (1997). Wechsler Memory Scale - Third Edition. San Antonio, TX: The Psychological Corporation.