MEASUREMENT & INSTRUMENTATION

Measurement

Definition - the rules for assigning numbers to objects/phenomena to represent quantities of attributes.

Assumptions - based on assumptions that:

Phenomena exist in some amount and that this amount can be measured

Phenomena are not constant, so they vary from situation to situation and from person to person

This variability of phenomena can be measured numerically

Purposes of measurement

By assigning numbers the researcher can differentiate between people or objects that possess varying degrees of the phenomena (critical attribute)

By assigning numbers the researcher connects numbers to the phenomena occurring in the natural/real world

Advantages

Decreases guesswork in gathering information and increases objectivity

Increases precision = accuracy of information

Establishes or uses a common language (temp high vs 101 F)

Measurement principles

Directness of measurement

Direct - length of stay

Indirect - attitudes

Measurement error - difference between what exists in reality and what is measured by the research instrument/tool

Levels of Measurement

Categorical - mutually exclusive and exhaustive categories

Nominal

mutually exclusive and exhaustive categories

Gender, marital status, occupation

Ordinal

mutually exclusive and exhaustive categories that are ranked (high to low, or most important to least important)

SES, attitudinal scales, Likert scales

Continuous - mutually exclusive and exhaustive categories that are ranked with equal intervals between each rank or unit of measurement

Interval

mutually exclusive and exhaustive categories that are ranked with equal intervals between each rank or unit of measurement

Body temperature, age

Ratio

mutually exclusive and exhaustive categories that are ranked with equal intervals between each rank or unit of measurement and with an absolute zero

Heart rate, respiratory rate, number of times pregnant

Converting data to another level of measurement

Data usually is converted to a lower level of measurment rather than a higher level

Continuous data converted to ordinal or nominal - eg age to age groups

Ordinal - educational level treated as nominal instead of ordinal

At times ordinal data measured on a Likert scale is treated as interval data.

This data is referred to as Aquasi-interval data@

Researcher does this so more powerful interval statistical tests may be used to test hypothesis

Determining the appropriate level of measurement

Is level of measurement appropriate for the type of data that is being sought and the hypothesis/question for the study?

What is the degree of precision desired when it is possible to consider the data at more than one level of measurement?

Errors of Measurement

Difference between true score in the real world, eg what actually exists and the observed score, eg obtained score collected by researcher

Keep in mind that we rarely actually know the true score

Factors contributing to errors of measurement

Situational contaminants - Hawthorne effect, concern for anonymity, researcher demeanor, location, time, temp during data collection

Response-set bias - social desirability, extreme responses, acquienscence with self-reports

Transitory personal factors - subject physical and emotional status during data collection, eg hunger, fatigue, anger, anxiety, pre-occupied with something

Variations in data collection - inconsistencies in data collection methods, eg changing coding, changing wording of interview questions, changing physiologic instruments or not calibrating instruments

Instrument clarity - poor quality of instructions, unclear self-report items or questions on tool

Item sampling - weak items on tool to measure attribute; eg of 100 possible items on tool 50 are selected; same person correctly answers 48 of 50 items on 1 version of the tool, but only answers 45 correctly on a second version of the tool

Instrument format - type of items or questions used in an interview or on printed tool; order of questions or items

Reliability of Measures

Stability

Test-retest reliability - correlation of 2 scores of 1 group with same instrument tested 2 weeks apart; tested then re-tested 2 weeks later (r=.80)

Reliability coefficient - scores of 1 group tested with parallel forms of the same instrument correlated (r = .80)

Internal consistency - homogeneity

Split-half method - compare odd numbered item with even numbered items

Coefficient alpha/ Cronbach=s alpha (.80)

Kuder-Richardson formula - KR-20

Equivalence

Inter-rater/inter-observer reliability

Inter-rater - consistency of 2 raters performance (.90)

Intra-rater - consistency of 1 rater's performance (.90)

Alternate forms (parallel forms) - construct 2 tools using the same outcomes, administer both tools to same group of subjects on same day and test for significant difference in scores

Validity of Measures

Content validity

Face validity - tool appears to measure phenomenon to lay person at face value

Expert validity - panel of experts agree that tool measures phenomenon

Criterion-related validity

Predictive validity - the extent to which future performance can be predicted by the past; eg hs gpa predict college gpa

Concurrent validity - extent to which a measure may be used to estimate an individual's present standing on the criterion; eg pulse oximeter & ABG oxygen saturation

Construct validity

Known groups technique (multi-contrasted group) - administer the tool to 2 groups known to be either extremely high or low on the phenomenon and compare their scores for significant difference

Multi-trait multi-method

Measure 2 or more different constructs using more than 2 methods for each construct at the same time to all subjects

Different measures of the same construct should have a high correlation if they do measure the same construct

Different measures of the same construct should have a low correlation if they do not measure the same construct

Factor analysis - assesses the various dimensions or factors of a phenomenon

Instrumentation

Use of existing instruments

Must fit with research hypothesis/questions, theoretical framework, and conceptual definitions

Both reliable and valid

Sources - Frank-Stromberg; Strickland & Waltz; Burrows

Must have authors= written permission to use or modify tool and use modification

Practicality of instrument in terms of cost, appropriateness for population, time required to complete it, physical and mental stamina of subjects, motor skills or language ability; and researcher training required to administer or score the instrument

Instrument development

Revise instrument with author written permission

Development of valid and reliable tool is very time consuming and may take years to accomplish

Testing required with multiple administrations, statistical analyses, and revisions

Instrument must be pilot tested before use in actual study or project

Pilot testing

Pretest of new instrument

Trial run of data collection to discover problems and revise to eliminate difficulties with the large sample

Ten to 25 subjects typical, dependent on the sample size

Sources of Error in Data Collection

Instrument inadequacies - lack of clarity in directions, items, etc.

Instrument administration biases

Environmental variations - location, temp, noise, interruptions, lack of privacy, lighting

Temporary subject characteristics during data collection process - anxiety, fatigue, hunger