Guest Column: How to sample case management performance
By Patrice Spath, RHIT
Brown-Spath & Associates
Forest Grove, OR
When evaluating the quality of services provided by case managers, it is not necessary to review every medical record or survey every patient. The total time involved in gathering data will be much less if you are using sampling to study the population. Plus, there is a shorter time lag between data collection and reporting.
With a smaller number of observations, it is possible to provide results much faster as compared to reviewing 100% of the total population.
Through the careful application of sampling techniques, you can efficiently gather a sufficient amount of information to make valid judgments about the quality of case management performance. Sampling is the process by which inference is made to the whole by examining a part. There are several sampling techniques to choose from. You’ll want to choose a method that ensures every patient in your study population has an equal chance of being chosen for study. For example: If there are a total of N patients in the population and you choose K of them to review, then each patient could have been chosen with a probability of K/N. Generally, you’ll want to use a random sampling method. Random sampling ensures that the sample is representative, on the average, of the entire study population.
Types of samples
Suppose you want to construct a process measure for completeness of record documentation by case managers using random sampling rather than 100% review. If you choose to use a pure random sample, you would randomly select records from the entire population (all patients seen by a case manager) for the observation period. The selection of a random sample usually is made with the help of random numbers. For instance, suppose the case managers saw 500 inpatients last month and you want to review a sample of 25 records.
The patients are numbered from 1 to 500. Select 25 random numbers that fall between 1 and 500. Random numbers can be generated through any number of computer programs, including most database and statistical analysis packages. The records that correspond to the random numbers are the ones selected for your review.
The rate of compliance with documentation requirements would then be calculated as the number of records with complete case management documentation divided by the total number of records reviewed.
This estimated compliance rate would be a fair representation of the chance that the next patient record will have appropriate documentation. This is an example of a pure random sample leading to an estimated compliance rate.
You could measure the same process (complete documentation) by taking a stratified sample of cases. The record population could be stratified (or divided) into many different categories, but for simplicity we’ll divide them into two "strata": records of patients 50 and older, and records of patients 49 and younger.
You’d then select your sample from each of these strata rather than the population as a whole using random sampling techniques. If you divide the population into several categories based on the case manager assigned to the patient, random sampling techniques would be applied to each category.
However, if you intend to compare rates of compliance among case managers, then larger sample sizes must be available in each stratum. It may be more practical to evaluate case managers as a group; otherwise, a large review effort will be required to ensure meaningful comparisons.
Sample size
Many researchers strive for a sample size that reflects a 95% confidence level. This should be your goal also. The confidence level measures the percentage of possible population values captured in the sample size.
At the 95% confidence level, you can be fairly certain that your sample will include 95% of all the possible values. As expected, higher confidence levels require larger sample sizes.
If you wanted to be 100% confident that all values will be captured in your study, it would be necessary to sample the entire population. (For a sample-size selection table that illustrates the confidence levels for samples from different size populations, click here.)
For example, suppose you want to be 95% confident that the sample of records you will review to determine case manager compliance with documentation requirement is representative of the entire population of records. You’d need to review 213 records of the total population of 500 records. The sample sizes are expected to yield a 5% error rate, meaning that the findings from your review of 213 records may be 5% higher or 5% lower than what actually is happening in all 500 records. Poorly designed data collection instruments and/or badly trained data abstractors will increase the standard error of the mean.
Selecting cases
There are two methods that can be used to select cases for your study population. One method requires that the population is already known (e.g., patients discharged last quarter). This is a retrospective method of sampling that can be used to identify study cases from a population of patients who already have received care. When the population of patients already is identified (e.g., all patients discharged last year), list their names or other identifiers from 1 to N. The order of the listing is irrelevant. Generate a random number from 1 to N; that number is applied to the list of patients, and the patient corresponding to the random number is "chosen" for the study. A second random number is generated; if it is the same as the first number, it is disregarded; if it is new, then it is used to identify the second patient chosen. This continues until the total number of patients selected for the study population is chosen.
The second method of sample selection is one that can be applied concurrently. For example, as patients are discharged, you can determine if data from their record should be gathered or if the case is not one that should be included in the study. When the population of patients is accruing as the sample is being drawn (e.g. all patients who will be discharged this quarter), the true size of the patient population is unknown. In this instance, the population usually is phrased in terms of percentage of patients whose records you wish to review. For example, assume that you wish to review records from 10% of the patients. Every time a patient appropriate for the study is identified, generate a random number between 0 and 100. If the random number is less than or equal to 10, then the patient is chosen for the study. If the random number is greater than 10, then the patient is not included in the study sample. When the next patient appropriate for the study is identified, a new random number is generated, and the process is repeated. At the end of the study time period, approximately 10% of the patient population will have been chosen to be part of the study.
Gathering data on a sample of cases rather than 100% of the population is a reasonable choice for many of the performance studies conducted by case managers. The decision to sample is usually based on whether or not collecting data about every single member of the population would be too costly or take too much time.
Random sampling is the acceptable method for making sure that the study population is a representative subset of the total population.
You have reached your article limit for the month. Subscribe now to access this article plus other member-only content.
- Award-winning Medical Content
- Latest Advances & Development in Medicine
- Unbiased Content