HEALTHCARE DOCUMENTATION
QUALITY ASSESSMENT AND MANAGEMENT
BEST PRACTICES
Revised July 2017
Task Force Members (2017)
Contributors
Vetting Group
Toolkit Review
Task Force Members (2010 Edition)
Introduction
→ →
→
The Plan, Do, Check, Act Cycle for Quality Assurance
4 Steps of PDCA
PLAN
PRINCIPLES OF QUALITY
Quality Guidelines
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Blanks
•
•
•
•
•
•
•
•
Resolving Blanks
•
•
•
•
•
•
Roles and Responsibilities
Clinician/Author
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Facility Staff
•
•
•
•
•
•
•
•
•
•
Healthcare Documentation Specialist
•
•
•
•
•
•
MTSO
•
•
•
•
•
•
•
Authenticator
•
•
DO
Quality Manager (document integrity)
Document Integrity Auditor
Support Staff
Concurrent Review
QUALITY ASSESSMENT POLICIES AND PROCEDURES
Retrospective Review
Flagged Documents
Feedback
HDS Assessment
Author Assessment
CHECK
Sample Selection
Error Categories
SAMPLING GUIDELINES
ERROR CATEGORIES, DEFINITIONS, SCORING, AND
ENSURING CONSISTENCY
ITEM
DESCRIPTION
EXAMPLE(S)
ITEM
DESCRIPTION
EXAMPLE(S)
ITEM
DESCRIPTION
EXAMPLE(S)
Scoring Quality Assurance Audits
• Supports the values of quality delivery.
• A huge disparity of line lengths can exist among documents. If not all HDS staff transcribe/edit
an equal number of short and long documents, this method may not provide an accurate picture
of the HDS’s skillset.
• Produces an errors-to-lines ratio that can provide a more accurate picture of the HDS’s skillset.
• A single critical error in a document with many lines may yield a percentage that allows the QA
audit to pass.
• A single critical error in a document with very few lines will not only fail the QA audit but skew
an average of several audits in a downward fashion.
Ensuring Accuracy and Consistency in Your QA Program
ACT
Developing an Action Plan
•
•
•
•
•
•
•
•
•
•
•
•
CONTINUOUS QUALITY IMPROVEMENT
•
•
•
•
•
•
INDUSTRY RECOMMENDATIONS
APPENDIX A: Sample 1 – Quality Assessment Score Sheet
HDS Name
Job #
Author
Work Type
Auditor
Date of Review
TYPE OF ERROR
ERROR VALUE
NUMBER OF
OCCURRENCES
FINAL
DEDUCTION
Critical Errors
1. Patient demographics
-3
2. Wrong work type, template, provider information
-3
3. Terminology misuse; wrong lab value
-3
4. Wrong medication, wrong dose/dosage
-3
5. Unapproved abbreviations
-3
6. Incomplete or missing text, inserted/omitted text
-3
7. Incorrect side/site; unauthorized substitution
-3
8. Failure to edit; failure to flag
-3
9. Failure to follow author instructions
-3
10. Inconsistency/discrepancy
-3
Noncritical Errors
1. Misspelled medication, terminology, names
-1
2. Transposition of proper names
-1
3. Incorrect word form
-1
4. Incorrect, inserted or omitted text
-1
5. Nonsense text; punctuation; failure to flag
-1
6. Sound alikes; protocol failure
-1
TOTAL DEDUCTIONS
ERROR TOTAL: _______________
FINAL SCORE: _______________
If the same error is repeated throughout the document, it is only counted once. Score of 98 is
considered passing.
COMMENTS:
APPENDIX A: Sample 2 – Quality Assurance Audit
HDS NAME: ID:
DATE: TYPE:
AUDITOR INITIALS:
Account/Job Number
Work Type
Points Possible
Minus Score
Equals Score
100
100
100
100
100
100
100
100
AVERAGE
#DIV/0!
If the same error is repeated throughout the document, the error is only counted once.
TYPE OF ERROR
#ERRORS
x VALUE
TOTAL
CRITICAL
Patient demographics
0
3
0
Work type/template/wrong provider
0
3
0
Wrong provider information
0
3
0
Terminology misuse
0
3
0
Wrong medication, dose/dosage
0
3
0
Wrong lab value
0
3
0
Unapproved abbreviations
0
3
0
Incomplete or missing text
0
3
0
Inserted or omitted text
0
3
0
Incorrect side/site
0
3
0
Failure to edit/failure to flag
0
3
0
Failure to follow author instructions
0
3
0
Inconsistencies/discrepancies
0
3
0
Unauthorized substitution
0
3
0
NONCRITICAL
Misspelled meds, terminology, names
0
1
0
Transposition of proper names
0
1
0
Incorrect word form
0
1
0
Incorrected, inserted or omitted text
0
1
0
Nonsense text; punctuation
0
1
0
Failure to flag; sound alikes
0
1
0
Protocol failure
0
1
0
TOTALS
FINAL SCORE: 98 and above PASS
< 98 NEEDS IMPROVEMENT
NOTE: Any critical errors found upon audit will automatically fail the audit.
APPENDIX B: Sample Quality Assurance Scoring Worksheet
JOB NUMBERS WT
DATE
TRANSCRIBED
DATE
REVIEWED
TOTAL
LINES
3-Point
Errors
1-Point
Errors
0.5-Point
Errors
0.25-Point
Errors
1459801, 1460168 and 6, 24 12/18, 12/19
1460549
2 12/20/16 12/28/16 198 0 0 1 1
1466708, 1466843 and 21, 2 1/1, 1/2
160607 30 1/3/16 1/11/16 211 0 0 0 1
1474534, 1474609 and 4, 2 1/15, 1/16
1475646 24 1/17/17 1/24/17 216 0 0 0 1
1481712, 1482295 and
6, 4 1/29, 1/30
1483455 24 2/1/17 2/7/17 231 0 0 0 0
1497143, 1497477 and 2,6 2/26, 2/27
1498439, 1498927 44, 24 2/28, 3/1 3/7/17 206 0 0 1 2
1506576, 1507959 6, 2 3/16, 3/19 3/21/17 247 0 0 0 0
TOTALS: 1309 0.00 0.00 1.00 1.25
Total lines counted, all reports = 1309
Total Errors = 2.25
Errors divided by lines = 0.00172
Error percentage rate = 0.17 %
ACCURACY % RATE = 99.8
Total lines transcribed this quarter: 105,531
Actual Percentage of
Lines Reviewed:
1.24%
Accuracy rate must be 98.5% or higher
Critical Errors -3 Noncritical Errors -1
1. Patient demographics 1. Misspelled medication, terminology, names
2. Wrong work type, template, provider information 2. Transposition of proper names
3. Terminology misuse; wrong lab value 3. Incorrect word form
4. Wrong medication, wrong dose/dosage 4. Incorrect, inserted or omitted text
5. Unapproved abbreviations 5. Nonsense text; punctuation; failure to flag
6. Incomplete or missing text, inserted/omitted text 6. Sound alikes; protocol failure
7. Incorrect side/site; unauthorized substitution
8. Failure to edit; failure to flag Minor Errors -0.5 -0.25
9. Failure to follow author instructions
10. Inconsistency/discrepancy Educational Feedback -0
TRANSCRIPTIONIST: SCORE, APPENDIX B
QUALITY ASSURANCE SCORING WORKSHEET
QUARTER: 1st QUARTER 2017
APPENDIX C: CCD QA Review Form
Reviewer:
Provider:
Critical errors – A critical error is any error in a patient care record that has the potential to:
1. Adversely impact patient safety.
2. Alter the patient’s care or treatment.
3. Adversely impact the accuracy of coding and billing.
4. Result in a HIPAA violation.
5. Adversely affect medicolegal outcomes.
Please correct all critical errors.
Noncritical errors – Noncritical errors impact document integrity but do not have the potential to
affect patient safety, care, or treatment, and/or do not alter the intended meaning of the author.
Error-free documents – number of documents with no critical or noncritical errors.
Explanation and copy of errors are listed below the table.
Date
Document
Type
Encounter
#
MR#
# of Error-
Free
Documents
Critical #
of Errors
Noncritical
# of Errors
Yellow – errors/concerns
Green – Corrections
Critical errors in red
Noncritical errors in blue
MRN#
Encounter #/Date
CCD QA Review Form—SAMPLE
Reviewer: JLD
Provider: Joe Shmoe, MD
Critical errors – A critical error is any error in a patient care record that has the potential to:
1. Adversely impact patient safety.
2. Alter the patient’s care or treatment.
3. Adversely impact the accuracy of coding and billing.
4. Result in a HIPAA violation.
5. Adversely affect medicolegal outcomes.
Please correct all critical errors.
Noncritical errors – Noncritical errors impact document integrity but do not have the potential to
affect patient safety, care, or treatment, and/or do not alter the intended meaning of the author.
Error-free documents – number of documents with no critical or noncritical errors.
Explanation and copy of errors are listed below the table.
Date
Document
Type
Encounter
#
MR#
# of Error-
Free
Documents
Critical #
of Errors
Noncritical
# of Errors
02/05/14
Progress
123456789
123456
1
02/05/14
Progress
987654321
9876542
1
2
02/05/14
DS
13467932
134625
2
02/06/14
Progress
147852369
852147
1
02/05/14
3
Yellow – errors/concerns
Green – Corrections
Critical errors in red
Noncritical errors in blue
MRN# 123456
Encounter# 123456789 2/5/2014:
REVIEW OF SYSTEMS
Gastrointestinal: Nausea, no vomiting, no diarrhea, no constipation. Abdominal pain: The pain is mild,
characterized as continuous. (Critical – Inconsistent with PE)
PE
Gastrointestinal: Soft, nontender (Critical – Inconsistent with ROS), nondistended. Normal bowel
sounds.
MRN# 9876542
Encounter# 987654321 2/5/2014:
Clinical Milestones
Appreciate oral surgery inout input (Noncritical – Wrong word form/spelling/typo)
Dx and Plan: Check echo to rule out ceompensated decompensated (Noncritical – Wrong word
form/spelling/typo) valve disease. There was no documentation of MR or AS on previous echo last year.
Review of Systems
Gastrointestinal: Abdominal pain: Left middle (see PE below). The pain is severe, characterized as
cramping/colicky.
PE-Objective
Gastrointestinal: Soft. Normal bowel sounds. Left upper quadrant (Critical – Inconsistent with ROS) tenderness.
Not distended.
MRN# 134625
Encounter# 13467932 2/5/2014:
Interval History
&! (Critical – incomplete or missing data) y/o woman admitted with syncopal episode that she had after bouts of
diarrhea. She became nauseated and had bowel and bladder incontinence, at which point hematochezia was
discovered. Admitted for evaluation by GI for hematochezia and by Neurology to rule out seizure.
MRN# 852147
Encounter# 147852369 2/6/2014:
HOSPITAL COURSE
Medical management: Postop left THA day #1 (Critical – Incorrect postop site. Patient had a left TKA.)
APPENDIX D: Statistically Valid Sampling
Sampling is an essential step in determining the quality of work that is being delivered without having to check
all the reports or jobs delivered. Good quality sampling is characterized by the sampling technique used and the
sample size picked. Obtaining a sample that is appropriate in both regards is critical to having a good
understanding of the quality of work delivered. Sampling must be done at a job level since accuracies are
measured at a job level.
Sampling Technique
Using a random sampling technique will ensure all jobs delivered have an equal opportunity to be picked as a
sample. Random sampling is both easy to use and can give an accurate representation of all jobs delivered.
Sample Size Determination
Determining the correct sample size will help us get an accurate measure of the quality of work while using the
resources required for sampling in an optimal manner. Using a larger sample size could provide us better
accuracy in determining the population (population is the set of all jobs delivered); however, this will exhaust
more resources for sampling. On the flip side, a smaller sample size would help conserve resources but may not
necessarily provide a good understanding of the quality of the population. Also, a smaller sample size is
susceptible to higher variation in the population (e.g., if the population contains jobs with accuracies that are
highly varied). Calculating the minimum sample size helps us determine the correct sample size required.
Minimum Sample Size Calculation
Minimum sample size (MSS) is calculated using this formula:
MSS = ((CI * SD)/P)
2
SD refers to the estimated standard deviation of
the population. Standard deviation is a measure of
variation. A low standard deviation indicates a low
variation. If the standard deviation is low then MSS
will also be low; the converse is also true.
CI refers to the confidence intervals (also known as
confidence level), which determines the
probability that the sample will represent the population. This should be set at 95%. Increasing the CI to greater
than 95% will increase the probability that the sample represents the population, but this will also increase the
minimum sample size required; the converse of the statement is also true, i.e., reducing the CI will result in
reduction of MSS.
P refers to precision, which means the accuracy level, in decimal points, that we would want when determining
the quality of the population. It is recommended that precision be set at 0.025 for healthcare documentation
jobs. What this means, essentially, is if the actual quality of the population is 99.50, the sample will give us an
accuracy estimate that will fall between 99.475 and 99.525.
MSS
minimum sample size
SD
standard deviation
CI
confidence intervals
or confidence level
P
precision
The spreadsheet embedded below provides a calculator for minimum sample size. Once the user inputs the
standard deviation, confidence level, and precision, the minimum sample size required is provided.
MSS Calculator.xls
Data Requirements to Determine Minimum Sample Size
To calculate the minimum sample size, it is necessary to get the estimated standard deviation of the population
(SD). To do this, collect the job level accuracies for all the jobs audited. Using the latest three (3) months of data
is ideal. Data should be collected only for unbiased samples. Biased samples like focused audits, special audits,
version audits, etc., which concentrate on specific authors, employees, or other specific variables should be
excluded from this data set. Standard deviation can be easily calculated in Microsoft Excel using the formula
“STDEV.”
Caveats
➢ MSS calculation should not be generalized; i.e., MSS calculated for a particular data set or population
should not be used for its subsets because each subset can have a different standard deviation.
o Example: If minimum sample size is calculated for a particular facility, then it should be used
only for that facility and should not be used for sampling a specific author, healthcare
documentation specialist, or department within that facility as the standard deviation can be
different for each.
➢ It is possible that in case of smaller facilities (or data sets) the minimum sample size calculated would
exceed the total number of jobs delivered (or total data points), or the minimum sample size may be too
large for a department or facility to audit. Employ subjective decision-making in such cases, keeping in
mind that the confidence level of such samples will be less than 95%.
➢ Each organization or MTSO will have to determine how they will use the resulting numbers in
performing their retrospective QA reviews, keeping in mind their budgetary constraints.
➢ The recommended number of reports to review may be spread over a period of time that aligns with
your QA staffing capabilities and budget constraints.
➢ Note regarding clinician QA: If no score is assigned to clinician reviews, these sampling guidelines
cannot be used.
Examples for Calculation of Minimum Sample Size by Facility/Client
Consider this scenario: We have two facilities (or MTSO clients) for which we need to determine the minimum
sample size, and we assume the information below is provided data of the job level accuracy of all audited jobs
for the last 3 months. Calculate the standard deviation in Microsoft Excel using the formula “STDEV.” Our
example data set is provided below.
SD
Facility1
99.89
99.56
99.76
99.87
100
100
99.87
99.89
99.6
100
99.84
0.1503
Facility2
99.95
99.61
100
99.45
100
99.95
99.87
99.32
100
100
99.82
0.2443
As given in the recommendations, the confidence level is set at 95% and precision is set at 0.025.
Facility 1: The standard deviation for Facility 1 is 0.1503. Using the calculator, we get the minimum sample size
for Facility 1 as 139 jobs.
Continuous Data
Inputs
Answer
Standard Deviation
0.1503
Confidence Level (e.g. 95%)
95.0%
Precision (e.g., ± 2 units)
0.025
Minimum Sample Size
139
This means for Facility 1 we need to sample at least 139 jobs to have a 95% confidence level that the sample
picked is a good representation of all jobs delivered.
Facility 2: The standard deviation for Facility 2 is 0.2443. Using the calculator, we get the minimum sample size
for Facility 2 as 367 jobs.
Continuous Data
Inputs
Answer
Standard Deviation
0.2443
Confidence Level (e.g. 95%)
95.0%
Precision (e.g., ± 2 units)
0.025
Minimum Sample Size
367
This means for Facility 2 we will need to sample at least 367 jobs to have a 95% confidence level that the sample
picked is a good representation of all jobs delivered. Here, a higher number of jobs need to be sampled for
Facility 2 as the standard deviation for Facility 2 is high. The standard deviation for Facility 2 is high because job
level variation in accuracies is higher for Facility 2 (varies between 99.32 to 100).
Examples on Calculation of Minimum Sample Size by Healthcare Documentation Specialist
Job 1
Job 2
Job 3
Job 4
Job 5
Job 6
Job 7
Job 8
Job 9
Job 10
SD
Employee 1
100
99.75
99.5
99.75
99.75
99.75
100
100
99.75
99.5
0.1845
Employee 2
98.5
98.25
99
98.75
98.25
98.5
99
98.25
97
98.25
0.5683
As given in the recommendations, the confidence level is set at 95% and precision is set at 0.025.
Employee 1: The standard deviation for Employee 1 is 0.1845. Using the calculator, we get the minimum sample
size for Employee 1 as 209 jobs.
Continuous Data
Inputs
Answer
Standard Deviation
0.1845
Confidence Level (e.g. 95%)
95.0%
Precision (e.g., ± 2 units)
0.025
Minimum Sample Size
209
This means for Employee 1 we need to sample at least 209 jobs to have a 95% confidence level that the sample
picked is a good representation of all jobs delivered.
Employee 2: The standard deviation for Employee 2 is 0.5683. Using the calculator, we get the minimum
sample size for Employee 2 as 1985 jobs.
Continuous Data
Inputs
Answer
Standard Deviation
0.5683
Confidence Level (e.g. 95%)
95.0%
Precision (e.g., ± 2 units)
0.025
Minimum Sample Size
1985
This means for Employee 2 we will need to sample at least 1985 jobs to have a 95% confidence level that the
sample picked is a good representation of all jobs delivered. Here, a higher number of jobs need to be sampled
for Employee 2 as the standard deviation for Employee 2 is much higher compared to Employee 1. The standard
deviation for Employee 2 is high because job level variation in accuracies is higher for Employee 2 (varies
between 97 to 99).
NOTE: If the sample size is very high and impractical, the confidence level could be lowered to achieve a more
practical sample size. In the Employee 2 example, if the confidence level were changed to 80%, the minimum
sample size would be reduced to 849.
Glossary Quick Reference
Population
Total number/the set of all jobs/reports measured
Standard Deviation (SD)
A statistic that indicates how tightly the data points are clustered around a
mean for a given process, which in turn indicates how much variation exists
Confidence Interval (CI)
or
Confidence Level (CL)
Refers to the confidence interval which determines the probability that the
sample will represent the population. Also referred to as margin of error.
Precision (P)
Refers to the accuracy level, like decimal points, that we would want when
determining the quality of the population
APPENDIX E: Glossary
Admission, Discharge, Transfer
(ADT) feed
An electronically generated list of patients and their corresponding
demographic information, typically used by the healthcare
documentation specialist to properly identify transcribed reports.
Auditor
A qualified and trained higher-level HDS who reviews the work of HDS
staff and/or clinician created documentation for essential quality
components as deemed necessary by a facility, organization, or business.
This work may be done with or without voice files.
Author
An individual who creates a sound file to be converted to text or who
generates a document using a variety of input methods, such as direct
computer entry (EHR) or front-end speech recognition. This individual
may also be referred to as a dictator, originator, clinician, or provider.
Authentication/Authenticator
Refers to the process by which the provider verifies what has been
captured in the record and affixes their signature to the report as proof
of that verification. According to The Joint Commission, authentication
must be done by the author of the record and cannot be delegated to
anyone else, regardless of the process for inclusion of signature.
Note: The dictator may not be the same as the authenticator, as ancillary
personnel may be employed to assist in dictation and information
capture.
CHDS
Abbreviation for Certified Healthcare Documentation Specialist.
CMT
Abbreviation for Certified Medical Transcriptionist.
Concurrent review
An audit of a document that occurs before the document is
authenticated.
Demographics
Information pertaining to the patient, such as name, date of birth,
medical record number, and encounter number.
Dictator
See Author.
Note: The dictator may not be the same as the authenticator, as ancillary
personal may be employed to assist in dictation and information
capture.
Facility
A hospital, clinic, physician practice, outpatient surgery center, dental
practice, long-term care or skilled nursing facility, birthing center, or
other organization that provides healthcare services. Other examples
include physical and occupational rehabilitation centers and dialysis
centers.
HDS Level 1
The healthcare documentation specialist, level 1, transcribes and/or
edits basic patient healthcare documentation dictated by physicians and
other healthcare practitioners. Level 1 individuals possess basic or entry-
level knowledge with little to no transcription or editing experience.
Nature of work performed would start at entry level and increase as
depth and breadth of knowledge, exposure to specialties, and dictators
and/or types of documentation can be produced while meeting
departmental quality and production expectations.
HDS Level 2
The healthcare documentation specialist, level 2, transcribes and/or
edits patient healthcare documentation dictated by physicians and other
healthcare practitioners. Level 2 individuals possess proficient
knowledge within certain areas of expertise and can meet departmental
expectations. Nature of work performed is for a specific medical
specialty or at a community hospital level with limited dictators and/or
types of documentation produced. AHDI certification is preferred (RHDS,
CMT, or CHDS).
HDS Level 3
The healthcare documentation specialist, level 3, transcribes and/or
edits patient healthcare documentation dictated by physicians and other
healthcare practitioners. Level 3 individuals possess proficient
knowledge in the field of healthcare documentation. Nature of work
performed crosses all medical specialties in a large acute care setting.
Individuals may perform QA tasks, mentor peers, and/or assist with
projects. AHDI certification is preferred (RHDS, CMT, or CHDS).
Macros
A single instruction that expands automatically into a set of instructions
to perform a particular task.
Medical specialty
In this context, a distinct field of study, such as cardiology, orthopedics,
gynecology, or psychology.
MTSO
Medical Transcription Service Organization
Normals
A term used to describe a shortcut for inserting standard text. Authors
may request the insertion of a specified standard text in lieu of
repeatedly dictating the same information.
May also be referred to as “standards” and “routines.”
Healthcare Documentation
Specialist (HDS)
An individual who transcribes traditional dictation by physicians and
other healthcare providers to document patient care. May also edit draft
text created by speech recognition software.
Originator
See Author.
Retrospective review
An audit of a document that occurs after the document has been
delivered to the client or the chart.
RHDS
Abbreviation for Registered Healthcare Documentation Specialist.
Account Specifications
Documentation describing a facility or client’s unique requirements and
preferences including technical data and issues of style. May also be
referred to as a Style Guide for facilities.
Speech Recognition
Computer technology that enables a device to recognize and understand
spoken words, by digitizing the sound and matching its pattern against
the stored patterns.
Template
A standardized layout for a given report type. A template may include
placement markers for patient demographic information as well as
formatted headings, subheadings and signature blocks. A clinician EHR
template may also draw in discrete data from within the patient’s
medical record to enhance the final document.
Turnaround time (TAT)
1. The interval of time measured from the time of document
completion by an author in the EHR to the time of authentication.
2. For transcribed/edited documentation, TAT is the interval of time
from completion of dictation to completion of document and filing
of that document to the EHR.
3. For an MTSO, interval of time measured from the time the dictation
is received until the document is finalized in the transcription
process.
See also Career Map Abbreviations.
REFERENCES
Fox-Acosta, K. “Health Information Documentation: Who Owns the Quality of the Information?” AHDI: Modesto,
CA. Plexus, October 2013.
Fox-Acosta, K. “Quality Medical Reports in Today’s World.” AHDI: Modesto, CA. Plexus, Sept/Oct 2014.
Doggett, S. and Smith, J. “Can Clinician-Created Documentation Work?” AHDI: Modesto, CA. Plexus, Sept/Oct
2014.
Zhou, Li. “Improving Health IT Safety Through the Use of Natural Language Processing to Improve Accuracy of
EHR Documentation.” AHRQ National Web Conference, AFYA, Inc., 7 February 2017, Online
Presentation.
