Evaluation Process

Supervision shall be a continuous process that occurs from the first day of the rotation through the last day.  Feedback from the supervisor to the intern regarding progress toward rotation training goals and professional development should occur on a regular and frequent basis.  Feedback from the intern to the supervisor regarding the intern’s training needs and experiences should also be frequent and regular.  Given the frequency of observational feedback and supervision, it is our goal that the formal feedback sessions do not contain any "surprises" for the intern.

The Psychology Internship Program continually assesses each intern’s performance and conduct.  Two formal evaluations will take place during the Core rotation:  a) a verbal mid-rotation evaluation session approximately mid-way through the Core rotation to determine the progress made toward intern training goals and to determine possible changes or alterations in the rotation to maximize the intern’s training; and b) the final written rotation evaluation, occurring during the last week of the rotation.  Three formal evaluations take place during the year-long Adjunct rotation: a) written evaluations at the fourth-month mark and the eighth-month mark to determine the progress made toward intern training goals and to determine possible changes or alterations in the rotation to maximize the intern’s training; and b) the final written rotation evaluation, occurring during the last week of the rotation.  When a written evaluation has been completed, verbal feedback to the intern from the supervisor should occur after the intern has read the formal written evaluation by the supervisor. 

During the intern orientation, Consortium expectations and goals for the intern training year will be presented to the new interns.  This will include discussions of the timeframe for evaluations, content of evaluations and review of evaluation forms.  Methods of assisting an intern who may not be making satisfactory progress will also be discussed.  Psychology Interns are informed that they are required to act in accordance with the American Psychological Association's ethical principles and standards for providers of psychological services and according to state law. Violations of these principles and standards may constitute grounds for dismissal from the program.
 
The Training Director and the Training Committee meet monthly and, as a group, regularly review the evaluation data for each intern.  The group meeting is chaired by the Training Director.  In collaboration with the group, the Training Director combines the evaluations and provides the interns with a verbal summary of their progress in the program.

Inadequate Performance

To maintain in good standing in the internship, interns must meet competency expectations, assessed with the use of the Evaluation Form.  Specifically, interns must be rated at a level of I (Intermediate) or higher on all written evaluations (see time points above) across all competency areas; No competency areas will be rated as E (Entry) or R (Remedial).  At the end of the training year, at least 50% of competency areas must be rated at a level of competence of HI (High Intermediate) or higher; No competency areas will be rated as E (Entry) or R (Remedial).  These performance levels are designed to assure that interns completing the program are prepared to function as qualified and competent entry-level practitioners who can provide an array of psychological services in a variety of settings.  If an intern’s performance falls below competency standards, performance improvement and due process procedures are followed, as outlined below.

Interventions by Level of Deficiency

Entry Level Skill.  If an intern receives an E (Entry) on a written evaluation, the supervisor is expected to discuss this with the intern, increase the intern's supervision and direct the intern to other appropriate resources to address the deficit area (e.g., assign readings).  The supervisor will keep a written record of the discussion with the intern and corrective steps agreed upon, will monitor the intern’s skill development, and will be provide biweekly feedback to the supervisee regarding progress.  Concurrently, the supervisor will notify the Training Director of the concern, consult the Training Director and Training Committee for suggestions regarding remediation, and provide monthly feedback to the Training Director regarding progress on the skill development.  As the skills of the intern are assessed from the beginning of a rotation, deficiencies are typically identified early on and discussed at the mid-rotation evaluation feedback meeting.  Indeed, if the supervisor believes an intern is deficient in any domain assessed by the Evaluation Form, they are expected to complete a written evaluation of the intern at the mid-rotation point, rather than verbal feedback alone.  If the deficiency is identified toward the end of a rotation, the plan to improve the intern’s skills is shared with the supervisor of the next rotation who will monitor the skill until it reaches a satisfactory level.  If the intern does not improve by the next evaluation point (e.g., if the deficit is identified at the mid-rotation point, it must improve to a satisfactory level by the end-of-rotation evaluation point), a formal remediation plan is developed (see Formal Remediation Plan: Processes and Procedures). 

Remedial Level Skill.  If an intern receives an R (Remedial) on a written evaluation, a remediation plan is required (see Formal Remediation Plan: Processes and Procedures).  The supervisor is expected to discuss his/her concerns with the intern, communicate his/her concerns with the Training Director, consult with the Training Committee for suggestions regarding remediation and develop a remediation plan to address the deficiencies.  In addition, the Training Director will consult with the intern’s graduate program Director of Clinical Training regarding the intern’s performance in their academic program and request suggestions for remediation.  The intern’s graduate program DCT is provided with a copy of the plan.  The intern will be notified, in writing, if any formal review is occurring and the Training Committee will receive any information or statement from the intern related to his/her response to the rating. 

If there is a substantial knowledge deficit in a competency area central to providing professional psychological services, the Training Committee will decide if the student is also displaying Problematic Performance (see steps for identifying and addressing Problematic Performance below). 

Formal Remediation Plan: Processes and Procedures

Once a formal remediation plan is necessary, the following steps will be followed:

1. A remediation plan will be developed.  This is a time-limited, remediation-oriented supervised period of training. It is designed to return the intern to an appropriate functioning level with the full expectation that the intern will complete the internship. Each remediation plan will include the following:
  a. A description of the intern's unsatisfactory performance
  b. Recommended actions needed from the intern to correct the identified problems
  c. Supportive intervention/modifications made to the intern's training program (e.g., increase supervision with the same or other supervisors, change focus or format of supervision, require coursework or readings, reduce caseload and recommend personal therapy)
  d. A time line for correcting the problem
  e. The action to be taken if the problem is not corrected

2.   If the remediation plan developed in Step 1 is unsuccessful in addressing the problematic performance and/or conduct within the timeframe identified, the Training Director will meet with the Training Committee to discuss further courses of action. These may include one of the following sanctions or actions:

a. Modified Remediation Plan -- It may be determined that continuing the remediation plan with specific modification is the most appropriate intervention (repeat Step 1). When the problem is considered severe, an intern may be required to complete a remediation plan and concurrently placed on probation.

b. Probation -- A probationary period would involve close supervision with active involvement from both the immediate supervisor(s) and Director of Training. All details, requirements, and expectations of the probation period will be in writing. The supervisor(s) and the Training Committee will meet a minimum of once a month during the probationary time to monitor the intern’s progress in addition to the intern’s weekly supervision. It is the decision of the Training Committee as to whether or not the intern, during this time, should continue providing direct services to patients. The Training Director will discuss the intern’s status with the graduate program Director of Clinical Training.

Probation is time limited and remediation-oriented. During this closely supervised training period, the Training Director and supervisor monitor the degree to which the intern addresses, changes, and/or otherwise improves the problem behaviors. During the probation period, the intern may be suspended from engaging in certain professional activities until there is evidence that the problem behaviors have been rectified.

The intern will be given written notice of the probation that includes the following information:
i. Description of the problematic performance and/or conduct, including specific incidents or complaints
ii. Specific recommendations for rectifying the problems
iii. The length of the probation period, during which the problem is expected to be rectified
iv. Procedures to ascertain whether the problem has been appropriately rectified.
 
3.   If the interventions outlined in Step 2 have been unsuccessful in addressing the skill deficits within the timeframe identified, the Training Director will meet with the Training Committee to discuss further courses of action, which may include:

i.   Continuation of the probation for a specific time period
ii.  Suspension whereby the intern is not allowed to continue to engage in certain professional activities until the skill deficit in question has improved
iii.  Inform the Intern that the Training Committee is recommending to the Training Director that the intern will not, if the behavior does not change,     successfully complete the internship.
iv. Termination-- Dismissal from the internship may occur if probationary attempts are deemed to have little or no behavioral impact or there are APA ethical violations and/or state legal violations. This action is decided by the Training Director and Training Committee and will also involve consultation with UMSL’s HR representative. The intern will be notified, if appropriate, in person and provided with a written letter of the decision to dismiss. The Training Director will report the decision to the intern’s graduate program training director. APA will also be informed in writing of this action. The intern may appeal the decision to terminate by writing a letter to the Training Director requesting an informal hearing.

Due Process

All of the above steps must be appropriately documented and implemented in ways that are consistent with due process procedures. The Internship Training Director will communicate with the graduate program in a timely manner when problems arise with an intern that are not readily resolvable at the internship site, that are recurrent, or that may lead to the institution of due process procedures or alteration in the intern’s program.  Ongoing communication will be maintained until the problem is resolved.  All formal actions taken by the program shall be communicated in writing to the intern and the intern's graduate department.  The nature and rationale of the decision and remediation procedures shall be indicated.  The intern shall receive copies of all formal communications regarding his or her performance.

The intern has the option to file a written grievance with the University.  Please see the Collected Rules and Regulations regarding the University’s formal grievance procedures.

Problematic Performance

It is possible that intern performance deficiencies may be classified as "problematic performance."

Definition

Problematic performance in professional functioning that is reflected in one or more of the following ways:
a)  an inability and/or unwillingness to acquire and integrate professional standards into    
     one's repertoire of professional behavior;
b)  an inability and/or unwillingness to acquire professional skills in order to reach an
     acceptable level of competence as defined by prevailing standards of care; or
c)  an inability and/or unwillingness to control personal distress, psychological
     dysfunction, and/or excessive emotional reactions that interfere with professional
     functioning.

The above concerns typically become identified as “problems” when they include one or more of the following characteristics:

a)  the intern does not acknowledge, understand, or address the problem when it is identified,
b)  the problem is not merely a reflection of a skill deficit which can be rectified by academic or didactic training,
c)  the quality of services delivered by the intern is sufficiently negatively affected,
d)  the problem is not restricted to one area of professional functioning,
e)  a disproportionate amount of attention by training personnel is required,
f)  the trainee's behavior does not change as a function of feedback, remediation efforts, and/or time,
g)  the problematic behavior has potential for ethical or legal ramifications if not addressed,
h)  the intern's behavior negatively impacts the public view of the agency,

Procedure

I.  Identification and Remediation of Problematic Performance

The Training Committee in executive session (without the intern representative) will review intern evaluations after each rotation in the regularly scheduled meeting.  If an inadequacy is identified in at least one major area (professional standards, professional skills, personal functioning) during this review, or if a supervisor requests immediate action to be taken (by the Training Director or site Training Coordinator) to address what is believed to be problematic performance, the following actions will be taken:

1)  the Training Committee shall meet within ten working days in a special session to decide on a course of action.  At least seven working days prior to the meeting, the intern shall be informed that the meeting will take place and offered the opportunity to provide the committee with relevant information regarding his or her response to the noted problematic performance.

2)  The Committee shall determine by 4/5 majority (with the Training Director voting) whether the intern meets the criteria for problematic performance:

      a)  If the intern is not found to demonstrate problematic performance:

             i)  No further action will be taken, or
             ii) If deficiencies exist that do not meet the criteria for problematic performance, the  Committee may notify current supervisor(s) that active    monitoring in addition to the regular evaluation process is required in the specific problem area identified with specific time limits to the monitoring.

      b)  If the intern is found to demonstrate problematic performance, the following action may be taken: 

            i)  A remediation plan shall be developed and implemented with a specified time frame for successful completion, or
  
            ii)  the intern may be temporarily suspended from his or her duties so that intervention beyond the scope of the internship may be pursued. 

3)  After the specified time period of probation, the Training Committee will determine if the intern has successfully completed the remediation program.


       a) If the intern has successfully completed the remediation program, the probation will be lifted and no further action taken.

       b) If the intern has not successfully completed the remediation program, the following actions may occur:

            i)  continuation of remediation plan with possible revision and specified time limits;

            ii)  temporary suspension from duties during which intervention beyond the  scope of the internship may be pursued. 

II.  Implementing Decisions.

If the intern is found to demonstrate problematic performance that is not resolved through remediation, the Internship Director shall take under the advisement the recommendation of the Training Committee and will follow the University of Missouri System’s Collected Rules and Regulations regarding employee discipline and possible termination. 

III.  Intern Appeal.

The intern has the option to file a written grievance with the University.  Please see the Collected Rules and Regulations regarding the University’s formal grievance procedures.

IV.  Communication with Graduate School.

The Internship Training Director will communicate with the graduate program in a timely manner when problems arise with an intern that are not readily resolvable at the internship site, that are recurrent, or that may lead to the institution of due process procedures or alteration in the intern’s program.  Ongoing communication will be maintained until the problem is resolved.  All formal actions taken by the program shall be communicated in writing to the intern and the intern's graduate department. The nature and rationale of the decision and remediation procedures shall be indicated.  The intern shall receive copies of all formal communications regarding his or her performance.