TSA Tools (click here)
Measuring proficiency of Technical Skill
Attainment (TSA) is a critical measure of Career and Technical Education program effectiveness. (Is the
cte program meeting the industry's needs for future employees/professionals?)
Advisory Committee's Role?
1. An Advisory
Committee (local, regional, or community college APPROVES the Technical Skill Assessment
that will be used. 2. The Advisory Committee APPROVES the LEVEL
which is designated as PROFICIENT. 3. The Advisory Committee serves as a 3rd PARTY
in approving the VALIDITY and RELIABILITY of assessments used.
Proficient
Level? (What LEVEL of skill performance is PROFICIENT?)
A standardized test or paper/pencil exam is a poor
measure of TSA because skill attainment implies authentic assessment of skills acquired and performed by students. Advisory committee members serve as industry-experts
to APPROVE the level of performance which is locally deemed as PROFICIENT. For high school CTE program, PERFECTION is
NOT realistic. Based upon comments I have heard from agricultue industry reps. they desire to hire
employees that have had sufficient exposure to industry skills and the ability to learn more. Based upon industry
comments, PROFICIENT should probably be a level at which students are PREPARED
TO LEARN MORE, without signficant remediation in the basics..
Crude Example: Depending upon
the grading system, a student that recieved an A in a welding class may be less proficient in the act of welding than
a student that recieved a D in a welding class. Authentic assessment is the key. Sometimes, hands-on learners
do not perform well on paper/pencil assignments, but can weld or replace a head-gasket.
Grades? Can we use grades to determine proficiency? YES...IF...AUTHENTIC
ASSESSMENT is part of the grade. For many years the answer has been ,"No, you may NOT use grades to determine
proficiency." The answer is still NO , UNLESS the grades include
a significant number of scores from AUTHENTIC ASSESSMENT(S). (Authentic Assessment Example= Measuring
a student's ability to weld a butt-weld by welding a butt-weld...RATHER THAN...only passing a paper/pencil test about
butt-welding.)
Valid and Reliable?
Valid?
Knowledge and skills identified by
industry experts as important to know to be successful in the business or industry. Some knowledge and
skills are NEED-TO-KNOW while some others are NICE-TO-KNOW. The Advisory
Committee would establish the NEED-TO-KNOWS that will be assessed for ALL students. (aka. Critical
Competencies) Low Validity Example: Some Agriculture careers require strong communication skills, therefore
high performing students (those with grades of B or above) will be assessed to determine their level of proficiency in communication.
High Validity Example: ALL Agricultural Careers require strong communication skills, therefore ALL students will
be assessed to determine their communication skills. An Advisory Committee-Approved RUBRIC will be used to assess communication
skills performed during a group activity. (i.e., teams of students will build picnic tables and compete against each
other)
Reliable? The
assessment can be measured the same way for each student, and the assessment measure the SAME knowledge and skills.
Students perform the same knowledge and skills in the same setting. Low Reliability Example: Some students conduct butt-welds with
a stick-welder, while other students perform a butt-weld with a wire-feed welder. Random scores are assigned to welds
using nothing other than personal opinion. A standard scoring Rubric is NOT utilized.
High Reliability Example: All students perform a butt-weld using identical dimensions of steel
pieces with the same model of welder. A scoring Rubric is used to score each weld, and decrease subjectivity
of assessment. BEFORE the assessment, all students will have received the scoring Rubric that will be used to score
their welds.
Evaluated for Proficiency? Students that were scored/graded on thier understanding/performance of critical competencies.
(Hint: 100% of students that completed 1.5 units within the same cte program sequence should have been Evaluated to
determine proficiency.)
Teacher Examples of TSA Assessments: (examples
that make sense and are not costly or difficult)
1. Agricultural shop class: The
course has a major emphasis is on communication skills, team work, and working to produce a quality product in
a safe and efficient manner. The students worked in equal sized groups to build a project which was scored using a communication
skills rubric. (75 minute test period.) 2. Food Science Class: Individually
students complete a series of activities to assess their competency levels. (a)
general knowledge test, (b) a product development presentation, (c) an aroma's test,
(D) a triangle test to identify which product is different than the others, and (E) a food
safety and sanitation component. 3. Horticulture Class: Individually the students will be take a three part assessment.
(a) a general knowledge exam, (b) design a landscape plan, and (C)
evaluate a design plan.
1. Agricultural shop class: The course has a major emphasis is on communication skills,
team work, and working to produce a quality product in a safe and efficient manner. The students worked in equal sized groups
to build a project which was scored using a communication skills rubric. (75 minute test period.) 2. Food Science Class: Individually students complete a series
of activities to assess their competency levels. (a) general knowledge test, (b)
a product development presentation, (c) an aroma's test, (D) a triangle test to identify
which product is different than the others, and (E) a food safety and sanitation component. 3. Horticulture
Class: Individually the students will be take a three part assessment. (a)
a general knowledge exam, (b) design a landscape plan, and (C) evaluate a design plan.
1.
Agricultural shop class: The course has a major
emphasis is on communication skills, team work, and working to produce a quality product in a safe and efficient manner.
The students worked in equal sized groups to build a project which was scored using a communication skills rubric. (75
minute test period.) 2. Food Science Class: Individually
students complete a series of activities to assess their competency levels. (a)
general knowledge test, (b) a product development presentation, (c) an aroma's test,
(D) a triangle test to identify which product is different than the others, and (E) a food
safety and sanitation component. 3. Horticulture Class: Individually the students will be take a three part assessment.
(a) a general knowledge exam, (b) design a landscape plan, and (C)
evaluate a design plan.
1. Agricultural
shop class: The course has a major emphasis is on communication skills, team work, and working
to produce a quality product in a safe and efficient manner. The students worked in equal sized groups to build a project
which was scored using a communication skills rubric. (75 minute test period.) 2. Food Science Class: Individually students complete a series of activities to
assess their competency levels. (a) general knowledge test, (b) a product
development presentation, (c) an aroma's test, (D) a triangle test to identify which
product is different than the others, and (E) a food safety and sanitation component. 3. Horticulture
Class: Individually the students will be take a three part assessment. (a)
a general knowledge exam, (b) design a landscape plan, and (C) evaluate a design plan.
1.
Agricultural shop class: The course has a major
emphasis is on communication skills, team work, and working to produce a quality product in a safe and efficient manner.
The students worked in equal sized groups to build a project which was scored using a communication skills rubric. (75
minute test period.) 2. Food Science Class: Individually
students complete a series of activities to assess their competency levels. (a)
general knowledge test, (b) a product development presentation, (c) an aroma's test,
(D) a triangle test to identify which product is different than the others, and (E) a food
safety and sanitation component. 3. Horticulture Class: Individually the students will be take a three part assessment.
(a) a general knowledge exam, (b) design a landscape plan, and (C)
evaluate a design plan.
CTE - TO Do List - FY2009 thru FY2012
1.
Program Self-Assement 2. 3rd Party Approval 3. TSA per course 4. POS 5. Relax have fun
1. Conduct a program self-assessment. - - takes approx. 15 minutes Your progam assessment should be utilized in determining legitimate
uses of Perkins dollars. Perkins dollars are not "your" dollars, they are intended to improve cte progams
based upon identified needs.
2. Utilize advisory committee as as
3rd party to APPROVE your local assessment tool(s) for technical skill attainment (TSA).
I have always said, "do not use grades." But if authentic assessment of competency performance
is included in your course grades, it now makes sense to determine TSA with course grades. (Which is probably easier, than
determining TSA for "completers".)
3. Measure and record TSA for each course.
NOT ALL competencies, only the competencies taught in the specific course.
Your local advisory committee approves the level of performance needed to be considered "proficient". TSA
is reported to the state (Feds) for concentrators...students that have completed 1.5 units,in the identified 3.0 units of
ag courses.
4. Work with community college to develop Plans of Study
(aka Programs of Study).
By 2012, 75% of your district's CTE programs must have Plans of Study.
5.
Relax and have fun, fighting the battle against ignorance.
Contrary to what you may have heard the changes in Federal Perkins are minor, not major.
Plans of Study are NOT required until 2012. Some ag programs were doing all of these requirements years ago, so nothing new for some,
other than dcoumenting course plans (Plan of Study) things on paper. If you are already
conducting dual-credit courses, the changes are basically whipped.
Significant Perkins
Changes for FY08 thru FY12
What MUST ALL CTE teachers do?
a. Identify "Occupational/Technical proficiency" a.
(Personally, I would recommend recording "occupational proficiency"
for each student in each course in the sequential program. Each CTE course MUST include competencies. If a student is proficient in 1.5 units of courses,
they ARE occupationally proficient. b. Virtually ALL other student data is collected through
Project EASIER via student files submitted to the state (ie., JMC reports, etc.).
DEFINITIONS:
COMPLETER is NO longer a key measurement tool. It is still used
for some special populations indicators, but it is NOT a key measurement tool for ALL students.
CONCENTRATOR - Concentrator replaces Completer as a key measurement tool.
At the Secondary level, a concentrator is any
student that has completed1.5 units in a single specific CTE program. (ie., ag business, bus, fcs, etc.)
At the Postsecondary level, a concentrator is any student that has 1.) been accepted into a CTE program; 2) completes 12 credits of technical
core in a specific CTE program; or 3.) completes a short-term CTE program of less than 12 credits, that terminates with an
award of an industry-recognized credential, or a certificate.
Occupational Proficiency must be reported on any student that qualifies as a concentrator.
Therefore, it may be wise to quantify occupational proficiency at the completion of each course. Personal "Gruis" recommendation: At the end of each
course, record whether students did or did not complete an acceptable level of course's COMPENTENCIES.
(Remember, according to state code, CTE programs MUST be competency-based.)
The old...90% level of competencies attained... IS GONE. In its place, a level
of successful ATTAINMENT must be a level identified by a third party or your local advisory committee as ACCEPTABLE.
Gruis Editorial: The law uses the word "test"; however, "test" could be a checklist
of competencies or skills that students successfully acquired. I define "successfully acquired" as ... "Students learned enough to
perform the competency at the next level (college or work)." Academic Proficiency (no change): ITED data is automatically collected via Project
EASIER. Career/Technical Program – At the Secondary (high school) level, students must be enrolled in at least
one Career and Technical Education (CTE) program that is comprised of three or more sequential units.
(“Sequential” does NOT infer that the courses must be taken in a specific order. “Sequential”
implies that the CTE courses must build upon the competencies required for occupational success within a specific pathway.)
Local school districts determine which CTE courses make up the program sequence. Arts & sciences or liberal arts courses (ie., math, English, etc.) are NOT
allowed as part of the 3 unit CTE course sequence. Because Iowa has comprehensive high schools, it is assumed that ALL students will complete enough math, science, social
studies, and language arts to graduate; however, these courses are NOT allowed to be identified as part of the 3 CTE unit
minimum for a CTE program. Federal
Perkins Law interpretations- - Welcome to the world of “gray”. Very few expenditures
are identified as “black-and-white” (legal or illegal) in Federal code. ALL legal expenditures do need to
meet the INTENT of Federal law; and therefore, a simple list of legal/ not-legal expenditures is impossible to generate. Title III, Part A, Section 311(a) of the Perkins Act: The section states that "Funds made available under this Act for career and technical education activities shall supplement, not
supplant, non-federal funds expended to carry out career and technical education activities and tech prep program activities."
Section 135(c)(7) Clearly states that funds made available to an eligible recipient may be used
for "leasing, purchasing, upgrading or adapting equipment, including
instructional aids and publications(including support for library resources) designed to strengthen and support academic and
technical skill achievement."
Textbooks, probably would meet the
definition for instructional aids and publications, but the recipient may be supplanting local funds with federal
funds. Thus the expenditure may not be an allowable use of funds under that section of the Act. However,
this would only be a ‘supplant’ violation, if the district is providing similar support for other programs
while withdrawing the support of local funds from the program in question. Supplement vs. Supplant The underlying intent of Federal Perkins law is to improve the
achievement of students in career and technical programs. Using Federal dollars to ‘supplement’ existing
programs is a legal use of Federal funds; using Federal dollars to ‘supplant’
local dollars is NOT legal. (CTE programs are composed of CTE
courses. Academic courses, such as math, English or science, are NOT part of a CTE program.) Text books are one of the "gray" areas in the Federal Perkins law;
there are no "black-and-white" rules on texts. So, here is our (Iowa DE) interpretation of Federal Perkins
law with regard to textbookss. (Just joking…but, if you don’t like our interpretation, please move your
school to another state; although, I can’t think of states that do not follow these same interpretations.) 1.
Consumable items are NOT legal expenditures. One-time-use items
such as student workbooks, welding rods, feed, etc. are NOT legal expenditures. 2. "Supplant" is NOT legal. If a district purchases
text books, with Perkins $, that have normally (in the past) been purchased with other school dollars; this would be considered
"supplanting" and is NOT legal. Example: It is NOT LEGAL to purchase
a course's ‘regular/standard’ textbook with Federal dollars. This is a tough call, because
some/many CTE courses do not have a specific ‘course’ textbook. Courses such as math often have a designated
‘course’ text book. Any texts that could be defined as a ‘course’ textbook are NOT legal expenditures. Crude Example: Standard Operating Procedures (SOP) -- Providing toilet paper is standard practice for operating a school, and would
NOT be an allowable expenditure utilizing Federal dollars. Purchasing any materials or items of this type would be considered
as standard-operating-procedures, and would NOT be allowable. Other examples of Standard-Operating-Procedures:
a. providing desks, b. providing chalk, c. providing a teacher, d. providing a classroom, etc. 3. “Supplement” is LEGAL. Using Federal Dollars to purchase a set of texts which serve as supplemental
resources or
additional resources beyond the regular course textbook is legal. If a specific textbook is the ONLY books used in a course, and the textbook id used daily/frequently throughout the course;
these would probably NOT be legal uses of Federal dollars. Example: If a school purchases GPS units, with Federal Perkins $,
a set of texts or materials purchased to assist students in using the GPS units would be allowable. If these texts
or printed materials are the only texts used in the course, they would NOT be allowable. Essentially, texts that function
as a “course’s” primary text would NOT be allowable. Supplemental materials or texts are allowable.
(Consumable items are NEVER allowable.) 4. Equipment
is a legal expentiture, IF it incorporates NEW
or ENHANCED technology. Examples:
Hammers -No: …New hammer technology… I don’t buy it.
Unless the purchase would be for a few hammers which are designed to meet the needs of students with physical disabilities.
Computers -Yes: Computers are typically out-of-date as soon as they arrive; therefore
computers are almost always allowable. Table Saw -Yes: Table saws
typically have new and improved features, such as safety features. Table
Saw Blades -No: One table saw blade to aid in the training of the new equipment is an acceptable Perkins
expenditure. A pack of 10 extra table-saw blades would not be allowable because saw blades, drill bits, etc.
are consumables.
The full version of the Project EASIER Plus CTE PowerPoint (from the Dec. 2007
ICNs) is linked below.
Project EASIER Plus - CTE Student Data Entry Steps for 2008
Perkins Basic Information
Perkins 101 (Powerpoint file)
All Aspects of an Industry (PDF file)
Project Easier - JMC Example
Occupational Proficiency Record Keeping Example
Iowa Vocational Education Standards?
3 Sequential units of CTE courses
State Reimbursement $
State Board Approval, etc.
Iowa Vocational Education Standards
Other CTE or Perkins Links
CTE Research Elliot & Knight (ppt file)
Office of Vocational and Adult Education (Federal)
National Research Center (CTE)
ACTE (Association for Career and Technical Education)
Vocational Information Center
Oklahoma Department of Career and Technology Education
Michigan Center for Career and Technical Education
Journal of Career and Technical Education
Washington Career and Technical Education
Career Clusters
FFA ONLY! Click on the link below to download the Perkins/FFA Report Template. This Annual
Report collects ALL data that is needed for your school's state CTE report via Project EASIER. (I suggest that you share the completed Annual FFA report with the person that completes your school's CTE report
data in Project EASIER.)
FY08 Annual FFA Report
|