User Rating: / 0
PoorBest 
os catalogue december 2013 - os catalogue december 2013

 

Assessment Tool Development Manual

 

 

I. Introduction

In the National TVET Strategy approved by the Council of Ministers in 2007, section---- outlines principles regarding occupational assessment and certification. This manual as a mandatory directive refers to this chapter, which guide the user on how to develop assessment item/tool of a particular occupation to be able to implement occupational assessment and certification.

The understanding of the Occupational Standards and its unit/s of competence as well as the different parts of each unit of competence would be very useful in developing an assessment item/tool. The manual also provides information on how the knowledge/written test and practical assessment tools should be developed based on a prepared evidence plan of an occupation and selected methods of gathering evidences.

The knowledge/written and practical assessment item/tool constitute an assessment package of which the candidate for assessment must be able to demonstrate his/her competence. The assessment packages are explained in this guide that refer to the Assessor’s Instructions package and the Candidate Instructions and assessment package.

The assessment is being conducted by a certified and accredited assessor in an accredited assessment center/venue duly approved and recognized by the CoC in consultation with assessor’s panel. The conduct of assessment is administered and supervises by the Center of competence in every region. The candidate who has proven competence in all the units of competence of an occupation through occupational assessment shall be awarded a National Qualification Certificate (NQC) and a Competence Certificate (CC) for only a unit or cluster of competence.

II. Definition of Terms

Assessment

Assessment means the process of collecting evidence and making judgments on whether competency has been achieved to confirm that an individual can perform to the standard expected in the workplace as expressed in the relevant endorsed industry/enterprise competency standards or the learning outcomes of an accredited course.

Assessment context

The environment in which the assessment will be carried out. This will include physical and operational factors, the assessment system within which assessment is carried out, opportunities for gathering evidence in a number of situations, the purpose of the assessment, who carries out the assessment and the period of time during which it takes place.

Assessment tool

Assessment Tools are the instruments and procedures used to gather and interpret evidence of competence. They serve the evidence-gatherer’s needs for objectivity and transparency and the candidate’s need for clarity and structure.

Ø Instrument -The activity or specific questions used to assess competence by the assessment method selected.

Ø Procedure- The information or instruction given to the candidate and the assessor about how the assessment is to be conducted and recorded.

Ø instrument(s) – the specific questions or activity developed from the selected assessment method(s) to be used for the assessment. (A profile of acceptable performance and the decision making rules for the assessor may also be included.)

Candidate

A candidate is any person presenting for assessment. The candidate may be:

· a learner undertaking training in an institutional setting

· a learner/worker undertaking training in a workplace

· a learner/worker wanting their skills recognized

· or any combination of the above.

Competency

The specification of knowledge, skill and attitude and the application of them to the standards of performance required in the workplace.

Competency based assessment

A form of standards-based assessment and involves comparing a person’s performance against an expected level of performance, known as a standard. The person must prove they know and/or can do what is required in the standard.  Judgment outcomes are “Competent or “Not Yet Competent”.

Dimensions of competency

The concept of competency includes all aspects of work performance and not only narrow task skills. The four dimensions of competency are:

· task skills

· task management skills

· contingency management skills

· job/role environment skills

Direct evidence - Evidence that can be observed or witnessed by the assessor.

This may include:

· observations by the assessor of the candidate carrying out work functions according to the occupational standards

· oral questioning of the candidate

· demonstration of specific competences needed to complete the work function

Element

An element is the basic building block of the unit of competency. Elements describe the tasks that make up the broader function or job, described by the unit.

Evidence

Information gathered and matched against a unit of competency to provide proof of competency.

Evidence guide

Designed to assist with the interpretation and assessment of the unit and outlines the key things to look for during assessment (critical aspects of competence), underpinning knowledge and its application (essential knowledge) and suggestions about which units may be assessed together (interdependent assessment of units). The evidence guide also suggests possible assessment strategies (assessment methods and resources) and describes the generic competencies that underpin effective workplace practice (key competencies).

Evidence Plan

Evidence plan is a planning tool.  It is the first step in developing an evidence gathering tool.  The evidence plan is a list of the evidence that the assessor must gather when conducting an assessment of a particular unit of competency.  It also identifies the evidence gathering methods that will be used.

Indirect evidence

Evidence of the candidate’s work results and performance that can be reviewed or examined by the assessor. This may include:

· assessment of technical qualities of finished product

· written test of underpinning knowledge

· review of previous work performed

· testimonials from employers

· reports from colleagues, clients and/or supervisors

· work diary / journal

· evidence of training

· examples of reports or work documents

Moderation

Moderation is a process which involves assessors in discussing and reaching agreement about assessment processes and outcomes in a particular industry or industry sector. This enables assessors to develop a shared understanding of the requirements of specific assessment packages, including the relevant competency standards and assessment guidelines, the nature of evidence, how evidence is collected and the basis on which assessment decisions are made.

Observation

A method which enables the assessor to observe directly what candidates can do.

Occupational Competence

Occupational competence means that a person is able to apply his/her knowledge, skills and attitude required to successfully complete work functions in the workplace in a range of situations and environments, in accordance with the occupational standard.

Occupational Standard

A standard defined by experts of an industry sector indicating the competencies and their respective performance criteria that are required, for a worker to perform the various tasks of the occupation in industry. Occupational assessment is solely based on the expected outcome – competence required to perform work functions at the workplace – as defined in Occupational Standards.

On-site assessment

A kind of assessment in which a worker in industry is assessed by methods of observation and product/ outcome evaluation, as he/she is actually performing his/her job. Only the competencies to be assessed which are components of his job are being assessed.

Performance criteria

Evaluative statements which specify what is to be assessed and the required level of performance. The performance criteria specify the activities, skills, knowledge and attitudes that provide evidence of competent performance for each element.

Portfolio

An assessment instrument that includes a collection of different types of evidence relating to the work being assessed. It can include a variety of work samples. Here, it is important that the evidence meet the requirements of sufficiency and currency.

Practical project

This is an activity that allows the candidate to demonstrate manual and/ or behavioral skills. The assessment here may be based on the end-result of the activity (the product or outcome), or the performance of the task (the process), or a combination of both. This is used where ‘Demonstration of practical Skill is required for the evidence.

Questioning

It is a common assessment technique, which is used in a number of situations. It does not involve a large investment of time, and responses to provide useful evidence of a candidate’s vocational/technical knowledge, understanding of workplace procedures, legislation and safety requirements

Qualification

In the vocational education and training sector, qualification refers to the formal certification issued by a CoC for a person who has achieved all the requirements for a qualification as specified in the occupational standard.

Range statement

Provides a range of possible workplace contexts, tasks or enterprise specific requirements that may be relevant. It gives information that will help to design the focus of the assessment.

Recognition of Prior Learning and Experience

This can be resorted to in the assessments of the core assessors of an industry sector, where long-experienced practitioners, especially supervisors and managers are considered.

Role Play

A common type of practical assessment tool that the candidate is presented with a situation, often a problem or an incident, to which he has to respond by assuming a particular role.

Simulation

Simulation is a form of evidence gathering that involves the candidate in completing or dealing with a task, activity or problem in an off-the-job situation that replicates the workplace context.  This is commonly observed in the assessments of candidates in the health sector, like nurses, caregivers, mid wives, emergency medical technicians or paramedics. Here the assessment task uses a mannequin in place of a real person.

Unit of competency

Unit of competency means the specification of knowledge and skill and the application of that knowledge and skill to the standard of performance expected in the workplace. Essential information needed to conduct occupational assessment is defined in the unit of competence.

Validation

Validation involves reviewing, comparing and evaluating assessment processes, tools and evidence contributing to judgments made by a range of assessors against the same standards. Validation strategies may be internal processes with stakeholder involvement or external validations with other providers and/or stakeholders.

III. Guiding Principles for the Development of Assessment Tools

3.1 Assessment Principles

When developing assessment tools, you need to ensure that the principles of assessment are met.  The assessment principles require that assessment is valid, reliable, flexible and fair.

· Validity refers to the extent to which the interpretation and use of an assessment outcome can be supported by evidence. An assessment is valid if the assessment methods and materials reflect the elements, performance criteria and critical aspects of evidence in the evidence guide of the unit(s) of competency, and if the assessment outcome is fully supported by the evidence gathered.

· Reliability refers to the degree of consistency and accuracy of the assessment outcomes. That is, the extent to which the assessment will provide similar outcomes for candidates with equal competence at different times or places, regardless of the assessor conducting the assessment.

· Flexibility refers to the opportunity for a candidate to negotiate certain aspects of their assessment (for example, timing) with their assessor. All candidates should be fully informed (for example, through an Assessment Plan) of the purpose of assessment, the assessment criteria, methods and tools used, and the context and timing of the assessment.

· Fair assessment does not disadvantage particular candidates or groups of candidates. This may mean that assessment methods are adjusted for particular candidates (such as people with disabilities or cultural differences) to ensure that the method does not disadvantage them because of their situation. An assessment should not place unnecessary demands on candidates that may prevent a candidate from demonstrating competence (for example, an assessment should not demand a higher level of English language or literacy than that which is required to perform to the workplace standard outlined in the competencies being assessed).

3.2 The Rules of Evidence

Well-designed assessment tools must comply with the rules of evidence, i.e. the tool must facilitate the gathering of evidence that is:

· valid (covers all requirements of the unit of competency)

· sufficient (enables the assessor to make a decision about competency over time and in different situations)

· current (competent performance is contemporary)

· authentic (is the candidate’s own work)

IV. Occupational Competence

4.1 Competency

People are considered to be competent when they are able to apply their knowledge, work values and skills to successfully complete work activities in a range of situations and environments, in accordance with the standards of performance expected in the workplace.

This view of competency:

· emphasizes outcomes

· focuses on what is expected of an employee in the workplace

· highlights the application of skills and knowledge to workplace tasks

· focuses on what people are able to do and the ability to do this in a range of contexts

Competency involves successful work performance.  As such it is usually seen to comprise four dimensions, namely:

· Task skills – this involves undertaking a specific workplace function[s].

· Task management skills – this involves managing a number of different tasks to complete a whole work activity.

· Contingency management skills – this involves responding to problems and irregularities when undertaking a work activity. This may involve dealing with:

Ø breakdowns

Ø changes in routine

Ø unexpected or atypical results or outcomes

Ø difficult or dissatisfied clients

· Job/role environment skills – this involves dealing with the responsibilities and expectations of the work environment when undertaking a work activity. This may involve:

Ø working with others

Ø interacting with clients and suppliers

Ø complying with standards operating procedures

Ø observing enterprise policy and procedures

4.2 Evidence Plan

This forms part of the documented assessment plan and details the evidence requirements of the assessment, information regarding who will collect the evidence and the time period involved.

The purpose of evidence plan is to:-

· understand what is expected from the candidate

· focus for the assessor in evidence gathering

· record assessment outcomes

· identify what action is needed to address skill and knowledge gaps

The key steps in constructing an evidence plan are:

· Select and read full unit of competency to be assessed

· Identify the evidence that a person who can competently perform the work activity should be able to provide

· Develop a list of the evidence that the person should be able to provide

· Identify the best way of collecting the evidence listed in the evidence plan

· Indicate which tools will be used to gather each item of evidence

· Group the various units of competence that can be included in one project or task, and place this information in a table of evidence plan

· Document evidence plan.

· Retain the Evidence Plan for use in constructing the evidence gathering tools

4.3 Evidence Gathering Methods

Evidence gathering methods describe the approach that is used by the assessor for collecting the evidence required for assessment. It includes:

a) Direct observation

b) Review a product/outcome

c) Questioning

a) Direct observation

· This is observing the candidate while he or she is carrying out tasks, real (i.e. in the actual workplace) or simulated (i.e. performing tasks/ projects in an Assessment Center which is reflective of an actual work situation).

· The following are covered during the observation activity:

Ø The procedure / work flow in performing the activity/ project

Ø The manual skills / techniques in accordance with the performance criteria

Ø The OHS requirements in the performance of the tasks

Ø The proper use of tools and equipment

Ø The economy of materials employed

Ø The practice of 5S in the performance of the tasks, etc

b) Review a product/outcome

· Reviewing the product/ outcome produced by the candidate after the task has been completed and measuring this against the specifications of the project or standard outcome requirements, taking into account tolerances allowed.

c) Questioning

· Asking questions orally or in writing which are answered orally or in writing. The questions could relate to the observation or to the product. This is done to check the understanding of the candidates’ why certain activities were carried out or test their ability to work within contexts required in the range statements or in other contingencies suggested by the assessment criteria.


V. Development of Assessment Tools

5.1 Develop Commonly used Knowledge Assessment Tools

Knowledge assessment tools may be in the form of written or oral question / structured interview. Both written items in the form of multiple choice and matching type questions and oral question/interview will have a passing score of 50%.

5.1.1 Developing Good Written Items

For development of written items a distinction can be made between so-called “general guidelines” which have to be applied to all types of items for written questions and guidelines specific to each type of written question item. These guidelines may have a dual purpose: They can guide item development teams when designing written items and they can be used as checklists for evaluating existing items (= quality assurance of item development).

i. General Hints

· An important aspect of the content area must be addressed

· Item must be clear, concise and complete

· Potentially insensitive content or language must be avoided

· Content should be accurate and up-to-date

· No clues should be given within one item, or to other items

· Item must be at the appropriate competence level

· Objective answers should be sought

· Purposefully misleading of examinees to answer incorrectly must be avoided

· Negative questions must be avoided

· The vocabulary level should be simple

· Correct grammar, punctuation and spelling must be used

· Only universal abbreviations should be used

· All symbols used must be defined and given their full term.

ii. Hints for Multiple Choice Items

· There are twenty one [21] questions in this section.

· The candidate selects the best answer for each question. He/she answers by writing the letter that corresponds to the answer in the answer sheet provided.

· If s/he wishes to change an answer, he/she places an X mark through the letter he has written and then write the letter of the preferred answer.

· A correct answer scores 1, an incorrect answer scores 0. No marks will be given for a question if more than one answer is supplied.

· There must be a clearly stated objective for each item

· Vocabulary should be as simple as possible

· Sentence structure should be simple and direct with correct grammar

· There must be only one correct or one best answer

· Essential information only in order to minimize examinee’s reading time

· Knowledge required should be neither too general nor too specific

· Stem should be in either question form or completion form

· Negative questions should be avoided as much as possible

· If unavoidable, put negative words in stem NOT in options.

· Highlight negation (e.g. underline, CAPITALISE, write in bold)

· Give repetitive phrases in stem not in options

· There must be four options: one correct answer + three distracters

· Options should be independent of each other

· Options should be consistent in length and in form

· There must be no verbal or structural clues between stem and options

· The use of: “all of the above” or “none of the above”, “never”, “always”, “only” and “all” should be minimized

· The correct answer blends in the distracters: neither longer nor shorter, more detailed or more abstract

· Distracters should be plausible and worded in varied form to each other

· Possible distracters can be common student errors or true statements applicable to other situations

iii. Hints for Matching Items

· There are fifteen [_15_] questions in this section.

· Match column A with column B. Select the letter of the correct answer from column B & write your answer on the provided space in front of the number in column A.

· If the candidate wishes to change his/her answer, he places an x mark on his/her previous answer and write the preferred answer.

· A correct answer scores 1, an incorrect answer scores 0. No marks will be given for a question if more than one answer is supplied

· Clear instructions on how to indicate choice should be given

· Lists of descriptions and options should be short and homogeneous

· Options should be at least four and plausible

· One list should be longer than the other list and the difference between two lists should be two

· Two lists should be differently identified (e.g. numbers and letters)

Matching items can be powerful in addressing competencies on higher levels of complexity (i.e. checking the problem solving capability of a candidate).

5.1.2 Developing Verbal/Oral Questioning

Questioning allows the assessor to probe to obtain clarification, confirmation or supplementation when needed.

Verbal questioning may also be a reasonable way to accommodate a candidate’s need for consideration of their language and literacy skill levels. Remember that the assessment should not demand higher literacy, language or communication skills than those required for the job itself.

The number of items to be developed for questioning shall be selective and not more than six.

i. General guidelines to develop verbal/oral questioning

· Keep questions short and focused on one key concept

· Ensure that questions are structured

· Test the questions to check that they are not ambiguous

· Use `open-ended questions such as `what if...?’ and `why...?’ questions, rather than closed questions

· Keep questions clear and straight forward and ask one at a time

· Use words that the candidate is able to understand

· Look at the candidate when asking questions

· Check to ensure that the candidate fully understands the questions

· Ask the candidate to clarify or re-phrase their answer if the assessor does not understand the initial response

· Confirm the candidate’s response by repeating the answer back in his/her own words

· Encourage a conversational approach with the candidate when appropriate, to put him or her at ease

· Use questions or statements as prompts for keeping focused on the purpose of the questions and the kind of evidence being collected

· Use language at a suitable level for the candidate

· Listen carefully to the answers for opportunities to find unexpected evidence

· Follow up responses with further questions, if useful, to draw out more evidence or to make links between knowledge areas

· Compile a list of acceptable responses to ensure reliability of assessments

5.2 Develop Commonly used Practical Assessment Tools

5.2.1 Develop Work Related Projects

When assessing work-related projects such as designing a product, writing a workplace document, solving a problem, conducting a presentation, developing a proposal for management, it is useful to design a project brief or instruction sheet. Projects can be designed for individuals or for groups to complete.

i. The project brief or instruction sheet should outline the following:

· the purpose of the project which elements of competency should be demonstrated through the project

· resources the candidate might use

· any particular performance expectations

· who will observe the performance or assess the product

· instructions for the candidate, including the timeframe and any other pertinent information

ii. In designing work related projects, due emphasis is given to the following:

· Identify and think of a project that will cover a cluster of unit of competence of  an occupation

· Identify unit of competencies to be covered under that project

· Identify critical aspect of competencies of a unit or an occupation must be demonstrated by the candidate and observable by the assessor

5.2.2 Develop Assessment Tools for Project/ Tasks Performed in Actual Industry Setting / On-Site Assessment

Determine the actual projects/ tasks to be done by the candidates on the scheduled dates. The design specifications must also be known, for quality assessment of product. This can be gathered from a conference with the industry managers/ supervisors of the company. This data can be gathered in the table, below:

Activities to be undertaken during the assessment schedule

Occupations covered as guided by the OS of the sector

Specific competencies involved

Required equipment, tools & materials

Table 1. Data Gathering on On-Site Assessment

Note: It may be possible that two or more occupations are involved in the actual job; hence, the tools to be prepared must be separate for these occupations. This is done to identify the competencies & occupational levels that are achieved and to be certified properly.

Assessor then prepares the evidence gathering plan, and subsequently, the other component parts of the assessment package.

5.2.3 Develop Assessment Tools for Structured Assessment Activities/Role plays

In cases where constructing a structured assessment activity to assess competency, it is needed to develop a range of assessment tools, which could include:

a) a scenario/outline of the situation

b) instructions for the candidate

a) The scenario

· The scenario can be a simple card outlining the scenario to the candidate, any other participants, and the assessor. The scenario has been developed to assess how effectively a worker can promote flexible work-practices to an employer, as required in the occupational standard.

· Scripts for people involved in the scenario/role-play

· It is needed to provide scripts for any participants who help to create the situation. The scripts shall be developed/ designed for a candidate in as structured activity to assess how effectively he/she can promote flexible work-practices to an employer, as required in the occupational standard.

b) Instructions for the candidate

In addition to the information that should be provided to candidates facing any assessment task, candidates should be advised about what is being assessed through the dramatization. If you plan to use any recording instruments, such as videos or tape recorders, your instructions should include this information, so that candidates are prepared for this so that they can be as relaxed as possible during the assessment.

5.2.4 Develop Assessment Tools for Workplace Simulation

Simulation may help the assessor for assessment decision making if this is an appropriate assessment method. This is followed by an example, which includes the instructions/procedures for an assessor and the assessment instrument, for an activity that simulates a hazardous situation.

a. Guidelines for workplace simulation

i. Before making a decision to use simulation, consider:

· occupational standard requirements and industry views on the use of simulation

· the benefits and limitations of using a simulation

· candidates’ characteristics and needs

· available workplace opportunities

· the cost of establishing and using simulated environments

· how the simulated assessment can be combined with other forms of evidence gathering such as logbooks, portfolios or work placements

· does simulation meet the principles of assessment in this unit or cluster

ii. Preparing the assessment event

· if you are assessing within a TVET institution, consider forming a partnership with local enterprises that may provide access to a workplace or equipment, authentic workplace documents or advice on how to create a realistic simulated environment

· review the whole qualification or units of competence to be assessed to build in opportunities for assessing whole work tasks or clusters of competencies

· where appropriate include opportunities to assess relevant generic competencies such as teamwork, communication, occupational health and safety and leadership

· include contingencies as part of the assessment design, for example, candidates might be required to deal with the pressures of telephones, time constraints and interruptions to workflow

· focus the assessment activity on processes as much as the end product

· apply operational procedures and occupational health and safety requirements as they would be in a real work setting

· validate methods, context and concepts with industry/workplace representatives to ensure the accuracy of the assessment approach

· prepare an observation checklist that clearly outlines the critical aspects

iii. Preparing the physical location

· consult with workplace/industry experts on what should be included

· check real workplaces to get ideas about current practice and ways of setting up work spaces and equipment

· where practical, alter the training environment so that it reflects a real workplace

· use equipment and other facilities that are as close to those used by industry as possible

iv. Preparing the candidate(s)

· give candidates a pre-assessment briefing outlining the assessment method, process and tools

· discuss the criteria against which their performance is to be assessed

· give candidates adequate information about the role they are to undertake and the significance of the event

v. Conducting the assessment

· where practical, involve industry experts in the assessment process and the decision making

· where appropriate, video the performance of the candidate

· use a checklist of critical aspects to focus on the observation of candidate performance

· use self-assessment, peer assessment and debriefing activities to add to the evidence gathered and help candidates develop reflective skills

5.3 Develop Direct Observation Method

Observation is an important method for competency-based assessment, which requires candidates to demonstrate not only what they know, but also what they can do. A number of tools can be developed to support this assessment method including:

a) observation checklist and questions to accompany checklist

b) Instructions to candidates and assessors

a) Observation checklist and questions to accompany checklist

An observation checklist is useful when observing performance in both real-work situations/on-site assessment or in simulated environments where candidates are able to demonstrate competencies. It enables the assessor or other evidence-gatherer to observe in a focused way, to take structured notes that can be referred to when making the assessment decision, to provide informed feedback to candidates, and to enhance the objectivity of the assessment decision.

The complexity of an observation checklist will mirror the complexity of the task/tasks to be observed. In the case of clustered assessment, the checklist will need to assist the assessor to handle the increased complexity of his/her task, or the sophistication of the skills s/he is assessing, particularly in the case of higher level qualifications. A simple list of ticks and crosses on a checklist without any other information isn’t helpful and would generally be questioned.

b) Instructions to candidates and assessors

It is needed to include clear instructions for the candidate and for the assessor either on the checklist or in a separate document:

· Candidates need to know exactly what is expected of them, and any materials that they are required to supply.

· Assessors need to know exactly what they are looking for, what resources are needed, and any other issues that need to be taken into account. They also need to know how to use the observation checklist.

· Observation checklists may be supported by a list of performance questions, which are derived from the Evidence Guides in the unit/ units of competency. These questions include dimensions of competency, such as contingency management skills (what would you do if……?), job/role environment skills (what are the procedures and policies for……?) and task management skills (what are your functions and how do you manage them when you. ………?).

Table 2. Recommended Assessment Methods & Tools

Instruments/ Tools

Methods

Remarks

Observation

Product /outcome evaluation

Questioning

Practical exercises of projects done in the AC/venue

X

X

X

Questioning only is done by the Assessor to support his Observation

Actual job performance on-site

X

X

X

Questioning only is done by the Assessor to support his Observation

Simulations, & demonstrations

X

X

This can be used for assessments of actions

under “safe” conditions (example the practice of CPR for emergency patient)

Role plays

X

X

This is useful in Secretarial occupation, Front Desk, etc

Portfolios

.

X

X

This may be used for the assessment of supervisors and managers of companies for accreditation as Core Assessors.

Knowledge test in written./ or Interview questions

X

X

This instrument is also used especially for underpinning knowledge and also for generic competencies


VI.
Validation and Moderation of Assessment Tools

6.1 Validation of Assessment Tools

Validation is a quality review process. It involves checking that the assessment tool produced valid, reliable, sufficient, current and authentic evidence to enable reasonable judgments to be made as to whether the requirements of the relevant aspects of the occupational standard had been met. It includes reviewing and making recommendations for future improvements to the assessment tool, process and/or Outcomes.

Validation of Assessment Tools proceeds from the following conditions:

1. The Assessment tools first of all should be developed by members of the Assessors Panel to relate the activities / projects to the actual situation in Industry and to decide whether these projects can provide evidence of possession of competence.

2. To ensure that the assessment tools developed meet the conditions cited in 1 will be further reviewed (validated) by a group of experts in the Panel, and possibly also by the members of the Regional Industry/Association of the Occupation.

6.2 Trial, Refine and Review Assessment Tools

1. To ensure that the assessment tools are consistent with the requirements of the occupational standard and that they maintain their currency, sufficiency and effectiveness, it is important that the tools are reviewed by fellow assessors and trialed prior to use.

2. Inviting feedback from the industry will hopefully confirm that the tools enable effective collection of evidence, and that the level of difficulty is appropriate to the qualification level. Differences of opinions provide an opportunity to discuss and resolve any ambiguities or misunderstandings before the tools are used with candidates.

3. Trialing the developed tools before they are used formally with candidates will enable to gauge the user friendliness of the format, the appropriateness of the literacy and numeracy levels, the clarity of the instructions and the practicality of the format for recording assessment evidence and judgments. It will also enable to evaluate the suitability of the times allowed for assessment tasks, and the tool’s overall cost-effectiveness.

4. During the trial, it is also possible to assess the tool’s degree of adaptability. This will be determined by its capacity to be adjusted in accordance with variations in context and the needs of candidates, while still ensuring valid and reliable assessment decisions.

6.3 Moderation: Quality Control

Moderation is the process of bringing assessment judgments and standards into alignment. It is a process that ensures the same standards are applied to all assessment results within the same Unit(s) of Competency. It is an active process in the sense that adjustments to assessor judgments are made to overcome differences in the difficulty of the tool and/or the severity of judgments.

Recommendations to achieve moderation in the use of Assessment tools:

1. The problem on differences in the difficulty of the tool may be overcome if the same assessment tool/ instrument shall be used for the same unit/s of competence involved in assessment. As suggested in the Directive, The FTA or the concerned Ministry of the sector, shall make sure that the Assessment tools used by the CoCs (for TVET completers), and the Regional Industry Boards (for Assessment of workers) are the ones authorized by them, with the specific instruction that the other versions being replaced shall no longer be used.

2. The concern on the differences present in the severity of assessor Judgments may be minimized when we employ at least a team of Assessors to conduct the Assessments. (The Directive suggested that the members of this team, is one from the Assessors Panel, one from the Regional Industry Board, and the other a representative of the CoC or the concerned Ministry). These three persons, who are experts/ or experienced with the occupation under assessment shall independently make their assessments, and compare their results, such that the final decision will be agreed by the team.

The recommended actions cited here, on validation and moderation procedures may improve the validity and reliability of our assessments as an important requirement for Certificatio

VII. Organizations in-charge with the development and use of Assessment Tools

The following describes the roles of the implementers of Assessment in the development and use of Assessment Tools.

Federal TVET Agency

· Coordinate with the Sector-leading ministries for the development of First Version of Assessment tools. Succeeding versions will be developed in the Regions, as indicated in the Directive.

· Capacitate the CoCs and the Regional Industry/ Associations in the Development of Assessment Tools.

· Takes charge of the moderation procedures for the assessment tools developed,

· Provide the CoCs with the recent versions of the validated and moderated Assessment Tools for the occupations that are covered in the assessment activities of the CoCs,

· The CoC shall only be provided with the Assessment tools for occupations where the availability of Assessors and Assessment Centers/ Venues have already been established.

· Maintain a data bank of finalized assessment tools.

Regional Industry / Associations

· Validate the Assessment Instruments/ tools developed by the Assessors

· Submits validated assessment tools/ instruments to the FTA for the Assessment of TVET Trainees/ completers.

· Submits validated Assessment Tools to the concerned Ministry for Assessment of Industry Workers

Center of Competence

· Capacitate the Regional Industry / Associations / Assessors Panel in the development of Assessment /Tools.

· Takes charge of the safekeeping and proper use of Assessment Tools provided by the FTA.

Assessors Panel

· Development of the Assessment Tools for the occupations

· The Assessors implementing assessments shall be members of this panel

· The Panel shall submit to the respective Regional Industry/Association, the developed tools for Validation.

· Responsible in upholding the integrity of the assessment tool by safeguarding the authenticity and security of all documents developed.

VIII. Other Assessment-Related Documents

Self- Assessment Guide: This is usually provided to the Candidate before he finalizes his application for assessment, which will guide him on what competencies he is qualified to apply.

Competence Assessment Agreement Form – This form indicates the agreement made by the candidate who acknowledges his consent to be assessed by the designated assessor.

Competency Assessment Result Summary – This is the final form for the assessor to indicate the result of the Assessment.

Candidates Feedback Form - This is a survey form to solicit information from the Candidates on the process and results of Assessments. The purpose of this is to gather information that will be helpful in improving the Assessment system.

Feedback form for the Assessor/s - This information-gathering form, like the previous one, is solicited as input to further improve the system.

Note: The Assessment methods and instruments/ tools, including the suggested forms, shall serve to guide the Assessors Panel in the development of Assessment Instruments. The Panel is responsible to determine the appropriate forms and instruments which it sees fit to the Assessment to be conducted

Annex

1. Sample written type of knowledge assessments

a) Samples of multiple choice items

1. Which of the following is not a protection device?

a. Circuit Breaker

b. Surge Arrester

c. Distance Relay

d. Ammeter

2. The frequency of our Electric power system is

a. 60 Hz

b. 50 Hz

c. 120 Hz

d. 180 Hz

3. Which of the following is not an Operating System

a. Windows XP

b. Microsoft Office

c. Linux

d. Windows 7

Column A

Column B

___1. Distance Protection

___2. UTP

___3. Transformer Radiator

___4. SCADA

___5. AVR

a. Used to regulate the level of voltage delivered

b. Supplies Electricity for  substation use

c. Used for Protection and Communication

d. Used for reactive power Compensation

e. used for cooling purpose

f. unit of reactive power measurement

g. Grounding set

h. unit of active power measurement

b) Samples of matching items

c) Suggested questions and answers for interview

Interview Questions

The candidate must answer all the questions within 10 minutes

Satisfactory response

Yes

No

Q1. What are some characteristics included in the definition of Business? Name at least three.

A11. An economic activity

A12. Production or procurement of goods and services

A13. Sale or exchange of goods and services for the satisfaction of human needs

A14. Dealings on goods and services on a regular basis

A15. Profit earning

A16. Uncertainty of return

A17. Element of risk

Q2.  What is meant by business risk?

A2. ( Best answer) This refers to the possibility of inadequate profits or even losses due to uncertainties or unexpected events.

Q3.  Name at least five basic factors that must be considered to start a business:

A31. Selection of line business

A32. Size of the firm

A33. Choice of form of ownership

A34. Location of the business enterprise

A35. Financing the proposition

A36. Physical facilities

A37. Plant layout

A38. Competent and committed workforce

A39. Tax planning

A310. Launching the enterprise

Q4.   What do you understand of the term “ Sole proprietorship”?

A4.  This refers to a form of organization where business is owned, managed and controlled by a single individual who bears all the risks and is the only recipient of all the profits.

Q5.   What do we mean by “ Partnership” in business?

A5.  This is defined as an association of two or more persons who agree to carry on a business together and share the profits as well as bear the risks collectively.

The candidate’s  knowledge on this unit of competence is:

Satisfactory q Not Satisfactory q

Signature of Assessor:

Date                            :

 


2. Self-Assessment Guide Template

Occupation

----------------------------------------------------------------

Level

-----

Instructions: -Read each of the units of competence listed below

-Place a tick in the appropriate box opposite each question to indicate your answer

Are you confident that you are capable to:

Yes

No

Units to be included in Project 1

1. Code & Unit Title

2. Code & Unit Title

3. Code & Unit Title

Units to be included in Project 2

1. Code & Unit Title

2. Code & Unit Title

3. Code & Unit Title

Units to be included in Project 3

1. Code & Unit Title

2. Code & Unit Title

3. Code & Unit Title

Generic Competencies:

1. Code & Unit Title

2. Code & Unit Title

3. Code & Unit Title

4. Code & Unit Title

5. Code & Unit Title

6. Code & Unit Title

Candidate’s Name:____________________ Candidate’s Signature:__________

Date:___________


3. Competence Assessment Agreement Template

Candidate’s Name:

Assessor’s Name:

Occupation

Level

Units of Competence to be Assessed: (Please put a tick on the space before the number of unit/s you want to be assessed)

Units to be included in Project 1

1. Code & Unit Title

2. Code & Unit Title

3. Code & Unit Title

Units to be included in Project 2

1. Code & Unit Title

2. Code & Unit Title

3. Code & Unit Title

Units to be included in Project 3

1. Code & Unit Title

2. Code & Unit Title

3. Code & Unit Title

Generic Competencies

1. Code & Unit Title

2. Code & Unit Title

3. Code & Unit Title

4. Code & Unit Title

5. Code & Unit Title

6. Code & Unit Title


Candidate to answer questions

Yes

No

1. Have the context and purpose of assessment been explained?

o

o

2. Have the qualification and units of competence been explained

o

o

3. Have the scope of assessment been explained?

o

o

4. Do you understand the assessment procedure and evidence to be collected?

o

o

5. Have your rights and appeal system been explained?

o

o

6. Have you discussed any special needs to be considered during assessment?

o

o

I agree to undertake assessment with the knowledge that evidence gathered will only be used for professional development purposes and can only be accessed by concerned assessment personnel.

Candidate’s Signature:

Date:

Assessor’s Signature:

Date:



4. Sample Assessor’s Package Template

Ministry of Education

Sector: - ----------------

Sub Sector: - ------------------------------

Occupation: _________________________________Level________

Assessor’s Package

The assessment covers the following units of competence:

  1. _______________________________________________
  2. _______________________________________________
  3. _______________________________________________
  4. _______________________________________________
  5. _______________________________________________
  6. _______________________________________________
  7. _______________________________________________
  8. _______________________________________________
  9. _______________________________________________

Contents of the Assessor’s Package:

1. Instructions for the competence Assessor

2. Evidence Plan

3. Project Information or Problem

4. Worksheet (If necessary)

5. Demonstration/Observation Checklist

6. Product Evaluation Sheet (If required in the practical project)

7. List of Equipment, Tools and Materials

Methods of Assessment

1. Observation/Demonstration with Interview Question (If necessary)

2. Written Assessment

Nature of assessment:

1. Qualification – based

- All units of competence are included in different practical projects and the accompanying knowledge tests for basic and common competencies.


INSTRUCTIONS FOR THE COMPETENCE ASSESSOR

Occupation

__________________________________Level _______

BEFORE THE ASSESSMENT

  1. Establish the assessment context and purpose of the assessment
  2. Familiarize self with the qualification or units of competence to be assessed

· Assessment context

· Assessor’s guide

· Assessment methods

· Product evaluation sheets

· Time limits

· Evidence to be collected

· Performance Criteria

· Assessment and Certification arrangements

  1. Ensure that the Assessment Center complies with the relevant occupational health and safety requirements as:

· Well lighted and well ventilated assessment room

· Risk areas properly marked

  1. Ensure that the following assessment resources are available in the Assessment Center:

· Complete set of safe and functional equipment and tools

· Complete set of supplies and materials

· Assessor’s guide and rating sheets (Assessment Package)

  1. Notify candidates of any resources they have to bring during the assessment

ON THE DAY OF ASSESSMENT

  1. Check the attendance of the candidates

· Check the candidates registered number

· Instruct candidates to sign on the attendance sheet

· Check accomplished self assessment guide

· Provide overview of the qualification to be assessed refer to (EOS)

  1. Orient the candidates (Time allotment is 15 minutes)

· explain the context and purpose of the assessment and the assessment process

· explain the Qualification or units of competency to be assessed and the evidence to be collected

· outline the assessment procedures to be undertaken

· assess the needs of the candidates and establish any allowable adjustments in the assessment procedure

· seek feedback regarding the candidate’s understanding of the occupational standard/units of competence being assessed, evidence requirements and assessment process

· make all necessary announcement just before the start of the assessment

· specify  the do’s and don’ts inside the assessment room

  1. Instruct candidates to accomplish the competency assessment agreement

· Provide the candidates with all materials, tools and equipment required to complete the tasks. Ensure that all tools should be of the same kind and quality

  • Administer the assessment in accordance with the outcome-based assessment methodology

· Give specific instructions to the candidates clearly and slowly and provide each candidate with a copy of the Candidate Instructions Guide and Worksheets

· Inform each candidate that s/he has fifteen (15) minutes reading time and the total time duration for each is indicated in the Project Information.

· Make certain that the candidate fully understands what is expected and knows the time limits allowed to finish each assessment project.

· Stay at the assessment area during the entire duration of the assessment activity and observe the candidate perform the tasks.

· Inhibit self from providing any assistance during the assessment or indicate in any way whether the candidate is or not performing the activity correctly.

· Intervene only for safety reasons

· Implement the evidence gathering process and ensure its validity, reliability, fairness and flexibility

· Judge the evidence in terms of the four dimensions of competency – task skills, task management skills, contingency management skills and job/ role environment skills

· Record details of evidence collected and make  judgment about the candidate’s competence based on the evidence and the relevant standards required  of the units of competence

· Make no alterations/erasures in the Observation Checklist

POST ASSESSMENT

  1. Provide feedback on the outcome of the assessment process.  This  includes providing the candidate with:

· clear and constructive feedback on the assessment decision

· give the candidates your feedback on the assessment result

Feedback should indicate whether the candidate is:

· COMPETENT

· NOT YET COMPETENT

On each of the competencies assessed

· information on ways of overcoming any identified gaps in competency revealed by the assessment

· information on reassessment process, if applicable

  1. Prepare the necessary assessment reports

· Record the assessment outcome using the prescribed Observation Checklist

· Make sure the report or rating sheets are properly signed by you and the candidate

· Maintain records of the assessment procedures, evidence collected and assessment outcome

· Endorse assessment results/outcomes to assessment centre manager

· Prepare recommendations for  the issuance of  national certificate/ certificate of competence


Sample Evidence Plan Template

Sector

 

 

Sub-Sector

 

 

Occupation :

 

 

Level

 

 

Units of competence to be assessed:

Appropriate Assessment Method (to be placed “X” )

Assessment tool/instrument / written /oral question using interviews, practical exercise /demonstration ,role play, simulation

 

Observation

Outcome evaluation

Knowledge Assessment (Oral or written)

Practical exercise

Oral/written

 

A. Practical Assessment

 

Project 1: ___________________________(project title)

 

Code & Unit Title ----------------------------

Code & Unit Title ----------------------------

 

Code & Unit Title ----------------------------

 

Code & Unit Title ----------------------------

 

Project 2: ___________________________(project title)

Code & Unit Title ----------------------------

 

Code & Unit Title ----------------------------

 

Code & Unit Title ----------------------------

 

Code & Unit Title ----------------------------

Project 3: ___________________________(project title)

 

Code & Unit Title ----------------------------

 

Code & Unit Title ----------------------------

 

Code & Unit Title ----------------------------

 

Code & Unit Title ----------------------------

 

B. Knowledge Assessment:

 

a) Written Test

 

b) Oral Questioning

 

 

NOTE: The OS must be the guide in preparing the plan, considering the various elements of the unit of competence and performance criteria indicated. Underpinning knowledge and skills must also be considered.


Project Information

Given the necessary tools, equipment and information, the candidate is expected to perform the following _________projects.

The assessor should ask oral questions on which s/he is not satisfied by the candidate’s performance

Assessment Venue: Well Equipped Assessment Centre or real work place

Total time for all projects --------- Hrs

Project 1

Instruction: Under this project the candidate is expected to perform the tasks based on the information provided.

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Competencies covered:

1. -------------------------------------------------------------------------------------

2. -------------------------------------------------------------------------------------

3. -------------------------------------------------------------------------------------

Other requirements: (optional) -----------------------------------------------------------------------

Total time for the project: ----------- hours

Project 2

Instruction: Under this project the candidate is expected to perform the tasks based on the information provided.

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Competencies covered:

1. -------------------------------------------------------------------------------------

2. -------------------------------------------------------------------------------------

3. -------------------------------------------------------------------------------------

Other requirements: (optional) -----------------------------------------------------------------------

Total time for the project: ----------- hours

Project 3

Instruction: Under this project the candidate is expected to perform the tasks based on the information provided.

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Competencies covered:

1. -------------------------------------------------------------------------------------

2. -------------------------------------------------------------------------------------

3. -------------------------------------------------------------------------------------

Other requirements (optional): -----------------------------------------------------------------------

Total time for the project: ----------- hours

Observation Checklist

Candidate’s Name:

Assessor’s Name:

Occupation

Level

Unit of Competency:

Unit Code, Unit Code, Unit Code, Unit Code

Date of Assessment:

Time of Assessment:

Instructions for the Assessor

· Observe the candidate.

· Describe the assessment activity and the date on which it was undertaken

· Place a tick in the box to show that the candidate completed each aspects of the activity to the standard expected in the enterprise.

· Complete the feedback sections of the form.

OBSERVATION

During the demonstration of skills, did the candidate:

Yes

No

Project 1: (Project title with unit code(s))

· _____________________________________________________

· _____________________________________________________

· _____________________________________________________

· _____________________________________________________

· _____________________________________________________

Project 2: (Project title with unit code(s))

· _____________________________________________________

· _____________________________________________________

· _____________________________________________________

· _____________________________________________________

· _____________________________________________________

Project 3: (Project title with unit code(s))

· _____________________________________________________

· _____________________________________________________

· _____________________________________________________

· _____________________________________________________

The candidate’s overall performance in the practical project was:

q SATISFACTORY q NOT SATISFACTORY


Product evaluation sheet for the Projects 1, 2 and 3 outputs

Workmanship

Project specification

Actual output

Tolerance

Remarks

Satisfactory

Not Satisfactory

The candidate’s output is:

q Satisfactory q Not Satisfactory

Signature of Assessor


Sample for Materials to be used in the Assessment:

Quantity

Item description

Specification

Tools & Instruments:

1. ------------------------------------------------------------------------

2. ------------------------------------------------------------------------

3. ------------------------------------------------------------------------

4. ------------------------------------------------------------------------

5. ------------------------------------------------------------------------

6. ------------------------------------------------------------------------

Note: Materials and tools may be modified by the Assessor, depending on the actual job performed, and location & the standard installation practices of the country.

 

5. Sample Candidate’s Package

Ministry of Education

Sector: - ___________________________________

Sub Sector: - _______________________________

Occupation: _______________________ Level_________

Candidate’s Package

The assessment covers the following units of competence:

  1. _________________________________________________
  2. _________________________________________________
  3. _________________________________________________
  4. _________________________________________________
  5. _________________________________________________
  6. _________________________________________________
  7. _________________________________________________
  8. _________________________________________________

Candidate’s instructions and worksheets

· Instructions to Candidate

· Methods of Assessment

1. Observation/Demonstration with Oral Questioning/Interview

2. Written assessment

Candidate name

Registration number

Name of assessment center

Name of training provider

Type of Assessment applied:

 

Qualification – based

 

Competency – based


INSTRUCTIONS TO THE CANDIDATE

Occupation:

___________________________________ Level ___

1. For Qualification-based assessment, you are to perform all the projects in this package together with the written tests.

2. For competence-based assessment, you are required to perform the specific project/s where the competencies to be certified are included.

3. Read the Project package/s and Written test (if applicable) properly.

4. All your questions and clarifications should be addressed to the assessor only.

5. Complete the task/project described in the project package.

6. You are given fifteen (15) minutes reading time for each project and the number of hours allocated for each project is indicated in the Project Information Sheet.

7. After you finish this assessment, you are required to submit all the project packages and written tests questions provided to you.

8. You are allowed to use your tools/ instruments with the permission of the Assessor, and this must be presented to the assessor before start of the test.

9. After you finish the assessment, you are required to sign the Competency Assessment Results Summary, and return to the stock room all the tools and materials borrowed.

10. You are also required to clean up your work station after completion of your work.

Project Information

Given the necessary tools, equipment and information, the candidate is expected to perform the following _________projects.

The assessor should ask oral questions on which s/he is not satisfied by the candidate’s performance

Assessment Venue: Well Equipped Assessment Centre or real work place

Total time for all projects ________ Hrs

Project 1

Instruction: Under this project the candidate is expected to perform the tasks based

on the information provided.

_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Competencies covered:

1. _______________________________________________________

2. _______________________________________________________

3. _______________________________________________________

Other requirements: (optional) _______________________________________________

Total time for the project: __________hours

Instructions:

_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Note (If any)

___________________________________________________________________________________________________________________________________________________________________________________________________________________________

Project 2

Instruction: Under this project the candidate is expected to perform the tasks based

on the information provided.

____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Competencies covered:

1. _______________________________________________________________________

2. _______________________________________________________________________

3. _______________________________________________________________________

Other requirements: (optional) _________________________________________________

Total time for the project: ___________ hours

Instructions:

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

_____________________________________________________________

Note (If any)

___________________________________________________________________________________________________________________________________________________________________________________________________________________________

Project 3

Instruction: Under this project the candidate is expected to perform the tasks based

on the information provided.

_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Competencies covered:

1. ____________________________________________________________________

2. ____________________________________________________________________

3. ____________________________________________________________________

Other requirements: (optional) ______________________________________________

Total time for the project: ________ hours

Instructions:

_______________________________________________________________________

_______________________________________________________________________

_______________________________________________________________________

_______________________________________________________________________

Note (If any)

_______________________________________________________________________

_______________________________________________________________________

_______________________________________________________________________

_______________________________________________________________________

Sample for Materials to be used in the Assessment:

Quantity

Item description

Specification

Tools & Instruments:

1. ______________________________________________

2. ______________________________________________

3. ______________________________________________

4. ______________________________________________

5. ______________________________________________

6. ______________________________________________

Note: Materials and tools may be modified by the Assessor, depending on the actual job performed, and location & the standard installation practices of the country.

6. Competency Assessment Results Summary

Candidate’s Name

Assessor’s Name

Occupation

Level

Assessment Centre

Date of Assessment

Nature of Assessment

 

 

Qualification-based             ,   Competence-based

C. Observation with Oral Questioning

Performance of the candidate (Pls tick)

COMPETENT

NOT YET COMPETENT

Not Applicable

A. Practical Assessment

Unit of Competences

Project 1: ___________________________(project title)

Code & Unit Title ----------------------------

Code & Unit Title

Code & Unit Title

Project 2: ___________________________(project title)

Code & Unit Title

Code & Unit Title

Code & Unit Title

Project 3: ___________________________(project title)

Code & Unit Title

Code & Unit Title

Code & Unit Title

B. Knowledge Assessment:

a) Written Test

b) Oral Questioning

General Comments [Strengths / Improvements needed]

Recommendation:

q For issuance of NQC (Satisfactory in all units of Competence and Knowledge test)

q For issuance of CC (Satisfactory In Some Core Competence)

q For re- assessment (All core competencies not satisfied)

Candidate’s signature:

Date:

Assessor’s signature:

Date:

 




 

Last Updated (Friday, 06 January 2017 13:58)