Beyond Certification

ANSI Personnel Certification Accreditation Workshop

I've just learned there are still several spots left in the upcoming April 2-3 Workshop on ANSI/ISO/IEC 17024 - Personnel Certification Accreditation which will be held in DC. If you are interested in attending, please contact ANSI immediately per the instructions below.

 ***

This workshop is designed to introduce organizations involved in personnel certification to the international standard ANSI/ISO/IEC 17024 Conformity assessment - General requirements for bodies operating certification of persons. This standard is increasingly recognized by the government, the certification industry and organized labor as an accepted benchmark for bodies managing the certification of persons.

 

Through presentations, small group discussions and exercises, you will learn:

 

• The requirements of ANSI/ISO/IEC 17024
• What content is required by each requirement, what types of documents and data may be helpful in documenting compliance with each requirement and how the requirements will be handled during an audit.
• The terminology, identifying related documents and resources that may be helpful in understanding the requirements.
• How to begin an informal self-assessment of your organization to identify the areas of improvement that may be needed to meet the requirements
• The ANSI accreditation process which is guided by the International Standard ISO/IEC 17011

The cost of participation in this two-day workshop is $795 for ANSI members and $895 Non-Members. A 10% discount is offered when more than one participant registers from an organization.

For further information about this course, please visit the Education and Training website or contact Dr. Vijay Krishna by phone 202-331-3614 or e-mail vkrishna@ansi.org.

More on Consensus

In his comment, Jamie Notter points us to his blog post which shares an excellent definition of consensus.  It describes two critical components of consensus: a high level of commitment to a chosen course of action and a strong shared understanding of the rationale for the decision.  Take a look at the post, it makes some important points.

Kevin Holland also points out that associations should not utilize a formal consensus process on every decision they make.  AGREED!!!  Talk about anti-nimble, and that's certainly not where associations need to be.  Some decisions just need to be made, and quickly, period.

In the post, I am specifically talking about developing industry standards, and there I do believe consensus is critical.  But, I also wanted to clarify that I am not advocating that all associations should become ANSI-accredited as developers of American National Standards.  That path is right for some, not for others.  My point is that the option is something all standard-setters should be aware of and give serious consideration as to whether or not it has value for them.  Of course any organization can develop processes consistent with the principles put forth by ANSI (and ISO, the International Organization for Standardization) without being formally acknowledged. However, for some, the benefit of third-party verification of quality is significant - and that's what accreditation can give you. 

Take the certification industry standards, for example.  There are many, including:

ANSI/ISO/IEC 17024: General Requirements for Bodies Operating Certification of Persons (2003) American National Standards Institute (ANSI), Washington, DC. 

Development, Administration, Scoring and Reporting of Credentialing Examinations (2004), Council on Licensure, Enforcement and Regulation (CLEAR), Lexington, KY.

Principles of Fairness: An Examining Guide for Credentialing Bodies (2002), National Organization for Competency Assurance and Council on Licensure, Enforcement and Regulation, Lexington KY.  

Standards for the Accreditation of Certification Programs (2002) National Commission for Certifying Agencies (NCCA) of the National Organization for Competency Assurance,  Washington, DC.

Standards for Educational and Psychological Testing (1999) of the American Educational Research Organization, American Psychological Organization and the National Council on Measurement in Education).

There is only one standard on this list (ANSI/ISO/IEC 17024) that I can be ASSURED was developed following the quality principles of consensus, balance, transparency, due process and others.  Does that mean the others aren't good standards?  No.  Does it mean the processes they followed were bad?  Of course not.  But, absent conformance to any formally documented standards development process, all we have to go on is our trust in the standards developer.  My question: in this era of consumer distrust, is that enough?  If you're a standards developer, that's a question to take to heart.

Intro to Computerized Adaptive Testing

If your certifying agency is considering computer adaptive testing, you might want to look into Assessment Systems Corporation's workshop on June 1 in Minneapolis.  The following topics will be covered:

• Rationale and basic assumptions of IRT
• Major IRT models and their parameters
• Estimating the latent trait, Θ, and its standard error
• Estimating item parameters
• Item and test/bank information
• Linking items to create an item bank for CAT
• Basic elements of CAT
• Types of CATs: CATs for equiprecise measurement, classification, measuring change
• What to consider before implementing CAT
• Implementing CAT
• Operational issues for some CATs: item exposure, enemy items, content balancing

Learn more here.

A New Guide for Organizations Considering Certification

Considering Certification:  A Guide to Making the Decision is finally available!  Many of you know I've been working on this in my spare time...and like most of you, I have none so it's taken a while. ;-)

It's a primer for organizations considering development of a certification program and provides detailed guidance on how to decide if your organization should create such a program.   What are the credentialing options?  Which is right for your organization?  How do you conduct a feasibility analysis?  How do you write a business plan?  How do you set up the governance structure?  What staffing do you  need?  These are just a sampling of the questions you'll get answers to.  It contains must-have information and advice not found anywhere else in print. 

Click here for more information, to preview the table of contents, or to purchase directly through the publisher.  Amazon will begin carrying it in October.

Here's what one of your colleagues has to say about it:

"Mickie has made a complex decision-making process simple.  We followed the systematic, knowledge-based process outlined in this guide and we saved immeasurable amounts of time, talent, treasure, and credibility while ultimately meeting the desired outcomes for the profession with a service that our members wanted."

       Joshua R. Goldman, Director of Education
       Association of College & University Housing Officers—International

Here's also what Josh has to say (tongue in cheek):  "I’m glad this didn’t exist two years ago; they would have had no need to hire me!" Thanks Josh, although they most definitely still need you to develop that quality certificate program for them!

Going Global?

Going global with certification is a topic of interest to many, but few resources exist to guide efforts.  Here's an article that's not new, but provides timeless advice.  Don't forget to click on the links at the end too - one covers legal issues to consider.  (Caveat:  you may need to be an ASAE & The Center member to view.)

Also important to realize, there is an important distinction between a U.S. association offering a credential to an international audience and one offering global credentialing.  The former occurs much more often.  In this case, a U.S.-based association creates a credentialing program based on U.S. practice and standards and allows individuals in other countries to apply.  In this case, those individuals will be required to know the U.S.-based terminology and standards even if they do not apply to them.  In addition, the program likely won’t cover some terminology and standards important to practice in their country.

In contrast, a truly global credentialing system studies the practice in every country where the certification will be offered, identifies the universal standards that apply to all, and creates the program based on these universal standards.  This, as you can imagine, is usually not an easy task for most professions since terminology, practice standards, and regulations often differ significantly by country.
The differences often lead associations to hybrid approach, including 1) creating the U.S.-based program first and later adopting other versions of it for use in other countries and 2) working with several nations to identify the universal set of standards, but then allowing individual countries to add standards to accommodate their local practice requirements. 

It can be helpful to distinguish the program types using the following terminology:

• A national program is one that is based on standards that were developed and adopted by one country, yet individuals from other countries are allowed to apply for the credential.
• An international program is one that is based on standards that were developed and adopted by one country and applied to other countries.
• A global program is one that is based on universal standards collectively identified by all the countries involved.
• A multinational program is one that is based on a combination of universal and local standards.  Representatives from different nations collectively define a universal set of standards, but they reserve the right to supplement those standards with ones that reflect their local conditions

National certification programs that allow international applicants are fairly common.  However, how they handle international applicants can vary.  Reciprocity and equivalency are forms to be familiar with:

• Reciprocity is mutual recognition by two countries of each others standards and/or requirements.
• Equivalency is the determination that a country’s standards and/or requirements are of equal content and quality of those of another country.

Retiring a Certification

Not a lot is written about why or how to retire (disband, discontinue) a certification.  Few certifying organizations have policies addressing this potential situation and without a policy, programs tend to linger on, especially when there's politics involved (and when isn't there?).

Here are the minimum components that should be included in a policy and associated procedures:

Circumstances leading to retirement consideration / decision - such as declining trends in number of applicants, number of certificants recertifying, revenue earned, value by stakeholders

Decision - who are the decision-makers and by what margin does a retirement motion get approved

Communications - to whom, how and when will the retired certification decision be communicated

Timeline - in general terms, when will the last application cycle / test administration occur (the key point here is to give sufficient notice to stakeholders)

Future Plan  - what happens to individuals already certified (must they cease using the designation at a given point in time, will they be able to maintain the credential indefinitely through ongoing recertification requirements, etc.)

If you are evaluating your certifications on a regular basis (you are, right?!), then you should readily have the data to trigger that certification retirement discussion, when warranted.

Certification Pricing

Read something today that suggested that as a general rule, the cost of certification should not exceed one percent of the target's gross annual salary.  Anyone using that rule of thumb?   For an average gross annual salary of $25,000, a fee of $250, for $50,000 a fee of $500, and $100,000 a fee of $1000.  At face value, it seems reasonable.  Sounds like a good place to start...but of course you need to consider your costs and make sure the fee covers those. 

I had a thought of how to turn this around for promoting.  Isn't certification worth investing just 1% of your annual salary?  Seems that''s hard to argue against that, assuming  your certification offers something to be valued.

Microsoft moving to Performance Based Testing

Here's an article describing how Microsoft is moving into more performance based testing for its certification programs.  The article does a nice job, I think, of explaining the benefits of performance based testing in terms everyone can understand.

Who is the Audience?

I've noticed recently that many certification articles/books written for certification NONexperts assume a fairly sophisticated understanding of certification/testing concepts which (duh) if the individuals already had, they wouldn't be reading the piece.

Here's my current favorite written to explain the test development and scoring process for individuals certified in a healthcare profession (org name witheld to protect the "innocent"):

"The scaled scores are not a "number correct" or "percent correct" scores.  Raw Scores are arithmetically transformed into scaled scores.  This conversion involves a simple linear transformation of raw scores (x) to scale scores (s) that takes the form of Total Raw Score times zslope plus zintercept."

Yep, simple. 

The worst part is that this is literally the last sentence of the written piece.  Not exactly the note I'd like to go out on.  Check your written materials for readability (for the audience targeted - not to you!)

FAQ: Training for Certification

Once again today I received one of the most frequently asked questions from execs considering developing certification: "Can taking our courses or training programs be a requirement for certification?"

The simplistic answer is yes, but only because there are no MANDATES that certifying agencies must follow in program design.  That is, you really are free to develop a credentialing program with whatever requirements you want and you can call it whatever you want (certification, registration, certificate, etc.)  However, just because you can doesn't mean you should.  You should know the implications.

First, mandating specific courses or training programs as a requirement for certification is in violation of the existing voluntary certification industry standards (ANSI and NCCA).   You're likely aware of organizations that require their courses as part of their certification programs.  In some instances, these programs – while called certification – may, in fact, be curriculum-based certificate programs (more on this later)...or they could be a blend between the two types of program.  In other cases, organizations may either not be aware of the industry quality standards or they have chosen not to comply with them.

Second, from the legal perspective, certification-related activities that are anticompetitive, discriminatory, based on subjective standards, or implemented without fair procedures are cause for antitrust concern.  In this case, a legal concern is a tying arrangement – that you are tying the certification product to an educational product (and requiring purchase of both).  Also, there is a fairness concern in requiring everyone to participate in (and pay for) coursework – regardless of whether they may need it.

Looking at the definitions from a purist perspective, if you are developing a program that includes both training and an assessment, that's called a certificate program.  The primary distinction between certification and curriculum-based certificates is in their focus. In certification, the focus is on assessing current experience, knowledge and skills. In a certificate, the focus is on training individuals to achieve a certain knowledge and skill base and then assessing attainment of it.   The programs meet entirely different goals so you need to identify your organization's goals before you determine which program type, if any, you should create.

For more information, check out the articles at this link.