Who is the Audience?

I've noticed recently that many certification articles/books written for certification NONexperts assume a fairly sophisticated understanding of certification/testing concepts which (duh) if the individuals already had, they wouldn't be reading the piece.

Here's my current favorite written to explain the test development and scoring process for individuals certified in a healthcare profession (org name witheld to protect the "innocent"):

"The scaled scores are not a "number correct" or "percent correct" scores.  Raw Scores are arithmetically transformed into scaled scores.  This conversion involves a simple linear transformation of raw scores (x) to scale scores (s) that takes the form of Total Raw Score times zslope plus zintercept."

Yep, simple. 

The worst part is that this is literally the last sentence of the written piece.  Not exactly the note I'd like to go out on.  Check your written materials for readability (for the audience targeted - not to you!)

FAQ: Training for Certification

Once again today I received one of the most frequently asked questions from execs considering developing certification: "Can taking our courses or training programs be a requirement for certification?"

The simplistic answer is yes, but only because there are no MANDATES that certifying agencies must follow in program design.  That is, you really are free to develop a credentialing program with whatever requirements you want and you can call it whatever you want (certification, registration, certificate, etc.)  However, just because you can doesn't mean you should.  You should know the implications.

First, mandating specific courses or training programs as a requirement for certification is in violation of the existing voluntary certification industry standards (ANSI and NCCA).   You're likely aware of organizations that require their courses as part of their certification programs.  In some instances, these programs – while called certification – may, in fact, be curriculum-based certificate programs (more on this later)...or they could be a blend between the two types of program.  In other cases, organizations may either not be aware of the industry quality standards or they have chosen not to comply with them.

Second, from the legal perspective, certification-related activities that are anticompetitive, discriminatory, based on subjective standards, or implemented without fair procedures are cause for antitrust concern.  In this case, a legal concern is a tying arrangement – that you are tying the certification product to an educational product (and requiring purchase of both).  Also, there is a fairness concern in requiring everyone to participate in (and pay for) coursework – regardless of whether they may need it.

Looking at the definitions from a purist perspective, if you are developing a program that includes both training and an assessment, that's called a certificate program.  The primary distinction between certification and curriculum-based certificates is in their focus. In certification, the focus is on assessing current experience, knowledge and skills. In a certificate, the focus is on training individuals to achieve a certain knowledge and skill base and then assessing attainment of it.   The programs meet entirely different goals so you need to identify your organization's goals before you determine which program type, if any, you should create.

For more information, check out the articles at this link.

Interesting Study Tips

Here is a certification test study tip I've never seen before:

"Think like a 60 year old white man."

This is a study tip included in a document posted on an ASAE & The Center online community for CAE candidates (you need to sign in).  Let me be clear that the community and the study tips are NOT provided by or affiliated with the CAE Commission.  ASAE & The Center created the community and any member is free to post what they want.

However, it's sad that such a statement is included on a test-taking tip sheet!  And, in defense of the CAE exam, it's not at all true.  The CAE Commission is not a group of 60 YOMM, nor is the item writing group.  I've participated on both (when was a 30something white female).  So, where is this impression coming from, I wonder?  Hopefully just the opinion of the creator of the list.

But it raises an important point.  As an association/certification exec, you should continually monitor the environment and see what is being said about you and your exam - good, bad or indifferent.  Maybe there are perceptions out there you need to address.

Certification and Associated Products

I mentioned in an earlier post that many in the certification industry are still debating the appropriateness of certifying agencies selling products or services other than certification.  The certification industry standards (like NCCA and ANSI) do NOT prevent selling products or service except under certain conditions like 1) requiring a separate product purchase as part of certification or 2) accrediting the education programs leading to the certification. 

However, some individuals believe certifying agencies should not sell education-related products and some believe they should not sell anything other than certification.  I don't get it.  If a product supports you in accomplishing your mission and goals, why shouldn't you sell it?  As an example, if your mission is public protection, why shouldn't you offer self-assessments to help your certificants indentify their learning needs?  Why shouldn't you offer an online searchable database of industry learning activities?  Why shouldn't you offer certificate programs in targeted areas where skills are lacking?  Aren't these products helping your cause? 

The usual argument is that it's a conflict of interest.  I don't see it.  Certainly, conflict can occur...if there is a professional association that also creates (or desires to create) these products.  But that's a competition (and political and turf) issue, not a conflict of interest.

So far I haven't heard a compelling argument for this "certification as the sole product" belief.  What do you think?

Do you need a hammer?

At the recent ASAE & The Center Great Ideas Conference, an attendee that I had just met asked me an interesting question; he said, "I've read a lot of your writings lately, and I can't tell so I was wondering....are you an advocate of certification or not?"

This shocked me initially because I guess I'd always thought of myself as a certification advocate, I'm a certification consultant for goodness sake.  But then clarity hit and I realized how many organizations have chosen to develop programs other than certification based on my counsel.  My response to him:  If certification is really right for the field, then I'm an advocate.  If it's not, then I'm not.  We proceeded to have a discussion about his certification program and the reasons certification may or may not be right for an organization, and what alternatives there are.  Then I asked him, "If you had NOT developed a certification program, would your field look any different than it does today."  A little sheepishly, he said no, was silent for a minute and then said, "I wish I'd have met you four years ago."  You see, there was a certification consultant who helped him build a  psychometrically sound and legally defensible program - unfortunately it's not really needed by the market.

So, here's my plea.  If you are considering developing a certification program, do NOT send out an RFP to certification consultants and/or testing agencies asking them to help you build it.  Why?  Because they just might help you build a really expensive program that has no impact!   Invest the time and resources to determine if certification would make the impact you  desire.  You might be surprised to find certification isn't the answer (or maybe it is).  There's a big toolbox out there -- certification, certificates, accreditation, awards, member classification systems, professional development, among many others.  Figure out what you're trying to build BEFORE you invest in one tool.  And if advisors are only selling one tool (usually certification), take great pause before signing that contract.    I don't believe anyone would intentionally try to sell you a program you don't need.  It's just that, if all you're selling is a hammer, everything tends to look like a nail.

Note:  many of the writings my colleague was referring to are included in the book and blog of the same name: We Have Always Done It That Way:  101 Things About Associations We Must Change.

Next Generation Credentialing

In case you didn't see it, Shannon Carter and I published an article on Next Generation Credentialing in the June Associations Now magazine.  Also, I'll be taking a concept touched on in the article - purpose-driven certification products and services - to the next level at a presentation at the 2007 NOCA conference:  Exploring New Frontiers in Credentialing, November 14 - 17, 2007 in San Antonio, TX.  There's no conference web page yet, but I'll post it when available.  Hope to see you there!

New Standards for Technology-delivered Assessments

ISO/IEC has released a new international standard: ISO/IEC 23988:2007 Information technology - A code of practice for the use of information technology (IT) in the delivery of assessments.

The aims of the standard are to provide a means of:

  • showing that the delivery and scoring of the assessment are fair and do not disadvantage some groups of candidates, for example those who are not IT literate;
  • showing that a summative assessment has been conducted under secure conditions and is the authentic work of the candidate;
  • showing that the validity of the assessment is not compromised by IT delivery;
  • providing evidence of the security of the assessment, which can be presented to regulatory and funding organizations (including regulatory bodies in education and training, in industry or in financial services);
  • establishing a consistent approach to the regulations for delivery, which should be of benefit to assessment centers who deal with more than one assessment distributer;
  • giving an assurance of quality to purchasers of "off-the-shelf" assessment software.

The 39 page in-depth standard can be purchased and downloaded from the ANSI store.  All associations considering or offering computer- or internet-based testing (and all associated service providers/vendors) would benefit from a review of this informative standard which, by the way, applies to both low and high-stakes assessments.

Requiring Training for Certification

I was asked today if an organization could require candidates for their certification program to purchase their training materials.  This is becoming a popular question.   As always, there's not a clear-cut answer.

If your organization has or is trying to create a professional certification program for the field, then it is not desirable to require purchase of training materials your organization has created.  Doing so would not be in compliance with the certification industry standards promulgated by the National Commission for Certifying Agencies or the American National Standards Institute for their accreditation programs for certifying bodies.   The problem is not with offering the training, it's with REQUIRING it.

If, on the other hand, your organization's program is a curriculum-based certificate program, then the training is an integral component of the program so it can and should be required. 

There's a final IF.  If your program is really a blend between certification and curriculum-based certificate, then the answer gets even greyer.  The answer then depends upon how the program really functions in the market as to whether or not it's a good idea to require study materials.  Generally, it's not.

Click here to learn more about the distinctions between professional certification and curriculum-based certificates.