Tuesday, August 15, 2017

A VERY brief note on certification..

I'm well-known for my distaste of certifications for software developers, trainers, and coaches.

What seems to be less-well-known is the reasoning behind that.

GETTING certifications is not the big problem.
  • I do not seek or accept certifications, but it's not because owning them is evil. 
  • Yes, a certification is a proxy for a reputation. 
  • Yes, I value reputation greatly and certification almost not at all.
  • No, it's not because the certifications are too easy to acquire. 
  • No, having a body I trust more doing the certification would not help.
  • No, I don't want to certify people or back a certification.
  • No, I'm not jealous because they're making a lot of money.
  • No, I am absolutely not against education in any way, shape, or form. 

Vendor Distraction

My problem is that once you become a certifying body then certifying more people becomes an important part of the revenue stream. You want to do so profitably so certifying more trainers to sell more certificates becomes important. I believe that within 5 years it becomes far more important for a business to serve the needs of the certification business than to serve the needs of the software industry.

The distraction is dangerous.

What makes the certifier successful is not what makes the industry successful.

Brand X selling 25% more certifications doesn't make the customers who are actually using Brand X processes even 1% more successful in their business.

We are supposed to be helping people solve their problems via software. Selling certificates won't necessarily help in that direction.

Ossification

Certification can't operate without standardization.

In a period of rapid transformation and growth, how reasonable is it to standardize? How good is it to hold a certificate in version 1.0 of a process that is now on version 10.23? If the topic changes since you learned it, then your learning becomes (essentially) useless - especially to the hiring manager.

If we're changing every year, should we standardize and therefore slow the rate of change, or should we wait and standardize once a single clear way of working is truly dominant and widely accepted?

I will argue that today's best practices become the antiquated flotsam of the past VERY quickly.

Our best practices will survive months, but certainly not years. Values may be settled, but this is the world of the web and of changing programming languages and market segments, Lean Startup, DevOps, and CD. What made sense a year ago still needs to be revisited and revised.

I say this even though I observe XP technical practices have aged very gracefully indeed, though their original formulations been augmented and sometimes replaced with different tooling. Nobody saw GIT and CD coming back then. Note that there is no certification for XP practices.

Bad Metrics

Having too difficult a certification becomes a problem because markets love success stories. How many millions have we certified? How many more people have been certified this year than last year? Are we having a banner growth year in our certification business?

People see the growth or decline of the training business as an indication of a growing or declining topic area. When we have a banner year, doesn't that mean our brand is having a banner year?  When we have a slowdown, doesn't that indicate our brand is dying off? For instance: Doesn't fewer Brand X certifications mean Brand X is dying? 

I think this is Goodhart's Law (and maybe Campbell's Law) at its worst.

Aging Materials

If certification can't freeze the march of time, then the materials are aging ungracefully and quietly in the background. As mentioned earlier (under ossification) this is certification-rot. Constantly upgrading materials (including online tests) is an ongoing cost.

Certification isn't the most profitable business in the world. How much time and money should a certifying body invest in updating materials every single year? Every quarter? Every month?

If a certifying body updates the materials, won't it require retraining or even (gasp) recertification of trainers? Won't it damage the value of certifications issued in the past?

Imagine that email:
You paid $2000.00 for a certificate last month. We regret to tell you that your certification will be deprecated as of the 12th of next month when the new training materials roll out.

You can sign up for a retraining at the reduced cost of $1000/seat. 
Thanks for your continued business.


Is the reasonable approach, then, to not update the materials or to not deprecate obsolete certificates?

Which is more valuable to the certifier?

Which to the certified?

Which to the people who hire certified individuals?

Uncertain Recency

Staying with the theme of ossification and rapid change along with aging materials, let's assume there is an agile method called SACRUFEE (invented here, pronounced "scruffy"). Let's say that the method has several years of use and that there is an official document that describes its current state. 

Further, let's assume that the document describing SACRUFEE is updated periodically to keep it up-to-date.

Now, the problem is that you may not know if the materials used in your training are current or not. You probably don't know and have not asked.

On your certificate, does it say which version of the SACRUFEE MANUAL was used in your training?

Were you certified in 2017 to be competent in the quintuply-replaced 2011 SACRUFE style?

What if your friends were certified in 2013 SACRUFEE and you were certified in 2015.4 SACRUFEE method? Aren't they carrying the same certification as you are? Would you be held to the older standard because the majority know it?


Uncertain Authority

Is the trainer who taught your SACRUFEE certified in 2017.b version of the SACRUFEE METHOD MANUAL?

  • Do they actually know the current material? 
  • Have they ever participated as material contributors in a SACRUFEE project? 
  • Did the people who trained them ever materially contribute in at least one SACRUFEE project? 
  • Is what you were taught really SACRUFEE at all, or just a few of the principles and practices thereof? 
  • Did you learn the trainer's personally modified version of SACRUFEE, or was it the official version with some personal notes added? 
  • Do you and your trainers really subscribe to the techniques, values, and process of SACRUFEE at all, or is this "just a job" so you adopt the brand and skip the mindset, techniques, and process? 


So...

I don't mind that you have a certificate.

I'm not even mad that you've issued some.

I am worried about the business model of certification and the effects it has on the certifying bodies. Certifiers are the ones courting a deep problem.

The phenomenon of certification as a way that one body can confer a proxy reputation on someone who has not done the work (and thereby earned a real reputation) is upsetting.

The ability of a certification body to choose which practitioners' reputations and experience are valid is more upsetting yet, something that happens for real as soon as hiring managers start requiring SACRUFEE certifications.

I'm willing to grant that the creators of a method (and their selected body of like-minded people) have the right to define the method and decide who is using it or not. There's nothing wrong with saying "what you are doing is not SACRUFEE, but is interesting and seems to have good results."

And the phenomenon of a body bestowing a reputation proxy for pay, in effect limiting the field to those who can afford to pay for the proxy, upsets me most deeply.

It's probably not you. It's a bigger issue. But let's talk about that.

1 comment:

  1. nice post.
    https://www.knowlathon.com/it-service-management-and-excellence/itil-direct-plan-improve/

    ReplyDelete