Churnalism - yet another evil in the field of benefits reporting

So @stevie_chambers decided to wind me up with this tweet yesterday morning :- 

Cisco Unified Computing System selected by Slumberland http://bit.ly/d1FDqd < saved $368k @ianhf :)

Now Steve is somebody that I regard as a friend and that I have immense respect for - despite us having differing views on the technology widget Steve's company sells I have more respect for Steve than the majority of his employers organisation put together.

So with that out of the way - I was simply staggered that he should choose to highlight such a terrible article & classic example of Churnalism to me - I can only put it down to 'new baby head', or perhaps a desire to see me off with ever increasing blood pressure? So let's just review the 'article' :) (note I'm not saying anything about the technology here at all - merely what & how 'benefits' are purported)

This is the kind of lazy, half arsed, factless, press release driven tripe that masquerading as 'reporting' that drives me (and I hope you) insane! So time for the usual list :-

• How long did this take to achieve?
• How exactly was it achieved, which how much effort & disruption?
• Re "saved" - where did they start from re SLAs, technology, organisation & processes?
• Compared to what & whom?
• Where is the before & after comparison of TCO and TTM?
• What was the investment required to achieve this?
• What was the ROI & IRR?
• What is the timeframe over which the benefit is calculated & reported?
• How much more would be possible with other ways, priorities & measures?
• What is the scale of the environment?
• What is the scope of the environment?

I'm not arguing that Slumberland haven't benefited, and that they haven't improved things for themselves - but as with all case-studies, customer references or benefits reports it is absolutely imperative to have full & total contextual disclosure of the before & after situations, the objectives, the success metrics and priorities.

As I previously blogged about here it really is time for the IT industry to grow up and get some simple quality & disclosure standards about reporting and marketing... This current level of information just doesn't help anybody...

Claims Justification

So often we hear press releases, blogs, whitepapers, tweets, marketing powerpoints or spoken (or whispered) claims concerning IT technology or (increasingly) architectures, about subjective & relative statements (faster, cheaper, more available, better, sexier etc) BUT

How often is any justification, edvidence or proof given for this?

I'd go so far as to state that claims without justification aren't just unhelpful to anyone, they are downright dangerous :-

• Sets false expectations, that can never realistically be achieved - thus positioning everybody for a dismal future of perceived failure
• Wastes customers time having to evaluate the claims and justify what that actual possibility is and the context required in order to achieve such claim (if at all possible)
• Detracts from the real positives that genuinely are out there in vendor land (honest there are some but you need to look very hard)
• Ruins the credibility of the parties making the claims
• Destroys the trust the customer has in the party making the claim, which means all future claims will have to be validated 

One of the better things of the evolution of humans is that in early times quacks made random & often fraudulent claims, however through scientific method we've moved into an era where claims made in that area are now fully validated before being accepted. Why does this appear to be so hard for IT technology providers?

So to be clear, if you make a claim on performance, availability, functionality, commercial benefits etc be sure to be able to justify the claim objectively with facts & evidence showing how the claim was formulated and any assumptions or prerequisites. Next you must be prepeared to share this information with exactly the same forum and audience you made the original claim to.

If you don't justify your claims expect me to a) shout loudly that you make fiction based claims, b) ignore your claims totally, c) regard your company & technology with considerable disdain...

There are words for people who make frequent claims without justification - and do you really want to have the life-span and popularity rating of a politician? If you can't, or won't, justify your claims then I'd very much prefer you don't make any...