I would say though, that we’ve tried to help buyers develop a short-list or recommendation, with the new introduction of scenarios, where we’ve graded each system according to how well they fit.

This means that buyers should be able to identify their projects with the scenarios and based on that narrow down the list of vendors.

In the early chapters of the report, there is a few pages on the actual selection process. One of the things I would argue is that the type of Excel based tool that you mention, adds very little value to the selection process. In particular I’m very critical of the weight and in general feel that much precious time is wasted discussion the ratings in Excel spreadsheets.

There are better and more efficient ways at arriving at the “best” CMS. If you could narrow it down to 2 – 3 vendors, I would strongly encourage buyers to do a prototype to learn more and educate themselves on the product.

James Robertson has some good free articles on this. A good place to start is http://www.steptwo.com.au/papers/kmc_evaluate/

Cheers, Janus