First, we vetted the 100 vendors to submit to a vendor product catalog, over 50 submitted which we used the data to pair down who were appropriate for the Wave report.
Hands-on lab evaluations: I spent up to 6 hours with each vendor in a windowless room to evaluate their product live using common customer scenarios. I grilled the executive team, and discussed their strengths and weaknesses.
Product demos. We asked vendors to conduct demonstrations of their products’ functionality. We used findings from these product demos to validate details of each vendor’s product capabilities.
Customer reference calls. To validate product and vendor qualifications, Forrester also conducted reference calls with up to three of each vendor’s current customers for a total of up to 27 customer calls.
We collected hundreds of screenshots, presentations, samples, reports and all of this information was entered in a multi-tab spreadsheet that accounts for thousands of cells, scoring, and detailed explanations which clients can use to toggle up and down specific needs as in some cases, specific feature needs may need to be highlighted over others.
In the bottom links, I’ve made my research process very transparent, and have indicate the other three other blog posts documenting this laborious research effort.
