Ian Ayres presents two main methods in his book “Super Crunchers” of rationally managing business: regression analysis and randomized tests – both of course with large samples. I’ve previously written about a specific correlation (within regressions) and how interesting research in that area would be. However, for academics without proper investors, this is mostly not the feasible approach for a paper, due to the lack of access to the data.
What does seem feasible, though, is randomized testing: You can easily set up test sites on the internet. It gets more complicated again when trying to generate enough traffic (i.e. testers) onto the page. And if you have specific demographic needs (as in my case managers), this can get rock the feasibility again.
Is evidence-based management really feasible?
I think this is often a problem with the concept of evidence-based management. I really, really like the idea. But most of the time, especially in innovative areas, you just don’t have the evidence due to untested terrain or too small sample sizes. And even if someone has just conducted an appropriate study, either the time to find it or the price of the study creates a higher cost than deciding wrongly and undoing your decision would. If a mistake would mean live-or-die, there is no evidence available for an appropriate price and conducting own research would be too costly as well, for heaven’s sake, find a way to never have to make that decision.
Reduce particularities with abstraction
But does this mean researchers without the means (such as me) should either leave the idea of regression or randomized experiments where the data sets are not readily available? I think sometimes you can bring the research onto an abstract level and thus remove the particularities. In my case (proving a manager’s opinion about something), I believe that a manager’s opinion is not so different from low level employees; depends on what the decision-making process is about, says my gut feeling. So in order to be able conduct a randomized experiment with regular low level employees as a representation of managers, I would first have to prove that the matter concerned invokes the same opinions with these employees as with managers. Is that feasible? With desk research (there already is a study available that proves this general concept), yes. And what if the secondary research proves that managers do think differently about this aspect? Thesis bust, I guess. I’ll need to find that study early on…