In My Humble Opinion, the ridicules of how some researchers and clients choose to deceive/convince themselves by massaging subjectivity into objectivity in order to support an opinionated cause is not a surprise. The industry of web-design is simply too complex and hard to grasp given its nature; human’s interaction with technology, that “scientific” research can’t be carried out without disputes and billion-dollar investments can’t be merely based upon conjectures of so called experts.
These elements essentially gave rise to all these debates and arguments. I personally have no issue be it ethical or methodologically about the conjuring of seemingly ill constructed research and its translation into the twisted truth that may seal web redesign deals. What should be of concern is the realism researchers construct the “scientific” approach and interpret their findings to be. Researchers should be given the authority and discretion to fully utilize their judgment based on years of experience, and not feel inferior in using such ways to frame their user research. However, great care must be taken in achieving representative samples and identifying key issues to be addressed, as wrong identification of problem is synonymous to ineffective solutions that waste company’s money. Though realism should be emphasized, but the attempt to build user personas to the extent of simulation of everyday lives of target consumers is interestingly unjustifiable in any economic sense. It’s as much a marketing research gimmick as it is unrepresentative, negating all principles of marketing research.
Another aspect that is harder to control but should be given equal weightage of attention is the interpretation process of key findings and statistical data. This can be said to be just as deterministic as the problem identification process in making the research a success. Eye-tracking and many other research tools that aid web-design may prove to be very useful in generating insights, but without in depth and experienced-based knowledge in the relevant field of research, it will be like letting a greenhorn driver maneuver a BMW, preventing the achievement of full potential.
Its use as a political tool in the corporate world hasn’t been a real shocker too. It’s intuitive that CEOs and senior management can never be comfortable with speculative recommendations founded upon qualitative findings. It may be a waste of money for redundant research to be carried in such purpose but think of the potential losses that may occur if redesigns are not passed on due to “uncomfortable” stakeholders.
The article has in its best effort tried to plea for greater acceptance of qualitative and “less-than-scientific” method in this field of web-design centered upon the end-user. Risks and sure-win recommendations simply does not co-exist in this arena.
Lastly, IMHO, the smoke (in title) should be referring to the illusionary and vague existence of solutions based upon scientific and quantitative data, while mirror (in title) tells us how self-reflective some problems in this field is, that more often than not, research is redundant and unnecessary in devising solutions.
