Cross posted from myResearchSpace
I have recently been looking at issues around best practice in web design. Best practice is often encapsulated as the perceived wisdom of professionals which has become enshrined in the collective mythology of the profession. An example of this in web design is that "users will give up on a website if they have to click more than 3 times"; a hypothesis with no experimental basis and with at least anecdotal research evidence (http://www.uie.com/articles/three_click_rule/) suggesting that it is false (there is little correlation between the number of times a user clicks and a user's successful accomplishment of a task).
With a growing sense of frustration at not being able to establish the relative credibility of any of the advice that was been given on best practice in web design, I came across a book published by the US Department of Health and Human Services called Research-Based Web Design & Usabilty Guidelines which uses research findings as the basis for web design guidelines. Apart from being beautifully presented and freely accessible online, it is the methodology used to present the guidelines which is truly impressive. Each guideline within the book is given 2 scores. The first is one for 'Relative Importance' rated by a panel of 16 reviewers, half of which were usability specialists and half web designers. The second score is for 'Strength of Evidence' for each guideline, again rated by a panel of researchers, authors and designers on the basis of a 5 point metric.
I wonder if there are lessons to be learned from this in the application of presenting research in a thesis?