I just found the following interesting opinions of Zubin Ghahramani in the presentation he gave at the Machine Learning Summer School 2005:
I have no idea why anyone would want to use non-subjective priors. Objective priors are fraught with inconsistencies and no modeling is truly objective anyway. If you want robustness make sure your [...]