On playing/testing the Trials of Darkness mod, I had the idea that, perhaps, it should be easy for modders to find out how their mod is being played, how people are making progress in it. Of course, there is only one way to truly find out: objective statistics tracking.
This would mean keeping track of where a player died, how much time he spend on certain parts, how many items he is carrying, what dialog choices he made, etc. And, apart from keeping track of these things, sending them over to the mod developer in some way.
This would massively ease the amount of work that playtesting entails, and it would give rise to higher quality mods. Players giving up in frustration after half an hour will be noted by the modder, and he will be able to see why and take corrective action, thereby giving a better experience for players.
Of course, there are some caveats. Anonymity should be guaranteed (I don't want people to know I struggled for ages trying to kill that measly wolf), perhaps even only aggregated statistics should be available (to avoid modders focusing on single playthroughs). People should be warned of and be able to opt out of statistics collecting. And it's a bit of work to implement I guess.
But I think it would be really, really insightful for the developers. To give an example, Valve is doing this all the time, and I believe their games are getting better for it.
What do you guys (and Ville) think?