Skewed Ratings

[bulbasteve]

Or readers could like, read the reviews, disregard the 100/100 or like, 99/100 ones, and use the nifty drop-down feature that breaks down exactly why the user rated the server the way they did.  If it seems credible that a 'new' server get rated a 50 or so on their economy or WoE competition, and that other values seem logical, one can most likely take that person's review to heart. 

The problem is that the rating effects your position on the list and the general score. Just because someone votes for 5 for economy and comepetition doesn't mean they hate the server and sure doesn't make it a bad review. And on both ends people will fudge the the ratings on the individual things to get at the general score they want.

Maybe a "tilt" could be added like they do on this game review sites that also break down reviews into the individual factors. That way people won't feel the need to embellish the truth on the individual ratings to get the score they want. So while you may have it as a 5 for economy you may use a tilt of 5 positively to make up for it.

Down side is ... spammer has more reason to make more accounts to spam ... i am already sick of ppl spamming good/bad reviews why can't they just do by the rule and make one review only.

Well the fact of the matter is that the first page of the listing (non special sorted) is going to get way more hits than page 6 or 7 so people either have to totally lie and give everything 100 or start faking reviews to force their server to show up higher on the list. In that case you may want to reconsider how many reviews are on a page and things like that. Or the fact that new reviews are the only things that show on the main page probably also encourages people to keep a steady stream of reviews coming since it gets eyes on the server page.

Though of course other more umm "controversial" servers just bring their drama with a rating war to the reviews, but I'm not sure what can be done about that one...

I believe the idea of volunteers taking care of it is a good idea. Only problem is finding people and make sure they're being fair. Hence I present an "improvement" of sorts to the idea; Why not put a team of 1-3 people in charge of recruiting and moderating a team of review moderators? Wouldn't really be more work for the admins, And I believe quite a bunch of the people on this forum would enjoy helping out.

Hmm, would that sounds like a good idea, would you have multiple reviews using a server at the same time to get more views on a server or just have these people in charge overseeing their reviews to make sure they aren't biased as hell?

Even with a team of volunteers it isn't like they have to be reviewing every server there is, maybe there should also be a submission aspect to this where people to recommend a server to review. So it isn't just whatever server randomly they draw out of a hat but ones that the reviewers are interested in seeing and have features, events or something that make it worthwhile to examine for everyone.

[Mieu]

The problem is that the rating effects your position on the list and the general score. Just because someone votes for 5 for economy and comepetition doesn't mean they hate the server and sure doesn't make it a bad review. And on both ends people will fudge the the ratings on the individual things to get at the general score they want.

Maybe a "tilt" could be added like they do on this game review sites that also break down reviews into the individual factors. That way people won't feel the need to embellish the truth on the individual ratings to get the score they want. So while you may have it as a 5 for economy you may use a tilt of 5 positively to make up for it.

Yeah, but the same perspective could be applied to the site-wide listings as well--servers with five total reviews and an average score of 99.999 (an exaggeration, of course) would probably be viewed with skepticism.  Likewise a server with a total of 100+ reviews and an average score of 75 might be worth a closer look.  I'm not saying there's no value whatsoever in the current rating system, just that it'd be a feat to come up with something better--that takes every aspect of a server into account while thwarting serial-100voters.  The 'tilt' idea is a good one, but it won't keep people from voting a 100/100 on a server stat with a positive tilt, and likewise pissed-off players voting low with a negative tilt simply for the sake of doing so, unless I'm misunderstanding the concept of tilting, in which case I'd appreciate you elabourating

Slaw's idea is a good one, but there's always going to be one thing to keep in mind--people's opinions as to what an 'acceptable' review is and what isn't is completely relative (sometimes).  While no one can deny that a expletive-ridden review scoring the server at rock-bottom is unacceptable, is it within the right of the moderators to say 'no, this server doesn't deserve 100, please lower the score.' Another shade of grey could be that a player might have a negative opinion of several aspects of the server.  They may write a brutal review, which isn't dishonest or rude, but totally their opinion.  One moderator might decide that the player's review remain, another one might think it too harsh and take it down.

Just being the devil's advocate.  Should I stop babbling now or what?

[bulbasteve]

The 'tilt' idea is a good one, but it won't keep people from voting a 100/100 on a server stat with a positive tilt, and likewise pissed-off players voting low with a negative tilt simply for the sake of doing so, unless I'm misunderstanding the concept of tilting, in which case I'd appreciate you elabourating

Well the problem is I'm sure some people actually DO think that servers deserve 100/100 or 1/100, but at least with the new system they dont have to lie to do it. So a person can simply look in the detailed description to see that they acknoweldge competition is a 5 in a good review or that balance is a 10 for a bad review.

I suppose another option for the two ends would be that it is actually required that the use the other idea here, that of the expanded review. That way a moderator can decide if they have fully supported a very low or very high score. If you just say "this serv0r suxxors" then the score will not count, but if you list your reasons for the score it will be accepted.

They may write a brutal review, which isn't dishonest or rude, but totally their opinion.  One moderator might decide that the player's review remain, another one might think it too harsh and take it down.

Guess you could just go the multiple reviewers for servers route. You know 3 on each or something so that you can see if there are disagreements and stuff (and just for the fact they may focus on different aspects of the game anyway).

[Slaw]

Hmm, would that sounds like a good idea, would you have multiple reviews using a server at the same time to get more views on a server or just have these people in charge overseeing their reviews to make sure they aren't biased as hell?

I think just having them go through already written server reviews and removing obviously biased ones would be good enough. Maybe lurk the different servers' forums and see if they give out items for reviewing and such?

[Mieu]

I love doing that, Slaw.  Its like watching a hundred different soap operas, without the TV!  Reality entertainment at its finest.

[Slaw]

Indeed. Trolling forums is win.

[bulbasteve]

I think just having them go through already written server reviews and removing obviously biased ones would be good enough. Maybe lurk the different servers' forums and see if they give out items for reviewing and such?

I'm not even sure that would work, the server owner would argue that they are still giving genuine reports, and the only real hard and fast rule for reviewing is that they don't do proxy ones. Because a review really is SUPPOSED to be biased, it's a review! Especially in bad reviews, sure if you think the GMs are horrible and noone should join the game is clearly biased, but that doesn't mean it isn't totally true.

At least if you force them to write a long review from now on for very high or low scores it may make them rethink if they should simply give it a more "fair" rating and force people to fully justify their reasons for making it very good or bad. Because going back and time and instating a new rule on what is biased or not I think can only lead to trouble...

[Slaw]

I'm not saying we should move in and remove any review that sound biased in some way or another. Just that it isn't that rare that a server with around 8 people on suddenly get 10 reviews all saying something in the lines of "This server is the greatest!" with all 100% score. That's the type of behavior we should try to take care of.

[bulbasteve]

hmm yeah sure that works, at least for the old ones...