| | TDJ
Registered: Dec 2001
Posts: 1879 |
csdb gem top 40
For those curious about how the csdb charts would look if negative votes were cast aside, I've written a special and completely useless article.
Read it at http://www.lyrical.demon.nl/csdb/gem40.html .. Don't mind the lack of pretty pictures or the ugly layout, it's the text that counts :)
And ofcourse I'm more than interested in your reactions, so bring it on! |
|
| | SIDWAVE
Account closed
Registered: Apr 2002
Posts: 2238 |
Well, it's very hard to make any fair system.
I don't understand your argument that demos are down-voted. If they get a 1 instead of nothing at all, that still counts more than nothing I guess..
There is also the issue, what you want the votes for ?
Personally, I've used them to download the 'best' demos, to have some sort of 'top' collection to use for my private parties.
About the scale, I think 3 should cover it.
Bad, good, the best.
|
| | yago
Registered: May 2002
Posts: 333 |
Quote:
The problem was ofcourse that it would take quite some time until there were enough members to get a somewhat representative chart. [setting up a gem voting on another server]
Thats also a Problem of CSDb. If you look at the Number of Votes, there are never more then 60 Votes for one Entry, which are very few compared to events like Breakpoint[1] or Assembly.
I think the "CSDb Demo of the Year" (or what was it called) is a nice Way to work around this, unfortunately the timing for the last one was too tight (at least for me, before I could transfer enough nice Stuff on my real C64, the "casting" was over, and I was annoyed because none of my favorite Productions appeared).
Wishlist Time: If the Demo of the Year is gonna be done again, I would like more Time for casting, and massively Announcement on all kind of Sites/Foren, and maybe even a gem voting for this special occasion.
I fear that gem voting alone is not enough to prevent up/down-voting, we should have a massive Amount of Votes to prevent such bullshit.
Zed Yago
[1] OTOH, I will never understand how "Return of Darkness" placed 2nd!
|
| | TDJ
Registered: Dec 2001
Posts: 1879 |
Quote: Well, it's very hard to make any fair system.
I don't understand your argument that demos are down-voted. If they get a 1 instead of nothing at all, that still counts more than nothing I guess..
There is also the issue, what you want the votes for ?
Personally, I've used them to download the 'best' demos, to have some sort of 'top' collection to use for my private parties.
About the scale, I think 3 should cover it.
Bad, good, the best.
Example: a demo has 5 votes, 4 times a 9 and one 10. This would give an average of 9.2, which would make it #1 here at csdb (for the sake of this argument let's pretend csdb doesn't use a weighted average). Let's say somebody doesn't like that demo to be on top, so he gives it a 1. The average will become 7.8, and the demo will disappear from the top 10. This is called downvoting, and most of the time it doesn't have anything to do with the quality of the demo itself.
Like I said in the article, I only care about votes given by people for demos they really like, hence I only take those in account. The goal is very simple: to get a list that hopefully will point people to demos they don't know. And I like playing with numbers, sue me :) |
| | TDJ
Registered: Dec 2001
Posts: 1879 |
Yago, ofcourse you're right: more votes means better representation of the average taste. But there simply are too many options you can vote for here, in that aspect the demo of the year was easier, as there were only a limited number of demos.
On the other hand, the used system meant that a lot of demos which should be in that list were missing. If the same system would be used this year it would mean a disaster, as Floppy & LCP already are responsible for at least 5 definitive entries on that list.
Anyway, the only way downvoting is possible with 'my' system is by giving a demo you don't like an 8 .. but at the same time that will probably mean a better score in the csdb chart :) |
| | yago
Registered: May 2002
Posts: 333 |
Quote:
Anyway, the only way downvoting is possible with 'my' system is by giving a demo you don't like an 8 .. but at the same time that will probably mean a better score in the csdb chart :)
I am the King of all Downvoters! ;-)
If you want to downvote using the gem system, you pick 3 crap Entries, so that the to be downvoted Release gets 0, while 3 crap Entries get much Points.
Even better is not to use crap entries, but in Quality comparable Stuff to the to be downvoted Release.
If the Demo of the Year gets enough Votes, it would constantly uprise the Number of total Votes.
BTW: The design of your "csdb gem top 40" looks familiar to me ;-)
|
| | TDJ
Registered: Dec 2001
Posts: 1879 |
Quote: Quote:
Anyway, the only way downvoting is possible with 'my' system is by giving a demo you don't like an 8 .. but at the same time that will probably mean a better score in the csdb chart :)
I am the King of all Downvoters! ;-)
If you want to downvote using the gem system, you pick 3 crap Entries, so that the to be downvoted Release gets 0, while 3 crap Entries get much Points.
Even better is not to use crap entries, but in Quality comparable Stuff to the to be downvoted Release.
If the Demo of the Year gets enough Votes, it would constantly uprise the Number of total Votes.
BTW: The design of your "csdb gem top 40" looks familiar to me ;-)
Eh no .. I don't take the total number of votes in account, but the number of votes for a release. So if you give gem-votes to 3 crap entries this would not influence the score of other demos. And the 3 crap entries would only appear in the list if you get at least 9 other people to vote for them as well. In which case I will personally disqualify those crap entries ;)
edit: I added an example to the article, I hope everybody understands now.
And I'm using the csdb style sheet on purpose :) |
| | SIDWAVE
Account closed
Registered: Apr 2002
Posts: 2238 |
Nomatter what, the people here using this have their own winners, and ig they are not in the official top list, I don't see anyone writing complaints about it.
We all know what are really the best demos.
Those are the ones that we load again and again, over time, because we need to hear that tune, in that screen, seing that gfx, reading that specific text, thinking about how the one-framer is made.. because it all makes sense to us personally.
Anyhow, I should not meddle in this debate I think.
I don't feel qualified.
Personally I like good gfx, music and code. And any combination is good for me, if just 1 of these are there.
(this makes me load demo again and again)
And I do like scrollers or text in general, and I think a demo without any message with meaning is pointless.
I also think Deus Ex Machina is the most boring demo I ever saw. Looks nice and all, but it's just missing something that makes it a real demo.
(now kill me if you want) :-)
From the top 10 i probably like the most right now and since some time: Oneder, Krestology, Dawnfall, Dutch Breeze.
But to me, none of them are better than the other, I simply watch them at any 'needed' time to get that feeling they put me in, and then they fulfill something in that moment, and so they are all equally great to me. Each in a different way.
So I shall say no more about voting, I can't use it for anything really.
:-) |
| | TDJ
Registered: Dec 2001
Posts: 1879 |
This is not a list of the best demos ever on the c64 - it is a list of the *most favourite* demos as voted by the csdb members. Sure, if you would take a survey at a big party, ask everybody to write down their 10 favourites you probably would get a whole different list, but that doesn't mean that this one has less merit.
I think the outcome of the system I used was rather interesting, so I wrote an article about it. Does it mean that I agree with the top 40? In no way, a lot of my favourites are missing, and at the same time I'm not too crazy about a lot of demos that are there (Insomnia wouldn't be in my top 40, or even top 100). But to me that's part of the fun: comparing your own taste to that of the others.
And maybe, just maybe, you'll notice an entry that you've never seen before, or a long time ago, and decide to take a look at it. And maybe, just maybe, you might discover another gem. Now wouldn't that be nice? :) |
| | Steppe
Registered: Jan 2002
Posts: 1510 |
<irony>
What Deus Ex Machina only on 17, closely followed by Batmania? Muahahaha! ;-)
</irony> |
| | Steppe
Registered: Jan 2002
Posts: 1510 |
But the project is interesting, keep it up TDJ! :-) |
| | TDJ
Registered: Dec 2001
Posts: 1879 |
Quote: <irony>
What Deus Ex Machina only on 17, closely followed by Batmania? Muahahaha! ;-)
</irony>
And why not? There's no doubt that Deus Ex Machina is a much better demo (no wonder, since it was made 11 years later by somebody who was already a top coder when Batmania was released) but bear in mind that a lot of people prefer the old style demos as well - hence they vote for demos such as Batmania, and maybe not for D.E.M.
Also, what good would it be if D.E.M. would be #1 on this chart as well? Remember: this is an alternative to the normal chart, giving a different view but using the same votes :)
I'm not sure where I will take this though - it's more of an experiment so far, so chances are that there will be no follow-up ;Z |
| | SIDWAVE
Account closed
Registered: Apr 2002
Posts: 2238 |
I think we need the old system used in the 80's charts.
Once a month it's voting time, and once a year it's demo of the year.
It gives a better reflection of what people think, at any given time. The chart becomes more 'correct' IMO.
Online 'floating' votes, well - you can always vote on an older demo, so the current events are probably not up to date. And maybe you don't see all the new demos.
What about monthly vote rollcall, and then you cant vote for that demo anymore, that way.
At end of year you can vote again, for best of the year.
And then you have an online vote, like system is now, that is just "one of the best demos I ever saw vote", and that can be given at any time, and then no more.
?
|
| | Hein
Registered: Apr 2004
Posts: 942 |
UP voting is the same as DOWN voting, only different direction. Who sais that someone who votes a 10 isn't as biased as someone voting a 1?
I'm curious though what makes it possible that 64allstars had 5x10,5x1 and 1x9 gives the average of 9.2. Later someone votes another 1, making 6x1, 5x10 and 1x9 all of a sudden the average is 2.3. What's the magic formula used on CSDb? |
| | TDJ
Registered: Dec 2001
Posts: 1879 |
Quote: UP voting is the same as DOWN voting, only different direction. Who sais that someone who votes a 10 isn't as biased as someone voting a 1?
I'm curious though what makes it possible that 64allstars had 5x10,5x1 and 1x9 gives the average of 9.2. Later someone votes another 1, making 6x1, 5x10 and 1x9 all of a sudden the average is 2.3. What's the magic formula used on CSDb?
As for the up/downvoting thing: completely true, that's why I have the "at least 10 gem-votes" rule.
And the 64 allstars thing is by far the most clear example of why the csdb voting system doesn't work, i.m.h.o. :) |
| | TDJ
Registered: Dec 2001
Posts: 1879 |
Quote: I think we need the old system used in the 80's charts.
Once a month it's voting time, and once a year it's demo of the year.
It gives a better reflection of what people think, at any given time. The chart becomes more 'correct' IMO.
Online 'floating' votes, well - you can always vote on an older demo, so the current events are probably not up to date. And maybe you don't see all the new demos.
What about monthly vote rollcall, and then you cant vote for that demo anymore, that way.
At end of year you can vote again, for best of the year.
And then you have an online vote, like system is now, that is just "one of the best demos I ever saw vote", and that can be given at any time, and then no more.
?
Okay, so if I understand you correctly you want 2 different voting systems: 1 for current demos, one for all-time demos. For the latter one you want to keep using the system already in place, and for the first one you want to use the 'once a month+once a year' system?
That might work (it would filter the relatively big amount of 'new' demos from the all-time chart, demos that are mostly there because people just saw them), but on the other hand: with so few demos being released in 2004 (less than 50 so far), why bother? Why not just select all eligible demos at the end of the year based on average csdb score (let's say all demos with at least a score of 6), make a zipfile, ask people to download that, take a look at the demos in it (wouldn't take more than a few hours max) and then give their top 3 or something? For demo of the year that would be sufficient, and it would surely be better than the current system of just taking the party winners .. |
| | Ben
Account closed
Registered: Feb 2003
Posts: 163 |
I personally have always perceived a unidimensional system inappropriate. I have once tried to rank musicians, but rather by introducing a 10 dimensional system and giving weights to each and every one of them. This system even included a dimension like 'employment/development' (how did a musician develop over time), and the obvious categories sounds, melody, use of player, impact/popularity, et cetera, and -yep, even- overal impression.
Actually using a multidimension system in the appropriate way is indeed quite laborious, but it would reflect more precisely what the qualities of particular demos are. I can imagine e.g. Krestology would end up high in the graphics and design category, but One-Der would score considerably higher in the code-technical difficulty category.
By introducing an overal impact or innovativeness category, the 'radicalness' of inventions can be easily captured irrespective of the time at which the inventions are done.
Just some thoughts..
<edit>
Please note, voters are also expected to provide the weights. Some people are mostly interested in technical code, others prefer the atmosphere over code, et cetera.. |
| | TDJ
Registered: Dec 2001
Posts: 1879 |
For my old system I also gave scores for different disciplines (code, gfx, music, even for how much of the demo was made by the group itself) and then calculate an overall value. But, apart from it being way too much work, it's also nonsense, I feel, as it is the total that counts. And have you ever seen a music review where they give different scores for music, lyrics, and the way the band dresses? :)
For my own personal ratings-database I use the 5 star system: 1 star = poor, 2 stars = fair, 3 stars = good, 4 stars = excellent and 5 stars = magnificent.
I use it for 2 different purposes: first of all to see what demos I will store on a real floppy so I can view them on the real c64 (all gems, demos that score 3 stars or more), and also for a new project that will start early next year, about the 'best' c64 demos ever, and the groups that made them.
Roughly translated: 1 star = 1-5 here on csdb, 2 stars = 6/7 here, 3 stars = 8, 4 stars = 9 and 5 stars = a 10.
Ofcourse it's not really about how good a demo is, but how much I like it. And that's what counts. Why would I rate a demo that's technically perfect, has good gfx + music but no soul, and is a bore to watch, a 10?
And thus we get back to my article, where 'favourite' votes are all that matter :) |
| | jailbird
Registered: Dec 2001
Posts: 1578 |
TDJ, more than a year has passed since your article was published and it seemed to me as hell of a good idea! Since it's not updated dinamically, I'd be really interested to see the current situation on the GEM chart... Any plans to continue the project and to write another article?
|
| | TDJ
Registered: Dec 2001
Posts: 1879 |
Actually I was just thinking about it the other day .. no real plans though but if more people might be interested I may give it a go. |
| | Ed
Registered: May 2004
Posts: 173 |
I am looking forward to more of this. Go ahead!
|
|
