Credit system seems unfair, anyone else thinks so?

Message boards : Number crunching : Credit system seems unfair, anyone else thinks so?

To post messages, you must log in.

Previous · 1 · 2

AuthorMessage
Profile dcdc

Send message
Joined: 3 Nov 05
Posts: 1832
Credit: 119,821,902
RAC: 13,431
Message 49444 - Posted: 6 Dec 2007, 15:05:09 UTC

parity between BOINC projects is impossible as the projects depend upon different strengths within the CPU (and beyond into memory etc). Rosetta could probably up the average credit - i.e. multiply the granted by 1.07 accross the board, but within the project the credit is generally very well aligned to that deserved.

If your claimed credits are lower than granted then this is probably due to high scores in the benchmarks (whetstone+dhrystone/2) which doesn't take into account things like cache size, although cache size has a very big impact on Rosetta performance. For example, a Duron and a Barton-core Athlon XP of the same speed will get the same benchmark, but the Athlon has 4x the cache and so is much better at Rosetta. Therefore it's the benchmarks that are wrong, and the granted is correct.
ID: 49444 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Nothing But Idle Time

Send message
Joined: 28 Sep 05
Posts: 209
Credit: 139,545
RAC: 0
Message 49457 - Posted: 6 Dec 2007, 23:09:37 UTC - in response to Message 49444.  

parity between BOINC projects is impossible as the projects depend upon different strengths within the CPU (and beyond into memory etc). Rosetta could probably up the average credit - i.e. multiply the granted by 1.07 accross the board, but within the project the credit is generally very well aligned to that deserved.

Don't want any "parity wars" but that 1.07(?) multiplier would be welcome. I go way back to Paul D. Buck (2005) who used to preach how one's credit is the symbol of one's contribution to a project in terms of time and money. In 2005 when I started I ran Einstein/Rosetta 50/50. After 2 years I have 78,000 Rosetta credits and 208,000 Einstein credits. Of course there were variations in resource shares over time, project uptimes and downtimes for both projects, and other variables I can't quantify, but by and large I personally believe the difference between my two totals is mostly because Rosetta has traditionally given less credit per unit time. Therefore, based on total credit Paul D. Buck would conclude that I contributed 27% of my resources to Rosetta and 73% to Einstein, and that just isn't true.



ID: 49457 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Astro
Avatar

Send message
Joined: 2 Oct 05
Posts: 987
Credit: 500,253
RAC: 0
Message 49458 - Posted: 7 Dec 2007, 0:09:26 UTC

I feel for ya. I used to follow this carefully, dedicated much time to Cross Project Parity, and have come to the conclusion that the onus lies with BOINC management to set the standard, and to provide the "path" to achieve it. However, Dr. Anderson doesn't want boinc to be the main anything. He doesn't wan't every project reporting/responsible to a "Boinc Central Command". I don't see how leaving it to the projects themselves will ever solve it. Boinc has to take charge on this issue.

Boinc did set a standard, and that was with the benchmarks calculation system. Rosetta by my figures and work is the project most following the already established standard.

I also agree some other project pay more/hour, so perhaps instead of raising Rosetta and a few others, users should be asking the other projects to lower theirs. Then you run into smaller projects who still use the Boinc benchmark and don't have the knowledge/ability to implement an fpops sytem, or their application/s won't support it. Or some use an average of the 2-3 wus returned, but with optimized third party boinc cc's out there raising the average and thus making Rosetta seem cheap on the credit front. Then theirs Einstein for much of 2005/2006 they were the most generous with their fixed credit system. And those running Seti with optimized apps also got a better credit/hour.

So, users would tell the other projects "Seti and Einstein pay more than you". So some new projects (ABCathome for example) are giving it away like peanuts at a "peanut days" celebration. Then other users tell other projects, "Well, I get X at ABC, Seti, and Einstein", give me more or I'll leave". Seti recently lowered theirs to try to get back to project parity and there was a big ruckus over that. Users didn't say "thanks seti, for overpaying me for all that time". People are just funny that way I guess. Also, If you don't use other projects AND pay close attention, then I suspect that many just don't understand it. They don't understand that theirs already a STANDARD for them to follow

Personally, I want credit to equal across projects so when I compare mine to someone elses, it might actually mean something. It's been so unequal for so darn long that Credit is actually meaningless as a measure of ones contribution. It's too late for boinc to fix it. They dropped the ball years ago on that one.
ID: 49458 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Angus

Send message
Joined: 17 Sep 05
Posts: 412
Credit: 321,053
RAC: 0
Message 49463 - Posted: 7 Dec 2007, 7:08:42 UTC
Last modified: 7 Dec 2007, 7:09:04 UTC

That horse left the barn a long time ago.

It's a total waste of resources to even talk about credit parity. Each project should do as they please. If they want to grant 10x the credit, so what?

If the stats sites quit comparing aplles to oranges to kiwi fruit, the whole thing would go away, and the projects would have to compete on their worthiness, not their credit granting scheme. Like the good ole days of Distributed Computing, before David Anderson screwed it up.
Proudly Banned from Predictator@Home and now Cosmology@home as well. Added SETI to the list today. Temporary ban only - so need to work harder :)



"You can't fix stupid" (Ron White)
ID: 49463 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Nothing But Idle Time

Send message
Joined: 28 Sep 05
Posts: 209
Credit: 139,545
RAC: 0
Message 49479 - Posted: 7 Dec 2007, 17:18:43 UTC - in response to Message 49463.  

That horse left the barn a long time ago.

You've enunciated your position on this topic many times over the years. Nevertheless, not everyone shares your view and I don't care to engage you on this matter.
ID: 49479 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
alpina

Send message
Joined: 19 Oct 05
Posts: 2
Credit: 26,658
RAC: 0
Message 49480 - Posted: 7 Dec 2007, 17:51:37 UTC

I completely disagree with you Angus. Credits are great because it motivates people to crunch for these projects. It makes crunching for a lot of people far more attractive than it would have been if there wouldn't have been a credit system. And it stimulates competition which benefits the projects.

I think your rant at David Anderson is unjustified. The BOINC platform is a fantastic concept; it makes it possible for the crunchers to switch between projects in seconds and to ad new projects in a minutes. It's also a big improvement for project leaders, it gives them a free platform to build their applications on and they can attract a lot of crunchers with little effort.

A credit system is a good thing, the BOINC-platform is a great thing, the only problem is that there is no way to impose a framework, which all projects have to oblige to, when they grant credits. The original benchmark idea was a good start but it turned out to be flawed(optimized clients and so on).

I think that fixed credits, i.e. the projects determine the granted credits, is the only way to go(but I'm not sure if every project can implement it). And more importantly, fixed credits should be equal across the different project which could be done if projects run the original BOINC benchmarks locally on some benchmark machines and grant fixed credits on that basis.
ID: 49480 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile dcdc

Send message
Joined: 3 Nov 05
Posts: 1832
Credit: 119,821,902
RAC: 13,431
Message 49490 - Posted: 8 Dec 2007, 10:35:03 UTC - in response to Message 49480.  

And more importantly, fixed credits should be equal across the different project which could be done if projects run the original BOINC benchmarks locally on some benchmark machines and grant fixed credits on that basis.

I agree with most of what you said, but unfortunately the above isn't possible. There is very little correlation between the benchmarks and how well suited a particular machine is to the project. 'Improving' the benchmarks is also not possible as which project do you make the benchmarks more similar to? Do they use SSE extensions - if so which ones?, consider cache size, RAM speed, FPU performance, Integer performance... The only reasonably accurate option would be to have one benchmark per project, which means we're back where we started with little cross-project parity.

Within Rosetta, the only way the credit accuracy could be improved is if the first submitted jobs were given pending credit and then it were granted once a reasonable number of jobs had been submitted. That would make little difference I would presume though.
ID: 49490 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
FluffyChicken
Avatar

Send message
Joined: 1 Nov 05
Posts: 1260
Credit: 369,635
RAC: 0
Message 49527 - Posted: 9 Dec 2007, 9:39:23 UTC - in response to Message 49480.  

I completely disagree with you Angus. Credits are great because it motivates people to crunch for these projects. It makes crunching for a lot of people far more attractive than it would have been if there wouldn't have been a credit system. And it stimulates competition which benefits the projects.

I think your rant at David Anderson is unjustified. The BOINC platform is a fantastic concept; it makes it possible for the crunchers to switch between projects in seconds and to ad new projects in a minutes. It's also a big improvement for project leaders, it gives them a free platform to build their applications on and they can attract a lot of crunchers with little effort.

A credit system is a good thing, the BOINC-platform is a great thing, the only problem is that there is no way to impose a framework, which all projects have to oblige to, when they grant credits. The original benchmark idea was a good start but it turned out to be flawed(optimized clients and so on).

I think that fixed credits, i.e. the projects determine the granted credits, is the only way to go(but I'm not sure if every project can implement it). And more importantly, fixed credits should be equal across the different project which could be done if projects run the original BOINC benchmarks locally on some benchmark machines and grant fixed credits on that basis.


I think you missunderstand Angus here. It's not the credits, it the cross project parity.
1) What a credit is here crunching Rossetta cannot be compared to what a credit crunched at seti etc are. You are crunching different problems, using the CPU in a different way... the apples to oranges.

2) Rosetta optimise their program as such... but it's called the standard program, at other places the members can modify the client and so improve it further they tend not to become the standard so even the exact same computer will give different credit. this increases the problem.

3) credit is just not a projects main priority.
Team mauisun.org
ID: 49527 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
DJStarfox

Send message
Joined: 19 Jul 07
Posts: 145
Credit: 1,250,162
RAC: 0
Message 49548 - Posted: 9 Dec 2007, 22:20:19 UTC
Last modified: 9 Dec 2007, 22:22:18 UTC

I think the credit system is a bit complicated. What makes it mildly frustrating is the lack of consistency between Windows and Linux application code. The simplest solution for the credit system would be to grant 10 credits/decoy. The problem with that is it won't honor the original BOINC example of cobblestone performance per minute. So, we are stuck with what we have.

Again, I would much rather the developers spend time on the technicalities of their applications (robustness, code-cleanup, memory usage, graphics in Linux, etc.) than on a new credit system.
ID: 49548 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 49551 - Posted: 10 Dec 2007, 4:01:00 UTC

10 credits per decoy sounds like a great idea... simple, easy to explain, fair.

...then you stop to notice that on some tasks it takes your machine 3 hours to complete a single decoy... and on other tasks it takes 6 minutes. Same machine, same background tasks... just different work from the project.

...So then you start thinking about seperating the different types of work from the project and some types are worth... say 1 credit per decoy for the 6 minute ones, and 30 credits per decoy for the 3 hour ones.

...but then within each type there is considerable variability as well. You get a random number that quickly figures out the model is not worth the time to go in to full atom refinement and someone else hits 10 in a row that all go through the refinement stage.

...and so you take the average with each type of work and use that as the credit measure... and that's what we've got.
Rosetta Moderator: Mod.Sense
ID: 49551 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Conan
Avatar

Send message
Joined: 11 Oct 05
Posts: 151
Credit: 4,244,078
RAC: 2,057
Message 49553 - Posted: 10 Dec 2007, 5:13:55 UTC

I do a number of projects (12 with credit) and Rosetta is about middle to low in terms of credit granted per hour.
It is a lot higher than Spinhenge or Predictor (my lowest) by between 2 to 7 cr/h.
It is a higher than The Lattice and also LHC by 1 to 4.
It is on a par with Ralph (as it should be).
It is lower than the standard Seti app and CPDN by 1 to 4 (depends on WU type).
It is lower than Einstein and Docking by 2 to 5.
And it is lower than Cosmology and QMC by 2 to 12 (depending on WU type).

So for me it does not stand out as an overly generous project for credit granting but it is far from the worst, hence middle of the road.

This has not affected the number of people crunching for the project so it would seem people crunch Rosetta because they want to. If I got more credit I doubt I would complain, but that is me, not everyone.

You will never get parity across projects and those that are trying I believe are wasting their time.
No project does exactly the same thing, therefore the work done per WU differs.
It can even differ in the same project, see these two following Rosetta examples that work on the same WU family so you would expect them to give similar results;
Result 123892323 (incase the result gets purged 63,257.17 seconds, 653.73 claimed, 17.46 granted, 2 decoys generated).
Result 123900291 (20,571.68 seconds, 72.14 claimed, 87.40 granted, 10 decoys generated).
(My preferance time is 21,600 seconds).
Which WU has done the most work? the one with 2 decoys or the one with 10?

If you use the 10 credits per decoy, suggested above, I still would not be happy with that first result (admittidly they don't happen often thank goodness).

Just going on how hot my processors get doing QMC compared to Rosetta, I would say that QMC makes my system work a lot harder than Rosetta does and I would expect more credit for the extra work it puts my computer through.

So at this stage I can live with what we have, if I want to chase credit then I have a number choices, if I want to chase science then I have all the choices that Bonic can offer.
ID: 49553 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 · 2

Message boards : Number crunching : Credit system seems unfair, anyone else thinks so?



©2024 University of Washington
https://www.bakerlab.org