Two Preferences

Message boards : Number crunching : Two Preferences

To post messages, you must log in.

AuthorMessage
Profile Gerry Rough
Avatar

Send message
Joined: 2 Jan 06
Posts: 111
Credit: 1,389,340
RAC: 0
Message 60399 - Posted: 31 Mar 2009, 2:07:32 UTC

Recently, I changed my global preferences to try to cut down on my cache of WUs: it was getting way too large. But now that I've changed things, now BOINC seems to have barely enough work to keep going.

My "connect to network" preference is 0 days, and my "maintain enough work" preference is 1 day. I thought that would do the trick, but I think I need some help.

Hence the question: What is the difference between the "Connect to network about every" preference, and the "Maintain enough work for an additional" preference? If you check the unofficial BOINC wiki page, they seem to have overlapping functions, although they are obviously not the same. They both seem to tell BOINC how much cache to have so that BOINC will not run out of work.

(Click for detailed stats)
ID: 60399 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 60400 - Posted: 31 Mar 2009, 3:52:04 UTC

I'm not exactly clear on the distinction either, so perhaps someone else can help us with that.

...but if you just want a more concise list of tasks, you can increase your runtime preference on Rosetta to up to 24 CPU hrs per task if you like. This will effect all currently downloaded tasks as well, so you want to be sure you are about to request a bunch of tasks and update the preference at the same time. So I generally suggest increasing the runtime preference value gradually over the course of a week or so. This lets BOINC see that tasks are taking longer and so it requests fewer of them per day of time it is trying to request (actually, your DCF is adjusting).
Rosetta Moderator: Mod.Sense
ID: 60400 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Paul D. Buck

Send message
Joined: 17 Sep 05
Posts: 815
Credit: 1,812,737
RAC: 0
Message 60405 - Posted: 31 Mar 2009, 7:29:16 UTC

Critical to answering this question is also the version of BOINC you are using. there are significant differences in the way that version 5.x, 6.5 (and below) and 6.6.x clients are managing the work queues.

I suspect that you are seeing the "flexing" of the work queue I commented on in Einstein@Home where BOINC can run down the cache and then suddenly change its mind and over-fetch work again. Again, critical to understanding is the BOINC version, the projects attached to, the resource shares of these project, and sometimes the phases of the moon.

The connect interval indicates to BOINC how often it is allowed to connect, with 0 indicating always on. This usually has the side effect of lowering the queue maintained. The maintain work should keep one days work on hand but the calculations don't seem to work well for multi-core systems particularly when work on hand has widely varying completion times (CPDN along side IBERCIVIS for example).

I use the same settings, 0 and 1.0 and when I look at my i7 today, right now, I have 16 Rosetta tasks (@3:07), 4 GPU Grid tasks (@5:47), 6 ABC (@8:17) tasks queued ...

Which is low because I run 12 tasks at the same time. 4 GPU Grid and 8 others. The 16 RaH tasks should take 6 hours, and the ABC tasks will take less than the 8 hours because 2 CPU cores should go idle. So, I have only 14 hours of work queued instead of the desired 24 ...

Oh, and I am running 6.5.0 ...

What is likely to happen is that several of the ABC tasks will be started and then all of a sudden BOINC is going to notice that it is under fed and download tasks for somewhere and refill the queues. Normally I run with more than 50 tasks (the list is off the screen) ...

Oh, last point, projects that run out of work or go off-line also screws up this big time. EaH and MW both went off the air this weekend and BOINC does not handle that as well as it should either ...
ID: 60405 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
mikey
Avatar

Send message
Joined: 5 Jan 06
Posts: 1895
Credit: 9,179,636
RAC: 3,204
Message 60406 - Posted: 31 Mar 2009, 8:31:39 UTC - in response to Message 60399.  

Recently, I changed my global preferences to try to cut down on my cache of WUs: it was getting way too large. But now that I've changed things, now BOINC seems to have barely enough work to keep going.

My "connect to network" preference is 0 days, and my "maintain enough work" preference is 1 day. I thought that would do the trick, but I think I need some help.

Hence the question: What is the difference between the "Connect to network about every" preference, and the "Maintain enough work for an additional" preference? If you check the unofficial BOINC wiki page, they seem to have overlapping functions, although they are obviously not the same. They both seem to tell BOINC how much cache to have so that BOINC will not run out of work.


I run with the number of 1.00 in the first box and 0.25 in the second box. I am using version 6.4.5, on this machine, and seem to have about 1.5 days worth of work. Of course it thinks my ABC units will take 11 hours each but they really only take 5 to 6 hours. So I guess in truth I only have about 24 hours or so worth of work, which is what my settings are.
ID: 60406 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Gerry Rough
Avatar

Send message
Joined: 2 Jan 06
Posts: 111
Credit: 1,389,340
RAC: 0
Message 60410 - Posted: 31 Mar 2009, 12:58:13 UTC - in response to Message 60405.  
Last modified: 31 Mar 2009, 13:01:38 UTC

The connect interval indicates to BOINC how often it is allowed to connect, with 0 indicating always on. This usually has the side effect of lowering the queue maintained. The maintain work should keep one days work on hand but the calculations don't seem to work well for multi-core systems particularly when work on hand has widely varying completion times (CPDN along side IBERCIVIS for example).

[snip]

Oh, last point, projects that run out of work or go off-line also screws up this big time. EaH and MW both went off the air this weekend and BOINC does not handle that as well as it should either ...


I have wondered about both of these variables for months, actually. You just said it better. :-/

My thought for curing my "low cache" ills (sounds like a cigarette) are to increase the connect interval and see what that does over the next week or so. I will try that first as the independent variable in my erstwhile endeavor.

Running 6.4.7, by the way.

(Click for detailed stats)
ID: 60410 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Paul D. Buck

Send message
Joined: 17 Sep 05
Posts: 815
Credit: 1,812,737
RAC: 0
Message 60419 - Posted: 31 Mar 2009, 17:24:18 UTC

Change it by something low like .1 or .2 and run for several days.

The problem with the cache "flexing" is that it can look good for a long time and then go south on you. Today, I have a scroll bar on the i7 with 13 Rosetta tasks (3:09) {36}, 7 ABC (5:37) {39}, 4 GPU Grid (12:14), 1 Genetic Life (4:35), 3 IBERCIVIS (1:28 and 0:22), and 12 MW (0:18) {4}

Which is 36+39+4+3.5+4 = 86 hours total / 8 cores = 10 hours queued. Which is still wrong ... :)

I just started with 6.6.20 on my Mac Pro to see if it is completely bad and may put it on the i7 today or tomorrow to see if it behaves better than other 6.6.x versions I have tried.
ID: 60419 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Gerry Rough
Avatar

Send message
Joined: 2 Jan 06
Posts: 111
Credit: 1,389,340
RAC: 0
Message 60505 - Posted: 5 Apr 2009, 23:19:21 UTC

A related question that I forgot to mention at the beginning of this thread is why is the connect interval not enforced by BOINC? If I tell BOINC to connect once every day, it connects every few hours; if I tell BOINC to connect every .1 days, you guessed it, it connects every few hours. I have yet to see any clear pattern with this preference, other than the results being more or less cache.

(Click for detailed stats)
ID: 60505 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Paul D. Buck

Send message
Joined: 17 Sep 05
Posts: 815
Credit: 1,812,737
RAC: 0
Message 60511 - Posted: 6 Apr 2009, 6:20:49 UTC - in response to Message 60505.  

A related question that I forgot to mention at the beginning of this thread is why is the connect interval not enforced by BOINC? If I tell BOINC to connect once every day, it connects every few hours; if I tell BOINC to connect every .1 days, you guessed it, it connects every few hours. I have yet to see any clear pattern with this preference, other than the results being more or less cache.

Sadly the developers treat most of these participant settings as advice that they are free to ignore as and when they will ... Task Switch Interval is another that is not really enforced.

In general, these settings are bounds that may or may not constrain the actions of BOINC ...

Heck, some have troubles with NNW being ignored on some projects ...
ID: 60511 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Mod.Sense
Volunteer moderator

Send message
Joined: 22 Aug 06
Posts: 4018
Credit: 0
RAC: 0
Message 60517 - Posted: 6 Apr 2009, 13:07:27 UTC
Last modified: 6 Apr 2009, 13:08:46 UTC

Gerry, if the predictions of when work will complete were perfect, then the setting would result in a connection once per interval to get work and report results. But, if you run short on work sooner then that, and have an internet connection, BOINC will go ahead and contact the projects to get more work rather then go idle.

It is an impossible task to schedule this. The tasks do not have consistent runtimes from many projects. There is no way to be certain how many hours the machine will be running BOINC during the next week. There are debts and resource shares as well as task deadlines and user preferences to be achieved. And the BOINC client is trying to balance all of these at the same time.

The basic idea of the setting is for use when you have a machine that only has an internet connection when you walk over to it and create one. If you know you will only be at this machine once a week, it gives you a way to give BOINC the clue that it should try to prepare to run for a week without a network connection (i.e. get a 7 day cache of work).

This is why it is important that the DCF issues get fixed. They are throwing off the number of seconds of work that is being requested. It is also why people often point out here on the Rosetta boards that tasks aren't completing with the desired runtime preference that has been established.

[edit]and FYI, Paul is talking about the BOINC developers in this case, not the Rosetta developers.
Rosetta Moderator: Mod.Sense
ID: 60517 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Paul D. Buck

Send message
Joined: 17 Sep 05
Posts: 815
Credit: 1,812,737
RAC: 0
Message 60518 - Posted: 6 Apr 2009, 16:11:07 UTC - in response to Message 60517.  

[edit]and FYI, Paul is talking about the BOINC developers in this case, not the Rosetta developers.

Yes, I was not at all clear about that ...
ID: 60518 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote

Message boards : Number crunching : Two Preferences



©2024 University of Washington
https://www.bakerlab.org