Message boards : Number crunching : How are pre-empted tasks scheduled?
Author | Message |
---|---|
Richard Send message Joined: 28 May 07 Posts: 7 Credit: 19,240 RAC: 0 |
I currently have 15 tasks "Waiting to run", at various stages of completion, two running, and 6 "Ready to start" Presumably the 15 have been pre-empted at some point, but why are incompleted tasks not finished before starting new ones? I assume, if the stay-in-memory option is set that they stay in memory and progressively more memory is taken up. This seems a bit odd, or have I misunderstood something? Is there a way of forcing a "finished unfinished work before starting new work" setting? |
Mod.Sense Volunteer moderator Send message Joined: 22 Aug 06 Posts: 4018 Credit: 0 RAC: 0 |
This is what you will see when BOINC is constrained on memory. It will begin a task, and as the tasks progresses (at least with Rosetta) it needs progressively more memory until it reaches the 130MB area you typically see. If BOINC is not allowed to use enough memory to run both tasks at the same time, it will suspend the one needing more memory and begin another one in the hope that it will be able to run in less memory... which it can... but typically only for a few minutes before it reaches the same memory requirement. Yes, you are correct all of those tasks are probably taking up space out in your swap file. I say probably because I am assuming all of the activity you have described occured since you last rebooted the machine. Yes, BOINC will attempt to complete what it has started before beginning something new. In fact, now that you presently have two tasks running, that was how it decided on the second task to run. Often when the machine is idle, BOINC is allowed more memory, so it will revisit these suspended tasks and eventually complete them. Either by running one at a time, or if idle time allows enough memory, by running two at a time. Suggest you revise your General Preferences, increasing the amount of memory that BOINC is allowed to use. P.S. Actually, come to think of it, you must have had the above occur, and then restarted the machine. Otherwise the status of these started tasks would show "waiting for memory" instead of "waiting to run". Rosetta Moderator: Mod.Sense |
Richard Send message Joined: 28 May 07 Posts: 7 Credit: 19,240 RAC: 0 |
This is what you will see when BOINC is constrained on memory. It will begin a task, and as the tasks progresses (at least with Rosetta) it needs progressively more memory until it reaches the 130MB area you typically see. If BOINC is not allowed to use enough memory to run both tasks at the same time, it will suspend the one needing more memory and begin another one in the hope that it will be able to run in less memory... which it can... but typically only for a few minutes before it reaches the same memory requirement. Thanks for your reply. The allowed memory (500MB) was more than twice the 130MB you mention. But I'll try increasing it to 1GB (i.e. 50% of my total). Basically I want to try to avoid having my day-to-day apps swapped out. Until starting Rosetta swap was never used, except when I was doing something very memory intensive (e.g. editing large graphics files), even when running SETI or Chess960. |
Richard Send message Joined: 28 May 07 Posts: 7 Credit: 19,240 RAC: 0 |
Also, I've noticed that if I abort a task that is supposed to be running but that isn't using any CPU time, a new task is started instead of one of the pre-empted tasks being resumed. |
Mod.Sense Volunteer moderator Send message Joined: 22 Aug 06 Posts: 4018 Credit: 0 RAC: 0 |
Well, my 130MB was intended to be an average. Some tasks are specifically designed to utilize more memory and are only sent to machines that have more then 512MB of memory. But I don't believe that figure takes in to account how much memory BOINC is allowed to use. Yes, if not using swap file is your key priority, you were already configured in a reasonable mannar. But, if you don't mind swapping a little, you can make the suggested change and get more BOINC crunching completed each day. The 130MB I mentioned was for one task, so you'd have to double that for your two cores. Yes, still less then your 500. Keep in mind that increasing to allow BOINC 1GB doesn't mean it will necessarily use it. Just means if it needs to that it is allowed. This will allow it to continue crunching the tasks it is working on, rather then stopping one with status "waiting for memory" and beginning a new one in hopes it can run in less. Rosetta Moderator: Mod.Sense |
Mod.Sense Volunteer moderator Send message Joined: 22 Aug 06 Posts: 4018 Credit: 0 RAC: 0 |
Also, I've noticed that if I abort a task that is supposed to be running but that isn't using any CPU time, a new task is started instead of one of the pre-empted tasks being resumed. This could be because BOINC Manager already knows how much memory the preempted task will require to run, and knows it doesn't presently have enough memory available. You are correct to expect BOINC to have a preference for completing existing started tasks prior to beginning new ones. But this isn't always possible, so the next choice is to start a new task. Rosetta Moderator: Mod.Sense |
Richard Send message Joined: 28 May 07 Posts: 7 Credit: 19,240 RAC: 0 |
This could be because BOINC Manager already knows how much memory the preempted task will require to run, and knows it doesn't presently have enough memory available. So, is there a proportion of 2GB that will guarantee that any task will run, that is less than 100%? I.e. is there a maximum size of task that is sent to machines like mine? I've already set the allocation to 50%, yet pre-empted tasks are not running. I guess I need to have a setting of 2X% to allow for 2 tasks that take X%. |
Mod.Sense Volunteer moderator Send message Joined: 22 Aug 06 Posts: 4018 Credit: 0 RAC: 0 |
Perhaps we've miscommunicated. Your machine will only actively work on one task per CPU core. Now that you've got the situation with several partially started tasks, and allowed more memory to be used, you should be set. But it will still take you some time to work through the partially completed work units, two at a time. Your 50% setting should be fine, as long as none of the other memory settings are struck as a limit. Rosetta Moderator: Mod.Sense |
Richard Send message Joined: 28 May 07 Posts: 7 Credit: 19,240 RAC: 0 |
Sadly this doesn't seem to be happening. What it seems to be doing is downloading new jobs. It tries to re-start a pre-empted job, but does no work on it (no CPU activity, although boinc manager says it's running) and when I abort this job it then starts one of the newly downloaded jobs. |
Richard Send message Joined: 28 May 07 Posts: 7 Credit: 19,240 RAC: 0 |
Indeed there doesn't seem to be any situation under which pre-empted tasks will restart computation. There does seem to be "Something Wrong" [TM] Boinc manager 5.8.17 Ubuntu 7.04 2.6.20-16-generic amd64 x2 2GB |
Mod.Sense Volunteer moderator Send message Joined: 22 Aug 06 Posts: 4018 Credit: 0 RAC: 0 |
I agree, that's not how it should be working. I think there have been some scheduling issues on Linux. Have you tried a current BOINC download? I believe the 5.10.xx series may have resolved some of those issues. Or going back to 5.8.16, which was the last non-beta release. Rosetta Moderator: Mod.Sense |
Richard Send message Joined: 28 May 07 Posts: 7 Credit: 19,240 RAC: 0 |
OK Thanks. I've found that a workaround is that re-starting the boinc-client when it fails to start computing on each new pre-empted pair of tasks gets things going again. I just have to keep doing it until the backlog has gone. |
Message boards :
Number crunching :
How are pre-empted tasks scheduled?
©2024 University of Washington
https://www.bakerlab.org