Excessive workunit fetch

Author	Message
Grant (SSSF) Send message Joined: 28 Mar 20 Posts: 1681 Credit: 17,854,150 RAC: 20,118	Message 103353 - Posted: 16 Nov 2021, 21:33:13 UTC - in response to Message 103340. Last modified: 16 Nov 2021, 21:35:05 UTC The regular Rosetta tasks also reserve 8gb of ram per task Complete and utter rubbish. Stop making things up. how much the Virtual Memory Size is and the Working set size is, you care about the one that shows the most memory. No you don't. Virtual memory is just that- virtual memory. It is used if there isn't enough physical RAM. The Working Set size is the one that matters. If the amount of Virtual Memory required becomes an issue, then you've got a much bigger problem with your system as a whole due to lack of free disk space than some value reported for a particular Rosetta Task. Grant Darwin NT ID: 103353 · Rating: 0 · rate: / Reply Quote

Mad_Max Send message Joined: 31 Dec 09 Posts: 209 Credit: 25,992,337 RAC: 13,353	Message 103653 - Posted: 2 Dec 2021, 0:52:52 UTC Last modified: 2 Dec 2021, 0:54:40 UTC It is a confirmed BOINC (not Rosetta) bug with excessive workunit fetch for a project if <max_concurrent> setting is used. It was identified about year ago or so. One of the latest bug reports to BOINC developer: https://github.com/BOINC/boinc/issues/4322 And finally a patch to this problem is in work now: https://github.com/BOINC/boinc/pull/4592 It will be included in one of the future BOINC releases. For now there are few workarounds until new fixed version of BOINC available: 1 - avoid using <max_concurrent> setting completely OR 2 - set work cache size to a really low values OR 3 - use R@H with other projects with stable WUs supply and WITHOUT <max_concurrent> setting applied to this "spare" projects (only to R@H) Because bug itself is a wrong calculation of amount of work BOINC already have in queue for projects limited by <max_concurrent> setting. It just count only up to <max_concurrent> number of tasks in queue and ignores the rest. So calculated work queue size does not increase no matter how many tasks BOINC has already loaded for a such project. And falls into an endless loop of getting new work. ID: 103653 · Rating: 0 · rate: / Reply Quote

Grant (SSSF) Send message Joined: 28 Mar 20 Posts: 1681 Credit: 17,854,150 RAC: 20,118	Message 103655 - Posted: 2 Dec 2021, 5:48:42 UTC - in response to Message 103653. And finally a patch to this problem is in work now: https://github.com/BOINC/boinc/pull/4592 It will be included in one of the future BOINC releases. Glad to see there might finally be a fix for this issue. Grant Darwin NT ID: 103655 · Rating: 0 · rate: / Reply Quote

[AF>Le_Pommier] Jerome_C2005 Send message Joined: 22 Aug 06 Posts: 42 Credit: 1,258,039 RAC: 0	Message 103702 - Posted: 4 Dec 2021, 14:43:14 UTC Thanks for this input Mad Max. I may give it a try, next year when FB is over :) ID: 103702 · Rating: 0 · rate: / Reply Quote

Greg_BE Send message Joined: 30 May 06 Posts: 5691 Credit: 5,859,226 RAC: 0	Message 103750 - Posted: 6 Dec 2021, 20:57:54 UTC You can get around that by adding the word project_ to the max concurrent. I have been using this for a few days now and it works perfect. [project_max_concurrent] N [/project_max_concurrent] ID: 103750 · Rating: 0 · rate: / Reply Quote

Greg_BE Send message Joined: 30 May 06 Posts: 5691 Credit: 5,859,226 RAC: 0	Message 103751 - Posted: 6 Dec 2021, 20:59:59 UTC - in response to Message 103353. The regular Rosetta tasks also reserve 8gb of ram per task Complete and utter rubbish. Stop making things up. how much the Virtual Memory Size is and the Working set size is, you care about the one that shows the most memory. No you don't. Virtual memory is just that- virtual memory. It is used if there isn't enough physical RAM. The Working Set size is the one that matters. If the amount of Virtual Memory required becomes an issue, then you've got a much bigger problem with your system as a whole due to lack of free disk space than some value reported for a particular Rosetta Task. Grant, hes not that far off. 7629.39 physical per task and 103.13 virtual (cages tasks) ID: 103751 · Rating: 0 · rate: / Reply Quote

Mad_Max Send message Joined: 31 Dec 09 Posts: 209 Credit: 25,992,337 RAC: 13,353	Message 103762 - Posted: 7 Dec 2021, 21:37:36 UTC - in response to Message 103751. Last modified: 7 Dec 2021, 21:43:55 UTC He said it about regular (non Python/virtual box) task. Regular R@H tasks RAM usage lies in 0.7-3 Gb range for almost all tasks (for >95% of tasks) and in 0.7-1.5 Gb usually (for >70%). So it is very far from the 8 GB per tasks. ID: 103762 · Rating: 0 · rate: / Reply Quote

[AF>Le_Pommier] Jerome_C2005 Send message Joined: 22 Aug 06 Posts: 42 Credit: 1,258,039 RAC: 0	Message 105860 - Posted: 7 Apr 2022, 17:21:35 UTC If I can read github correctly, it is said that this patch will be published in boinc 7.20.0 ?? ID: 105860 · Rating: 0 · rate: / Reply Quote

Mad_Max Send message Joined: 31 Dec 09 Posts: 209 Credit: 25,992,337 RAC: 13,353	Message 106180 - Posted: 11 May 2022, 0:38:59 UTC Yes, it already included in BOINC ver 7.20.0 (and later). But v. 7.20 itself is not finished yet. ID: 106180 · Rating: 0 · rate: / Reply Quote