Message boards : Number crunching : New Memory Requirements?
Author | Message |
---|---|
Chris Holvenstot Send message Joined: 2 May 10 Posts: 220 Credit: 9,106,918 RAC: 0 |
Did we get a new batch of work with some higher memory requirements? Over the past day I have frequently seen tasks waiting for memory. I had not seen this before on my systems. A sample task name would be: rb_06_15_205_770_rs_stg0_lrlxjcst_t000__casp9_SAVE_ALL_OUT_21438_15644_0 It wants about 800 meg (which on a six core system adds up quickly) If this is going to be the new norm I need to run to Fry's ASAP |
Murasaki Send message Joined: 20 Apr 06 Posts: 303 Credit: 511,418 RAC: 0 |
Some of the targets we are being asked to calculate for the CASP9 competition have huge structures that require a great deal of memory. How many targets are going to be that big will probably not be known until the last target is released by the competition organisers. There will still be lots of work with more reasonable memory requirements as part of the normal research activities, but I expect this won't be the last batch of high-memory work units we see. If you can afford more memory and are willing to upgrade to help Rosetta then great. However I will be keeping my current configuration for now and just continue processing whatever Rosetta chooses to fire at me. |
Bikermatt Send message Joined: 12 Feb 10 Posts: 20 Credit: 10,552,445 RAC: 0 |
Chris I am playing with a new box I just built and it has been running win7 64 for the last 24 hours or so. Task manager shows all of the rosetta work units vary between the high 200s to the low 400s for memory usage. The box is not running any of the casp tasks right now though and as Murasaki said, I remember reading some where that those tasks would be using more memory. Anyway, the same box was running Linux for a few hours before I loaded win and I seem to remember quite a few of the tasks running up into the 600s, but I was not looking at task names so those could have been casp tasks. The machine was running a lot better with Linux so I will be reloading this morning after it finishes up its last tasks. I will let you know for sure after it is reloaded. Also, on boxes that I use for crunching only, I set the memory usage up to 90% for "in use" and "idle". I noticed sometimes that if a box was right at the limit it would start waiting for memory even when I was just checking the system out. I also noticed on my six core box that when I added 3 GT240s for running GPU grid tasks it started waiting for memory when trying to run nine tasks at once (duh). I had to bump it up to eight GB it it now will run all nine tasks without issue. Four GB for six cores had been working well for me on my six core box so went I built this most recent machine I planned on 4 GB for every six cores. I'm planning on building another 24 core box at the end of this summer and with more tasks out there that use higher memory I think I will just go for a GB per core from now on. It seems like quite often a machine will download quite a few of the same types of tasks in a row. I think this has to do with how the tasks are generated. I have noticed quite often when one of my machines has to download a lot of tasks at the same time, they are quite often the same type of task. Anyway, if you get a whole batch of the higher memory requiring tasks less than one GB per core might not cut it anymore. |
Chris Holvenstot Send message Joined: 2 May 10 Posts: 220 Credit: 9,106,918 RAC: 0 |
Matt - of course Linux runs batter <sly grin> |
mikey Send message Joined: 5 Jan 06 Posts: 1896 Credit: 9,387,844 RAC: 9,807 |
Also, on boxes that I use for crunching only, I set the memory usage up to 90% for "in use" and "idle". I noticed sometimes that if a box was right at the limit it would start waiting for memory even when I was just checking the system out. The newer version of Boinc have a setting to turn this off, go to the Boinc Manager, Advanced, Preferences, processor usage and change the setting 'while processor usage is less than' to zero, put the number not the word in there. By default this is set to 25, and if your pc does anything that uses the processor Boinc will not crunch until you have more free time. It will say 'waiting for memory' if this is a problem, it is for most of us. This is a setting for those that want to do something else while they crunch and Boinc is not backing off enough. |
mikey Send message Joined: 5 Jan 06 Posts: 1896 Credit: 9,387,844 RAC: 9,807 |
Matt - of course Linux runs batter <sly grin> You mean it makes brownies too?! |
Bikermatt Send message Joined: 12 Feb 10 Posts: 20 Credit: 10,552,445 RAC: 0 |
I just found the 24 core box running 5 casp 9 tasks out of 24. They were all using around 600MB per task so I must have seen a few of them running the other day. The Box is using 13GB right now, it runs around 10 GB with no casp tasks. I will definitely go with one GB per core from now on. Linux is really running great on the box also. I have actually just started using Linux in the last few months. I tinkered a little bit with it eight years ago when I was a computer tech but never really got into it. Today I have one xp and two win7 licenses,as far as I am concerned I will never never need to buy a windows license again. |
Matthew Love Send message Joined: 24 Nov 05 Posts: 73 Credit: 44,948 RAC: 0 |
Also, on boxes that I use for crunching only, I set the memory usage up to 90% for "in use" and "idle". I noticed sometimes that if a box was right at the limit it would start waiting for memory even when I was just checking the system out. What reason did they give for adding this feature? |
mikey Send message Joined: 5 Jan 06 Posts: 1896 Credit: 9,387,844 RAC: 9,807 |
Also, on boxes that I use for crunching only, I set the memory usage up to 90% for "in use" and "idle". I noticed sometimes that if a box was right at the limit it would start waiting for memory even when I was just checking the system out. Dr. A does not have to give answers and normally does not to us peons, we are not worthy. The Boinc Developers mailing list sometimes gets answers but not always. Dr A has been doing Boinc for a long time and has developed some notions that not everyone else agrees with, but Boinc runs and for the most part works just fine. A guy named Rom, also a programmer and works for Dr. A, is MUCH more communicative but takes his orders from Dr. A, so doesn't always get his way. He had been on the boards before but usually leaves after ending up trying to defend Dr. A's decisions. How can anyone defend what their boss is thinking when the boss doesn't always include you in those thoughts?! People on the boards don't always understand that and it has gotten ugly before. Anyway back on topic, has anyone had any more problems with memory issues since going to version 6.10.56? Has anyone used the next Beta version 6.10.57? Does it give the same memory issues? |
Chris Holvenstot Send message Joined: 2 May 10 Posts: 220 Credit: 9,106,918 RAC: 0 |
The rotund guy in the back of the room raises his hand ... When I initially asked the question I was running 6.10.56. Have not seen this repeat on the system I bumped up to 8 gig. I have the memory sicks to do another tonight. Maybe tomorrow if I come down with a terminal case of lazyness. I am going to guess that a good rule of thumb is going to be "install 1 gig per core" Thanks for giving me some insight into what you have seen. |
mikey Send message Joined: 5 Jan 06 Posts: 1896 Credit: 9,387,844 RAC: 9,807 |
I think that is a good rule of thumb for Windows especially and probably Linux just to make it easier. Linux handles less memory MUCH better than Windows but that means it may handle more memory better too. |
Mod.Sense Volunteer moderator Send message Joined: 22 Aug 06 Posts: 4018 Credit: 0 RAC: 0 |
I would say that sounds like a good guideline, regardless of whether Rosetta is being run. And I would also say that while the formal requirements have not changed, and the actual usage varies, that should be enough to assure all cores keep crunching even if trudging through an unfortunate combination of tasks. Rosetta Moderator: Mod.Sense |
darwincollins Send message Joined: 1 Oct 09 Posts: 7 Credit: 5,586,679 RAC: 0 |
I just found the 24 core box running 5 casp 9 tasks out of 24. What type of box is it that it can handle 24 cores? Please describe. Thanks |
dcdc Send message Joined: 3 Nov 05 Posts: 1832 Credit: 119,891,919 RAC: 1,902 |
I just found the 24 core box running 5 casp 9 tasks out of 24. I believe Bikermatt has a 2x12 core Opteron box... |
Message boards :
Number crunching :
New Memory Requirements?
©2025 University of Washington
https://www.bakerlab.org