I (Mod.Sense) wanted to start a thread just for the recent memory issues. There are many. I'm going to start with what I know and try to address as many of the related questions as possible. This information will likely go in to the FAQs once we hash out the topic in this thread.

Please post any questions you have which I haven't addressed below.
===================================

Q1: Why am I getting no work? The message says my computer doesn't have enough memory, but I've checked the system requirements page, and my system meets the requirements.

A1: Your system may meet the 256MB minimum requirement, but Rosetta often creates work units that require more memory to run. These involve longer protein strands, and/or more then one protein with DOCking and HINGE tasks.

At present, the BOINC servers do not have a smooth method for Rosetta to organize the work and isolate work units for systems with only the minimum memory from those which require more. The result is that there are times when the server is unable to locate any work units for minimum memory systems.

There are several approaches to addressing this so you can continue to crunch Rosetta. One is for the Project Team to try and keep a mixture of both types of work units available on the server to improve the chances that work is available for minimum memory systems. The Project Team has already taken action to do this.

Another approach is to keep a larger cache of work ready to run on your machine. This is done in the General Preferences with the setting for "connect to network about every ... days". Keeping a larger cache helps assure you have work to crunch, even during server outages, a lack of appropriate work units, or problems with your internet connection. It also gives you more lead time prior to downloading a new Rosetta version, which may let you schedule that to a time of your chosing, rather then when you go to get your next task(s).

A third approach is to attach to other BOINC projects with a small resource share, so your computer can work on other projects during any period when you are unable to get Rosetta work that is appropriate for your machine. There are several protein and medically related projects to chose from. Some even use the Rosetta program being developed and enhanced with your help here at Rosetta@home.
===================================

Q2: Do the large memory work units take longer to run? How does this effect my work unit runtime preference?

A2: It doesn't effect the runtime preference you have established in your Rosetta Preferences. The same rules pertain as to how long to crunch the task, and that you must crunch at least one model. One slower systems, some types of tasks may take longer then the 3 hour default runtime preference to complete that first model. BOINC will continue running the work unit, but it will be a little confused about predicting how much longer it will require to complete. This is because the estimated completion time is really only recomputed at the end of a model, and your machine hasn't reached one yet.
===================================

Q3: My machine runs Linux and can get done in 256MB what takes Windows twice as much. Why can't my 256MB Linux machine get the tasks that require the large memory Windows machines?

A3: You may be correct, but BOINC doesn't work that way. The memory requirement designated on a task pertains to all platforms that might download it. The BOINC server programs do not support designating a different memory requirement for each platform.
===================================

Q4: Rosetta is losing my machines because I don't feel it is fair for them to require more memory.

A4: We certainly don't want to lose your support. In fact, that is the reason we've destinguished two different task sizes. We want to assure that running Rosetta will not interfere with your use of your computer, and not overly work the disk drive.

If we did not seperate the high memory tasks, then systems without enough memory would be overly worked in attempting to run the tasks. The only way then to assure smooth operation for all Rosetta tasks would be to increase the minimum memory for the entire project. So creating two distinct types of tasks, based on the memory requirements observed in test runs is a much better way to assure more people can participate, and do so with minimal disruption to their computer use.

By creating two types of tasks, we're able to keep the minimum memory requirement as low as possible, and yet still pursue the advanced science projects which require more memory to run well.
===================================

Q5: Why don't they fix the Rosetta program so it uses less memory?

A5: Changes have already been made to reduce the amount of memory used. And it is possible that further changes can be made in the future as well. But we need to get the science done first and see if our new approaches are producing the desired results before we go too deep in to optimizing how the new science routines run.
===================================

Q6: But I do have 512MB of memory, why am I not able to get the large memory tasks?

A6: There are a couple of things to check. One is whether some of your memory is used by your graphics adapter (can someone post details on how to check this? I'll incorporate them here once I get them).

The other thing to check is your BOINC General Preferences. There are two settings that pertain to physical memory, the percentage you want BOINC to use when the computer is in use, and the maximum percetage you want BOINC to use when the computer is idle.

If your settings only allow BOINC to use 90% of your physical memory, then if your machine is 512MB, you are allowing BOINC to use up to 460MB. The HINGE tasks require slightly more then the 460MB result. If you'd like to run them, then BOINC will need your preferences to reflect that you are willing to allow a higher percentage of memory, at least when the computer is idle.
===================================

Q7: Why doesn't BOINC just use 100% of memory when the computer is idle? What's the point of configuring less?

A7: If you want to leave some room on your machine for your other applications to remain in memory, you might set the idle memory usage to less then 100%. This will help your response time when you go back to using your computer.

If you find that your computer is too slow to activate again once you've been away for a while, you may want to gradually reduce your setting for the percentage of memory to allow BOINC to use while the computer is idle. Reduce it until you've feel your struck a good compromise between getting more BOINC work done, and having your computer ready when you are to get your other work done. This setting is in your General Preferences.
===================================

Q8: But my machine has been running fine for months, why the problem now? Why don't they just fix it?

A8: These memory settings have been in the preferences for some time. They define which tasks your machine will download. But only after upgrading to the 5.8 BOINC clients did these limits actually get enforced as the tasks are running. So, while you may not have changed your settings, if you upgraded to a 5.8 version of BOINC, you went from a version that did not enforce these preferences, to a version that does enforce them. This may be why you see tasks in a status of "waiting for memory" (see Q13 below), when you didn't previously. And sometimes only see one CPU active (see Q12 below).

The other recent change that coincided with the BOINC changes is that the Project Team has been working on a lot of advanced science lately and studying more very large proteins. So, more of the large memory tasks are being created then in the past.
===================================

Q9: Is this a trend towards the large memory tasks?

A9: No. The project will have tasks of both sizes going forward, and do what they can to help assure that minimum memory systems can find minimum memory tasks to download.
===================================

Q10: Are you telling me I have to buy more memory if I want to continue to support Rosetta?

A10: Certainly not. It is up to you how you wish to use your computer. And as discussed above, additional memory beyond the 256MB minimum is not a requirement.
===================================

<added March 7>
Q11: I've watched my machine's memory usage as the normal memory work units run, and they do not use the 256MB that is stated as their requirement. Why state a 256MB requirement when the tasks do not take that much memory to run?

A11: Since each task is exploring an unknown landscape, it is not possible to determine ahead of time exactly how much memory or CPU time will be required to complete a specific model. The 256MB (and higher) "requirement" is in place to allow for the exceptional case where a model may take more memory then is typical for that type of work unit.
===================================

Q12: I've revised my settings to allow BOINC to use more memory and now received two of the HINGE work units. But now BOINC will not run both tasks at the same time. Why not?

A12: BOINC is actually monitoring the memory usage of the tasks as they run. When a Rosetta task starts, it uses a small amount of memory while it initializes the work unit and the first model. Then, as the model computes, memory is used and freed up as the computation progresses. Towards the end of a model, the task is typically using significantly more memory then at the beginning.

If your computer has multiple CPUs, or is hyperthreaded (HT), BOINC will normally run a task on each CPU. The number of CPUs BOINC should use can be controlled via your General Preferences. However, when BOINC notices the configured memory settings are exceeded, it will pause one of the tasks until enough memory is available.

Some of the HINGE tasks can consume nearly 400MB of memory as they run. To run two of them requires twice as much memory for that phase of the computation. So, even if your machine has 512MB of memory and you allow 100% of it to be used by BOINC, you can still run in to combinations of work units that are unable to keep all CPUs busy at all times. The tasks will complete, but only when the memory usage of the first task is reduced (which might occur as it starts a new model), or when the computer is idle, if your configuration allows BOINC to use more memory during idle time.
===================================

Q13: Why does BOINC start on one task, then interrupt it saying it is "waiting for memory" and start another?

A13: When BOINC finds itself exceeding your configured memory settings, it will stop work. If your computer has multiple CPUs, it may only have to stop one of the tasks to fall back to within your configured memory guidelines. If another task is available, BOINC begins on that one. If that second task uses less memory, then perhaps both could run within your configured memory usage guidelines. BOINC has no way to know, until it starts running it, how successful it will be in that effort. If it then sees this new task taking more memory, which may take a minute or two of runtime, then it will hold it as well, until memory becomes available (i.e. until another BOINC process frees up memory).
===================================

Q14: Why has my RAC declined since these HINGE tasks started running on Rosetta?

A14: The potential reasons for RAC decline are too numerous to detail all in one place. But the above memory issues may be one cause. Your RAC is based on the rate at which you return completed work, and the credit you receive for that work. If your credit awarded as compared to your credit claimed is in about the same ratio as it has been with other work units, then the RAC decline is likely due to:
1) lack of work for minimum memory machines, as described in Q1
2) not enough memory to keep all CPUs busy, as described in Q12
3) BOINC memory use limited, as described in Q6
===================================

Q15: So why can't they just fix it to use the memory it needs to without all this fussing around with settings?

A15: They did! The BOINC defaults help assure smooth operation. All of the above is just an explaination of how BOINC works, and how you can control it to work differently. Configuring things and changing settings are just options available for those that want to crunch even more.
===================================

---

Rosetta Informational Moderator: Mod.Zilla

I've added 5 more Qs & As this morning. Hope this helps everyone understand what is happening. Please post with any questions. Or points that should be further clarified.

---

Rosetta Moderator: Mod.Sense

The other thing to check is your BOINC General Preferences. There are two settings that pertain to physical memory, the percentage you want BOINC to use when the computer is in use, and the maximum percetage you want BOINC to use when the computer is idle.

These settings have been in the preferences for some time, but were not actually used on the 5.4 BOINC client. So, only recently when people started upgrading to the 5.8 BOINC clients did these limits actually get enforced.

When downloading work, these settings are checked on the server side, not the client side. Thus, even people using older BOINC clients will need to boost the "use at most when idle" setting in order to get big work for a 512MB machine.

If the low-mem crowd forces the project programmers to reduce the memory footprint would this also possibly cause us to produce fewer decoys in our specified run time? That is, could I produce fewer decoys per run just so that other participants could get a HINGE task on their machine? This is not a selfish question, just curious about the ramifications of giving people what they want.

AMD, are you saying that the server takes the General Preference for memory against the last reported memory of that host, and uses the result to determine what can be downloaded?

I guess as I wrote that, I was picturing the enforcement at runtime which is new to the 5.8 versions.

---

Rosetta Moderator: Mod.Sense

"Idle time", I think you are asking if a trade-off would be possible where all work units could be made to run in 256MB memory, perhaps at the cost of doing less models per hour.

Such a trade-off is not possible. You'd essentially have to write things out to disk to save memory, and yes that would slow the actual work. It would also increase disk activity to undesireable levels.

In theory, it is possible to run any memory intensive application on any machine with enough virtual storage. But it would run very slowly, because it would constantly be bringing memory back from the swap file for processing. And it would result in an immense number of disk operations (i.e. almost constantly), which could ultimately impact the life of the disk drive. So, the larger memory requirement is a better idea all around for a healthy run environment.

---

Rosetta Moderator: Mod.Sense

AMD, are you saying that the server takes the General Preference for memory against the last reported memory of that host, and uses the result to determine what can be downloaded?

Not AMD, but answers anyway. ;)

Yes, the Scheduling-server has always checked the computers reported memory, and now it also uses the 2 new global preferences, if both busy_frac and idle_frac > 0, finds the highest, and uses this to multiply with reported memory. Oh, and if highest > 1, it's limited to 1, in case someone tries to report too high. ;)

Even v5.4.9 doesn't use these 2 Global preferences, they still gets downloaded from server and back to server in Scheduler-request, and is therefore used.

Only if project runs outdates Scheduling-server, or, the preferences haven't been initialized so still is zero, is not the preferences being used by Scheduling-server. In this case, the computers memory is used to limit if can get wu or not.


The client-side pausing and aborting of wu on the other hand only happens if runs v5.7.xx or later...

---

"I make so many mistakes. But then just think of all the mistakes I don't make, although I might."

A1:

Another approach is to keep a larger cache of work ready to run on your machine.

But don't set higher than in This table
For 10-day deadline-wu, this means 4.24 days is max.

A4:

If we did not create the high memory task type, then systems without enough memory would be overly worked in attempting to run the tasks.

Uhm, if these wu's hadn't been created, no-one would run them either... :)

We'd have to increase the minimum memory for the entire project to assure that only people that will be able to run the tasks efficiently are getting them.

"Running efficiently" wouldn't be the problem, instead low-memory-machines would just abort the work when tried to use more than the max specified in preference, if running v5.8.xx that is.

A8: As discussed in #6 above, BOINC client software only recently started enforcing the configured memory settings. So, while you may not have changed your settings, if you upgraded to a 5.8 version of BOINC, you went from a version that did not enforce these preferences, to a version that does enforce them.

#6 and #8 already commented on, it's more to do with Rosetta@home not having the preferences until resently, and possibly also running too old Scheduling-server...

A12:

Also worth mentioning, when computer active, default is allow only 50%, while when idle allow 90%.

---

"I make so many mistakes. But then just think of all the mistakes I don't make, although I might."

Thanks for this, it is a good start on a useful set of FAQs

In A7, you might like to give the example that with BOINC using 100% or memory, even basic windows features like the Start button can take a minute or two to react. Some users find this disconcerting, others don't mind. If you don't like this behaviour, experiment with dropping the max % while machine is idle 1% at a time 98%, 97% ... till the machine feels responsive.

Noteh also that even with a smaller setting, if you have a big spreadsheet open that can take memeory away from the Start button, etc, as can any large program. This setting only regulates BOINC's use of the memory, not that of programs you have left dormant.

R~~

Thanks for the comments. I've made some further revisions in the original Q&A section. In some cases I further clarified the question to better fit the answer I was trying to explain.

I noticed that part of how I got my self in to trouble was using the word "problem", and each reader has a different perspective on exactly what the problem is. So by further clarifying the questions to define each specific angle of the problem, I hope I've made things more clear now.

---

Rosetta Moderator: Mod.Sense

I wanted to bump this thread, since we're now seeing more high memory tasks coming in the downloaded work. Please review the original post of this thread for details on memory requirements and how BOINC and Rosetta handle them.

---

Rosetta Moderator: Mod.Sense

And they told me that "3GB of RAM was excessive" lol.

Photoshop alone eats up about 1.4 GB while working on big graphics.

---

And they told me that "3GB of RAM was excessive" lol.

Photoshop alone eats up about 1.4 GB while working on big graphics.

Yes, and if one can afford to go out to a restaurant and eat dinner then one can also afford to buy additional RAM in lieu of calories.

And they told me that "3GB of RAM was excessive" lol.

Photoshop alone eats up about 1.4 GB while working on big graphics.

Yes, and if one can afford to go out to a restaurant and eat dinner then one can also afford to buy additional RAM in lieu of calories.

It's all about choices isn't it, eat or buy ram? Personally if that were my choice, it isn't, but if it were, NO Boinc would be my choice! Seriously if someone has to make a choice between eating and buying ram, stop crunching and spend your money more wisely!! The rest of us will just have to pick up the slack.

---

And they told me that "3GB of RAM was excessive" lol.

Photoshop alone eats up about 1.4 GB while working on big graphics.

Cpu's too! Used to be one was plenty, now Photoshop can use 2 or more if you have them! I have read that Photoshop loads 2 copies of every photo you are working on into memory. One to keep and one to work on, making changes on the 2nd one and only overwriting the first when you tell it too. The first is there so you don't have to reload your original when you want to just go back and start over.

---

Hi.

F.Y.I. at the moment i have running.

Rosetta Mini 1.45 = 162 MB

P.O.E.M. = 369 MB

P.O.E.M. = 363 MB

MalariaControl = 26 MB

Total system = 1.200 MB

Not to bad.

pete.

---