Problems and Technical Issues with Rosetta@home

Message boards : Number crunching : Problems and Technical Issues with Rosetta@home

To post messages, you must log in.

Previous · 1 . . . 138 · 139 · 140 · 141 · 142 · 143 · 144 . . . 317 · Next

AuthorMessage
Jean-David Beyer

Send message
Joined: 2 Nov 05
Posts: 204
Credit: 6,927,641
RAC: 9,107
Message 103591 - Posted: 29 Nov 2021, 13:51:54 UTC - in response to Message 103586.  

Yes, you have to get them when you can. But I pick up a few more from time to time, so I should make it until tomorrow.
Hopefully they will throw some more in the hopper.

I see there are lots of them in the hopper,

Rosetta 3674 63066 6.88 (0.24 - 56.39) 2050

and my machine asks for some, but does not get any.

Mon 29 Nov 2021 08:40:33 AM EST | Rosetta@home | update requested by user
Mon 29 Nov 2021 08:40:38 AM EST | Rosetta@home | Sending scheduler request: Requested by user.
Mon 29 Nov 2021 08:40:38 AM EST | Rosetta@home | Requesting new tasks for CPU
Mon 29 Nov 2021 08:40:40 AM EST | Rosetta@home | Scheduler request completed: got 0 new tasks
Mon 29 Nov 2021 08:40:40 AM EST | Rosetta@home | No tasks sent
Mon 29 Nov 2021 08:40:40 AM EST | Rosetta@home | Project requested delay of 31 seconds
Mon 29 Nov 2021 08:41:15 AM EST | Rosetta@home | Sending scheduler request: To fetch work.
Mon 29 Nov 2021 08:41:15 AM EST | Rosetta@home | Requesting new tasks for CPU
Mon 29 Nov 2021 08:41:16 AM EST | Rosetta@home | Scheduler request completed: got 0 new tasks
Mon 29 Nov 2021 08:41:16 AM EST | Rosetta@home | No tasks sent

ID: 103591 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Peter Humphrey

Send message
Joined: 26 Jul 18
Posts: 5
Credit: 4,347,407
RAC: 4
Message 103592 - Posted: 29 Nov 2021, 16:35:24 UTC - in response to Message 103553.  

In that case I may decide I can't afford to stay with this project. It's far too much of a memory hog; I've suspended it while I debate with myself. Who'd have though that 64GB RAM would be too little, even with 24 processors?
ID: 103592 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jim1348

Send message
Joined: 19 Jan 06
Posts: 881
Credit: 52,257,545
RAC: 0
Message 103593 - Posted: 29 Nov 2021, 17:11:46 UTC - in response to Message 103592.  

Who'd have though that 64GB RAM would be too little, even with 24 processors?

If you are willing to jump through some hoops (though they are actually rather easy), there is a way, by running multiple BOINC instances.
https://boinc.bakerlab.org/rosetta/forum_thread.php?id=6893&postid=103516#103516

You then run as many as you can (8 for example) in each instance. That works because you only need a lot of memory to download them, not run them.
Here is how to set it up:
https://www.overclock.net/threads/guide-setting-up-multiple-boinc-instances.1628924/

I already had a second BOINC instance set up on a Ryzen 3900X with 96 GB of memory, so I can use all 24 cores (12 per instance).
Also, I added a second BOINC instance to a Ryzen 3950X with 128 GB of memory.
They are all under Ubuntu 20.04.3, but it works on Windows as well. It is just a bit easier to start up automatically in Linux.

You can use three BOINC instances (or more) if you need to.
ID: 103593 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jean-David Beyer

Send message
Joined: 2 Nov 05
Posts: 204
Credit: 6,927,641
RAC: 9,107
Message 103594 - Posted: 29 Nov 2021, 22:40:16 UTC - in response to Message 103553.  

I messed around with app_config but that can make a mess of things.


I am using this, and that sets my upper bound to three at a time. What would be the symptoms of the mess to which you refer?

[/var/lib/boinc/projects/boinc.bakerlab.org_rosetta]$ cat app_config.xml 
<app_config>
   <project_max_concurrent>3</project_max_concurrent>
</app_config>

ID: 103594 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5691
Credit: 5,859,226
RAC: 0
Message 103595 - Posted: 29 Nov 2021, 22:58:37 UTC - in response to Message 103594.  

I messed around with app_config but that can make a mess of things.


I am using this, and that sets my upper bound to three at a time. What would be the symptoms of the mess to which you refer?

[/var/lib/boinc/projects/boinc.bakerlab.org_rosetta]$ cat app_config.xml 
<app_config>
   <project_max_concurrent>3</project_max_concurrent>
</app_config>



project? I have never seen that before.
Most of us tried to use max_concurrent and then got buried in tons of tasks we could never complete by their deadlines.
ID: 103595 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5691
Credit: 5,859,226
RAC: 0
Message 103596 - Posted: 29 Nov 2021, 23:01:43 UTC

Something does not make sense
Says 2,000 tasks queued.
Had a look at the schedulers...0 on all projects.
So are the 2,000 not released yet or have they all been taken and the system did not update?
ID: 103596 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jim1348

Send message
Joined: 19 Jan 06
Posts: 881
Credit: 52,257,545
RAC: 0
Message 103597 - Posted: 29 Nov 2021, 23:38:29 UTC - in response to Message 103595.  

project? I have never seen that before.
Most of us tried to use max_concurrent and then got buried in tons of tasks we could never complete by their deadlines.

Project_max_current will limit the total number of work units running for all projects.

But either one of them can cause the problem of excessive downloads.
https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=5720&postid=45319
https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=5720&postid=45323
ID: 103597 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Falconet

Send message
Joined: 9 Mar 09
Posts: 354
Credit: 1,282,599
RAC: 86
Message 103598 - Posted: 30 Nov 2021, 1:11:40 UTC - in response to Message 103596.  

The queue only updates once around every 4 hours while the server status page is around 30 minutes or so.
ID: 103598 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5691
Credit: 5,859,226
RAC: 0
Message 103603 - Posted: 30 Nov 2021, 7:13:14 UTC - in response to Message 103597.  

project? I have never seen that before.
Most of us tried to use max_concurrent and then got buried in tons of tasks we could never complete by their deadlines.

Project_max_current will limit the total number of work units running for all projects.

But either one of them can cause the problem of excessive downloads.
https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=5720&postid=45319
https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=5720&postid=45323



So it comes down to this, any attempt to limit the amount of tasks will cause excessive downloads.
What if you rolled back in versions of BOINC?
ID: 103603 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Bryn Mawr

Send message
Joined: 26 Dec 18
Posts: 412
Credit: 12,584,961
RAC: 10,887
Message 103608 - Posted: 30 Nov 2021, 9:16:35 UTC - in response to Message 103603.  

project? I have never seen that before.
Most of us tried to use max_concurrent and then got buried in tons of tasks we could never complete by their deadlines.

Project_max_current will limit the total number of work units running for all projects.

But either one of them can cause the problem of excessive downloads.
https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=5720&postid=45319
https://lhcathome.cern.ch/lhcathome/forum_thread.php?id=5720&postid=45323



So it comes down to this, any attempt to limit the amount of tasks will cause excessive downloads.
What if you rolled back in versions of BOINC?


Not will, can.

I’ve been running project_max_concurrent on most projects for several years with no excess downloads.
ID: 103608 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Peter Humphrey

Send message
Joined: 26 Jul 18
Posts: 5
Credit: 4,347,407
RAC: 4
Message 103610 - Posted: 30 Nov 2021, 11:53:55 UTC - in response to Message 103593.  

You then run as many as you can (8 for example) in each instance. That works because you only need a lot of memory to download them, not run them. Here is how to set it up:


I don't understand. When I run boincmgr it shows several jobs as "waiting for memory". How can adding yet more of them release memory? And why would boinc need more memory to download a job than to run it?

(This is Gentoo Linux.)
ID: 103610 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Falconet

Send message
Joined: 9 Mar 09
Posts: 354
Credit: 1,282,599
RAC: 86
Message 103611 - Posted: 30 Nov 2021, 12:04:53 UTC - in response to Message 103610.  

You then run as many as you can (8 for example) in each instance. That works because you only need a lot of memory to download them, not run them. Here is how to set it up:


I don't understand. When I run boincmgr it shows several jobs as "waiting for memory". How can adding yet more of them release memory? And why would boinc need more memory to download a job than to run it?

(This is Gentoo Linux.)



Read my post. I hope it helps.

https://boinc.bakerlab.org/rosetta/forum_thread.php?id=6893&postid=103572
ID: 103611 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jim1348

Send message
Joined: 19 Jan 06
Posts: 881
Credit: 52,257,545
RAC: 0
Message 103612 - Posted: 30 Nov 2021, 14:02:46 UTC - in response to Message 103611.  

Falconet has the right answer. But I would only add that it is the project that sets the memory requirements, not BOINC.
If they say more, then BOINC just obeys. (The memory isn't released, it is just reserved.)

And adding a second BOINC instance gives you another bite at the apple. One BOINC instance doesn't see what the other one is doing.
So if the pythons ever do require more memory to run, that could cause problems. But we are a long way from that at the moment.
ID: 103612 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5691
Credit: 5,859,226
RAC: 0
Message 103613 - Posted: 30 Nov 2021, 14:47:25 UTC - in response to Message 103612.  
Last modified: 30 Nov 2021, 14:50:22 UTC

Falconet has the right answer. But I would only add that it is the project that sets the memory requirements, not BOINC.
If they say more, then BOINC just obeys. (The memory isn't released, it is just reserved.)

And adding a second BOINC instance gives you another bite at the apple. One BOINC instance doesn't see what the other one is doing.
So if the pythons ever do require more memory to run, that could cause problems. But we are a long way from that at the moment.




I thought there was a memory limiter command?

I have stopped with Python as it is killing LHC ATLAS from running.

How do you set up another instance? This is really new territory for me.
ID: 103613 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jim1348

Send message
Joined: 19 Jan 06
Posts: 881
Credit: 52,257,545
RAC: 0
Message 103614 - Posted: 30 Nov 2021, 17:13:20 UTC - in response to Message 103613.  

I thought there was a memory limiter command?

I have stopped with Python as it is killing LHC ATLAS from running.

How do you set up another instance? This is really new territory for me.

There are memory limits set by the project (probably both maximum and minimum). You might be able to change them, but I have not looked into it.

Yes, I have two BOINC instances on three Ubuntu machines. I use them as necessary, not just for Rosetta.
The main difference for Windows is that you have to start up the second instance manually to make the icon visible, though I did manage to get it to work via Task Scheduler.
But that was on Win7. I have not tried it on Win10, and it may be different.
ID: 103614 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Thomas Köller

Send message
Joined: 27 Nov 21
Posts: 1
Credit: 293,470
RAC: 0
Message 103615 - Posted: 30 Nov 2021, 17:39:09 UTC

On the computing preferences page, I cannot remove the check mark next to 'Suspend GPU computing when computer is in use'. Or, to be more precise, I can but it reappears as soon as I save my changes. Why is that?
ID: 103615 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jim1348

Send message
Joined: 19 Jan 06
Posts: 881
Credit: 52,257,545
RAC: 0
Message 103616 - Posted: 30 Nov 2021, 18:00:01 UTC - in response to Message 103615.  

On the computing preferences page, I cannot remove the check mark next to 'Suspend GPU computing when computer is in use'. Or, to be more precise, I can but it reappears as soon as I save my changes. Why is that?
That is a bit curious. I do not see it here. Maybe a blocker of some sort on your web browser? I would try a different one.
ID: 103616 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Peter Humphrey

Send message
Joined: 26 Jul 18
Posts: 5
Credit: 4,347,407
RAC: 4
Message 103617 - Posted: 30 Nov 2021, 18:06:14 UTC - in response to Message 103611.  

Read my post. I hope it helps.

https://boinc.bakerlab.org/rosetta/forum_thread.php?id=6893&postid=103572


It does - thanks. Isn't it usual for a program to reserve memory only when it's needed?
ID: 103617 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Profile Greg_BE
Avatar

Send message
Joined: 30 May 06
Posts: 5691
Credit: 5,859,226
RAC: 0
Message 103618 - Posted: 30 Nov 2021, 18:23:05 UTC - in response to Message 103614.  

I thought there was a memory limiter command?

I have stopped with Python as it is killing LHC ATLAS from running.

How do you set up another instance? This is really new territory for me.

There are memory limits set by the project (probably both maximum and minimum). You might be able to change them, but I have not looked into it.

Yes, I have two BOINC instances on three Ubuntu machines. I use them as necessary, not just for Rosetta.
The main difference for Windows is that you have to start up the second instance manually to make the icon visible, though I did manage to get it to work via Task Scheduler.
But that was on Win7. I have not tried it on Win10, and it may be different.



Is there a dummies page with a simple explantion to set this up?
ID: 103618 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Jim1348

Send message
Joined: 19 Jan 06
Posts: 881
Credit: 52,257,545
RAC: 0
Message 103620 - Posted: 30 Nov 2021, 18:35:42 UTC - in response to Message 103618.  

Is there a dummies page with a simple explantion to set this up?

With BOINC, it only gets worse.
https://boinc.berkeley.edu/wiki/Client_configuration#Options
ID: 103620 · Rating: 0 · rate: Rate + / Rate - Report as offensive    Reply Quote
Previous · 1 . . . 138 · 139 · 140 · 141 · 142 · 143 · 144 . . . 317 · Next

Message boards : Number crunching : Problems and Technical Issues with Rosetta@home



©2025 University of Washington
https://www.bakerlab.org