Why not do twice as much work?

Message boards : Number crunching : Why not do twice as much work?
Message board moderation

To post messages, you must log in.

AuthorMessage
Profile MarkRBright

Send message
Joined: 12 Mar 20
Posts: 4
Credit: 110,006
RAC: 0
United Kingdom
Message 625 - Posted: 27 Jun 2020, 8:49:32 UTC

Am I missing something here or is perhaps this project missing something?
If you look at the extract from my log file below you can hopefully see that this project could get through twice as many work units as it does. As it stands, when it gets a task, it is always immediately followed by a "Project requested delay of 303 seconds". The task itself only takes about 140 seconds from start to completion of the subsequent upload. The result is that it waits about 160 seconds before it reports it as complete and requests another Work Unit. This is despite having "report completed tasks immediately" set in my cc_config.xml file.
Is there a way of me overriding this sleep time to something more fitting for my computers? Or is it controlled by the Project?
If so, can the projects Admin reduce the delay number to something more in line with the time it takes to do the job?
Looking at the delays of the other projects that I currently do work for, the delay appears to be anything from 7 to 90 seconds with your 303 being exceptional. Clearly the smaller the number, the less idle time for your project on all of your crunchers machines. The flip side is that there would be more calls to your servers, so I accept it's a balance. In my case I would suggest that about 150 seconds would ensure fairly constant crunching which would literally allow me to do twice as much work for your project, and would hopefully not cause your servers to go into meltdown.
Perhaps there any other way round this?
Yours optimistically
Mark

27/06/2020 08:50:24 | iThena | Requesting new tasks for CPU
27/06/2020 08:50:26 | iThena | Scheduler request completed: got 1 new tasks
27/06/2020 08:50:26 | iThena | Project requested delay of 303 seconds
27/06/2020 08:50:28 | iThena | Starting task PERF_TESTS_0_8916908_2_30_0
27/06/2020 08:52:40 | iThena | Computation for task PERF_TESTS_0_8916908_2_30_0 finished
27/06/2020 08:52:43 | iThena | Started upload of PERF_TESTS_0_8916908_2_30_0_r1417084583_0
27/06/2020 08:52:45 | iThena | Finished upload of PERF_TESTS_0_8916908_2_30_0_r1417084583_0
27/06/2020 08:55:31 | iThena | Sending scheduler request: To report completed tasks.
27/06/2020 08:55:31 | iThena | Reporting 1 completed tasks
27/06/2020 08:55:31 | iThena | Requesting new tasks for CPU
27/06/2020 08:55:33 | iThena | Scheduler request completed: got 1 new tasks
27/06/2020 08:55:33 | iThena | Project requested delay of 303 seconds
27/06/2020 08:55:36 | iThena | Starting task PERF_TESTS_0_8920207_2_30_0
ID: 625 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Rysiu
Project administrator
Project developer
Project tester
Project scientist
Avatar

Send message
Joined: 25 Aug 19
Posts: 391
Credit: 1,119,833
RAC: 852
Poland
Message 637 - Posted: 28 Jun 2020, 22:15:16 UTC

Hi MarkRBright!

The time you specified was increased several months ago due to the server load.

Perhaps this time will be reduced...
However, you should expect to increase the processing time, which will also automatically eliminate the problem.
ID: 637 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
mikey

Send message
Joined: 12 May 20
Posts: 29
Credit: 449,912
RAC: 366
United States
Message 640 - Posted: 29 Jun 2020, 11:47:49 UTC - in response to Message 625.  

Am I missing something here or is perhaps this project missing something?
If you look at the extract from my log file below you can hopefully see that this project could get through twice as many work units as it does. As it stands, when it gets a task, it is always immediately followed by a "Project requested delay of 303 seconds". The task itself only takes about 140 seconds from start to completion of the subsequent upload. The result is that it waits about 160 seconds before it reports it as complete and requests another Work Unit. This is despite having "report completed tasks immediately" set in my cc_config.xml file.
Is there a way of me overriding this sleep time to something more fitting for my computers? Or is it controlled by the Project?
If so, can the projects Admin reduce the delay number to something more in line with the time it takes to do the job?
Looking at the delays of the other projects that I currently do work for, the delay appears to be anything from 7 to 90 seconds with your 303 being exceptional. Clearly the smaller the number, the less idle time for your project on all of your crunchers machines. The flip side is that there would be more calls to your servers, so I accept it's a balance. In my case I would suggest that about 150 seconds would ensure fairly constant crunching which would literally allow me to do twice as much work for your project, and would hopefully not cause your servers to go into meltdown.
Perhaps there any other way round this?
Yours optimistically
Mark


There is a way just load up some virtual machines and do multiple workunits at the same time. It does mean you will also have to run another project in each virtual machine, that's a requirement for running these NCI workunits and the data hit could be huge if you are paying for your data to be transferred but you could setup one virtual machine for each cpu core you have, don't know about virtual cores, and run multiple units at the same time. It does take alot of ram to setup them up but it's doable for most people.
The number one user here is getting over 300K credits per day, that would be ALOT of physical machines as I'm getting around 2K credits per day with 18 pc's!!!
ID: 640 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Aurum

Send message
Joined: 10 Feb 21
Posts: 6
Credit: 175,668
RAC: 101
United States
Message 1049 - Posted: 10 Feb 2021, 2:51:51 UTC

I just joined iThena and I've been reading many of the posts. I get the impression one should only run one iThena project per local network, but that might've been one of the older projects such as DIMES or another.
Should I install iThena on each computer in my local network or just one???
TIA
ID: 1049 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Rysiu
Project administrator
Project developer
Project tester
Project scientist
Avatar

Send message
Joined: 25 Aug 19
Posts: 391
Credit: 1,119,833
RAC: 852
Poland
Message 1051 - Posted: 10 Feb 2021, 20:45:57 UTC

You can install the client on multiple computers.
The limitation here is the capacity of your connection.
It's worth analyzing such an idea especially on CNode application ;) you can use multiple computers.
ID: 1051 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Aurum

Send message
Joined: 10 Feb 21
Posts: 6
Credit: 175,668
RAC: 101
United States
Message 1052 - Posted: 11 Feb 2021, 0:52:48 UTC - in response to Message 1051.  

You can install the client on multiple computers.
The limitation here is the capacity of your connection.
It's worth analyzing such an idea especially on CNode application ;) you can use multiple computers.
I'm asking if you learn more by my running iThena on more than one computer on the same local network.

I proved that running iThena on all my computers grinds my internet connection to a halt. My kids are in covid distant learning and they were convinced the internet was dead.
ID: 1052 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Number crunching : Why not do twice as much work?

© 2019-2021 iThena. All rights reserved. | Private Policy

Page generated on 28 Nov 2021, 11:20:37 UTC in 0.0659 seconds.