Server is out of disk space

Author	Message
morgan Send message Joined: 30 Sep 19 Posts: 8 Credit: 586,552 RAC: 214	Message 626 - Posted: 27 Jun 2020, 11:19:02 UTC Upload is failing, getting ; Server is out of disk space. ID: 626 · Rating: 0 · rate: /

Steve Dodd Send message Joined: 4 Apr 20 Posts: 25 Credit: 747,002 RAC: 463	Message 628 - Posted: 27 Jun 2020, 22:45:43 UTC Someone needs to kick the db_purge processes. :) ID: 628 · Rating: 0 · rate: /

Rysiu Project administrator Project developer Project tester Project scientist Send message Joined: 25 Aug 19 Posts: 409 Credit: 1,240,185 RAC: 0	Message 631 - Posted: 28 Jun 2020, 17:04:36 UTC I make changes and fix errors... ID: 631 · Rating: 0 · rate: /

Rysiu Project administrator Project developer Project tester Project scientist Send message Joined: 25 Aug 19 Posts: 409 Credit: 1,240,185 RAC: 0	Message 634 - Posted: 28 Jun 2020, 18:35:31 UTC I'm cleaning the space on one of the partitions... The process may take some time... I'm monitoring the problem. ID: 634 · Rating: 0 · rate: /

Rysiu Project administrator Project developer Project tester Project scientist Send message Joined: 25 Aug 19 Posts: 409 Credit: 1,240,185 RAC: 0	Message 635 - Posted: 28 Jun 2020, 21:52:50 UTC Storage problem solved. ID: 635 · Rating: 0 · rate: /

mikey Send message Joined: 12 May 20 Posts: 29 Credit: 1,980,776 RAC: 2,209	Message 638 - Posted: 29 Jun 2020, 2:22:26 UTC - in response to Message 635. Storage problem solved. Thank you very much!!! ID: 638 · Rating: 0 · rate: /

kotenok2000 Send message Joined: 22 May 20 Posts: 11 Credit: 244,008 RAC: 581	Message 670 - Posted: 8 Jul 2020, 0:29:05 UTC It happened again ID: 670 · Rating: 0 · rate: /

Dr Who Fan Volunteer moderator Send message Joined: 7 Apr 20 Posts: 533 Credit: 1,235,562 RAC: 1,097	Message 671 - Posted: 8 Jul 2020, 0:37:23 UTC - in response to Message 638. Last modified: 8 Jul 2020, 0:40:22 UTC Yep Full 7/7/2020 19:33:48 \| iThena \| [error] Error reported by file upload server: can't open file PERF_TESTS_0C_5090894_2_30_0_r1041391138_0: No space left on device Server Status as of 7 Jul 2020, 23:28:35 UTC Tasks ready to send 107148 Tasks in progress 4604 Workunits waiting for validation 367189 Workunits waiting for assimilation 647718 Workunits waiting for file deletion 11 Tasks waiting for file deletion 5 Transitioner backlog (hours) 21.83 sample_assimilator (ithena_cnode) boinc-server Not Running sample_assimilator_perf (ithena_perf) boinc-server Not Running ID: 671 · Rating: 0 · rate: /

Rysiu Project administrator Project developer Project tester Project scientist Send message Joined: 25 Aug 19 Posts: 409 Credit: 1,240,185 RAC: 0	Message 672 - Posted: 8 Jul 2020, 16:59:37 UTC Yes. The problem has come up again. I'm trying to control the situation. ID: 672 · Rating: 0 · rate: /

Conan Send message Joined: 24 Sep 19 Posts: 108 Credit: 932,808 RAC: 1,109	Message 674 - Posted: 9 Jul 2020, 10:19:32 UTC Might be why uploads are stalling for some time before finally uploading and the web pages are slow to load. Conan ID: 674 · Rating: 0 · rate: /

Dr Who Fan Volunteer moderator Send message Joined: 7 Apr 20 Posts: 533 Credit: 1,235,562 RAC: 1,097	Message 677 - Posted: 10 Jul 2020, 18:02:22 UTC Still seeing the server full / can't upload error messages. 7/10/2020 13:02:15 \| iThena \| [error] Error reported by file upload server: can't open file PERF_TESTS_0C_6370752_2_30_0_r890170402_0: No space left on device Server still appears to be overworked/overloaded: Task data as of 10 Jul 2020, 16:44:02 UTC Tasks ready to send 329294 Tasks in progress 5381 Workunits waiting for validation 90763 Workunits waiting for assimilation 1775037 Workunits waiting for file deletion 2 Tasks waiting for file deletion 16 Transitioner backlog (hours) 26.80 ID: 677 · Rating: 0 · rate: /

Rysiu Project administrator Project developer Project tester Project scientist Send message Joined: 25 Aug 19 Posts: 409 Credit: 1,240,185 RAC: 0	Message 678 - Posted: 10 Jul 2020, 19:51:46 UTC Now the problem is on the assimilator deamon. The problem with the disk space is a little different. It's not typically disk space. I made some design/implementation bugs in the design. The problem is the number of files on the partition. I'm currently removing this problem. There may still be 'No space left on device' errors. I think this problem will be gone by this Sunday. ID: 678 · Rating: 0 · rate: /

Dr Who Fan Volunteer moderator Send message Joined: 7 Apr 20 Posts: 533 Credit: 1,235,562 RAC: 1,097	Message 679 - Posted: 10 Jul 2020, 19:58:31 UTC - in response to Message 678. Rysiu, Thanks for the quick update! Keep up the good work and hope you can get the gremlins out of the project. ID: 679 · Rating: 0 · rate: /

mikey Send message Joined: 12 May 20 Posts: 29 Credit: 1,980,776 RAC: 2,209	Message 680 - Posted: 10 Jul 2020, 23:35:06 UTC - in response to Message 678. Now the problem is on the assimilator deamon. The problem with the disk space is a little different. It's not typically disk space. I made some design/implementation bugs in the design. The problem is the number of files on the partition. I'm currently removing this problem. There may still be 'No space left on device' errors. I think this problem will be gone by this Sunday. THANK YOU!!! ID: 680 · Rating: 0 · rate: /

Conan Send message Joined: 24 Sep 19 Posts: 108 Credit: 932,808 RAC: 1,109	Message 724 - Posted: 25 Jul 2020, 3:25:30 UTC G'Day Rysiu, Just letting you know that this issue has raised it's ugly head again. Can't upload any work. Conan ID: 724 · Rating: 0 · rate: /

UBT - Timbo Send message Joined: 10 May 20 Posts: 4 Credit: 30,372 RAC: 0	Message 725 - Posted: 25 Jul 2020, 7:52:06 UTC - in response to Message 724. Same issue here: 25/07/2020 08:51:05 \| iThena \| [error] Error reported by file upload server: can't write file PERF_TESTS_0D_9653604_2_30_0_r1586974986_0: No space left on server ID: 725 · Rating: 0 · rate: /

M0CZY Send message Joined: 13 Nov 19 Posts: 20 Credit: 213,226 RAC: 117	Message 800 - Posted: 11 Sep 2020, 14:52:17 UTC - in response to Message 725. Fri 11 Sep 2020 15:50:05 BST \| iThena \| [error] Error reported by file upload server: Server is out of disk space ID: 800 · Rating: 0 · rate: /

Dr Who Fan Volunteer moderator Send message Joined: 7 Apr 20 Posts: 533 Credit: 1,235,562 RAC: 1,097	Message 801 - Posted: 11 Sep 2020, 17:33:37 UTC Last modified: 11 Sep 2020, 17:34:07 UTC Looks the the SEVER IS OUT OF SPACE AGAIN! Add 5 hours to times below to get UTC 9/11/2020 12:07:37 \| iThena \| [error] Error reported by file upload server: can't write file PERF_TESTS_0D_5705906_2_30_0_r1975168869_0: No space left on server 9/11/2020 12:07:37 \| iThena \| Temporarily failed upload of PERF_TESTS_0D_5705906_2_30_0_r1975168869_0: transient upload error 9/11/2020 12:07:37 \| iThena \| Backing off 03:09:33 on upload of PERF_TESTS_0D_5705906_2_30_0_r1975168869_0 ID: 801 · Rating: 0 · rate: /

Dr Who Fan Volunteer moderator Send message Joined: 7 Apr 20 Posts: 533 Credit: 1,235,562 RAC: 1,097	Message 802 - Posted: 11 Sep 2020, 21:20:24 UTC Not getting better. I currently have 426 tasks waiting for validation and growing, still getting the "Error reported by file upload server: can't write file". SERVER STATS seem wonky: Computing status Task data as of 11 Sep 2020, 20:23:48 UTC Tasks ready to send 204312 Tasks in progress 7320 Workunits waiting for validation 129 Workunits waiting for assimilation 27 Workunits waiting for file deletion 49 Tasks waiting for file deletion 52 Transitioner backlog (hours) 26.86 ID: 802 · Rating: 0 · rate: /

Dr Who Fan Volunteer moderator Send message Joined: 7 Apr 20 Posts: 533 Credit: 1,235,562 RAC: 1,097	Message 804 - Posted: 12 Sep 2020, 18:16:35 UTC My failed uploads finally went through but my Validation pending shot up to 353 tasks. Server Status page still showing HIGH hours of backlog and db_purge2 (ithena_perf) - boinc-server is Not Running: Task data as of 12 Sep 2020, 17:47:50 UTC Tasks ready to send 837294 Tasks in progress 4996 Workunits waiting for validation 133 Workunits waiting for assimilation 39 Workunits waiting for file deletion 0 Tasks waiting for file deletion 11 Transitioner backlog (hours) 39.53 Hope the welcome and mention of this as a NEW project on the BOINC BOARDS will not crash the system when the influx of new users happens. ID: 804 · Rating: 0 · rate: /