| 5 Jun 2020 |
 cdesai | and what about using shallow clones | 17:11:58 |
 Bubu | checkupdates needs potentially all the histoy | 17:12:23 |
| Bubu | as for the gitlab CI setup, I'm not sure if we have any influence there | 17:12:41 |
| Bubu | it's a single spinning 3TB drive | 17:13:49 |
| Bubu | on there is a qcow2 image on an lvm partion | 17:14:07 |
| Bubu | which backs the VM | 17:14:13 |
| Bubu | but I'd also be guessing IO starvation | 17:17:26 |
| * Bubu is tempted to upgrade the machine to buster | 17:19:07 |
| Bubu | that'll sure fix everything | 17:19:37 |
| Bubu | let's see if it can handle 2 parallel jobs at least | 17:21:48 |
| cdesai | yeah the qcow2 overhead would add up | 17:21:53 |
| cdesai | if possible I'd prefer raw disks for wherever the data is being stored. / root could be qcow2 to help with admin / backups / etc | 17:22:15 |
| Bubu | I can probably convert that to raw | 17:23:36 |
 jochensp | how much work would it be to make checkupdates parallel instead? | 17:24:59 |
| Bubu | cdesai: there's a setting for shallow clones for gitlab CI:
The number of changes to be fetched from GitLab when cloning a repository. This can speed up Pipelines execution. Keep empty or set to 0 to disable shallow clone by default and make GitLab CI fetch all branches and tags each time.
| 17:25:42 |
| Bubu | Does rebase still work when using this? | 17:25:58 |
| jochensp | yes | 17:26:07 |
| cdesai | Bubu: to be clear I meant shallow clone for the other repos, not fdroiddata itself | 17:26:29 |
| Bubu | jochensp: not that much probably, but there's an annoying detail about srclibs being able to be used as repos which means a bunch of apps sharing one git repo dir | 17:27:06 |
| Bubu | which then can't be done in parallel | 17:27:24 |
| jochensp | so it would be something like doing all srclibs first and then the rest in parallel? | 17:28:04 |
| jochensp | or could it be even different srclibs versions per app? | 17:28:24 |
| Bubu | In reply to @freenode_jochensp:matrix.org
so it would be something like doing all srclibs first and then the rest in parallel? that should work | 17:28:51 |
| jochensp | looks like.. lovely | 17:29:07 |
| Bubu | I did this yesterday before going with the multiple CI pipelines | 17:29:19 |
| Bubu | there were odd concurrency problems when doing it naively | 17:29:58 |
| jochensp | so you have code for that? | 17:30:00 |
| Bubu | like fdroid wanting to do two git commits at the same time | 17:30:20 |
| Bubu | network failures for some reason | 17:30:27 |
| cdesai | one thing that might help is having a mirror of all the source repos needed, and then using --reference | 17:30:43 |
