| 25 Jun 2020 |
 _hc | its a lot longer than 3s a package | 14:55:19 |
| _hc | some take minutes | 14:55:25 |
 Bubu | weird | 14:55:39 |
| _hc | don't really know why | 14:55:52 |
| _hc | just observed that | 14:55:59 |
| Bubu | I've run it over /repo (which has like 10000 apks) and 3s was the average time i got | 14:56:03 |
| _hc | I do know all the malware companies have custom code to optimize this kind of stuff | 14:56:13 |
| _hc | maybe that was with an SSD? | 14:56:32 |
| Bubu | most apks are under 1s for me | 14:56:36 |
| _hc | this is on gcc149, which is a spinner disk | 14:56:38 |
| Bubu | yeah pretty fast ssd | 14:56:49 |
| _hc | in which case, faster code won't help muc | 14:56:58 |
| Bubu | is it pegging the cpu? | 14:57:36 |
| _hc | unless apkanalyzer is doing stupid stuff | 14:57:40 |
| Bubu | *all cpus | 14:57:40 |
| Bubu | for me it got my system to about 60% load without doing any parallelization | 14:58:04 |
| _hc | I'm running multiple instances | 14:58:30 |
| Bubu | ah, okay | 14:58:41 |
| Bubu | makes sense | 14:58:43 |
| _hc | probably too many | 14:58:56 |
| Bubu | apkanalyzer definitely has some multithreading already going on | 14:58:58 |
| Bubu | _hc: can you take a minute and double check what I'm doing here? https://gitlab.com/fdroid/fdroiddata/-/issues/2073#note_368282439 | 14:59:37 |
| _hc | in related news, I have this "faup" URL extraction process based on grepping strings and feeding it to faup. | 14:59:56 |
| _hc | its been pegging 32 cores for ~2 months, mostly with grep | 15:00:14 |
| _hc | sometimes, single grep instances use up to 80GB RAM! | 15:00:23 |
| _hc | sure, I'lllook | 15:00:32 |
 cdesai | worth trying https://github.com/BurntSushi/ripgrep or https://github.com/ggreer/the_silver_searcher | 15:02:39 |
| _hc | have you tried them? | 15:06:47 |
| cdesai | yes, I use rg exclusively, and was using ag before I found that | 15:07:15 |
| _hc | is ripgrep a drop-in grep replacement? | 15:09:20 |
