tldr/pages/common/parallel.md

37 lines
1009 B
Markdown
Raw Normal View History

# parallel
2015-12-30 04:55:04 +00:00
> Run commands on multiple CPU cores.
2019-06-04 10:00:18 +01:00
> More information: <https://www.gnu.org/software/parallel>.
2015-12-30 04:55:04 +00:00
- Gzip several files at once, using all cores:
2015-12-30 04:55:04 +00:00
`parallel gzip ::: {{path/to/file1 path/to/file2 ...}}`
2015-12-30 04:55:04 +00:00
- Read arguments from `stdin`, run 4 jobs at once:
2015-12-30 04:55:04 +00:00
`ls *.txt | parallel -j4 gzip`
- Convert JPEG images to PNG using replacement strings:
2015-12-30 04:55:04 +00:00
`parallel convert {} {.}.png ::: *.jpg`
- Parallel xargs, cram as many args as possible onto one command:
2015-12-30 04:55:04 +00:00
`{{args}} | parallel -X {{command}}`
- Break `stdin` into ~1M blocks, feed each block to `stdin` of new command:
2015-12-30 04:55:04 +00:00
`cat {{big_file.txt}} | parallel --pipe --block 1M {{command}}`
2015-12-30 04:55:04 +00:00
- Run on multiple machines via SSH:
2015-12-30 04:55:04 +00:00
`parallel -S {{machine1}},{{machine2}} {{command}} ::: {{arg1}} {{arg2}}`
2024-02-08 07:03:52 +00:00
- Download 4 files simultaneously from a text file containing links showing progress:
`parallel -j4 --bar --eta wget -q {} :::: {{path/to/links.txt}}`
- Print the jobs which `parallel` is running in `stderr`:
2024-02-08 13:45:33 +00:00
`parallel -t {{command}} ::: {{args}}`