Re: [gentoo-user] Any way to run multiple commands from single script in parallel?

[Andreas Fink]

On Mon, 14 Mar 2022 11:13:13 +0100
"J. Roeleveld" <joost@××××××××.org> wrote:

> Hi,
>
> I often put multiple commands into a single file/script to be run in sequence.
> (each line can be executed individually, there is no dependency)
>
> Is there a tool/method to execute multiple lines/commands simultaneously? Like
> having 3 or 4 run together and when 1 is finished, it will grab the next one in
> the list?
>
> I would prefer this over simply splitting the file as the different lines/
> commands will not take the same amount of time.
>
> Thanks,
>
> Joost
>
>
>

At the end there's a very rudimentary bash script to do this. I did not
do much debugging (probably it fails already if max_jobs>#list_of_jobs).
Anyway it's just making use of sending jobs to the background and
"communicating" through a FIFO pipe (which you might want to delete at
the end).
#!/bin/bash

list_of_jobs=("sleep 3" "sleep 5" "sleep 1" "sleep 10" "sleep 4")
max_jobs=2
my_fifo=/tmp/my_job_fifo
write_to_fifo="yes"

function run_job () {
    eval "$@"
    if [[ $write_to_fifo == "yes" ]]; then
        echo "Writing to fifo ($@)"
        echo 1 > ${my_fifo}
    fi
    echo Finished job "$@"
}

function read_and_start_job() {
    next_job_idx=0
    while [[ ${#list_of_jobs[@]} -gt $next_job_idx ]]; do
        while IFS= read -r line ; do
            echo "next_job_idx=${next_job_idx} total=${#list_of_jobs[@]}"
            if [[ $next_job_idx -lt ${#list_of_jobs[@]} ]] ; then
                job="${list_of_jobs[${next_job_idx}]}"
                echo "Executing: ${job}"
                run_job ${job} &
                let next_job_idx++
            else
                echo "Set write_to_fifo=no"
                write_to_fifo="no"
            fi
        done < ${my_fifo}
    done
    write_to_fifo="no"
    wait
}

rm -Rf ${my_fifo}
mkfifo ${my_fifo}
read_and_start_job &
while [[ ${max_jobs} -gt 0 ]] ; do
    let max_jobs--
    echo 1 > ${my_fifo}
done
wait