Understanding and Using the PowerShell 7 ForEach Parallel Option

powershell hero img

Loops are one of those crucial logic components that every programming language uses extensively and PowerShell is by no means any different. A loop allows you to iterate through a collection of objects performing one or more operations on each.

The PowerShell ForEach-Object cmdlet allows you to create loops. This cmdlet is simple and to the point, but by default it is sequential. The next value in the loop is only processed after the previous one has returned. Up until PowerShell 7.0, you haven’t been able to perform an operation in parallel, which means multiple values at the same time, natively using the widely used ForEach-Object language feature. Introduced in PowerShell 7.0 Beta 3, the new ForEach-Object -Parallel feature changes that.

The Parallel option on the ForEach-Object cmdlet allows you to run a ForEach-Object loop against multiple values at the same time. With very minimal refactoring you can add this new functionality onto existing ForEach-Object loops and see how much of a speed increase you can get.

One special note is that this is not the same as the foreach loop. ForEach-Object is intended to be used over pipelined objects where foreach is intended to be used as in conventional languages. The pipeline is a central PowerShell feature that passes full objects, one after another, to subsequent commands. A very powerful feature, it is well suited to the ForEach-Object command, but not so much for the foreach loop.

ForEach-Object Before Parallel

As described above, the ForEach-Object is intended to iterate through objects on the pipeline. To demonstrate that let’s iterate over a collection of ten integer objects and time how long it takes. Using the | (pipe) character, we are passing each integer object and outputting the object. As you can see below it moves very quickly.

$Collection = 1..10

(Measure-Command {
  $Collection | ForEach-Object {
    $_
  }
}).TotalMilliseconds
# Result: 4.8004

What if we introduce a one-second sleep into each loop? This is a much more realistic example as many of the operations you might perform, may require several steps or have operations themselves that take some time to complete. Examples may be retrieving user properties, testing a computer network connection, or copying a file. In this case, we are just going to sleep for one second on each iteration.

$Collection = 1..10

(Measure-Command {
  $Collection | ForEach-Object {
    Start-Sleep -Seconds 1
    $_
  }
}).TotalMilliseconds
# Result: 10008.0906

As you can see, it’s pretty much exactly ten seconds, and in this case, scales linearly. If we had a very large collection of objects, hundreds to thousands, this can very quickly become extremely time consuming and may exceed the time you have allotted for the script or operation to run.

Simple collections that have short-lived operations may not really benefit from the ability to run in parallel. They might execute so fast that any speedups are negligible. But there are many cases where loops that perform complex operations on collection values would really benefit from being able to speed up its execution. For that let’s see what parallelizing our ForEach-Object loops can do.

Parallelizing ForEach-Object

If you have a large number of objects in a collection that take one or more seconds to complete, then really how much can the parallel option speed this up? Let’s take the second example from before and simply add the new parallel option on. Additionally, we are going to write out the object we are operating on to the host.

$Collection = 1..10

(Measure-Command {
  $Collection | ForEach-Object -Parallel {
    Start-Sleep -Seconds 1
    Write-Host $_
  }
}).TotalMilliseconds

3
1
2
4
5
6
8
7
9
10
# Result: 2391.4714

It took what was a consistent ten-second operation down to under three seconds! Also important to note is the order of the returned objects. As you can see they are not sequential. This is because it is running five operations at the same time, by default, and returning them as soon as it’s done.

You might notice I’ve mentioned that it runs five operations at the same time by default. You are free to change that by using the newly added -ThrottleLimit parameter. If you have compute-intensive tasks then this limit shouldn’t exceed the available number of cores. If your operations are such that they spend a lot of time waiting, such as retrieving data from Active Directory, then you are free to up the limit to a higher number as it won’t affect system resources nearly as much.

Should I use this everywhere?

The short answer is, probably not. There are plenty of cases where you might want the loop to proceed sequentially and others where it just moves so fast naturally that adding all the overhead of the parallel functionality actually increases speed rather than decreases it.

There are also the cases, that depending on what you are doing, might be detrimental to the environment. Such cases tend to be ones that are tough on a system or network resources. An example would be transferring files, if you do too many in parallel, you will saturate your connection and may actually slow everything down.

Using ForEach-Object Parallel Today!

As of this writing, you can install PowerShell 7.0, in its beta form, with the general availability coming out shortly. Now that you have seen how to easily refactor your code using the new parallel parameter on ForEach-Object, you can undoubtedly find many cases where this will greatly speed up your code.

Related Articles: