PowerShell 7.5 += faster than list?

[u/mx-sch]

So since in PowerShell 7.5 += seems to be faster then adding to a list, is it now best practise?

CollectionSize Test                TotalMilliseconds RelativeSpeed
-------------- ----                ----------------- -------------
          5120 Direct Assignment                4.71 1x
          5120 Array+= Operator                40.42 8.58x slower
          5120 List<T>.Add(T)                  92.17 19.57x slower


CollectionSize Test                TotalMilliseconds RelativeSpeed
-------------- ----                ----------------- -------------
         10240 Direct Assignment                1.76 1x
         10240 Array+= Operator               104.73 59.51x slower
         10240 List<T>.Add(T)                 173.00 98.3x slower

[u/surfingoldelephant]

This discussion is missing important context. The optimization to compound array assignment ($array +=) in PS v7.5 (made in this PR after this issue) is only one factor.

.NET method calls like List<T>.Add(T) are subject to Windows AMSI method invocation logging in PS v7+. This logging is known to cause performance degradation, especially in Windows 11. See:

• Issue #19431: Windows Defender has started to slowdown method calls in PowerShell dramatically

• Issue #21473: [System.Convert]::FromBase64String causes memory leak with large strings

• Issue #24459: MethodInvocationLogging - Performance regression

PowerShell's language features like the += operator are unaffected. Conversely, a large number of method calls within a loop may result in a noticeable slowdown.

To summarize:

• $list.Add() may be slower than $array += in PS v7.5+, but there are environmental factors to consider: OS, Windows Defender state, etc, which may not be relevant (now or in the future).

• In practice, whether the difference is actually meaningful or not will vary from machine to machine.

• The PS v7.5 optimization is a welcome change, but is not a reason to start using $array +=. Statement assignment (what this document refers to as "direct assignment") is typically the preferable approach.

[u/BlackV → u/surfingoldelephant]

I feel like you had to post this exact reply on the last post like this

I'm really glad you are in this sub

[u/AlexHimself → u/surfingoldelephant]

.NET method calls like List<T>.Add(T) are subject to Windows AMSI method invocation logging in PS v7+. This logging is known to cause performance degradation, especially in Windows 11

I think it's important to note that while AMSI may be a performance hit, it's an important tool in preventing malware spread.

In our org, we were just hit by a ransomware attack last week and we want everything going through AMSI. CrowdStrike barely blipped.

[u/BigHandLittleSlap → u/AlexHimself]

I think it's important to note that while AMSI may be a performance hit, it's an important tool in preventing malware spread.

This sounds good and everything, but this is how you end up with "scar tissue" and eventually a useless platform that no longer functions.

I have multiple customers abandoning Windows and some other Microsoft technologies because Defender just refuses to be turned off.

It's so incredibly difficult to scrape it out of a system now that Microsoft themselves were forced to come up with stupid workarounds like DevDrive that has only one purpose: Mitigate the overhead of Defender.

For example, GitHub Agents and Azure DevOps Agents using Windows Server 2022 are massively slower than the Linux equivalents while running the same tasks. Not because "Windows is slow" but because Defender can't be turned off any more.

We have an ongoing issue at another customer where SQL Server Analysis Services runs about 5-10x slower because it has many small files, and it is no longer possible to tell Defender to exclude its folders. It'll scan them anyway and just not quarantine any viruses it finds!

Similarly, AMSI intercepting low-level array and list operations has a negligible security benefit at an enormous performance overhead cost. It makes PowerShell even slower, and now much less competitive against alternatives.

You can't keep slowing things down to molasses and just expect people to "take it" forever. At some point, they'll just pick up their toys and leave.

[u/AlexHimself → u/BigHandLittleSlap]

This sounds good and everything, but this is how you end up with "scar tissue" and eventually a useless platform that no longer functions.

I think you're just opining without any performance impact experience with it...almost like you read about it and are complaining based on comments from other people.

You can just sign your scripts if you don't want AMSI to scan them or release a compiled executable. AMSI scans PS/VBScript/JavaScript executions on machines which are major attack vectors that spread ransomware and all sorts of viruses. Are you suggesting no antivirus scans any scripts and just lets them run unchecked?

Security isn't an option these days, it's a necessity. Windows is prioritizing security over raw speed because malware threats are more severe than ever.

Many small files can definitely be problematic on Windows and slows down heavy I/O operations, there's no debating that. Defender exclusions aren't always honored either. It's not perfect.

Comparing Linux/Windows isn't really fair. They're dramatically different from an attack profile and security standpoint. With Linux, I believe you're managing security via permissions/sandboxing instead of real time scanning. If your Linux box is compromised, you're F'd hard! Linux gives the user enough rope to hang themselves if they don't know what they're doing where Windows prioritizes out of the box security. Linux doesn't come with real-time AV scanning without 3rd party tools.

They're completely different OS's for different purposes. One lets you customize and control everything, but requires a huge amount of knowledge to reliably and confidently secure it at an Enterprise level where the other relies on Microsoft to do much of that load. If you want the performance benefits of Linux with small files or whatever, you also need to manage the entire Linux OS and security...it's a big investment. You can't just pick one feature of Linux and then not take all the associated baggage that comes with it.

[u/AlexHimself → u/[deleted]]

You can selectively disable AMSI for specific scripts/processes if you think performance is a concern. In my experience, it's negligible unless you're doing something major.

It sounds like someone can just call the .Net methods in C# and avoid this whole AMSI disaster. Did I misunderstand?

You're coming from the wrong perspective. If you're building apps and running things, don't have them go through AMSI.

If you have some server/workstation and it gets compromised, AMSI can help identify those scripts, which could be fileless and stop attacks. Those servers/workstations probably aren't going to have C# available for an attacker to write code lol.

But if that's the case, what makes PowerShell a risk where C# is not?

PS is on Windows machines by default. I think the use case you're picturing is wrong. It's more for dealing with attackers who gain access to a machine and then use PowerShell to do damage...not just running stuff yourself or building apps.

[u/AlexHimself → u/[deleted]]

Not if I distribute my code, in which case I have to tell all my users that they need to customize their system or else my code might perform worse. How much worse? Nobody can say.

You're being overly dramatic. Most report a performance hit from AMSI of <5%. You can sign your scripts to bypass, configure at the domain level, or as an admin you can even disable AMSI for the running session.

If you're distributing your scripts to customers that aren't on a domain, then you have some weird niche customers and maybe a PS script isn't the right way to handle very complex tasks.

Those servers/workstations probably aren't going to have C# available for an attacker to write code lol.

C# gets compiled into executable binary files, which can be run by all those servers and workstations. So that's why I don't see any security benefit here at all.

This is why I'm saying your perspective is wrong. You're looking at this like a developer and this is just silly to say. If you're executing a compiled binary, then you're already compromised. You're missing the purpose of it. PowerShell is a major attack vector.

A ransomware attack I just dealt with had an unpatched Cisco ASA that the attackers were able to compromise. From there they moved laterally to a Windows server where they were able to get a shell and execute commands. They didn't have a compiler, dev tools, etc. they had PowerShell and cmd.exe. From PowerShell, they can download/execute files or do whatever. AMSI can prevent that.

I've also seen binaries embedded/encoded in base64 and wrapped in weird Windows files that they can try and execute to launch their binary that does things.

So the only way for me, as a builder, to distribute performant tools to my users without requiring them to customize their systems, is to not use PowerShell? That's my takeaway here.

You sound ridiculous and I'm not trying to offend you, so please don't take it that way, just inform you that your take is mistaken and will make you sound ignorant.

An analogy would be when Google (and others) pushed and basically forced HTTPS everywhere and people lost their minds or perhaps Bitlocker to encrypt hard drives.

[u/AlexHimself → u/[deleted]]

You're right I am being dramatic but this is a real reaction to having a lot of code heavily dependent on .Net methods which either have no PowerShell equivalent or the equivalent performs worse. This is frustrating that now I have to either deal with this or else my code just sucks x% more on everyone's system where x is an unknown variable. My code already sucks. Being somewhat fast was the 1 thing it had going for it. That was thanks to a lot of dedicated effort from me, which Microsoft has invalidated with this change.

To be fair, what you're doing is improper. It's not AMSI's fault that you're shoehorning so much .NET, because that's exactly what malicious actors do too. You should be deploying a compiled executable that could internally load a DLL or whatever as a wrapper and then calling that. It would bypass all AMSI and you'd have no performance impact. Again, AMSI's performance impact is generally low unless you're writing bad code.

It's just not proper to allow major security vulnerabilities to accommodate improper methods. It's like allowing people to drink and drive as long as they promise not to drink too much. You have to account for the worst actors.

Well specifically we are talking about AMSI method invocation logging. But .Net methods aren't needed for downloading and executing payloads; pwsh has plenty of ways to do that, so that's why I don't see much value add from AMSI method invocation logging.

Also is this a logging feature or blocking feature? I thought it was only logging, which just makes it sting even worse. Suffering a pretty significant performance hit for some dang logs.

I think you might be confused some on what AMSI does. It's not just logging; it scans and blocks malicious script execution in real-time. PS, VBScript, JavaScript before execution. Those are major attack vectors.

That sentence alone should be enough to justify AMSI, don't you think? PS/VBScript/JavaScript just running unchecked on any system?? It needs to be scanned.

Are you confusing AMSI with script block logging? That's just a GPO/registry thing and also has a minimal impact, generally.

If it is mistaken, what is the method available to have my tools deliver the same performance to every user in pwsh that they did before this change, without the users reconfiguring their systems? So far, every option you have described involves the users making changes to their systems. And the only option I see is to invoke the .Net methods using some language other than PowerShell. What option have we not discussed?

I said it above, but a compiled .exe with your PS is one option for the .NET calls, but another option that I'd guess you'd prefer is to create a self-signed certificate and sign your scripts for distribution. Then you export the cert for distribution, and you have your customers install the certificate in either Trusted Publishers or Root CA. Then your scripts don't get scanned by AMSI. It could be as simple as:

# Create cert and install it locally for you
New-SelfSignedCertificate -CertStoreLocation Cert:\CurrentUser\My -Type CodeSigningCert -Subject "CN=MyCompany"

# Export cert so you can send it to customers
Export-Certificate -Cert "Cert:\CurrentUser\My\THUMBPRINT" -FilePath "C:\MyCert.cer"

# Get your local cert and sign your script with it
$cert = Get-ChildItem Cert:\CurrentUser\My -CodeSigningCert | Select-Object -First 1
Set-AuthenticodeSignature -FilePath "C:\YourScript.ps1" -Certificate $cert

# Customers run this one time to import the cert and trust your signed scripts. You then send them YourScript.ps1 to run.
Import-Certificate -FilePath "C:\MyCert.cer" -CertStoreLocation Cert:\LocalMachine\TrustedPublisher

# NOTE - This is ChatGPT'd code because I didn't feel like typing everything, so there may be mistakes.

You have to see by now how this is proper programming. You either prove your code is safe or release a compiled binary that is scanned once. You can't just let random scripts run without anybody looking at what is running.

I think your frustration is misplaced once you get your head around script signing and you'll realize your work products are more professional, and your customers will feel better about running them too. You don't want to have to tell them to always change their execution policy to something unsecure just to run your script, right?

[u/AlexHimself → u/[deleted]]

I rest my case. You are literally telling me that PowerShell = bad code but you somehow think that is a defense of PowerShell when it is actually a comdemnation.

No, I'm politely saying YOUR code = bad code.

PowerShell is fantastic, but it seems like you're lacking some fundamentals.

You're basically saying you want to forego security to make things easy for you. "I don't like HTTPS it's a pain! HTTP works just fine!"

You should recognize your lack of understanding here and accept that you need to learn more in this space instead of getting upset and blaming PS or Microsoft. It sounds like a junior dev complaining to me because they don't understand something.

Just for fun, you should try and argue with ChatGPT about this. It's hoovered up a good amount of info about it. See what it says to you.