@powercode Please have a look at the test failure.

Awesome. Out of curiosity: Do you have time measurements of cases where this improvement is noticeable?

@bergmeister It's mostly in allocations - it generates lots of extra strings, like 2 strings per item in the pipeline.

I'm reasoning more in terms of steadily removing silly allocations, and fixing the worst offenders perf-wise, and eventually we will have a more snappy shell.

The image below show the dotmemory view of strings allocated, grouped by callstack, when looking at the first 50000 items in the windows dir. It wasn't the biggest, but it was one of the bigger, and it was low hanging fruit. Does it make sense?

Thanks @powercode. Yes, makes sense. I agree that continous improvement will make it better over time. :)

@bergmeister It may seem like I'm just doing random changes, but I actually measure things now and then :)

dotTrace and dotMemory are almost always running on my machine.

@powercode I was not questioning it, I was just curious for my own education (because knowing this also means that one will know the scenarios where PSCore is stronger than Windows PowerShell meaning that I could recommend an upgrade to clients).
Would you mind having a look at issue #7603 with Import-Csv please which is causing actual OutOfMemory problems in some workflows where CSV files are a couple of GB large.

@bergmeister Windows PowerShell is still more efficient than PowerShell Core
in many scenarios. Main reason that .Net Framework engine works differently than .Net Core one. Optimizing individual cmdlets would be fine, but maybe we need something more general.

@bergmeister I'm currently working on optimizations for the formatting system, the filesystem provider, etc that makes it out-perform windows powershell with a huge margin, often like 4x. And with a memory footprint reduction of the similar size.

Import-Csv is problematic, since our property abstraction is heavier that I would like it to be. Especially in cases like that, where we actually have a table, we wouldn't need to store all the metadata for each object. I haven't given it so much thought - just passed by it last fall when doing related work.

I have an idea that I like to try out for Import-CSV - could drastically reduce the memory footprint.
That is to generate a dynamic assembly, with a class containing the fields of the CSV.

I was thinking about compiling to classes too and found some problems in the approach. One problem could be solved by #8852 (not trivial). After finding IDataView #8855 I suppose this is preferred way we should start research.