How to write processes to an XML file with Powershell?

Question

Exporting processes to a CSV file seemingly takes a snapshot and then just writes it out. Yet the XML variant creates a log file which it seemingly writes to indefinitely.

How can I write the results of the csv.ps1 script to an XML file?

csv.ps1

Get-Process | Export-Csv -Path .\Processes.csv -NoTypeInformation

Processes.csv, after running csv.ps1:

"writeback","0",,"0","0","0","0",,"System.Diagnostics.Process (kthreadd)",,"0",,,,,"Process","Microsoft.Win32.SafeHandles.SafeProcessHandle","30","-20",,"False","2019-03-16 7:37:18 a.m.",,"30",".","9223372036854775807","0","System.Diagnostics.ProcessModuleCollection","0","0","0","0","0","0","0","0","0","0","0","0","False","RealTime","0","0","writeback","3","0",,"System.Diagnostics.ProcessThreadCollection",,"0","0","False",,,,"0","0",,,"0","","True","00:00:00","00:00:00","00:00:00",,
"xbrlapi","3233",,"0","0","0","0",,"System.Diagnostics.Process (gnome-session-b)",,"0",,,,,"Process","Microsoft.Win32.SafeHandles.SafeProcessHandle","3419","0",,"False","2019-03-16 7:45:43 a.m.",,"3419",".","9223372036854775807","0","System.Diagnostics.ProcessModuleCollection","0","0","0","0","0","0","0","0","0","0","0","0","False","Normal","0","0","xbrlapi","3","3233",,"System.Diagnostics.ProcessThreadCollection",,"0","0","False",,,,"0","0",,,"0","","True","00:00:00","00:00:00","00:00:00",,
"xdg-desktop-por","12972","10","573689856","8712192","0","0","/usr/libexec/xdg-desktop-portal","System.Diagnostics.Process (systemd)",,"0.06",,,,,"Process","Microsoft.Win32.SafeHandles.SafeProcessHandle","12972","0",,"False","2019-03-16 10:29:22 a.m.",,"12972",".","9223372036854775807","0","System.Diagnostics.ProcessModuleCollection","0","0","0","0","0","0","0","0","0","0","0","0","False","Normal","0","0","xdg-desktop-por","3","12972",,"System.Diagnostics.ProcessThreadCollection","10","573689856","573689856","False",,,,"8712192","8712192",,"System.Diagnostics.ProcessModule (xdg-desktop-portal)","0","","True","00:00:00.0300000","00:00:00.0600000","00:00:00.0300000",,
"xdg-desktop-por","13006","10","474832896","16887808","0","0","/usr/libexec/xdg-desktop-portal-gtk","System.Diagnostics.Process (systemd)",,"0.16",,,,,"Process","Microsoft.Win32.SafeHandles.SafeProcessHandle","13006","0",,"False","2019-03-16 10:29:23 a.m.",,"13006",".","9223372036854775807","0","System.Diagnostics.ProcessModuleCollection","0","0","0","0","0","0","0","0","0","0","0","0","False","Normal","0","0","xdg-desktop-por","3","13006",,"System.Diagnostics.ProcessThreadCollection","10","474832896","474832896","False",,,,"16887808","16887808",,"System.Diagnostics.ProcessModule (xdg-desktop-portal-gtk)","0","","True","00:00:00.0200000","00:00:00.1600000","00:00:00.1400000",,
"xdg-document-po","3375",,"482291712","5652480","0","0",,"System.Diagnostics.Process (systemd)",,"0.08",,,,,"Process","Microsoft.Win32.SafeHandles.SafeProcessHandle","3375","0",,"False","2019-03-16 7:45:42 a.m.",,"3375",".","9223372036854775807","0","System.Diagnostics.ProcessModuleCollection","0","0","0","0","0","0","0","0","0","0","0","0","False","Normal","0","0","xdg-document-po","3","3375",,"System.Diagnostics.ProcessThreadCollection",,"482291712","482291712","False",,,,"5652480","5652480",,,"0","","True","00:00:00","00:00:00.0800000","00:00:00.0800000",,
"xdg-document-po","12985","8","482291712","5857280","0","0","/usr/libexec/xdg-document-portal","System.Diagnostics.Process (systemd)",,"0.03",,,,,"Process","Microsoft.Win32.SafeHandles.SafeProcessHandle","12985","0",,"False","2019-03-16 10:29:22 a.m.",,"12985",".","9223372036854775807","0","System.Diagnostics.ProcessModuleCollection","0","0","0","0","0","0","0","0","0","0","0","0","False","Normal","0","0","xdg-document-po","3","12985",,"System.Diagnostics.ProcessThreadCollection","8","482291712","482291712","False",,,,"5857280","5857280",,"System.Diagnostics.ProcessModule (xdg-document-portal)","0","","True","00:00:00.0100000","00:00:00.0300000","00:00:00.0200000",,
"xdg-permission-","3382",,"255184896","4739072","0","0",,"System.Diagnostics.Process (systemd)",,"0",,,,,"Process","Microsoft.Win32.SafeHandles.SafeProcessHandle","3382","0",,"False","2019-03-16 7:45:43 a.m.",,"3382",".","9223372036854775807","0","System.Diagnostics.ProcessModuleCollection","0","0","0","0","0","0","0","0","0","0","0","0","False","Normal","0","0","xdg-permission-","3","3382",,"System.Diagnostics.ProcessThreadCollection",,"255184896","255184896","False",,,,"4739072","4739072",,,"0","","True","00:00:00","00:00:00","00:00:00",,
"xdg-permission-","12992","7","255184896","4780032","0","0","/usr/libexec/xdg-permission-store","System.Diagnostics.Process (systemd)",,"0",,,,,"Process","Microsoft.Win32.SafeHandles.SafeProcessHandle","12992","0",,"False","2019-03-16 10:29:22 a.m.",,"12992",".","9223372036854775807","0","System.Diagnostics.ProcessModuleCollection","0","0","0","0","0","0","0","0","0","0","0","0","False","Normal","0","0","xdg-permission-","3","12992",,"System.Diagnostics.ProcessThreadCollection","7","255184896","255184896","False",,,,"4780032","4780032",,"System.Diagnostics.ProcessModule (xdg-permission-store)","0","","True","00:00:00","00:00:00","00:00:00",,
"Xorg","1369",,"490586112","98332672","0","0",,"System.Diagnostics.Process (lightdm)",,"131.97",,,,,"Process","Microsoft.Win32.SafeHandles.SafeProcessHandle","1369","0",,"False","2019-03-16 7:38:02 a.m.",,"1369",".","9223372036854775807","0","System.Diagnostics.ProcessModuleCollection","0","0","0","0","0","0","0","0","0","0","0","0","False","Normal","0","0","Xorg","3","1369",,"System.Diagnostics.ProcessThreadCollection",,"490586112","490586112","False",,,,"98332672","98332672",,,"0","","True","00:00:43","00:02:11.9700000","00:01:28.9700000",,
"Xorg","12550",,"499658752","84373504","0","0",,"System.Diagnostics.Process (lightdm)",,"44.97",,,,,"Process","Microsoft.Win32.SafeHandles.SafeProcessHandle","12550","0",,"False","2019-03-16 10:29:11 a.m.",,"12550",".","9223372036854775807","0","System.Diagnostics.ProcessModuleCollection","0","0","0","0","0","0","0","0","0","0","0","0","False","Normal","0","0","Xorg","3","12550",,"System.Diagnostics.ProcessThreadCollection",,"499658752","499658752","False",,,,"84373504","84373504",,,"0","","True","00:00:15.0800000","00:00:44.9700000","00:00:29.8900000",,

xml.ps1

Get-Process | Export-Clixml pi.xml

After running xml.ps1 the pi.xml ends like this:

              <S N="BaseAddress">140154072023040</S>
              <I32 N="ModuleMemorySize">208896</I32>
              <S N="EntryPointAddress">0</S>
            </Props>
          </Obj>
        </IE>
      </Obj>
    </Props>
  </Obj>
</Objs>

(Yes, I suppose I could convert the CSV to XML, I'm more asking how to use this XML facility from directly within powershell without additional steps.)

I think I want a non-streaming option, if that's the correct terminology.

See also:

What's your overall goal? Just an XML variant of the CSV file? Or would you rather *do* something with the `System.Diagnostics.Process` objects you receive from `Get-Process`? — Tomalak, Mar 17 '19 at 18:28
Well, I was really just looking for a reliable way to generate `xml`, @Tomalak. But, the data *could* certainly be of value. — Thufir, Mar 18 '19 at 00:57
Hi, I took your question at face value, but if you aren't wedded to specifically _CLI XML_ format, see my edit which I think may help. Thanks for tidying your question, too! — FSCKur, Mar 19 '19 at 23:23

FSCKur · Accepted Answer · 2019-03-19T23:22:19.330

Edit - At first, I answered the question as originally presented, around "why am I having issues with Export-Clixml, but see my edit further down for a solution involving ConvertTo-Xml.

Interesting question! I believe the answer is not related to streaming but actually to performance and serialisation.

Background:

CSV is flat - it will not serialise structured data. Therefore, it will probably complete quickly, although with low fidelity.
XML is structured - the XML export WILL serialize nested properties, and if those properties themselves have nested properties, then you may end up pulling quite a lot of information from the system.

There's a limit to how clever the serialiser can be, and it will likely accomplish this with multiple sequential calls for each object that comes down the pipeline. (It's a common problem in application development that serialising an object can pull a large graph from the data layer just for a single object. Bear in mind that Export-Clixml is a general-purpose serialiser; one could imagine writing one just to serialise the output from Get-Process, which might be optimised to minimise out-of-process calls.)

I tested in pwsh on Windows. If I pipe Get-Process | Export-Clixml processes.xml then the number of objects I serialise to file is a function of how long I wait before I press Ctrl-C. On my feeble company laptop it's in the order of a second per process at the default depth.

In Powershell on Windows, a ProcessInfo object (what you get back from Get-Process is a wrapper around a native object, which is exposed through COM. It's a handle in .NET to something that is outside of .NET. For this, read out-of-process calls - which are much slower than same-process calls. If you were piping pure .NET objects, then it would be a simple dereference to get the nested properties, and everything would complete quickly.

For each process, it is going to COM to get the loaded DLLs, then it is going to COM to get the threads... these are expensive calls on Windows and, probably, on *nix too.

Answer:

Set the depth on Export-Clixml.

If I specify a depth of 1: Get-Process | Export-Clixml -Depth 1 processes.xml then I get all the running processes within a second or two. But I don't get the full serialisation of nested properties.

In general, serialisation is a trade-off between fidelity and performance.

Edit 2019-03-19

I see you have added a clarifying comment - you want to get to XML, not specifically CLI XML.

Export-Clixml (and the related [System.Management.Automation.PSSerializer]::Serialize()) output XML that is valid, but heavily optimised for Powershell serialisation. If you are sending data to some general service, you're more likely to get mileage from ConvertTo-Xml, which gives you a more typical document structure for inter-process exchange.

> $Xml = Get-Process a* | ConvertTo-Xml -Depth 4      # name begins with 'a'
> $Xml

xml                            Objects
---                            -------
version="1.0" encoding="utf-8" Objects


> $Xml.Save('C:\dev\processes.xml')
> Get-Content 'C:\dev\processes.xml' -TotalCount 10
<?xml version="1.0" encoding="utf-8"?>
<Objects>
  <Object Type="System.Diagnostics.Process">
    <Property Name="Name" Type="System.String">AgentUI</Property>
    <Property Name="SI" Type="System.Int32">1</Property>
    <Property Name="Handles" Type="System.Int32">347</Property>
    <Property Name="VM" Type="System.Int64">2204067930112</Property>
    <Property Name="WS" Type="System.Int64">36667392</Property>
    <Property Name="PM" Type="System.Int64">50425856</Property>
    <Property Name="NPM" Type="System.Int64">29800</Property>

Interesting. If you nest it to a depth of one then you get the results back. Would it then be feasible to run each depth as its own iteration, so that a file is generated for level: one, two, etc? But would that even be usable? It would have to get stitched back together.... — Thufir, Mar 18 '19 at 00:59
It would be a waste of time unless you really need it, and wouldn't address the underlying issue (if I'm right). Can you do some experimentation to see if it is a performance thing? Try `Get-Process | select -first 1 | Export-Clixml processes.xml` and see if it completes within, say, 15 seconds. (IMO acceptable performance would be 200-500ms). Also try `Get-Process | select -first 1 | ConvertTo-Json` for comparison. You can also try piping through `select -Property # properties that you care about` to see if that helps. It's not impossible you've found a bug, this may narrow it down. — FSCKur, Mar 18 '19 at 19:11

How to write processes to an XML file with Powershell?

1 Answers1