Summary: Microsoft PFE, Gary Siepser, talks about outputting data via the pipeline in this exciting conclusion to a five part series.
Microsoft Scripting Guy, Ed Wilson, is here. Today brings an end to a week of guest blog posts by Gary Siepser. You can read the following posts to catch up with the previous days:
- Back to the Basics Part 1: Learn About the PowerShell Pipeline
- Back to the Basics Part 2: Learn about the Pipeline and "Getters"
- Back to the Basics Part 3: Do Something with Your Data
- Back to Basics Part 4: More Ways to Manipulate Data
In Parts 3 and 4 of this series, we talked about a number of different things you can do with data coming down the pipeline. In Part 5, we are going to talk about what you do at the end of a pipeline. This can often be to output or present your data in some way. We’ll focus on formatting your objects to view them a certain way, getting them into simple text files, and exporting to other formats (like CSV and XML).
After you have some data in your pipeline and you have manipulated it the way you want, the last part of the pipeline is to output or present your data. You have likely already noticed that if you choose not to do anything, you will still see your data. Any time you have stuff in the pipeline, if you let that stuff hit the end of the pipeline, Windows PowerShell is going to show it to you in some way.
This is due to the default formatting system in Windows PowerShell. Formatting is simply putting this structured data (your objects) into a more humanly readable form. This usually takes the form of a table or a list.
This default look at our data is great, and I am really glad it is there, but very often we need to see that data presented in a different way. We use the formatting cmdlets to control exactly what that readable rendering should look like. We have the Format-Table, Format-List, and Format-Wide cmdlets to take care of this.
The most common format type I find myself using is a table. Tables are great for comparing object values to each other in a fairly concise way. Format-Table is our best friend for this. Like most of the Object cmdlets we have already covered, the format cmdlets are really easy to use. You simply need to tell Format-Table or Format-List what properties you want to be shown. Format-Wide focuses on only one property at a time.
Format-Table has a parameter that I see used all the time called –AutoSize. Generally, it squeezes everything to the left as tight as it can to fit more in. It also helps with values that are being truncated like the ones you see in the previous example.
Sometimes though, if you have really wide values in your columns, -AutoSize can backfire and cause columns on the right to be left off completely because there is not enough space. Notice the in the following example that the Status column didn’t make the result because of–AutoSize.
Over the years I have encountered a number of folks who would like to see –AutoSize turned on by default because they use it almost every time they use Format-Table. Introduced in Windows PowerShell 3.0, you can run the following command to set the parameter defaults:
$PSDefaultParameterValues['Format-Table:Autosize'] = $true
Now, when you run Format-Table, the output will always be “autosized”! This setting will be lost when you close Windows PowerShell, but you can set it in a profile script to have it run each time you launch Windows PowerShell. For more information, see Help about profiles (Get-Help About_Profiles –ShowWindow).
Tables are great, but they are limited by the amount of width you have. There is a –Wrap parameter that can help with the width issue, but still there is only so much space in a table. This is where a list comes in to play.
I think of Format-List as the report writer. It gives you a vertical list of objects and properties. Each property has as much width as it needs and will wrap to next line if it needs to. Like Format-Table, Format-List needs to know which properties you want to show up.
When you are exploring objects beyond using Get-Member (discussed in Part 3), you can use Format-List followed by an asterisk ( * ) to represent all properties. Take a look at these examples for Format-List:
Format-Wide is really useful when you need to see only one thing, such as the Name property, but you want to squeeze as many objects on the screen as you can. Like the Format-Table cmdlet, there is an –AutoSize parameter to allow Format-Wide to maximize the numbers of columns without truncating your data.
Now that we know how to format our data to see exactly what we want to see, let’s see about how we can record this into a text file. There are a couple of different choices for this. I prefer the Out-File cmdlet, but there is also a Set-Content cmdlet.
The biggest difference between the cmdlets is that Out-File can come after formatting in the pipeline, but Set-Content cannot. Set-Content only works well when the objects coming through the pipeline are already strings. Out-File records whatever you could normally see in the Windows PowerShell console, so I think it’s the better cmdlet, especially for beginners.
Text files are great for log type files and simple long term keeping, but sometimes we need a richer data format. Many IT admins are already familiar with a format called CSV. CSV format is a text file that contains rows of values that are delimited by a character (a comma by default). However, the delimiter can be any character. Commas, tabs, and pipes seem to be the most common, in my experience.
We discussed the Import-CSV cmdlet in Part 2. Now we will look at the Export-CSV cmdlet. It’s a really simple cmdlet. Whatever gets piped in to it gets converted to simple text in the CSV file you create. It will export all the properties in the pipeline to that CSV file.
Remember, if you look at your objects without formatting in Windows PowerShell, you are going to see the default formatting we learned about earlier in this post. This is usually a reduced set of the properties for your objects—generally in some decent looking format.
Export-CSV will export ALL the properties for your objects, so you might end up with an unexpected amount of data in the CSV file. The great news is, if you have been following this series, you already know how to deal with this. The Select-Object cmdlet that we covered in Part 3 allows you to reduce the data to only what you want in the pipeline prior to running the Export-CSV:
You can see in the first example that there is a lot of data in the CSV file. This is because process objects in the pipeline have a large number of properties and Export-CSV lists them all. In the second example, you can see how the data is simplified because Select-Object is in the pipeline prior to the export, and it stripped the data to only the three selected properties. You can also see in the second example that I exported the CSV file by using a tab as the delimiter instead of the default comma.
Another confusing thing about Export-CSV is the #TYPE ... line at the top of file. This line is used specifically when Windows PowerShell imports that file, but it could cause issues if you plan to use the CSV file with some other system that won’t understand the line. Export-CSV has a parameter switch called –NoTypeInformation that you can use to cause Windows PowerShell to leave off that #TYPE ... line.
Many of the things we run in Windows PowerShell are lost when Windows PowerShell is closed. Now that we are storing data in the file system (which persists beyond a closed Windows PowerShell window), we might notice that this is an interesting way we can “save” or “persist” our objects that are using a CSV file. One limitation is that all values import as strings from a CSV file. A format that gets around this limitation is XML, and you can use the Export-CliXML cmdlet.
The Export-CliXML cmdlet works like Export-CSV, but it uses a much richer and more complex XML format for the file. This format retains more information about the objects, including the data type of the properties.
When you look at these examples, you can see how we can export to a rich format like XML (we are only looking at the first 10 lines), and then bring that data back in to Windows PowerShell and use it as if we just ran the original Get-Process.
There are so many things you can get done with your pipeline. This post has investigated some of the output abilities, but take time to explore all the core commands in Windows PowerShell.
I’ve been using Windows PowerShell for years. On a daily basis, I still feel like I discover cool new things I can do with Windows PowerShell, and especially the pipeline.
As I bring this series to a close, I hope you can see that the pipeline in Windows PowerShell is not simply a way avoid learning how to script. The pipeline is this wonderfully intuitive result of well-designed, single-purpose commands, coupled with the object-based nature of Windows PowerShell. It’s a great way for you to accomplish so much with so little. And you can do it in a way that is extremely readable and easy to modify and maintain going forward.
Go forth and solve the world’s problems one pipeline at a time. There is some Windows PowerShell geekiness for you!
~Gary
Thanks, Gary, for such a great series this week.
I invite you to follow me on Twitter and Facebook. If you have any questions, send email to me at scripter@microsoft.com, or post your questions on the Official Scripting Guys Forum. See you tomorrow. Until then, peace.
Ed Wilson, Microsoft Scripting Guy