Both Windows PowerShell and PSCX made it into Scott Hanselman’s 2009 Ultimate Developer and Power Users Tool List for Windows.  w00t!

• Supports both PowerShell version 1.0 and 2.0
• Installer updated to allow you to choose between A) install PSCX  default profile, B) append PSCX init script to your existing profile or C) don't mess with my profile.
• PSCX settings provider so that there is no impact on your global variables

• Read-Archive
• Expand-Archive
• Tail-File (with ability to wait for new output e.g. tail -f)
• Get/Push/Pop-EnvironmentBlock for saving and restoring the state of your environment variables

New Scripts

• Invoke-BatchFile (eg InvokeBatchFile vcvarsall x86 and yes the environment variables created in the batch file persist into the PowerShell session)

Occasionally I need to tail some very large log files over the network.  In this scenario, the standard PowerShell approach:

PS> Get-ChildItem *.log | %{ Get-Content $_ | Select –Last 1 }

just isn’t practical for performance reasons.  So the final drop of PSCX 1.2 will have a new cmdlet called Tail-File that is optimized for tailing the end of large (even huge) log files.  Here’s a performance comparison of the two approachs:

19# measure-command { tail-file \\Keith-PC\C\*.txt -count 1 }

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 0
Milliseconds      : 7
Ticks             : 70813
TotalDays         : 8.19594907407407E-08
TotalHours        : 1.96702777777778E-06
TotalMinutes      : 0.000118021666666667
TotalSeconds      : 0.0070813
TotalMilliseconds : 7.0813

20# measure-command {Get-ChildItem \\Keith-PC\C\*.txt | %{ Get-Content $_ | Select -Last 1 }}

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 17

Milliseconds      : 656
Ticks             : 176567027
TotalDays         : 0.000204359984953704
TotalHours        : 0.00490463963888889
TotalMinutes      : 0.294278378333333
TotalSeconds      : 17.6567027
TotalMilliseconds : 17656.7027

That is a speed-up of approximately 2500x.  That is pretty significant especially if you need to do this to a number of large log files.  For reference, the above test tailed two medium size log files (total size: 37.4 MB):

-a---     5/23/2009  6:33 AM      4173558 dirlist.txt
-a---     5/23/2009  6:33 AM     35053946 filelist.txt

Tail-File handles ASCII and Unicode encoding as well UTF8 as long as it contains only ASCII characters i.e. it will not choke on the UTF8 byte order mark.  Oh yeah, it also supports active tailing (tail –f) via the Wait parameter.  Use Ctrl+C to return control back to the console.

This is a continuation of PSCX Tips Part 1.  Here are some more things you can do with the PowerShell Community Extensions.

If you do .NET software development and partially strong name your assemblies then you know that at some point in your official build process you need to finish the strong-naming process.  When we have done this, we cycle through all the exes and dlls in a common output dir and apply the full strong name.  Problem is that we have some native binaries mixed in with the managed binaries.  So we need some quick way to test if a file is a managed assembly.  This is where Test-Assembly comes in handy:

PS> Get-ChildItem | Where {$_ –match '\.(dll|exe)$' –and (Test-Assembly $_)} |
    Foreach {sn.exe /Ra $_.fullname $pathToKeyFile}

On 64-bit Windows you may want to know whether a binary is a 32-bit or 64-bit binary.  That’s easy with Get-PEHeader:

PS> Get-PEHeader .\wordpad.exe

Type                    : PE64
LinkerVersion           : 9.0
OperatingSystemVersion  : 6.1
ImageVersion            : 6.1
SubsystemVersion        : 6.1
SizeOfCode              : 604672
SizeOfInitializedData   : 3915776
SizeOfUninitializedData : 0
AddressOfEntryPoint     : 65624
BaseOfCode              : 4096
BaseOfData              : 0
ImageBase               : 4294967296
SectionAlignment        : 4096
FileAlignment           : 512
Win32VersionValue       : 0
SizeOfImage             : 4538368
SizeOfHeaders           : 1536
Checksum                : 4583061
Subsystem               : Windows
DllCharacteristics      : 33088
SizeOfStackReserve      : 524288
SizeOfStackCommit       : 8192
SizeOfHeapReserve       : 1048576
SizeOfHeapCommit        : 4096
LoaderFlags             : 0

Need to test if the current user is elevated on Vista or Windows 7?  Use Test-UserGroupMembership e.g.:

PS> Test-UserGroupMembership -GroupName Administrators
False

Need to know what the 8.3 filename is for a file?  Use Get-ShortPath e.g.:

PS> Get-ShortPath '.\Windows Azure SDK'
C:\PROGRA~1\WI4C87~1

Need a touch like utility to change a file’s various time fields?  Try Set-FileTime e.g.:

PS> Set-FileTime .\test.csv -Modified (Get-Date).AddDays(-2)

Need to compute the hash for a string or an entire file using various algorithms e.g.:

PS> "hello world" | Get-Hash

Algorithm: MD5

Path       :
HashString : E42B054623B3799CB71F0883900F2764

PS> Get-Hash .\uszip.wsdl -Algorithm SHA1

Algorithm: SHA1

Path       : C:\Users\Keith\uszip.wsdl
HashString : 74910AE77B4089496D5EF829DF3F0485BF855EB9

Need to convert between different OS style line endings in files?  Check out these cmdlets;

• ConvertTo-MacOs9LineEnding
• ConvertTo-UnixLineEnding
• ConvertTo-WindowsLineEnding

Need to convert from or to Base64?  Try out the ConvertTo-Base64 and ConvertFrom-Base64 cmdlets e.g.:

PS> [byte[]](1..255) | ConvertTo-Base64
AQIDBAUGBwgJCgsMDQ4PEBESExQVFhcYGRobHB0eHyAhIiMkJSYnKCkqKywtLi8wMTIzNDU2Nzg5
Ojs8PT4/QEFCQ0RFRkdISUpLTE1OT1BRUlNUVVZXWFlaW1xdXl9gYWJjZGVmZ2hpamtsbW5vcHFy
c3R1dnd4eXp7fH1+f4CBgoOEhYaHiImKi4yNjo+QkZKTlJWWl5iZmpucnZ6foKGio6Slpqeoqaqr
rK2ur7CxsrO0tba3uLm6u7y9vr/AwcLDxMXGx8jJysvMzc7P0NHS09TV1tfY2drb3N3e3+Dh4uPk
5ebn6Onq6+zt7u/w8fLz9PX29/j5+vv8/f7/

When parsing legacy exe text, do you ever need to skip either the first couple of lines or perhaps the last line (or both)?

Skip-Object is very convenient here because you don’t need to know the length of the sequence you are dealing with.  Here’s an example:

PS> 1..10 | Skip-Object -First 2 -Last 1 -Index 4,6
3
4
6
8
9

That’s enough for this post but there’s still plenty more in PSCX.  Stay tuned.

We recently released a beta of the PowerShell Community Extensions that supports PowerShell V2 in addition to V1.   This version also supports 64-bit versions of Windows.  The primary purpose of PSCX is to flesh out the functionality of PowerShell such that tools like cygwin and MKS Toolkit aren’t needed.  We aren’t completely there with PSCX yet but there is quite a bit of useful functionality in PSCX.  Here are some examples.

Enhanced CD function has browser like backwards and forwards history:

PS> cd $pshome;cd $pscx:home;cd $pscx:ScriptsDir;cd~
C:\Windows\System32\WindowsPowerShell\v1.0
C:\Users\Keith\Pscx\Trunk\Src\PscxSnapin\bin\Debug
C:\Users\Keith\Pscx\Trunk\Src\PscxSnapin\bin\Debug\Scripts
C:\Users\Keith

PS> cd

     # Directory Stack:
   --- ----------------
     0 C:\Users\Keith
     1 C:\Windows\System32\WindowsPowerShell\v1.0
     2 C:\Users\Keith\Pscx\Trunk\Src\PscxSnapin\bin\Debug
     3 C:\Users\Keith\Pscx\Trunk\Src\PscxSnapin\bin\Debug\Scripts
->   4 C:\Users\Keith

# Use cd - and cd + to move backwards and forwards through stack
PS> cd -
C:\Users\Keith\Pscx\Trunk\Src\PscxSnapin\bin\Debug\Scripts

PS> cd +
C:\Users\Keith

# Or specify an entry number to jump to directly
PS> cd –1
C:\Windows\System32\WindowsPowerShell\v1.0

# cd up several dir levels at once
PS> cd ....

Sending PowerShell output to the clipboard:

PS> Get-Process | Out-Clipboard

# The following uses the alias ocb and sets output width to prevent text truncation
PS> Get-ChildItem env: | ocb –width 9999

# To get the raw data without any PowerShell formatting
PS> $pwd | Write-Clipboard 

Inspecting the contents of a file ala od.exe  (octal dump):

# View the first 16 bytes of powershell.exe
PS> Format-Hex $pshome\powershell.exe –count 16

Address:  0  1  2  3  4  5  6  7  8  9  A  B  C  D  E  F ASCII
-------- ----------------------------------------------- ----------------
00000000 4D 5A 90 00 03 00 00 00 04 00 00 00 FF FF 00 00 MZ..............

# Using the fhex alias to inspect a text file to see if it has the UTF-8 BOM
PS> fhex $pshome\FileSystem.format.ps1xml -count 48

Address:  0  1  2  3  4  5  6  7  8  9  A  B  C  D  E  F ASCII
-------- ----------------------------------------------- ----------------
00000000 EF BB BF 3C 3F 78 6D 6C 20 76 65 72 73 69 6F 6E ...<?xml version
00000010 3D 22 31 2E 30 22 20 65 6E 63 6F 64 69 6E 67 3D ="1.0" encoding=
00000020 22 75 74 66 2D 38 22 20 3F 3E 20 0D 0A 3C 21 2D "utf-8" ?> ..<!-

Formatting XML:

# Ever have an XML string display as one long line, try this out
PS> "<a><b><c><d/></c></b></a>" | Format-Xml # Alias is fxml
<a>
  <b>
    <c>
      <d />
    </c>
  </b>
</a>

Testing XML for well-formedness and against a schema:

PS> "<a><b><c><d/></c></b></a>" | Test-Xml
True
PS> "<a><b><c><d/><c></b></a>" | Test-Xml -verbose
VERBOSE: The 'c' start tag on line 1 does not match the end tag of 'b'. Line 1, position 19.
False

PS> $schemaPaths
C:\Program Files\Microsoft Visual Studio 9.0\Xml\Schemas\1033\Microsoft.Build.xsd
C:\Program Files\Microsoft Visual Studio 9.0\Xml\Schemas\1033\MSBuild\Microsoft.Build.Core.xsd
C:\Program Files\Microsoft Visual Studio 9.0\Xml\Schemas\1033\MSBuild\Microsoft.Build.CommonTypes.xsd
PS> Test-Xml .\ConsoleApplication1.csproj -SchemaPath $schemaPaths
True

Applying a XSL transform to a file:

PS> Convert-Xml My.xml –XsltPath My.xsl –EnableScript

There is a lot more functionality in PSCX such as the ability to create, read and expand ZIP and TAR files.  But that is enough of a teaser for this post.  Stay tuned for more info on what’s in PSCX.

Last night we released a beta of the PowerShell Community Extensions 1.2.  The release supports Windows PowerShell V2 and has better support for 64-bit Windows.  PowerShell MVP/Rockstar Oisin Grehan has added two very useful cmdlets to this release: Read-Archive and Expand-Archive which works not only on TAR and ZIP files but also on ISO files.  Give it a spin.  You can download it here.  Be sure to let us know if you run into any problems either via the PSCX discussion forum or issue tracker.  BTW, there are two known issues on x64 Windows.  The cmdlets Get-PEHeader and Test-Assembly don’t work.  After a week or so, we’ll look at officially releasing 1.2.

A reader asked me about using PowerShell to process image files.  While PowerShell doesn’t natively support doing this you could perform this task by using PowerShell’s support for .NET which supports image resizing.  However if you have the PowerShell Community Extensions (PSCX) installed you can do this pretty easily using cmdlets like so:

PS> Import-Bitmap .\Foo.jpg | Resize-Bitmap -Percent 50 | 
>>    Export-Bitmap -path fooSmall.jpg -Format jpeg

Can you feel the power?!

psmdtag:cmdlet - Bitmap manipulation

Today I was posting about an issue I was having with Team Foundation Server 2008 HTML alert emails that get generated for check-ins.  The second column in the details area of the email is way too narrow.  I was able to copy/paste the HTML into Expression Web so I could get to the HTML.  I wanted to post that HTML into the MSDN Forum post editor but, well, it doesn't like you putting in non-escaped HTML.  Fortunately PowerShell and PSCX are at my beckon call.  Here is all I had to do to escape my HTML so I could paste it into the HtmlView of the MSDN forum editor:

PS> Get-Clipboard | Foreach {[System.Web.HttpUtility]::HtmlEncode($_)} | Out-Clipboard  

I just love tools that save me time!

MoW wrote up a nice post on invoking XPATH expressions.  Check it out here.  Just wanted to let the PSCX users out there know that the equivalent of MoW's

PS I:\PowerShell> Function invoke-XpathExpression ([xml]$xml,$expression) {                                             
>>   $xn = $xml.PSBase.CreateNavigator()                                                                                
>>   $xn.Evaluate($expression)                                                                                          
>> }                                                                                                                    
>>    

                                                                                                                  
PS I:\PowerShell> # Example using the function                                                                          
PS I:\PowerShell>                                                                                                       
PS I:\PowerShell> invoke-XpathExpression -xml (type gl.xml) -exp "sum(GroceryList/Item/Price)"                          
29.15                                                                                                  

in PSCX would be:

PS> Select-Xml gl.xml -xpath '/GroceryList/Item/Price' | measure value -sum


Count    : 4
Average  :
Sum      : 29.15
Maximum  :
Minimum  :
Property : Value

Note that the PSCX cmdlet Select-Xml is oriented towards "selecting" node-sets hence the name.  Unfortunately xpath expressions that don't result in node-sets will error.  No worries though because PowerShell's Measure-Object cmdlet (measure alias provided by PSCX) can compute the sum easily.

OK so that was a bit shorter but what's the big deal.  Here's the deal.  If your XML uses XML namespaces then this all gets a good bit harder to deal with yourself.  Not impossible mind you.  I have written a number of posts on handling XML that uses XML namespaces but with Select-Xml it is pretty simple.  For instance, let's tweak the XML ever so slightly:

<GroceryList xmlns="tempuri.org">
  <Item>
    <Dept>Produce</Dept>
    <Name>Orange</Name>
    <Price>3.20</Price>
  </Item>
  <Item>
    <Dept>Meat</Dept>
    <Name>Steak</Name>
    <Price>13.20</Price>
  </Item>
  <Item>
    <Dept>Produce</Dept>
    <Name>Lettuce</Name>
    <Price>1.34</Price>
  </Item>
  <Item>
    <Dept>Meat</Dept>
    <Name>Ham</Name>
    <Price>11.41</Price>
  </Item>
</GroceryList>

Note the default namespace declaration on the root element.  Now the previous XPath expressions won't work but here is all we need to do with Select-Xml to make this work:

PS> Select-Xml gl.xml -xpath '/ns:GroceryList/ns:Item/ns:Price' -Namespace "ns=tempuri.org" |
>> measure value -sum
>>

Count    : 4
Average  :
Sum      : 29.15
Maximum  :
Minimum  :
Property : Value

All we needed to do was provide the namespace and a temp prefix (ns) to use in the xpath query.  Note that the -Namespace parameter will take an array of strings that match this format: "<prefix>=<namespace>".

Here are the slides that I presented at today's LiveMeeting.  Thanks to Marco Shaw for setting up the group!

If you are using PSCX 1.1.1 and are running on Vista and your not too attached to your own prompt function, I encourage you to try out EyeCandy.Vista.ps1.  This works best if you are running the PSCX profile.  If so, just modify the following lines like so:

# ---------------------------------------------------------------------------
# You can modify every aspect of the PSCX prompt appearance by
# creating your own eye-candy script.
# ---------------------------------------------------------------------------
## $PscxEyeCandyScriptPreference = '.\EyeCandy.Jachym.ps1'
## $PscxEyeCandyScriptPreference = '.\EyeCandy.Keith.ps1'
$PscxEyeCandyScriptPreference = '.\EyeCandy.Vista.ps1'

With the EyeCandy.Vista settings there is no mistaking an elevated PowerShell session from a non-elevated session.  :-)  The elevated session turns the background color to the DarkRed.  Now it turns out that in my humble opinion DarkRead isn't dark enough to get a good contrast between the background color and error output color.  This can be easily fixed though.  Fire up an elevated PowerShell.  If you are using the PSCX profile, just execute "su".  When the new PowerShell window appears it should have a dark red background.  Select the system menu (upper left corner by the PowerShell icon) and then select Properties.  Now select the Colors tab and then the Screen Background radio button.  Select the DarkRed color and change the "Red:" value to 80 as shown below:

This gives a pretty good contrast for error output and makes darn sure you know when you are in an elevated PowerShell session on Vista.

The PSCX team is getting real close to releasing PSCX 1.1.1.  This is primarily a bug fix release where we have fixed the limitation of the ADSI provider stopping at 1000 objects plus ~30 other issues.  There are a few new features:

NEW CMDLETS
    Get-ExportedType

NEW FUNCTIONS AND FILTERS
    Collect
    Elevate

NEW SCRIPTS:
    Get-ViewDefinition.ps1
    Out-Speech.ps1
    Select-Random.ps1

NEW VARIABLES
    $RegexLib   

NEW ALIASES
    su (Elevate)

Get-ViewDefinition is handy if you are curious if there are alternate views for an object.  Right now the only way to find this out is via the help assuming the help covers this.  With Get-ViewDefiniton you can find out easily:

6> get-process | select -first 1 | Get-ViewDefinition

Name       : Priority
Path       : C:\Windows\System32\WindowsPowerShell\v1.0\DotNetTypes.format.ps1xml
TypeName   : System.Diagnostics.Process
SelectedBy : System.Diagnostics.Process
GroupBy    : PriorityClass
Style      : Table

Name       : StartTime
Path       : C:\Windows\System32\WindowsPowerShell\v1.0\DotNetTypes.format.ps1xml
TypeName   : System.Diagnostics.Process
SelectedBy : System.Diagnostics.Process
GroupBy    :
Style      : Table

Name       : process
Path       : C:\Windows\System32\WindowsPowerShell\v1.0\DotNetTypes.format.ps1xml
TypeName   : System.Diagnostics.Process
SelectedBy : System.Diagnostics.Process
GroupBy    :
Style      : Wide

We are testing right now.  I hope to have this version released by the end of June (hopefully sooner).  Unfortunately like the other PSCX instals, you will have to uninstall your current version of PSCX first.  Too bad we don't have an MSI/installer guru on the PSCX team.  I thought I had the VS Setup project beaten into submission so it would allow us to upgrade version 1.1 but alas when we tested this, the updated PSCX snapin doesn't get registered.  PowerShell doesn't like it when the version info it has registered with it doesn't match the PSSnapin that gets loaded.  Oh well, perhaps we can crack this nut when put out the release after 1.1.1.