Keith's profileKeith Hill's BlogPhotosBlogListsMore Tools Help
    August 02

    Tail-File Cmdlet Coming in PSCX 1.2

    Occasionally I need to tail some very large log files over the network.  In this scenario, the standard PowerShell approach:

    PS> Get-ChildItem *.log | %{ Get-Content $_ | Select –Last 1 }

    just isn’t practical for performance reasons.  So the final drop of PSCX 1.2 will have a new cmdlet called Tail-File that is optimized for tailing the end of large (even huge) log files.  Here’s a performance comparison of the two approachs:

    19# measure-command { tail-file \\Keith-PC\C\*.txt -count 1 }

    Days              : 0
    Hours             : 0
    Minutes           : 0
    Seconds           : 0
    Milliseconds      : 7
    Ticks             : 70813
    TotalDays         : 8.19594907407407E-08
    TotalHours        : 1.96702777777778E-06
    TotalMinutes      : 0.000118021666666667
    TotalSeconds      : 0.0070813
    TotalMilliseconds : 7.0813

    20# measure-command {Get-ChildItem \\Keith-PC\C\*.txt | %{ Get-Content $_ | Select -Last 1 }}

    Days              : 0
    Hours             : 0
    Minutes           : 0
    Seconds           : 17

    Milliseconds      : 656
    Ticks             : 176567027
    TotalDays         : 0.000204359984953704
    TotalHours        : 0.00490463963888889
    TotalMinutes      : 0.294278378333333
    TotalSeconds      : 17.6567027
    TotalMilliseconds : 17656.7027

    That is a speed-up of approximately 2500x.  That is pretty significant especially if you need to do this to a number of large log files.  For reference, the above test tailed two medium size log files (total size: 37.4 MB):

    -a---     5/23/2009  6:33 AM      4173558 dirlist.txt
    -a---     5/23/2009  6:33 AM     35053946 filelist.txt

    Tail-File handles ASCII and Unicode encoding as well UTF8 as long as it contains only ASCII characters i.e. it will not choke on the UTF8 byte order mark.  Oh yeah, it also supports active tailing (tail –f) via the Wait parameter.  Use Ctrl+C to return control back to the console.

    Comments

    Please wait...
    Sorry, the comment you entered is too long. Please shorten it.
    You didn't enter anything. Please try again.
    Sorry, we can't add your comment right now. Please try again later.
    To add a comment, you need permission from your parent. Ask for permission
    Your parent has turned off comments.
    Sorry, we can't delete your comment right now. Please try again later.
    You've exceeded the maximum number of comments that can be left in one day. Please try again in 24 hours.
    Your account has had the ability to leave comments disabled because our systems indicate that you may be spamming other users. If you believe that your account has been disabled in error please contact Windows Live support.
    Complete the security check below to finish leaving your comment.
    The characters you type in the security check must match the characters in the picture or audio.
    Keith Hill has turned off comments on this page.

    Trackbacks

    Weblogs that reference this entry
    • None