Summary: Boe Prox shows us how to view the entries in the USN change journal.
Honorary Scripting Guy and Windows PowerShell MVP, Boe Prox, here today, filling in for my good friend, the Scripting Guy. Today I'm continuing on from yesterday’s post, Connect to USN Change Journal. In today’s post, I'll look at the actual entries of the journal, which will show us information about the files and folders that are being created, modified, and deleted.
Much like yesterday, I need to create some methods through reflection in addition to some Structs and Enums to handle some of the work that I will be doing. Some of this will look familiar because I created these when I set up for the change journal connection.
I won’t be showing all of the code here because it is literally 18 pages of building up everything. Instead, you can find the script in the Script Center Repository: Demo Script to View USN Change Journal Entries. I will show the rest of the code to make the connection and then to pull the entries.
First off, let’s make the connection to the change journal so we can have the handle available to us. Unlike what I used yesterday, I'll make some of my code into functions because these have the potential to be used more than once. Here are the functions I use to make the journal connection:
Function OpenUSNJournal {
Param ($DriveLetter = 'c:')
$FileName = "\\.\$DriveLetter"
$Access = [System.IO.FileAccess]::Read -BOR [System.IO.FileAccess]::Write
$ShareAccess = [System.IO.FileShare]::Read -BOR [System.IO.FileShare]::Write
$FileMode = [System.IO.FileMode]::Open
$VolumeHandle = [PoshChJournal]::CreateFile(
$FileName,
$Access,
$ShareAccess,
[intptr]::Zero,
$FileMode,
0,
[intptr]::Zero
)
If ($VolumeHandle -eq -1) {
Write-Warning ("CreateFile failed: 0x{0:x}" -f [System.Runtime.InteropServices.Marshal]::GetHRForLastWin32Error())
} Else {
$VolumeHandle
}
}
Function GetUsnJournal {
[OutputType('System.Journal.UsnJournal')]
Param ($DriveLetter = 'C:')
$JournalData = New-Object USN_JOURNAL_DATA
[long]$dwBytes=0
$VolumeHandle = OpenUSNJournal -DriveLetter $DriveLetter
If ($VolumeHandle -AND $VolumeHandle -ne -1) {
$return = [PoshChJournal]::DeviceIoControl(
$VolumeHandle,
[EIOControlCode]::FSCTL_QUERY_USN_JOURNAL,
[ref]$Null,
0,
[ref]$JournalData,
[System.Runtime.InteropServices.Marshal]::SizeOf([type][USN_JOURNAL_DATA]),
[ref]$dwBytes,
[intptr]::Zero
)
If ($Return) {
$JournalData.pstypenames.insert(0,'System.Journal.UsnJournal')
$JournalData
}
} Else {
Write-Warning "Unable to get handle of volume <$($DriveLetter)>!"
}
}
Now we can move forward with connecting to the journal:
$VolumeHandle = OpenUSNJournal -DriveLetter $DriveLetter #Using ‘C:’ as $DriveLetter
If ($VolumeHandle) {
$JournalData = GetUsnJournal -VolumeHandle $VolumeHandle
}
Now we have our handle and the journal information, which will prove useful in a short bit.
I want to ensure that I pull all of the possible journal entries, so I set up a variable that covers all of the Reason Masks:
$ReasonMask = [USN_REASON]([USN_REASON].GetEnumNames())
Now we can start the fun of working through the entries of the change journal. I need to create a buffer that will be used to track where we are with the entries. I am also using the journal data that I collected earlier to ensure that I am starting at the first USN entry. This ensures that I am not needlessly looking for data that isn’t there.
Write-Verbose 'Creating buffer'
$DataSize = [System.Runtime.InteropServices.Marshal]::SizeOf([type][uint64]) * 0x4000
$DataBuffer = [System.Runtime.InteropServices.Marshal]::AllocHGlobal($DataSize)
[void][PoshChJournal]::ZeroMemory($DataBuffer, $DataSize)
$AvailableBytes = 0
$ReadData = New-Object READ_USN_JOURNAL_DATA
$ReadData.StartUsn = $JournalData.FirstUsn
Write-Debug "Starting USN: $($ReadData.StartUsn)"
$ReadData.ReasonMask = $ReasonMask
$ReadData.UsnJournalID = $JournalData.UsnJournalID
$ReadDataSize = [System.Runtime.InteropServices.Marshal]::SizeOf($ReadData)
$ReadBuffer = [System.Runtime.InteropServices.Marshal]::AllocHGlobal($ReadDataSize)
[void][PoshChJournal]::ZeroMemory($ReadBuffer, $ReadDataSize)
[System.Runtime.InteropServices.Marshal]::StructureToPtr($ReadData, $ReadBuffer, $True)
$ReadMore = $True
$Page = 0
I am looking to query the journal for entries based on what I have already defined with the buffer and starting USN entry. Note that I am using the same DeviceIOControl method, but instead, I am using EIOControlCode::FSCTL_READ_USN_JOURNAL, which says that I want to read the entries instead of only viewing the journal information.
While ($ReadMore) {
$Page++
$return = [PoshChJournal]::DeviceIoControl(
$VolumeHandle,
[EIOControlCode]::FSCTL_READ_USN_JOURNAL,
$ReadBuffer,
$ReadDataSize,
$DataBuffer,
$DataSize,
[ref]$AvailableBytes,
[intptr]::Zero
)
Assuming that I was successful in my query and there is data available to read, I will start processing the data. Basically, I will take the bytes returned and convert that into an object by using a couple of custom functions:
- NewUSNEntry (which uses a function called GetFilePath)
- ConvertToUSNReason.
These functions are used to pull the data into a Struct to make the results humanly readable. Here is the source code for these functions:
Function ConvertToUSNReason {
[cmdletbinding()]
Param(
$ReasonCode
)
$List = New-Object System.Collections.ArrayList
Switch ($ReasonCode) {
($ReasonCode -BOR 0x00000001) {[void]$List.Add('USN_REASON_DATA_OVERWRITE')}
($ReasonCode -BOR 0x00000002) {[void]$List.Add('USN_REASON_DATA_EXTEND')}
($ReasonCode -BOR 0x00000004) {[void]$List.Add('USN_REASON_DATA_TRUNCATION')}
($ReasonCode -BOR 0x00000010) {[void]$List.Add('USN_REASON_NAMED_DATA_OVERWRITE')}
($ReasonCode -BOR 0x00000020) {[void]$List.Add('USN_REASON_NAMED_DATA_EXTEND')}
($ReasonCode -BOR 0x00000040) {[void]$List.Add('USN_REASON_NAMED_DATA_TRUNCATION')}
($ReasonCode -BOR 0x00000100) {[void]$List.Add('USN_REASON_FILE_CREATE')}
($ReasonCode -BOR 0x00000200) {[void]$List.Add('USN_REASON_FILE_DELETE')}
($ReasonCode -BOR 0x00000400) {[void]$List.Add('USN_REASON_EA_CHANGE')}
($ReasonCode -BOR 0x00000800) {[void]$List.Add('USN_REASON_SECURITY_CHANGE')}
($ReasonCode -BOR 0x00001000) {[void]$List.Add('USN_REASON_RENAME_OLD_NAME')}
($ReasonCode -BOR 0x00002000) {[void]$List.Add('USN_REASON_RENAME_NEW_NAME')}
($ReasonCode -BOR 0x00004000) {[void]$List.Add('USN_REASON_INDEXABLE_CHANGE')}
($ReasonCode -BOR 0x00008000) {[void]$List.Add('USN_REASON_BASIC_INFO_CHANGE')}
($ReasonCode -BOR 0x00010000) {[void]$List.Add('USN_REASON_HARD_LINK_CHANGE')}
($ReasonCode -BOR 0x00020000) {[void]$List.Add('USN_REASON_COMPRESSION_CHANGE')}
($ReasonCode -BOR 0x00040000) {[void]$List.Add('USN_REASON_ENCRYPTION_CHANGE')}
($ReasonCode -BOR 0x00080000) {[void]$List.Add('USN_REASON_OBJECT_ID_CHANGE')}
($ReasonCode -BOR 0x00100000) {[void]$List.Add('USN_REASON_REPARSE_POINT_CHANGE')}
($ReasonCode -BOR 0x00200000) {[void]$List.Add('USN_REASON_STREAM_CHANGE')}
($ReasonCode -BOR 0x80000000) {[void]$List.Add('USN_REASON_CLOSE')}
}
$List -join ', '
}
Function NewUsnEntry {
[cmdletbinding()]
Param (
[intptr]$UsnRecord,
[string]$DriveLetter,
[IntPtr]$VolumeHandle
)
#region Constants
$FILE_ATTRIBUTE_DIRECTORY = 0x00000010
#endregion Constants
#region Marshal to Struct
$USN_RECORD = [System.Runtime.InteropServices.Marshal]::PtrToStructure($UsnRecord, [type][USN_RECORD])
#endregion Marshal to Struct
#region Data Conversions
$Name = [System.Runtime.InteropServices.Marshal]::PtrToStringUni([intptr](
$UsnRecord.ToInt64()+$USN_RECORD.FileNameOffset),
($USN_RECORD.FileNameLength/2)
)
#endregion Data Conversions
#region Object Creation
If (($USN_RECORD.FileAttributes -BAND $FILE_ATTRIBUTE_DIRECTORY) -eq 0) {
$IsFile = $True
$IsFolder = $False
} Else {
$IsFile = $False
$IsFolder = $True
}
[pscustomobject]@{
Name = $Name
FullName = GetFilePath -PFileRefNumber $USN_RECORD.ParentFileReferenceNumber -VolumeHandle $VolumeHandle -DriveLetter $DriveLetter -File $Name
TimeStamp = [DateTime]::FromFileTime($USN_RECORD.TimeStamp)
Reason = ConvertToUsnReason $USN_RECORD.Reason
RecordLength = $USN_RECORD.RecordLength
FileReferenceNumber = $USN_RECORD.FileReferenceNumber
ParentFileReferenceNumber = $USN_RECORD.ParentFileReferenceNumber
USN = $USN_RECORD.USN
FileAttributes = [System.IO.FileAttributes]($USN_RECORD.FileAttributes)
IsFile = $IsFile
IsDirectory = $IsFolder
}
#endregion Object Creation
}
Function GetFilePath {
Param (
[int64]$PFileRefNumber,
[IntPtr]$VolumeHandle,
[string]$DriveLetter,
[string]$File
)
$OBJ_CASE_INSENSITIVE = 0x40
$FILE_OPEN = 0x1
$FILE_OPEN_FOR_BACKUP_INTENT = 0x4000
$FILE_OPEN_BY_FILE_ID = 0x2000
[long]$AllocationSize = 0
$UnicodeString = New-Object UNICODE_STRING
$ObjectAttributes = New-Object OBJECT_ATTRIBUTES
$IoStatusBlock = New-Object IO_STATUS_BLOCK
$FileHandle = [intptr]::Zero
$Buffer = [System.Runtime.InteropServices.Marshal]::AllocHGlobal(4096)
$RefPtr = [System.Runtime.InteropServices.Marshal]::AllocHGlobal(8)
$ObjectAttributeSize = [System.Runtime.InteropServices.Marshal]::SizeOf($ObjectAttributes)
$ObjAttIntPtr = [System.Runtime.InteropServices.Marshal]::AllocHGlobal($ObjectAttributeSize)
[void][System.Runtime.InteropServices.Marshal]::WriteInt64($RefPtr, $PFileRefNumber)
$UnicodeString.Length = 8
$UnicodeString.MaximumLength = 8
$UnicodeString.Buffer = $RefPtr
[void][System.Runtime.InteropServices.Marshal]::StructureToPtr($UnicodeString, $ObjAttIntPtr, $True)
$ObjectAttributes.Length = [System.Runtime.InteropServices.Marshal]::SizeOf($ObjectAttributes)
$ObjectAttributes.ObjectName = $ObjAttIntPtr
$ObjectAttributes.RootDirectory = $VolumeHandle
$ObjectAttributes.Attributes = $OBJ_CASE_INSENSITIVE
$Access = [System.IO.FileAccess]::Read
$ShareAccess = [System.IO.FileShare]::Read -BOR [System.IO.FileShare]::Write
$FileMode = [System.IO.FileMode]::Open
$Return = [PoshChJournal]::NtCreateFile(
[ref]$FileHandle,
$Access,
[ref]$ObjectAttributes,
[ref]$IoStatusBlock,
[ref]$AllocationSize,
0,
$ShareAccess,
$FILE_OPEN,
($FILE_OPEN_FOR_BACKUP_INTENT -BOR $FILE_OPEN_BY_FILE_ID),
[intptr]::Zero,
0
)
If ($Return -eq 0) {
$Return = [PoshChJournal]::NTQueryInformationFile(
$FileHandle,
[ref]$IoStatusBlock,
$Buffer,
4096,
[FILE_INFORMATION_CLASS]::FileNameInformation
)
If ($Return -eq 0) {
$NameLength = [System.Runtime.InteropServices.Marshal]::ReadInt32($Buffer,0)
$BufferPtr = New-Object IntPtr -ArgumentList ($Buffer.ToInt64()+4)
$Path = [System.Runtime.InteropServices.Marshal]::PtrToStringUni($BufferPtr,($NameLength/2))
"{0}{1}\{2}" -f $DriveLetter,$Path,$File
}
}
[void][PoshChJournal]::CloseHandle($FileHandle)
[void][System.Runtime.InteropServices.Marshal]::FreeHGlobal($Buffer)
[void][System.Runtime.InteropServices.Marshal]::FreeHGlobal($ObjectAttributeSize)
[void][System.Runtime.InteropServices.Marshal]::FreeHGlobal($RefPtr)
}
Now let’s process all of this data!
If ($return) {
Write-Verbose "Processing USN entries" -Verbose
$Uint64Size = [System.Runtime.InteropServices.Marshal]::SizeOf([type][uint64])
$UsnRecord = New-Object intptr -ArgumentList ($DataBuffer.ToInt64() + $Uint64Size)
Write-Verbose "Initial Bytes: $($AvailableBytes)" -Verbose
While ($AvailableBytes -gt 60) {
$UsnEntry = NewUsnEntry -UsnRecord $UsnRecord -DriveLetter $DriveLetter -VolumeHandle $VolumeHandle
$UsnEntry.pstypenames.insert(0,'System.ChangeJournal.UsnEntry')
$UsnEntry
$UsnRecord = New-Object IntPtr -ArgumentList ($UsnRecord.ToInt64() + $UsnEntry.RecordLength)
$AvailableBytes = $AvailableBytes - $UsnEntry.RecordLength
Write-Debug "Available Bytes: $($AvailableBytes)"
}
} Else {
Write-Warning 'Issue occurred reading Usn entries!'
Break
}
We can finally see the results of our data collection!
Pretty wild to see some of the stuff that is available to view here, right? Depending on the size of your change journal, it could take a while to cover all of the changes. Additionally, the starting point could be a few days ago. You could adjust the starting USN number based on some other entries, but keep in mind that if the number isn’t exact with this script, it will most likely throw an error.
We aren’t finished with the code yet. I also have the capability to page through the results if needed. I am also checking to see if any more data is available. And lastly, I'll ensure that I clean up after myself to avoid any issues with memory and not closing open connections.
$NextUSN = [System.Runtime.InteropServices.Marshal]::ReadInt64($DataBuffer,0)
Write-Debug "Next USN: $($NextUSN) - Journal Next USN: $($JournalData.NextUsn)"
$NoMoreData = $NextUsn -ge $JournalData.NextUsn
If ($NoMoreData) {
Write-Verbose 'Checking for more data'
While ($NextUsn -ge $JournalData.NextUsn) {
Start-Sleep -Milliseconds 500
$JournalData = GetUsnJournal -VolumeHandle $VolumeHandle
}
}
While ($Choice -notmatch 'c|q') {
$Choice = Read-Host "Page: $($Page) - Press C to display next page or Q to Quit"
}
If ($Choice -eq 'c') {
Write-Verbose "Using next Starting USN: $($NextUSN)"
[System.Runtime.InteropServices.Marshal]::WriteInt64($ReadBuffer, $NextUSN)
Remove-Variable Choice -ErrorAction SilentlyContinue
} Else {
Write-Verbose "Halting operation"
$ReadMore = $False
[System.Runtime.InteropServices.Marshal]::FreeHGlobal($ReadBuffer)
[System.Runtime.InteropServices.Marshal]::FreeHGlobal($DataBuffer)
[void][PoshChJournal]::CloseHandle($VolumeHandle)
}
$Choice = $Null
}
}
I won’t lie. There is a lot of stuff happening here and definitely not a lot of time to cover everything in great detail. Fortunately, the script that I have available to download in the Script Center Repository can show present the data to you. If you want an even easier approach (because working with PInvoke and reflection isn’t for everyone), you are in luck. I have a module available to use, and I will explain that tomorrow!
We invite you to follow the Scripting Guys on Twitter and Facebook. If you have any questions, send email to the Scripting Guy at scripter@microsoft.com, or post your questions on the Official Scripting Guys Forum. Until then, see ya!
Boe Prox, Windows PowerShell MVP and Honorary Scripting Guy