Tuesday, September 15, 2015

Powershell script to Zip, Archive and Move files

I have written a very handy Powershell script that helps Zip, Archive and Move files across Fileshares in your network.

This script is pretty useful in scenarios where you want to automate compression, zip and archival of these files into a separate fileshare location for back-up purposes. This script recursively identifies file-types to archive, zips, compresses and moves them to the specified file location. Zip operation reduces the actual file size by around 90%

Powershell script: ArchiveFiles.ps1

=======================================================================

param
(
    [string]$sourceDir,
    [string]$destinationArchiveDir,
    [string]$retentionDays,
    [string]$fileExtension,
    [int]$waitTime
)

[void][System.Reflection.Assembly]::LoadWithPartialName("System")
[void][System.Reflection.Assembly]::LoadWithPartialName("System.IO")
[void][System.Reflection.Assembly]::LoadWithPartialName("System.IO.Compression")
[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer")

    ##Script debugging parameters
    #$sourceDir = "\\SharepointFix\Archive Fileshare\Source\"
    #$destinationArchiveDir = "\\SharepointFix\Archive Fileshare\Destination\"
    #$retentionDays = "7"
    #$fileExtension = "*.log"
    #$waitTime = 200000
 
## Check Input parameters ##
if([string]::IsNullOrEmpty($sourceDir) -or [string]::IsNullOrEmpty($destinationArchiveDir) -or [string]::IsNullOrEmpty($retentionDays) -or [string]::IsNullOrEmpty($fileExtension) -or [string]::IsNullOrEmpty($waitTime))
{
    Write-Warning "Input parameters are not specified. Ensure input paramters are defined in the .BAT file"
 
    exit;
}

# Section 1: Gets Script Directory

Function Get-Script-Directory
{
    $scriptInvocation = (Get-Variable MyInvocation -Scope 1).Value
    return Split-Path $scriptInvocation.MyCommand.Path
}

# Section 2: Functions to create a zip. Also checks and creates zip if it does not exist

Function Move-Zip($zipfilename)
{
    try
    {
        if(-not (Test-Path($zipfilename)))
    {
    Set-Content $zipfilename ("PK" + [char]5 + [char]6 + ("$([char]0)" * 18))
    (dir $zipfilename).IsReadOnly = $false
    }

    $shellApplication = New-Object -com shell.application
    $zipPackage = $shellApplication.NameSpace($zipfilename)
     

    foreach($file in $input)
    {
    $zipPackage.MoveHere($file.FullName)
    Start-Sleep -milliseconds $waitTime
    }
     }
     catch
     {
         Write-Warning ("Exception thrown at: " + $_)
         #Write Exception to Log file
         Write-Output "Exception thrown at: " $_ | Out-File $logname -append
     }
}

#Log File name
$logName = (Get-Script-Directory) + "\ArchiveFiles-" + $(Get-Date -Format MM-dd-yyyy-HH-mm) + ".log"
$year_month_date = Get-Date -Format yyyyMMdd

## Parameters ##
Write-Host "Fileshare Source: " $sourceDir
Write-Host "Destination Archive: " $destinationArchiveDir
Write-Host "Retention Period (days): " $retentionDays

 try
   {
        ## Check whether Source File exists at Directory location
        if (Test-Path -path $sourceDir)
        {
            $LastWrite = (Get-Date).AddDays(-$retentionDays)
         
            $files = dir $sourceDir -Include $fileExtension -recurse | Where-Object {$_.LastWriteTime -le $LastWrite -and !($_.PSIsContainer)} | ForEach-Object {
                    $zipfilename = $destinationArchiveDir + $_.Name + ".zip"
                    $_ | Move-Zip($zipfilename)
                }
        }
        else
        {
            Write-Warning ("Source Path not found: " + $sourceDir)
        }
   }
   catch
   {
         Write-Warning ("Exception thrown at: " + $_)
         #Write Exception to Log file
         Write-Output "Exception thrown at: " $_ | Out-File $logname -append
   }
         
Write-Host "Script execution completed"

=======================================================================

You can automate the usage of the above script by calling it from BATCH file. Modify your script parameters in ArchiveFiles.bat

@ECHO OFF
cd /d %~dp0
PowerShell -NoProfile -ExecutionPolicy Bypass -Command "& '.\ArchiveFiles.ps1' '\\Server1\Archive Fileshare\Source\' '\\Server2\Archive Fileshare\Destination\' '7' '*.log' 50000"

Script Usage Parameters:

·         Script File: '.\ArchiveFiles.ps1'

·         Source Location: '\\Server1\Archive Fileshare\Source\'  - These are the files you wish to Archive. You can specify your own file share location.

·         Archival Location: '\\Server2\Archive Fileshare\Destination\'  - This location is where the files will be finally zipped, moved and archived. Specify your own archival location.

·         File Retention (days): '7'  - This param means that all Files except for last 7 days will be archived. You can specify your own value here. 0 means all files will be archived.

·         File Extension '*.log'  - This parameter means that only file extensions with .log will be archived, you can specify .log, .txt etc. To archive all types of files you could just specify *

·         Zip Operation Wait Time - 5000  (in milliseconds)- Zip operation generally takes ½ second to couple of minutes depending on file that needs to be compressed, zipped and moved over to the archival drive. As an example a 15 GB file it takes around 8 mins to zip and move.

Finally all above parameters can be configured in the ArchiveFiles.bat. This .bat file can be scheduled as a Windows Task as well.  If we encounter permissions/script related errors, scripts generates .log file for exceptions.

Hope this powershell script addresses your archival needs. Feel free to comment in case of any questions.


2 comments:

  1. Hi Mehuil

    Nice post
    Is it possible to archive the day wise logs instead individual?
    And it took almost 4 and half hours to archive 900 MB data. Can we minimize the time?

    Many Thanks
    Niyaz

    ReplyDelete
  2. Every file is zipped in single folder i need all files zipped in single folder

    ReplyDelete