Tuesday, September 15, 2015

Powershell script to Zip, Archive and Move files

I have written a very handy Powershell script that helps Zip, Archive and Move files across Fileshares in your network.

This script is pretty useful in scenarios where you want to automate compression, zip and archival of these files into a separate fileshare location for back-up purposes. This script recursively identifies file-types to archive, zips, compresses and moves them to the specified file location. Zip operation reduces the actual file size by around 90%

Powershell script: ArchiveFiles.ps1




    ##Script debugging parameters
    #$sourceDir = "\\SharepointFix\Archive Fileshare\Source\"
    #$destinationArchiveDir = "\\SharepointFix\Archive Fileshare\Destination\"
    #$retentionDays = "7"
    #$fileExtension = "*.log"
    #$waitTime = 200000
## Check Input parameters ##
if([string]::IsNullOrEmpty($sourceDir) -or [string]::IsNullOrEmpty($destinationArchiveDir) -or [string]::IsNullOrEmpty($retentionDays) -or [string]::IsNullOrEmpty($fileExtension) -or [string]::IsNullOrEmpty($waitTime))
    Write-Warning "Input parameters are not specified. Ensure input paramters are defined in the .BAT file"

# Section 1: Gets Script Directory

Function Get-Script-Directory
    $scriptInvocation = (Get-Variable MyInvocation -Scope 1).Value
    return Split-Path $scriptInvocation.MyCommand.Path

# Section 2: Functions to create a zip. Also checks and creates zip if it does not exist

Function Move-Zip($zipfilename)
        if(-not (Test-Path($zipfilename)))
    Set-Content $zipfilename ("PK" + [char]5 + [char]6 + ("$([char]0)" * 18))
    (dir $zipfilename).IsReadOnly = $false

    $shellApplication = New-Object -com shell.application
    $zipPackage = $shellApplication.NameSpace($zipfilename)

    foreach($file in $input)
    Start-Sleep -milliseconds $waitTime
         Write-Warning ("Exception thrown at: " + $_)
         #Write Exception to Log file
         Write-Output "Exception thrown at: " $_ | Out-File $logname -append

#Log File name
$logName = (Get-Script-Directory) + "\ArchiveFiles-" + $(Get-Date -Format MM-dd-yyyy-HH-mm) + ".log"
$year_month_date = Get-Date -Format yyyyMMdd

## Parameters ##
Write-Host "Fileshare Source: " $sourceDir
Write-Host "Destination Archive: " $destinationArchiveDir
Write-Host "Retention Period (days): " $retentionDays

        ## Check whether Source File exists at Directory location
        if (Test-Path -path $sourceDir)
            $LastWrite = (Get-Date).AddDays(-$retentionDays)
            $files = dir $sourceDir -Include $fileExtension -recurse | Where-Object {$_.LastWriteTime -le $LastWrite -and !($_.PSIsContainer)} | ForEach-Object {
                    $zipfilename = $destinationArchiveDir + $_.Name + ".zip"
                    $_ | Move-Zip($zipfilename)
            Write-Warning ("Source Path not found: " + $sourceDir)
         Write-Warning ("Exception thrown at: " + $_)
         #Write Exception to Log file
         Write-Output "Exception thrown at: " $_ | Out-File $logname -append
Write-Host "Script execution completed"


You can automate the usage of the above script by calling it from BATCH file. Modify your script parameters in ArchiveFiles.bat

cd /d %~dp0
PowerShell -NoProfile -ExecutionPolicy Bypass -Command "& '.\ArchiveFiles.ps1' '\\Server1\Archive Fileshare\Source\' '\\Server2\Archive Fileshare\Destination\' '7' '*.log' 50000"

Script Usage Parameters:

·         Script File: '.\ArchiveFiles.ps1'

·         Source Location: '\\Server1\Archive Fileshare\Source\'  - These are the files you wish to Archive. You can specify your own file share location.

·         Archival Location: '\\Server2\Archive Fileshare\Destination\'  - This location is where the files will be finally zipped, moved and archived. Specify your own archival location.

·         File Retention (days): '7'  - This param means that all Files except for last 7 days will be archived. You can specify your own value here. 0 means all files will be archived.

·         File Extension '*.log'  - This parameter means that only file extensions with .log will be archived, you can specify .log, .txt etc. To archive all types of files you could just specify *

·         Zip Operation Wait Time - 5000  (in milliseconds)- Zip operation generally takes ½ second to couple of minutes depending on file that needs to be compressed, zipped and moved over to the archival drive. As an example a 15 GB file it takes around 8 mins to zip and move.

Finally all above parameters can be configured in the ArchiveFiles.bat. This .bat file can be scheduled as a Windows Task as well.  If we encounter permissions/script related errors, scripts generates .log file for exceptions.

Hope this powershell script addresses your archival needs. Feel free to comment in case of any questions.

Tuesday, February 24, 2015

SharePoint Pre-Upgrade/Migration Checklist

Here are excerpts published by Metalogix. It covers a lot of Pre-upgrade/migration scenarios that could help track down or prevent upgrade/migration issues. I have personally applied a lot of these in my projects in the past, especially environment clean-up activity, where we would ideally delete Orphan sites/site collections, unused sites, unused groups/user accounts, track down large site collections, break them down to smaller site collections, archive old records and apply database level maintenance plans.

Here are the key excerpts from the White Paper:

No migration to SharePoint 2013 should be taken lightly. SharePoint content has grown at a remarkable rate of 75% annually. With that growth, your company’s users demand 100 percent up time and expect few or zero lapses in their ability to access business-critical sites. The IT department, meanwhile, must ensure that only the right people can access certain content while overseeing a smooth, error-free transition that will minimally impact their workload.

Its a simple message: “Clean up your current SharePoint environment today and planning out your future migration to SharePoint 2013 will save you time, money and headaches. The complexity lies, however, in determining what to cleanup and how to plan for a successful migration.

Cleanup offers two major gains for SharePoint 2013 migration. First, it will increase the performance of your current environment by removing resource-intensive tasks. Second, it will expedite the overall time required to move from your current SharePoint environment to SharePoint 2013.
No SharePoint environment would be complete without users. During the pre-migration analysis, identifying inactive and active users and groups not only helps you understand their post-migration expectations, it also gives administrators an opportunity to ask why these users are not using SharePoint for their collaboration needs. The result is less overhead and improved communication with your SharePoint users as you migrate to SharePoint 2013. The following sections explain how to identify these users

Including your influential users in the planning and implementation stages of your migration can prove beneficial to the success of the migration. Users can provide both “on-the-ground” feedback during the migration and communicate their SharePoint 2013 requirements and expectations.

Start by identifying the most active SharePoint users and sites in your environment by writing Powershell scripts. Communicate with these users throughout the process and help get early “buy-in” so that they will in turn communicate any changes or new features to their team. Some of the examples of Powershell scripts that SharePoint Admin/developer could write are Site Risk Assessment to gauge largest sites/site collections, content DB size, orphan sites, unused sites, large lists, complex workflows, orhpaned users/groups etc.

One often overlooked area of the cleanup process is removing orphan users. These are employees who are no longer with the company and were removed from Active Directory but still have a SharePoint user account and associated permissions for sites or content.Orphan users may also have active SharePoint alerts or My Sites. While orphan users can no longer access the content, if they’ve created alerts, My Sites, and other mechanisms in your environment, those can impact SharePoint’s hardware resources. Cleaning out these users before your migration will help teams gain back resources and better predict SharePoint user growth.

When new site collections are created, often the default SharePoint Groups (owners, contributors, etc.) are also created (intentionally or not). Frequently, only a few of those groups are used.  This information is valuable as you plan for your migration and can help reduce the number of Client Access Licenses (CALs) needed and reduce your user footprint. It’s also an opportunity to reach out to inactive users who are still with the organization to ask why they’re not using SharePoint.

Unused sites that contain almost no content or have very limited activity are excellent candidates for archiving as you prepare the migration process. You can find under-utilized sites within any scope (Site Collection, Web Application or even the whole farm) and filter the report to display certain site definitions.

Site Collections (or sites) often outgrow their original size. This suggests that users are actively engaged in SharePoint. It also points to a potential resource issue. Very large site collections can impact performance which in turn will likely hinder your migration plans.

Typically, it makes sense to split the Site Collection during the migration process. Or you may want to split the content so that some of your users can migrate to SharePoint 2013 and start using it, while other groups use SharePoint 2010.

As part of a migration plan, you might decide to reorganize your sites and promote some of them to site collections based on their size. For example, all sites above 100 GB might be candidates for promotion to Site Collection.

Another opportunity for cleanup is to identify small site collections and demote them to subsites. Often user requests for sites are satisfied by creating site collections. Over time, it might become apparent that those site collections could or should have been created as subsites of a different site collection. Another outcome of this analysis might be that you choose to just delete those site collections since they have very little use.

Microsoft’s best practices for numbers of items in a list or folder are often exceeded due to the unanticipated growth of SharePoint. Using the Most/Least Storage reports generated out of PowerShell, you can identify large lists with excessive number of items. During the migration you are then in a position to split those lists.
You can also use the List Property report to analyze the columns to determine whether they are approaching the Microsoft recommended list view thresholds (2,000 or more items with 20 or more columns).

With Powershell, you can generate the List and Library Storage report includes last accessed and/or last modified metadata. With that knowledge, you can choose to archive some of the content as part of the migration effort and/or the migration can be configured to skip older content that is no longer needed.
Another effective way to maintain control over content growth is to implement policies around versioning. Using Powershell scripts, you can gain insight into the storage impact of the document versions. The more versions an item has, the slower the migration of the item will be. With Powershell, you can use Set List Properties to configure versioning across any scope in the farm. The script can then configure SharePoint to automatically prune the versions from the content.

It’s not uncommon for SharePoint to be the repository for the same document in multiple places. To improve the migration performance and reduce user confusion, write a script to remove Duplicate Files

Powershell scripts can be written to move the Audit Log data from source content databases and stored in a standalone SQL database. This provides access to audit information even after you’ve moved to the new SharePoint version. Once the data is in the archive, you can use SQL Server Reporting Services or other tools can also be used to create audit reports after migration. This ensures a continuous audit trail even after the farm is migrated.

Web parts are used everywhere your SharePoint environment probably includes Microsoft web parts, custom web parts, and third party web parts.

As you plan your migration, it’s important to get an accurate assessment of the impact of any custom web parts used in your environment. This step will help you determine whether to update a custom web part so it works in 2013 or deprecate the component as its functionality is no longer needed or has been replaced by a new capability in SharePoint 2013.

Web part usage analysis will also assist in determining if support/maintenance on third party web parts should be continued or not. We can write Powershell scripts to report out Webparts on the page.

Web Part analysis helps you can gain global insight into which web parts are being used and which sites they are being used on. The report can even filter for the Closed or Hidden web parts.

The Web Part report can be run by site administrators or distributed to them so that they can understand which components will be migrated and which will not. This will set end user expectations and prevent miscommunication during migrations.

The Workflow repotrs could be generated via Powershell to find out details about workflow definitions, instances and their state. This information is particularly important for migration since it displays the details for all the in-progress workflows that cannot be migrated.

Content types are a powerful tool in SharePoint that enable organizations to organize, manage, and handle content in a consistent way across site collections. Gaining insight into the custom content types and where they are being used will help determine how the data will be managed once you migrate. Investigating the use of Content types will also help to highlight opportunities for creating enterprise content types that will provide broader scope.

Content Type report can be generated and filtered to show all content types or ones that are defined but not used. This information can be used to clean up the content types prior to the migration.

Using Powershell scripting reports, we can report out SharePoint sites/subsites that have broken inheritance and have unique set of permissions. This gives more insight on the complexity of permissions structure and also determines whether these sites should have ideally broken permission structure or not.

Alternatively you could use Metalogix tools known as ControlPoint and Content Matrix to report out all the above Pre-migration checklist parameters.