lørdag den 9. maj 2015

Project Server 2010 -> 2013 migration - Large databases -> Slow Convertto-SPProjectDatabase

The last couple of days i have tried to migrate a Project Server 2010 to Project Server 2013. The Project Server 2010 contained some quite large databases compared to what I normally encounter.
The shrinked database sizes was:
 - Archive:      75 GB
 - Draft:        40 GB
 - Published:    50 GB
 - Reporting:    20 GB
 - Content:       2 GB
As you properly already know the migration process contains the following steps:

 Mount-SPContentDatabase –Name ProjectServer_PWA –WebApplication http://projectserver/ -NoB2BSiteUpgrade  
 $wa = get-SPWebApplication http://projectserver  
 $wa.UseClaimsAuthentication = $true  
 $wa.Update()  
 (Get-SPWebApplication http://projectserver).migrateUsers($true)  
 Set-SPSite –Identity http://projectserver/TECHPWA -SecondaryOwnerAlias "CONTOSO\Administrator"  
 Upgrade-SPSite -Identity http://projectserver/TECHPWA -versionupgrade  
 Convertto-SPProjectDatabase -WebApplication http://projectserver -Dbserver projectserver -ArchiveDbname TechPWA_Archive -DraftDbname TechPWA_Draft –PublishedDbname TechPWA_Published –ReportingDbname TechPWA_Reporting –ProjectServiceDbname TechPWA_ProjectWebApp  
 Mount-SPProjectDatabase –Name "ProjectServer_PWA_ProjectWebApp" –WebApplication http://projectserver  
 Upgrade-SPProjectDatabase -Name "ProjectServer_PWA_ProjectWebApp" -WebApplication http://projectserver  
 Mount-SPProjectWebInstance -DatabaseName "ProjectServer_PWA_ProjectWebApp" -SiteCollection " http://projectserver/TechPWA" -DatabaseServer "projectserver"  
 Upgrade-SPProjectWebInstance -Identity "http://projectserver/TechPWA"  
 Enable-SPFeature -Identity TechPWAsite -URL "http://projectserver/TechPWA"  
 Test-SPProjectWebInstance –Identity http://projectserver/TechPWA  

At the step where I consolidate the four Project Server 2010 databases (Convertto-SPProjectDatabase) I started to encounter some problems.

Firstly the new ProjectWebApp database started at the size of 5 MB and slowly increased with 5 MB every 30 sec until it reached 40 MB.
Now the size jumped to 54.000 MB and everything stalled. The database size was static, the memory did not increase, the CPU was idle and the Powershell just keep hanging. for this reason i tried to restart the Powershell command several times also on different servers, always the same result.
After around 1½ day the Convertto-SPProjectDatabase finally finished with an error:

 Convertto-SPProjectDatabase : INIT METADATA ERROR!  
 At least one mismatch found between SQL schema for view  
 MSP_EpmProject_UserView and reporting metadata. Schema column: Project Number.  
 Metadata column: NULL  
 Call to MSP_Epm_CheckIntrinsicAttributes failed for Projects  
 INIT METADATA ERROR.  
 At line:1 char:1  
 + Convertto-SPProjectDatabase -WebApplication http://projectserver -Dbserver  
 ppm2013 ...  
 + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~  
 ~~~  
   + CategoryInfo     : InvalidOperation: (Microsoft.Offic...baseImpleme  
   ntor:ConvertToProjec...baseImplementor) [ConvertTo-SPProjectDatabase], Sql  
  Exception  
   + FullyQualifiedErrorId : Microsoft.Office.Project.Server.Cmdlet.PSCmdletC  
   onvertToProjectServiceDatabase  

I do not know the reason for the error (somthing with the schema for a custom field column not being correct). However, simply modify the stored procedure "MSP_Epm_CheckIntrinsicAttributes" to never return an error solved the problem.
So if you encounter this problem, simply execute the modified version of the Stored Procedure below:

 USE [ProjectServer_ProjectWebApp]/[ProjectServer_Reporting]
 GO  
 /****** Object: StoredProcedure [dbo].[MSP_Epm_CheckIntrinsicAttributes]  Script Date: 5/9/2015 4:02:14 PM ******/  
 SET ANSI_NULLS ON  
 GO  
 SET QUOTED_IDENTIFIER ON  
 GO  
 ALTER PROCEDURE [dbo].[MSP_Epm_CheckIntrinsicAttributes]  
   @EntityTypeUID       [uniqueidentifier],   
   @AttributeIsTimephased   [FLAG]          
 AS  
 BEGIN  
   DECLARE @TableName     [NAME]  
   DECLARE @UserViewName   [NAME]  
   DECLARE @OlapViewName   [NAME]  
   DECLARE @ErrorMessage   nvarchar(1000)  
   DECLARE @SchemaColumn   [NAME]  
   DECLARE @MetadataColumn  [NAME]  
   IF @AttributeIsTimephased = 0  
   BEGIN  
    SELECT @TableName = EntityNonTimephasedTableName  
    FROM MSP_EpmMetadataEntities  
    WHERE MSP_EpmMetadataEntities.EntityTypeUID = @EntityTypeUID  
    SET @UserViewName = @TableName + '_UserView'  
    SET @OlapViewName = @TableName + '_OlapView_00000000-0000-0000-0000-000000000000'  
    SELECT TOP 1  
      @SchemaColumn = SchemaInfo.ColumnName,  
      @MetadataColumn = MetadataAttribute.ColumnName  
    FROM  
    (  
      SELECT ColumnName = syscolumns.name  
      FROM dbo.syscolumns  
      INNER JOIN dbo.sysobjects ON  
       syscolumns.id = sysobjects.id  
      WHERE sysobjects.name = @UserViewName  
    ) AS SchemaInfo  
    FULL OUTER JOIN  
    (  
      SELECT ColumnName = MSP_EpmMetadataAttribute.AttributeColumnName  
      FROM MSP_EpmMetadataAttribute  
      WHERE  
       MSP_EpmMetadataAttribute.EntityTypeUID = @EntityTypeUID  
       AND MSP_EpmMetadataAttribute.AttributeIsTimephased = 0  
       AND dbo.MFN_Epm_ExtendedAttributeIsExposedInView(MSP_EpmMetadataAttribute.AttributeTypeUID, 1) = 1  
    ) AS MetadataAttribute ON  
      SchemaInfo.ColumnName = MetadataAttribute.ColumnName  
    WHERE  
      SchemaInfo.ColumnName IS NULL  
      OR MetadataAttribute.ColumnName IS NULL  
    IF (@SchemaColumn IS NOT NULL) OR (@MetadataColumn IS NOT NULL)  
    BEGIN  
      SET @ErrorMessage = 'At least one mismatch found between SQL schema for view ' + @UserViewName + ' and reporting metadata. Schema column: ' + ISNULL(@SchemaColumn, 'NULL') + '. Metadata column: ' + ISNULL(@MetadataColumn, 'NULL')  
      GOTO LblError  
    END  
    SELECT TOP 1  
      @SchemaColumn = SchemaInfo.ColumnName,  
      @MetadataColumn = MetadataAttribute.ColumnName  
    FROM  
    (  
      SELECT ColumnName = syscolumns.name  
      FROM dbo.syscolumns  
      INNER JOIN dbo.sysobjects ON  
       syscolumns.id = sysobjects.id  
      WHERE sysobjects.name = @OlapViewName  
    ) AS SchemaInfo  
    FULL OUTER JOIN  
    (  
      SELECT ColumnName = MSP_EpmMetadataAttribute.AttributeColumnName  
      FROM MSP_EpmMetadataAttribute  
      WHERE  
       MSP_EpmMetadataAttribute.EntityTypeUID = @EntityTypeUID  
       AND MSP_EpmMetadataAttribute.AttributeIsTimephased = 0  
       AND dbo.MFN_Epm_ExtendedAttributeIsExposedInView(MSP_EpmMetadataAttribute.AttributeTypeUID, 0) = 1  
    ) AS MetadataAttribute ON  
      SchemaInfo.ColumnName = MetadataAttribute.ColumnName  
    WHERE  
      SchemaInfo.ColumnName IS NULL  
      OR MetadataAttribute.ColumnName IS NULL  
    IF (@SchemaColumn IS NOT NULL) OR (@MetadataColumn IS NOT NULL)  
    BEGIN  
      SET @ErrorMessage = 'At least one mismatch found between SQL schema for view ' + QUOTENAME(@OlapViewName) + ' and reporting metadata. Schema column: ' + ISNULL(@SchemaColumn, 'NULL') + '. Metadata column: ' + ISNULL(@MetadataColumn, 'NULL')  
      GOTO LblError  
    END  
    SELECT TOP 1  
      @SchemaColumn = SchemaInfo.ColumnName,  
      @MetadataColumn = MetadataAttribute.ColumnName  
    FROM  
    (  
      SELECT ColumnName = syscolumns.name  
      FROM dbo.syscolumns  
      INNER JOIN dbo.sysobjects ON  
       syscolumns.id = sysobjects.id  
      WHERE sysobjects.name = @TableName  
    ) AS SchemaInfo  
    FULL OUTER JOIN  
    (  
      SELECT ColumnName = MSP_EpmMetadataAttribute.AttributeColumnName  
      FROM MSP_EpmMetadataAttribute  
      WHERE  
       MSP_EpmMetadataAttribute.EntityTypeUID = @EntityTypeUID  
       AND MSP_EpmMetadataAttribute.AttributeIsTimephased = 0  
       AND MSP_EpmMetadataAttribute.PresentInTable = 1  
    ) AS MetadataAttribute ON  
      SchemaInfo.ColumnName = MetadataAttribute.ColumnName  
    WHERE  
      SchemaInfo.ColumnName IS NULL  
      OR MetadataAttribute.ColumnName IS NULL  
    IF (@SchemaColumn IS NOT NULL) OR (@MetadataColumn IS NOT NULL)  
    BEGIN  
      SET @ErrorMessage = 'At least one mismatch found between SQL schema and reporting metadata. Schema column: ' + ISNULL(@SchemaColumn, 'NULL') + '. Metadata column: ' + ISNULL(@MetadataColumn, 'NULL')  
      GOTO LblError  
    END  
   END  
   ELSE  
   BEGIN  
    SELECT @TableName = EntityTimephasedTableName  
    FROM MSP_EpmMetadataEntities  
    WHERE MSP_EpmMetadataEntities.EntityTypeUID = @EntityTypeUID  
    SET @UserViewName = @TableName + '_UserView'  
    SET @OlapViewName = @TableName + '_OlapView_00000000-0000-0000-0000-000000000000'  
    SELECT TOP 1  
      @SchemaColumn = SchemaInfo.ColumnName,  
      @MetadataColumn = MetadataAttribute.ColumnName  
    FROM  
    (  
      SELECT ColumnName = syscolumns.name  
      FROM dbo.syscolumns  
      INNER JOIN dbo.sysobjects ON  
       syscolumns.id = sysobjects.id  
      WHERE sysobjects.name = @UserViewName  
    ) AS SchemaInfo  
    FULL OUTER JOIN  
    (  
      SELECT ColumnName = MSP_EpmMetadataAttribute.AttributeColumnName  
      FROM MSP_EpmMetadataAttribute  
      WHERE  
       MSP_EpmMetadataAttribute.EntityTypeUID = @EntityTypeUID  
       AND MSP_EpmMetadataAttribute.AttributeIsTimephased = 1  
       AND dbo.MFN_Epm_ExtendedAttributeIsExposedInView(MSP_EpmMetadataAttribute.AttributeTypeUID, 1) = 1  
    ) AS MetadataAttribute ON  
      SchemaInfo.ColumnName = MetadataAttribute.ColumnName  
    WHERE  
      SchemaInfo.ColumnName IS NULL  
      OR MetadataAttribute.ColumnName IS NULL  
    IF (@SchemaColumn IS NOT NULL) OR (@MetadataColumn IS NOT NULL)  
    BEGIN  
      SET @ErrorMessage = 'At least one mismatch found between SQL schema for view ' + @UserViewName + ' and reporting metadata. Schema column: ' + ISNULL(@SchemaColumn, 'NULL') + '. Metadata column: ' + ISNULL(@MetadataColumn, 'NULL')  
      GOTO LblError  
    END  
    SELECT TOP 1  
      @SchemaColumn = SchemaInfo.ColumnName,  
      @MetadataColumn = MetadataAttribute.ColumnName  
    FROM  
    (  
      SELECT ColumnName = syscolumns.name  
      FROM dbo.syscolumns  
      INNER JOIN dbo.sysobjects ON  
       syscolumns.id = sysobjects.id  
      WHERE sysobjects.name = @OlapViewName  
    ) AS SchemaInfo  
    FULL OUTER JOIN  
    (  
      SELECT ColumnName = MSP_EpmMetadataAttribute.AttributeColumnName  
      FROM MSP_EpmMetadataAttribute  
      WHERE  
       MSP_EpmMetadataAttribute.EntityTypeUID = @EntityTypeUID  
       AND MSP_EpmMetadataAttribute.AttributeIsTimephased = 1  
       AND dbo.MFN_Epm_ExtendedAttributeIsExposedInView(MSP_EpmMetadataAttribute.AttributeTypeUID, 0) = 1  
    ) AS MetadataAttribute ON  
      SchemaInfo.ColumnName = MetadataAttribute.ColumnName  
    WHERE  
      SchemaInfo.ColumnName IS NULL  
      OR MetadataAttribute.ColumnName IS NULL  
    IF (@SchemaColumn IS NOT NULL) OR (@MetadataColumn IS NOT NULL)  
    BEGIN  
      SET @ErrorMessage = 'At least one mismatch found between SQL schema for view ' + QUOTENAME(@OlapViewName) + ' and reporting metadata. Schema column: ' + ISNULL(@SchemaColumn, 'NULL') + '. Metadata column: ' + ISNULL(@MetadataColumn, 'NULL')  
      GOTO LblError  
    END  
    SELECT TOP 1  
      @SchemaColumn = SchemaInfo.ColumnName,  
      @MetadataColumn = MetadataAttribute.ColumnName  
    FROM  
    (  
      SELECT ColumnName = syscolumns.name  
      FROM dbo.syscolumns  
      INNER JOIN dbo.sysobjects ON  
       syscolumns.id = sysobjects.id  
      WHERE sysobjects.name = @TableName  
    ) AS SchemaInfo  
    FULL OUTER JOIN  
    (  
      SELECT ColumnName = MSP_EpmMetadataAttribute.AttributeColumnName  
      FROM MSP_EpmMetadataAttribute  
      WHERE  
       MSP_EpmMetadataAttribute.EntityTypeUID = @EntityTypeUID  
       AND MSP_EpmMetadataAttribute.AttributeIsTimephased = 1  
       AND MSP_EpmMetadataAttribute.PresentInTable = 1  
    ) AS MetadataAttribute ON  
      SchemaInfo.ColumnName = MetadataAttribute.ColumnName  
    WHERE  
      SchemaInfo.ColumnName IS NULL  
      OR MetadataAttribute.ColumnName IS NULL  
    IF (@SchemaColumn IS NOT NULL) OR (@MetadataColumn IS NOT NULL)  
    BEGIN  
      SET @ErrorMessage = 'At least one mismatch found between SQL schema and reporting metadata. Schema column: ' + ISNULL(@SchemaColumn, 'NULL') + '. Metadata column: ' + ISNULL(@MetadataColumn, 'NULL')  
      GOTO LblError  
    END  
   END  
   RETURN 0  
 LblError:  
   --PRINT @ErrorMessage  
   RETURN 0  
 END  

This stored procedure exists both in the 2010 reporting database and in the ProjectWebApp database. I modified it in both databases, but I guess it is the one in ProjectWebApp that is coursing problems.

2 kommentarer:

  1. Great tutorial, Loved this blog. Thanks aand keep sharing this kind of stuffs here. A2hosting Review

    SvarSlet
  2. Hi, thanks for your blog, i'm faced with this problem now and cant understand you write "This stored procedure exists both in the 2010 reporting database and in the ProjectWebApp database. I modified it in both databases, but I guess it is the one in ProjectWebApp that is coursing problems" But there are 4 bases draft, reporting, archive and published. Which one of them do you call ProjectWebApp in 2010?

    SvarSlet