28 May 2014

Cube Partitioning

Each measure group in a cube is divided into partitions, where a partition defines a portion of the fact data that is loaded into a measure group. SQL Server Analysis Services (SSAS) on SQL Server 2008 R2 Standard Edition allows only one partition per measure group, while multiple partitions are allowed in the Enterprise Edition. 

Partitions are completely transparent to the end user, but they have an important impact on performance and scalability. For example, partitions can be processed separately and in parallel. They can have different aggregation designs, although this was not implemented in System Center 2012 – Service Manager.

You can reprocess a partition without affecting all the other partitions in a measure group. Also, SSAS automatically scans only the partitions that contain the necessary data for a query, which can vastly improve query performance.

Cube partitioning is performed on every data warehouse maintenance job run, which is hourly by default. The specific process module that runs is named Manage Cube Partitions. It always runs after the Create Mart Partitions step.

This dependency data is stored in the infra.moduletriggercondition table.

The main dynamic link library (DLL), which handles partitioning, is in the warehouse utility DLL, Microsoft.EnterpriseManagement.Warehouse.Utility, in the PartitionUtil class. Specifically, there is a ManagePartitions () method in the class that handles all partition maintenance. 

The data warehouse maintenance DLL, Microsoft.EnterpriseManagement.Warehouse.Maintenance, and the data warehouse online analytical processing (OLAP) DLL, Microsoft.EnterpriseManagement.Warehouse.Olap, both call into Microsoft.EnterpriseManagement.Warehouse.Utility to handle partitions during maintenance and cube deployment

This is why actual partition handling is in the common warehouse utility DLL to avoid duplicating logic or code.

Cube Partitioning Maintenance performs the following tasks:
  • Create partitions
  • Delete partitions    
  • Update partition boundaries


To do this, the Structured Query Language (SQL) table etl.TablePartition is read to determine all the fact partitions that have been created for a measure group. The following actions occur:

Start cube processing for each measure group in the cube

Get all partitions from the Each measure group in a cube is divided into partitions, where a partition defines a portion of the fact data that is loaded into a measure group. 

SQL Server Analysis Services (SSAS) on SQL Server 2008 R2 Standard Edition allows only one partition per measure group, while multiple partitions are allowed in the Enterprise Edition. 
Partitions are completely transparent to the end user, but they have an important impact on performance and scalability.

For example, partitions can be processed separately and in parallel. They can have different aggregation designs, although this was not implemented in System Center 2012 – Service Manager. 

You can reprocess a partition without affecting all the other partitions in a measure group. Also, SSAS automatically scans only the partitions that contain the necessary data for a query, which can vastly improve query performance.

Cube partitioning is performed on every data warehouse maintenance job run, which is hourly by default. The specific process module that runs is named Manage Cube Partitions. It always runs after the Create Mart Partitions step. 

This dependency data is stored in the infra.moduletriggercondition table.

The main dynamic link library (DLL), which handles partitioning, is in the warehouse utility DLL, Microsoft.EnterpriseManagement.Warehouse.Utility, in the PartitionUtil class. Specifically, there is a ManagePartitions () method in the class that handles all partition maintenance.

The data warehouse maintenance DLL, Microsoft.EnterpriseManagement.Warehouse.Maintenance, and the data warehouse online analytical processing (OLAP) DLL, Microsoft.EnterpriseManagement.Warehouse.Olap, both call into Microsoft.EnterpriseManagement.Warehouse.Utility to handle partitions during maintenance and cube deployment.

This is why actual partition handling is in the common warehouse utility DLL to avoid duplicating logic or code.

Cube Partitioning Maintenance performs the following tasks:
  • Create partitions
  • Delete partitions
  • Update partition boundaries
To do this, the Structured Query Language (SQL) table etl.TablePartition is read to determine all the fact partitions that have been created for a measure group.

The following actions occur:
  • Start cube processing for each measure group in the cube
  • Get all partitions from the etl.TablePartition table for the measure group
  • Delete any partitions that exist in the measure group but that are missing from the etl.TablePartition table
  • Add any new partitions that have been created and that exist only in the etl.TablePartition table
  • Update any partition that might have changed by matching each partition to the RangeStartDate and RangeEndDate in the etl.TablePartition table
Remember the following about cube processing:

Only measure groups that are targeted at facts contain multiple partitions in SQL Server 2008 R2 Standard Edition. By default, all measure groups and dimensions contain only one partition. Therefore, the partition does not have any boundary conditions.

The partition boundaries are defined by a query binding that is based on date keys that match up to the date keys for the corresponding fact partition in the etl.TablePartition table.

Get all partitions from the etl.TablePartition table for the measure group table for the measure group

Delete any partitions that exist in the measure group but that are missing from the etl.TablePartition table

Add any new partitions that have been created and that exist only in the etl.TablePartition table
Update any partition that might have changed by matching each partition to the RangeStartDate and RangeEndDate in the etl.TablePartition table

Remember the following about cube processing:

Only measure groups that are targeted at facts contain multiple partitions in SQL Server 2008 R2 Standard Edition. By default, all measure groups and dimensions contain only one partition. Therefore, the partition does not have any boundary conditions.

The partition boundaries are defined by a query binding that is based on date keys that match up to the date keys for the corresponding fact partition in the etl.TablePartition table.

Named Calculation -- System Center cube

You can use named calculations to define new attributes on a dimension that a custom measure can later target. This makes it possible for you to extend the dimensional schema and customize the schema to fit your exact needs. The following example is from the SystemCenterWorkItemsCube:

<NamedCalculation ID="IncidentsPastTargetResolutionTime" Target="IncidentDW!IncidentDim" ColumnType="Int">
<Calculation>(case when ( (([Status] = 'IncidentStatusEnum.Resolved' OR [Status] = 'IncidentStatusEnum.Closed') AND ResolvedDate &gt; TargetResolutionTime) OR (([Status] != 'IncidentStatusEnum.Resolved' AND [Status] != 'IncidentStatusEnum.Closed') AND GETUTCDATE() &gt; TargetResolutionTime)) then 1 else 0 end )</Calculation>
</NamedCalculation>
In this example, the Incident dimension contains data, such as the status of the incident and the target resolution time. However, there is no native measure that calculates the number of incidents that exceeded the target resolution time, although this type of data is very useful for a systems administrator. You can create this scenario using a named calculation and aggregate the data so that a custom measure can target the new attribute and then present the information to an end user.

Remember that Service Manager supports only NamedCalculation targeting dimensions. NamedCalculation cannot target facts. The following table describes named calculation attributes.

Attribute
Required
Values
Definition
ID
Yes
String
Name of the named calculation.
Target
Yes
ManagementPackDimension
The target dimension for the measure
ColumnType
Yes
(Int, Double)
The Structured Query Language (SQL) type of the column
Type
No
(Count, Sum)
The type of  the measure


Cube Extension

The primary purpose of the Cube Extension element is to make it possible for you to modify the OLAP cube after the cube has deployed onto SSAS, without having to uninstall and re install the cube. In situations in which the OLAP cube has been fully processed with years of data, recreating the cube is time consuming because all partitions have to be fully reprocessed.

The Cube Extension element can define the following elements:

Each customization that is defined in a Cube Extension element can also be defined in a SystemCenterCube object. The only customization that is not allowed is the addition of facts or measure groups and substitutions to the cube.
  • The CUBE operator adds a summary row for every combination of groups specified. It also adds a summary row to the end of the result set that summarizes the entire result set.
  • When you use the CUBE operator, you can’t use the DISTINCT keyword in any of the aggregate functions.
  • You can use the GROUPING function with the CUBE operator to determine if a summary row has a null value assigned to a given column

Action -- System Center Cube

Actions are events that you can trigger on an OLAP cube when you are accessing data in the cube. Only drill-through actions are supported by Service Manager. The following is an example of an action:

<Action ID="DrillThroughOnWICreatedByUser" MeasureGroupName="CreatedByUser" ActionType="DrillThrough">
<DrillThroughColumns CubeDimension="WorkItemCreatedByUser_UserDim">
<Property PropertyName="FirstName" />
<Property PropertyName="LastName" />
<Property PropertyName="Company" />
<Property PropertyName="Department" />
<Property PropertyName="Office" />
</DrillThroughColumns>
</Action>
The following table describes actions attributes.
Attribute
Required
Values
Definition
ID
Yes
String
Name of the drill-through action
Measure Group Name
Yes
String
Targeted measure group of the action
Action Type
Yes
(Drill Through)
Type of action. Only drill-through actions are supported by Service Manager.
Cube Dimension
Yes
String
The cube dimension that is the target of the action, which must be a slicer on the Measure Group
Property Name

Key Performance Indicator -- System Center Cube

Organizations and businesses can use key performance indicators (KPIs) to quickly estimate the health of an enterprise by measuring its progress toward a predefined goal. Each KPI has a target value and an actual value. The target value is a quantitative goal that is critical to the success of the organization.

Large amounts of data are filtered to one discrete value that can be used to monitor performance and progress towards goals and benchmarks. Some examples of KPIs are a college having a goal that 90% of their students graduate within four years or a basketball team with a goal of causing the opposing team to shoot less than 50 percent for a game. You can use a scorecard to show a group of KPIs, providing in one instantaneous snapshot the overall health of a business.

The following is an example KPI:

<KPI ID="IncidentResolutiuonKpi" >
<Caption> The ratio of incidents resolved </Caption>
<Value>IIF(([Measures].[IncidentDimCount])&gt; 0,([Measures].[IncidentsResolvedCount]/[Measures].[IncidentDimCount]),null)</Value>
<Goal>1.0</Goal>
<GreenThreshold> 0.75</GreenThreshold>
<YellowThreshold>0.5 </YellowThreshold>
<Direction>Up</Direction>
<StatusGraphic>Thermometer</StatusGraphic>
</KPI>
The following table describes KPI attributes.


Attribute
Required
Values
Definition
ID
Yes
String
Name of the KPI
Caption
Yes
String
Description of the KPI
Value
Yes
String
MDX script defining the numeric value of the KPI
Goal
Yes
String
The target value of the KPI
Green Threshold
Yes
String (between 0.1 and 1)
Any number that is above or below this threshold, depending on the direction, is marked as green in the status symbol.
Yellow Threshold
Yes
String (between 0.1 and 1)
Any number that is above or below the threshold, depending on the direction, but does not meet the green threshold is marked as yellow in the status symbol. A number that does not meet the yellow threshold is marked as red in the status symbol.
Direction
Yes
(Up, Down)
If the direction is up, any numbers above the green or yellow threshold are marked with the corresponding symbol. Similarly for down, numbers below the green or yellow thresholds are marked with the corresponding symbol.
Status Graphic
Yes
(Shapes, TrafficLight, RoadSigns, Gauge, ReversedGauge, Thermometer, Cylinder, Faces, VarianceArrow)
The symbol that will represent the KPI.