Denny Lee's Windows Live Space: Analysis Services

Great Data Mining resource

Mon, 02 Jun 2008 17:40:26 GMT

To learn more information about Data Mining, check out the Rafal Lukawiecki's TechNet Spotlight at: http://www.microsoft.com/emea/spotlight/event.aspx?id=99

Data Mining and Business Intelligence for Enterprises
In this one-day seminar, Rafal Lukawiecki aims to show IT Professionals how data mining can be used in IT infrastructure to support real business scenarios demystifying the perception that Data Mining is complex, untested or only for specialists. This has become possible since Microsoft has taken the technology to new levels making it accessible to all. In four easy-to-understand yet packed with practical information sessions you will learn about what Data Mining and Business Intelligence can do for you, how to deploy and manage it, how to use it, and how to make it available to other parts of your IT environment.

Introduction to Data Mining

Working with Data Mining

Using Data Mining with your IT Systems (Part 1)

Using Data Mining with your IT Systems (Part 2)

Enjoy!

Analysis Services Distinct Count Optimization

Thu, 17 Apr 2008 22:15:59 GMT

Finally, after about 7 months of analysis, customer validation, and just general randomization - I have finally propped up the Analysis Services Distinct Count Optimization whitepaper which is available here: http://sqlcat.com/whitepapers/archive/2008/04/17/analysis-services-distinct-count-optimization.aspx

Quick summary of this whitepaper:

Distinct count (such as unique visitor counts on a Web site) calculations provide valuable information but come with a number of performance challenges. This white paper describes tests that were performed to determine how best to optimize these calculations and includes best practices based on the test results.

Analysis Services: For Distinct Count measure NULL = 0

Fri, 02 Nov 2007 07:07:55 GMT

If you are to look at the table of values where you want to do a distinct count on the ID column

ID	ColA
NULL	blah
NULL	blah
0	blah
3	blah
1	blah
1	blah
2	blah
2	blah

Within SQL, you will get a value of 4 in which there are four distinct values, 0, 1, 2, 3 and NULL is not counted.

But within Analysis Services, if you were to put a distinct count measure on top of the ID column, you would also get a value of 4 for four distinct values, but those distinct values are 1, 2, 3, and 0. In this situation, NULL and 0 are the same value. This is a little bit more apparent if your table had the values of

ID	ColA
1	blah
2	blah
2	blah
3	blah
NULL	blah

Within SQL, you will get a value of 3 (distinct values of 1, 2, and 3) but within AS, you will get a value of 4 (distinct values of 1, 2, 3, and NULL or 0). There are various philosophical reasons why this is happening ranging from the fact that there are actually quite a few definitions for what NULL is (I really don't want to get into that) and within AS we're looking at things from a multi-dimensional point of view (i.e. from the dimension members) hence the exclusion of a NULL value implies exclusion of the rows. Regardless of the philosophical point of view on this, one should just be aware of this when working with their distinct count measures.

Oh note, thanks to John Lam for reminding me that if you turn on the NullProcessing property it will preserve the NULL value - i.e. the first table will have 5 distinct values of 0, 1, 2, 3, and NULL.

Using ByAttribute or ByTable Processing Group Property with Analysis Services 2005

Fri, 19 Oct 2007 20:30:16 GMT

For any of you whom are considering using the ByTable (vs. ByAttribute) Processing Group property, I've propped up a blog called Using ByAttribute or ByTable Processing Group Property with Analysis Services 2005 with help from Richard Tkachuk, Akshai Mirchandani, and Eric Jacobsen. Basically, while using ByTable can at times be helpful, you really need to be careful on how you use it. Read the blog for more info, eh?!

Incorrect Impersonation Information may crash or hang Analysis Services

Fri, 12 Oct 2007 16:48:12 GMT

If you are processing your AS cube using impersonation mode, if you incorrectly place the impersonation information as MYDOMAIN/UserName instead of MYDOMAIN\UserName, you may crash or hang Analysis Services. A quick cursory examination noted that with Sybase and DB2 relational data sources, AS ended up crashing. If SQL Server or Oracle is your relational data source, then the processing is exceedingly slow to the point where the server was hanging; in the end eventually failing.

Thanks to John Lam, Nicholas Birke, and Wei Zhang

Precision Considerations for Analysis Services Users Whitepaper now available

Tue, 09 Oct 2007 21:42:41 GMT

For those of you whom want to know a little more about precision considerations for Analysis Services users, please check out this recently released whitepaper at http://www.microsoft.com/downloads/details.aspx?familyid=bae8beec-9892-4ecd-a9db-292254895f9c&displaylang=en.

This white paper covers accuracy and precision considerations in SQL Server 2005 Analysis Services. For example, it is possible to query Analysis Services with similar queries and obtain two different answers. While this appears to be a bug, it actually is due to the fact that Analysis Services caches query results and the imprecision that is associated with approximate data types. This white paper discusses how these issues manifest themselves, why they occur, and best practices to minimize their effect.

Authors: Denny Lee and Eric Jacobsen

Security Issues when Profiling Multiple Analysis Services databases

Thu, 02 Aug 2007 01:19:05 GMT

Problem: A user with full permissions within a database cannot see any profiling events when the user has full permissions to two or more AS databases and is not a server admin. The user will be able to trace any query related events for the first database via Profiler but he/she cannot capture any similar events for the other databases.

Reason: This is a known limitation because the trace is normally a feature restricted to server Admins. DB admins can only see the trace output whenever it is safe to assume from a security perspective that the events produced are for a specific database only. If its not possible to guaratee events related to a single databse, then the events are not exposed to the user.

Alternative Solution: An alternative solution to this problem is to go ahead to create multiple profiler traces, one for each database. When you connect to SSAS from the Profiler, you will be able to choose to profile any of the databases that you have full admin access. In this case, you will open up multiple profilers, one for each database and now you are profiling the AS databases desired.

Thanks to contributions by John Lam, Edward Melomed!

Renaming an Analysis Services 2005 cube

Fri, 20 Jul 2007 23:36:09 GMT

There is a question on how to rename an Analysis Services 2005 cube since you cannot do that within SSMS (you can rename a database but you cannot a cube). If you want to rename a cube prior to its deployment, you can alter it through XMLA. To do this:

<Cube ...><ID>Blah</ID><Name>Foo</Name>...

As you can see from above, the <Name> node can be changed so that while the ID for this cube is still "blah", now your name can be "Foo".

But if you want to change the name of your cube after its already been deployed, there are an additional number of steps that will need to be performed. First you will need to go the Olap database folder, e.g.

C:\Program Files\Microsoft SQL Server\MSSQL.2\olap\Data\[Database Name].0.db

and then access the cube XML definition file within that folder, such as: [Cube Name].0.cub. The XML within this file will look like (with a cube name of “Blah”)

<Cube><Name>Blah</Name><ID>Blah</ID>...

So you need only to change the name of the <Name> node (e.g. now “Foo”) to:

<Cube><Name>Foo</Name><ID>Blah</ID>...

You will also need to stop/restart the AS service for it to register the new name.

SSAS 2005 Error Codes and Messages

Wed, 18 Jul 2007 21:17:57 GMT

Thanks to Edward Melomed and Eric Jacobsen for these quick tips!

In order to view the full list of error codes and messages for SQL Server Analysis Services 2005, you can open up Visual Studio and look at the resources within the msmdsrv.rll file. A quick and handy way to view the RLL file is to use Mark Russinovich's Strings utility from Sysinternals - you can find it at: http://www.microsoft.com/technet/sysinternals/Miscellaneous/Strings.mspx. Note that this RLL file has the tracedefinition90.xml file embedded inside it.

Analysis Services 2005 Trace Event Definitions

Wed, 18 Jul 2007 07:50:33 GMT

In the Best Practices whitepaper Analysis Services Processing Best Practices (http://www.microsoft.com/technet/prodtechnol/sql/bestpractice/ssaspbpr.mspx), I had noted in the Troubleshooting Best Practices > Create a system-wide trace file section that you could create a Analysis Services Processing Trace via XMLA. Many thanks again to Dave Wickert for providing this example.

But I had forgot to note where one can get all of the trace event definitions from. To do this, go to the [%Olap Data Folder%]\bin\Resources\1033 folder (by default [%Olap Data Folder%] is C:\Program Files\Microsoft SQL Server\MSSQL.2\Olap) and open up the file tracedefinition90.xml. In there, you can find all of the trace event definitions.

Distinct Count Optimization for Analysis Services 2005

Fri, 13 Jul 2007 23:49:56 GMT

In reference to my blog on Distinct Count Optimization at http://denster.spaces.live.com/blog/cns!125D53A08EC75357!1156.entry, I would like to expand it slightly specifically for Analysis Services 2005. Based on customer visits and more extensive testing, I have found that following most of the advice in my original blog entry will help improve distinct counts.

But, for Analysis Services 2005, it will be better to partition where the distinct value (e.g. users) are placed in the same partition. This is different from Analysis Services 2000 where the partitioning strategy was to take that same distinct value and spread it across as many partitions as possible. For AS2k5, there was significant improvement when using this new partitioning strategy. Over the next couple of months, I'll update this blog entry when I finally get a best practices whitepaper to provide more details on this.

Scale Out Querying with Analysis Services

Sat, 16 Jun 2007 04:13:23 GMT

For more information on scale out querying architecture with Analysis Services, a new whitepaper can be found on the SQLCAT Best Practices Technet site:

Scale-Out Querying with Analysis Services
http://www.microsoft.com/technet/prodtechnol/sql/bestpractice/scoqryas.mspx
Denny Lee, Nicholas Dritsas

Analysis Services Best Practices Papers published

Tue, 12 Jun 2007 05:38:30 GMT

Check out two new Analysis Services Best Practice whitepapers:

Analysis Services Processing Best Practices

http://www.microsoft.com/technet/prodtechnol/sql/bestpractice/ssaspbpr.mspx

Analysis Services Query Performance Top 10 Best Practices

http://www.microsoft.com/technet/prodtechnol/sql/bestpractice/ssasqptb.mspx

Optimizing your distinct count calculation in Analysis Services

Sun, 19 Nov 2006 03:38:28 GMT

Optimizing your distinct count calculation in Analysis Services

Introduction

A distinct count (e.g. unique user or visitor counts) provides valuable information but it is not an easy problem to resolve. An example of this problem is your grocery "market basket" scenario where a single visitor purchased various dairy, meat, and vegetable items. While there may be eight (for example) items purchased, these items were all purchased by one person. Understanding that one person purchased many items can be very revealing in the context of business intelligence. Using a web analytics example, if you have a high activity / visitor web site that has a lot of traffic in comparison to visitors (e.g. a financial market site where a set of number of users are viewing many web pages to do market research), you have a very "sticky" web site with relatively loyal users. On the other hand, if you have a low activity / visitor web site that has a lot of users but not so much traffic in comparison (e.g. an auto site where many visitors view many different auto web site to find the best deal in cars), you have a less “sticky” web site with relatively un-loyal visitors. Both business scenarios reveal different insight about the types of visitors to your web site business allowing you to make some actionable business items.

Often distinct counts are solved via custom applications and pipelines; more prevalent advanced solutions utilize data mining techniques. Helping out the cause for many business intelligence analysts, the distinct count measure was introduced within Analysis Services 2000 (AS2k). It was based originally on the dimension structure within AS2k and provided fast access to distinct count calculations for small-to-medium size volumes of data. Much expanded in Analysis Services 2005 (AS2k5), the distinct count measure was redesigned based on the AS2k5’s hierarchy and attribute structure. This allowed the distinct count measure to be calculated on much larger volumes of data.

Saying this, typically users of Analysis Services will want to apply the distinct count measure on enterprise size amount of data (loosely defined as 20 million rows/day or greater). While Analysis Services 2005 provides much better functionality out of the box including:

· No longer having only 3Gb of memory to work with (AS2k could use a maximum of 3Gb of memory irrelevant of how much memory the server has)

· The 64-bit version of AS2k was never working that usable or workable (AS2k5 rocks!)

· The attribute/hierarchy model of AS2k5 handles the Cartesian problem (any measure needs to be calculated for every possible dimension member in combination with every other dimension member) in a very different way than a dimensional model.

there are still some limitations of the distinct count measure. This is due to the fact that:

· Distinct count measures require quite a bit of memory to store information on the property that is being counted (e.g. a user). This is eased by using 64-bit AS2k5 with its hexadecimal amounts of memory space but one only has so much memory before your server “runs out”.

· The attribute/hierarchy design of the AS2k5 allows distinct counts to be calculated by referring to the hierarchy or path of your dimension. That is, the hierarchy of a dimension (e.g. Browser > Internet Explorer > Internet Explorer 7.0 > Internet Explorer 7.0 SP1) allows a distinct count to be calculated faster by incrementing the count up the hierarchy.

· For the vast majority of the time, distinct counts are calculated at run-time vs. processing time.

The advantage of Analysis Services (and any OLAP system in general) is that it can pre-aggregate quite a bit of data. That way, when a query is sent to the OLAP cube the information is already calculated for you. Analysis Services proprietary data design also allows the engine to quickly calculate metrics that are not even pre-calculated. The problem is that with distinct counts it is not simple to pre-aggregate the various combinations of distinct counts by the many dimensions. And even if you are able to pre-aggregate various distinct counts, it is impossible to do this for all dimension members. After all, if you were to perform all of the possible cross-correlations for dimension members with each other – the result set would be far greater than the original data set. Because of these issues, distinct counts are typically calculated at run-time – i.e. at the moment the user asks the question.

Understanding these issues allows us to note ways of optimizing the OLAP distinct count. These techniques are applicable to both AS2k and AS2k5; though the terminology used here is for AS2k5. The basic run down of these distinct count optimization techniques are to:

· Create a separate measure group for each distinct count measure

· Create custom aggregations

· Create a partitioning strategy for your OLAP cube allowing yourself to “distribute” the data.

Create a separate measure group for each distinct count measure

As noted above, a separate measure group should be created for the distinct count measure containing only the distinct count measure. Create a cube to join the “distinct” measure group with all of the other measure groups. This does mean that will have multiple copies of the exact same measure group except that they have different measures within in them (especially the distinct count measures). The reason to do this is that:

· Distinct count measure is designed architecturally different from other measures like sum, count, min, or max.

· Noting this architectural difference, adding other measures with the distinct count will exponentially increase the size of the cube thus resulting in slower processing and slower query times.

Create your own custom aggregations

Creating custom aggregations can force the Analysis Services engine to pre-calculate some aggregations while not calculating others. While this can be done for any and all measures, this is rather effective for distinct count measures. This is because distinct counts are typically the most complicated at the summary level (the highest level of the hierarchy) hence creating custom aggregations at that level will allow the Analysis Services engine to calculate these at processing time (hence much faster queries). For example, within the context of web analytics users will typically query for day, week, and monthly reports. Hence if one created an OLAP cube for web analytics, one can build custom aggregations for those time periods.

Using the view of the sample Budget cube from AS2k, notice from above that you can set the Time dimension to aggregate at the “Bottom Level Only” instead of “Standard”. In this particular case, this would tell the Analysis Services processing engine to design aggregations for the “month” level only instead of the standard approach.

Create a partitioning strategy for your OLAP cube

As noted above, by creating a partitioning strategy for your OLAP cube, you will be able to “distribute” the data across multiple files thus allowing more threads to calculate the distinct counts faster. Typically, for maintenance, processing, and querying purposes – if you have a medium volume or larger amount of data within your fact tables – you will typically partition the data by regular intervals. For example, some enterprise-size OLAP cubes will be partitioned by day because the vast majority of the queries are done at the day level.

For AS2k, we had determined an optimization technique that is quite counter-intuitive. On top of creating partitions by a time period, you create additional partitions by some other dimension thus partitioning the data even further (within AS2k5, this implies that you need to do query binding vs. table binding for your partitions). What is counter-intuitive is that you would create partitions where the distinct IDs (e.g. unique visitors) are spread out through all of the partitions. That is, partition by dimension(s) that have the distinct IDs repeated in most/all partitions. The above is counter-intuitive in that most would try to partition by a dimension that minimized the distinct ID replication throughout the partitions which may be the case for AS2k5. Either way, at query time, this design forces the Analysis services engine to utilize multiple threads to calculate distinct counts instead of just one thread. That is, now you have distributed query calculations in parallel. Empirical analysis noted a four to ten time improvement in distinct count queries by using this design.

Partitioning Strategy Example

Going back to the grocery market basket analysis mentioned above, the question to be answered is “how many customers do I have that have purchased multiple products within a grocery store?” Using this example, the partitioning choices can be:

· None

· Partition by Users (minimize ID replication throughout partitions)

· Partition by Products (replicate ID users throughout as many partitions as possible)

Below is a graphical representation of three people (beige, orange, and blue) purchasing seven different products (a-g). If you use no partitioning strategy, a single thread will go through these 15 “facts”to calculate there are:

· 3 distinct users

· 1 user that purchased product g and another f

· 2 users that purchased b

Note, there is one thread for one partition to analyze the data.

Another option is to partition by users to minimize replication of users within these partitions:

Notice, there are now three threads to help perform this calculation. As you can see, this particular partitioning strategy will be much faster than having no partitioning. As well, within the context of AS2k5, this actually may be the optimal solution due to the redesign of Analysis Services to an attribute/hierarchy model.

The optimal partition strategy for AS2k (and potentially for AS2k5) is to partition by a dimension in which the users are spread evenly across all partitions.

Referring to the above graphic, now we have seven threads to quickly determine the distinct counts.

As you can see from the above example, by partitioning so that the distinct users are spread across multiple partitions, there will be more thread and less data for those threads to calculate the distinct counts. You will need to be careful with your partitioning strategy – you can actually run the risk where you peg the processors if there are too many threads running (hence the importance of multi-proc OLAP machines). In the above example, the CPU utilization for a single enterprise-size query by a single user was that of:

· One partition: 25%

· Three partitions: 50%

· Seven partitions: 75-80%

While these numbers are approximations and the CPU was pegged for milliseconds, as your system becomes larger and is utilized by more people, it will be that much more important to take into account of this.

New INTERSECT and EXCEPT operators in Sql Server 2005 T-SQL

Fri, 24 Feb 2006 21:13:01 GMT

This is a great explanation of using the new SQL 2k5 operators of INTERSECT and EXCEPT instead of inner and outer joins.

Quote

Raymond Lewallen : New INTERSECT and EXCEPT operators in Sql Server 2005 T-SQL

MDX: Count number of dimension members

Mon, 16 Jan 2006 19:37:29 GMT

If you are trying to count the number of dimension members via MDX, where your measure is [Measure].[Item Quantity] and dimension is [Item].members, you can write your MDX query as:

with
member [Measures].[blah] as 'Count(Crossjoin({[Item].members}, {[Measures].[Item Quantity]}), EXCLUDEEMPTY)'
select {
[Measures].[Item Quantity], [Measures].blah
} on columns, {
[Item].children
} on rows
from [Cube]

Analysis Services 2005 Usage Based Optimization and OlapQueryLog

Wed, 23 Nov 2005 08:30:55 GMT

The Usage-based Optimization feature is alive and well within SQL Server 2005 Analysis Services (AS2k5). But by default, the QueryLog feature (which samples and records the queries sent to AS2k5 is turned off. To turn it on, you will need to:

1) Create a SQL database (e.g. QueryLog) on the Analysis Services server (Actually, you can put it on another SQL server if you like, but the preferred option is to place it on the SQL server on the box).

2) Through SQL Management Studio, connect to your Analysis Services, right click, and click on "Properties".

3) Within the Analysis Services Properties dialog, scroll down the properties, and first look for:

Log \ QueryLog \ QueryLogConnectionString

Click on the Value column and it will provide you a dialog to point the connection string to the SQL database. Point to the SQL dB (e.g. QueryLog) that you had just created.

4) Then scroll to the property

Log \ QueryLog \ CreateQueryLogTable

Flip the Value column from false to true.

5) Click on OK and then you will have created the OlapQueryLog table within the QueryLog (if that's what you called it) database. The name of the table is actually the parameter:

Log \ QueryLog \ QueryLogTableName

Now that you have created your query log table, you can send queries to your OLAP cubes and then when its time to perform usage-based optimization:

1) Connect to your Olap cube via the SQL Management Studio

2) Go down the hierarchy of Measure Group > Partition, right-click the partition in question and then you will be provided the "Usage-based Optimization Wizard".

From there, you can use that wizard to optimize your Olap cubes. The "SQL Server Books Online" provides good reference material for the wizard and can view some of the screenshots of it right now at Mosha's blog at: http://www.sqljunkies.com/WebLog/mosha/archive/2005/06/23/usage_based_optimization.aspx

Analysis Services 2005 Drillthrough

Wed, 16 Nov 2005 01:02:39 GMT

Drillthrough functionality is quite different in AS2k5 vs. AS2k. The latter originally queries the SQL server directly when you want to drillthrough to the data. In AS2k5, the drillthrough data is placed directly into the OLAP cube so you can drillthrough without ever connecting to the SQL database (assuming that is your source). Saying this, this functionality is on by default and does increase the size of the OLAP database. But performance is not affected and now you have this additional functionality. For more information, T.K. Anand wrote an excellent paper on this at:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnsql90/html/sql2k5_anservdrill.asp

To provide an example against the AdventureWorks OLAP database (that comes with SQL Server 2005 Samples), you can query the Adventure Works Olap dB using the MDX statement:

select {
[Measures].[Internet Order Quantity]
} on columns, {
[Product].[Product Categories].children
} on rows
from [Adventure Works]

where it outputs the Internet Order quantities for all of the different categories. If you want to drillthrough, for example, just the Clothing category (one of the children of the [Product].[Product Categories] hierarchy, you need only to run the following MDX statement:

drillthrough maxrows 100
select
{[Measures].[Internet Order Quantity]} on 0,
{[Product].[Product Categories].[Clothing]} on 1
from [Adventure Works]
return
[$Product].[Model Name] as [Model Name],
[$Customer].[Full Name] as [Customer Name],
[$Customer].[Country-Region] as [Country Region],
[$Measures].[Internet Order Quantity] as [Orders],
[$Measures].[Internet Sales Amount] as [Sales],
[$Measures].[Internet Total Product Cost] as [Cost]

Note that you didn't need to turn any drillthrough functionality on with the AdventureWorks OLAP dB - its already on. Cool, eh?!

How to handle many-to-many relationships in Yukon?

Sat, 22 Oct 2005 00:05:10 GMT

It is tempting to try to use parent-child relationships to handle many-to-many relationships but Yukon has introduced a many-to-many relationship functionality that's really cool. For example, using a geography hierarchy noted below, France and Spain both belong to a hierarchy of "Western Europe" and "EU"

Western Europe
       Switzerland (SZ)
       France (FR)
        Spain (SP)
       Lithuania (LI)

EU
France (FR)
Spain (SP)

And the problem with many-to-many relationships is that you risk double-counting, double-summing of in this scenario. Well, to resolve this, you can use the reference of:

http://www.sqljunkies.com/WebLog/sqlbi/archive/2004/10/04/4447.aspx

Instead of the above examples of Customer, Account and CustomerAccount; you would create a Region (Western Europe, EU), Country (SZ, FR, SP, LI), and a CountryRegion table which maps the many-to-many relationships between the country and region.

How to get MDX "top" and "bottom"

Wed, 19 Oct 2005 22:56:36 GMT

Often, you will want to get the top or bottom X number of dimension members based on the dimension hierarchy. Within Analysis Services, you have the functions of TopCount, TopSum, and TopPercent (and the equivalent of Bottom functions) but these are based on a Measure. And what you want is the top child or bottom child of a member based on the dimension order - not on the Measure order.

Well, to do this:

with
member [Measures].[test] as '1'
select {
[Measures].[My Count]
} on columns, {
topcount(descendants([Domain].[Domain Hierarchy], 2, LEAVES), 1),
bottomcount(descendants([Domain].[Domain Hierarchy], 2, LEAVES), 1)
} on rows
from [Cube]

The trick for this is that:
The Descendants function will insure to get all of the members of the dimension hierarchy. The two (2) is the number of levels you want to go down; you'll want to specify what you want.

The problem in this case with TopCount and BottomCount (because in this case you're looking for the SQL equivalent of top and bottom against the dimension) is that its based on a measure. So you'll notice the TopCount and BottomCount is applied against a measure I created ([Measures].[test]) which has a value of 1.

Since all of the values are 1, then the TopCount and BottomCount will simply go with the internal hierarchy which then is the equivalent of FirstChild.FirstChild and LastChild.LastChild in these cases respectively because I had specified 1 as the last parameter of TopCount/BottomCount. You can specify a higher number to get the top and bottom 5 for example.

OLAP and SQL Firewall Configuration

Thu, 23 Jun 2005 02:50:00 GMT

OLAP and SQL Firewalls

For a client or middle-tier to access You will need to open the following firewall ports:
SQL Server: 1433

OLAP Services: 2393, 2394, 2725

For more information, please refer to:
Q301901 or
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/csvr2002/htm/cs_dp_typical_mjjt.asp

For Yukon (SQL Server 2005), you will need to open as well:

Analysis Services 2005: 2383 (default port), 2382 (redirector port)

Calculate Memory Used by Olap

Wed, 22 Jun 2005 07:14:10 GMT

Calculate Memory used by OLAP
To calculate memory, the formula is:
DimSize = (M x (61 + 4L + Sa + Sk))) + 4P + Sn

where

M: Number of members in the dimension.
L: Number of levels in the dimension, including the ALL level.
Sa: Average size required to hold the member names.
Sk: Size required to hold the member key.
P: Number of member property settings in the dimension for all levels.
Sp: Size required to hold the distinct member property values for all member levels.

Olap's Lazy Aggregation Error

Wed, 22 Jun 2005 07:13:00 GMT

Lazy Aggregation Error
If you get an “Error during lazy aggregation of partition %PartName% in cube %CubeName%, this is typically caused by a lazy aggregation error. To resolve this, you can:

Is the registry key ProcessReadSegmentSize at least 60% of physical memory on the machine?
How much disk space available for the temporary directory?

Resolution of either one of these things should remove the Lazy Aggregation Error you are seeing.

How to archive an OLAP database (not using msmdarch.exe)

Wed, 22 Jun 2005 06:33:52 GMT

How to archive an OLAP database (not using msmdarch.exe)

To provide an unsupported method to archive an Olap database instead of using msmdarch.exe.

You can backup the file folder containing the Olap data (i.e. c:\Program Files\Microsoft Analysis Services\Data) and the Olap repository SQL database (assumes you migrated the Olap repository to a SQL database). Note, this will backup the entire server.
To backup a specific OLAP database, you can:
1. Archive the c:\Program Files\Microsoft Analysis Services\Data\OlapDB folder and the c:\Program Files\Microsoft Analysis Services\Data\OlapDB.odb file.
2. Backup the Olap Repository SQL database OR copy the pertinent rows from the said database from the OlapObjects table. Note, the table is a parent-child dimension table.

Insure you do these processes at the same time and after all OLAP writing is done (e.g. no processing, lazy aggregations are finished, etc.)

Renaming OLAP database (Unsupported)

Wed, 22 Jun 2005 06:23:56 GMT

Renaming OLAP database (Unsupported)

Below are the unsupported steps to rename and OLAP database.

Archive the OLAP database and SQL repository (assumes you migrated the OLAP repository to SQL).
Stop Analysis Server
Rename the folder that contains the Olap dB (e.g. rename c:\Program Files\Microsoft Analysis Services\Data\blah to c:\Program Files\Microsoft Analysis Services\Data\blah2)
Go into the Olap repository SQL dB , and change the name “blah” to “blah2” w/in the OlapObjects table. Examples include:
- The column ObjectName needs to be changed from “blah” to “blah2”
- The ObjectDescription column needs to be changed from '<DATABASE Name = “blah">....' to '<DATABASE Name = “blah2">...'
Rename of the *.odb file within the OlapdB folder (e.g. rename c:\Program Files\Microsoft Analysis Services\Data\blah.odb to c:\Program Files\Microsoft Analysis Services\Data\blah2.odb)
Start Analysis Server

OLAP Distinct Count - how to make it faster

Wed, 22 Jun 2005 02:59:08 GMT

Distinct Count

To get the full performance out of Analysis Services for distinct count, insure that a distinct count measure is in its own cube and you can perform custom aggregations for the dimensions within the cube. But the most powerful way to improve performance is to use a partitioning strategy.

The idea is to create partitions where the distinct IDs are spread out through all of the partitions (i.e. partition by dimension(s) that have the IDs repeated in most/all partitions); which is slightly counter-intuitive. At query time, this design forces AS2k to utilize multiple threads to calculate distinct count instead of just one thread; i.e. distributed query calculations in parallel. From real-world scenarios, there was a 4-10 time improvement w/ this design.

Note, you willneed to be careful with your partitioning strategy – you can actually run the risk where you peg the processors if there are too many threads running; hence the importance of multi-proc OLAP machines.