Analysis :: Dynamic Values In XMLA To Incrementally Update Cube Processing
Nov 10, 2015
I am trying to incrementally update a Cube to get near real time data for the end users. Currently we have a Sql server agent Job that does a FullProcess on the Cube. The Cube consists of a single Measure group which is simply one named query containing inner joins of all the dimensions and fact tables in the underlying relational database. The end users have a lot to upload during the day and they would like us to refresh the cube (near real time) to ensure the adjustments are loaded so that they could reconcile their daily PnLs. We have a MeasureId added which is an auto increment column in the Measures table.
I am trying to schedule the below XMLA query in Sql server agent Job and schedule it to run every 15mins or even less (if possible). However it seems to be not working and keep throwing all sorts of errors.
DECLARE @LastMeasureId AS INT, @myXMLA nvarchar(max)
SELECT @LastMeasureId = "[Measures].[Maximum Measures Id]" FROM
OpenRowset(
'MSOLAP',
'DATA SOURCE=L68F728326574; Initial Catalog=GMDR;',
'SELECT NON EMPTY {[Measures].[Maximum Measures Id]} ON COLUMNS FROM [GMDR]');
I have a cube that we are processing nightly via an Analysis Service Processing Task in SSIS. In order to increase the performance of the processing time, we elected to use a lot of rigid dimension attributes, and do a full process of everything in the SSIS task. The issue that I am having is that after that task completes, I need to go into Visual Studio to deploy the cube becuase we are unable to browse or use the cube. This issue seemed to start once we changed the SSIS Analysis Service Processing Task to do a full process on the dimensions, rather than an incremental.
I would expect that once development is done, and it is processed and deployed, that is it. My thinking is that the SSIS task should just update the already deployed cube,
I'm just starting to learn SSIS and would like some advice on how to handle something I encounter frequently. I often have to connect to a remote FTP site which contains a large number of files. Each day a number of files are added (old files are not purged). So each day I need to download all files which have been added since last time I checked, store on a file server, then load to a SQL database.
I've written a DataFlow using a script which generates a list of files and dumps to a raw file and doing the same for FTP directory listing wouldn't be too hard - I could then feed these into a third DataFlow to work out what the new files were, then a third to download the files to a temp directory etc.
But is there a cleaner way of doing this (I'm not adverse to scripting or even writing my own components - but for maintainability reasons I'd like to keep this as "out of box" as possible)? How are other people approaching this task?
I have designed a cube. It has two fact tables and some dimensions. Fact table to fact table is many to many relationship.
For example
FactMain DataKey(PK), StartDateKey, PostCodeKey, TotalCost FactBridge DataKey(FK), ProductKey(FK), Position - PrimaryKey on DataKey + ProductKey + Position DimProduct ProductKey(PK), ProductCode
Cube is built successfully, processed successfully.When I try to process the cube from agent job, I am getting error "Attribute key not found: tablename, value..." I have added a job step to run AnalysisServices Command. I have taken the command from cube process script(taken from manually process the cube and take script generated). I used ProcessAffectedObjects = "true" in the script. When I checked the tables, the key does exist. Why am I getting this error?
I am getting following errors when i processed Cube in Production.An error occurred while the dimension, with the ID of 'DIM_PARTICIPANT', Name of 'DIM_PARTICIPANT' was being processed. End Error
An error occurred while the 'PARTICIPANT NAME' attribute of the 'DIM_PARTICIPANT' dimension from the 'XL_GCS_SelfServices' database was being processed. End Error
An error occurred while the dimension, with the ID of 'v d Transaction', Name of 'DIM_TRANSACTION' was being processed. End Error
An error occurred while the 'INVOICE NUMBER' attribute of the 'DIM_TRANSACTION' dimension from the 'XL_GCS_SelfServices' database was being processed. End Error.
But i implmented same in Dev and QA server, there is no issues found.
I am working on SQL 2012.We have a SSAS Cube build. On top of it client use Excel to connect to SSAS Cube and See the reports.My Cube process every hour and take almost a 1-2 min to Process.When ever End user/Client See or refresh the Excel report (Which use Cube as Source),WHEN CUBE IS PROCESSING ,they get an error that Source is not available
We Have tried best but cannot bring down the Processing time of the cube to 3-5 Second , so that End user don't face report refresh issue at the moment of cube processing
Requirement :In Case End User see the report while Cube is processing from back end , Instead of Error they should see some customize Msg which we can provide some thing Like "Data is Refreshing , please wait ".
I Have one cube, First time its Processed successfully and browsed fine. Now i was change in the cube Calculations/somewhere, now again process the process failed. I am unable to browse my cube. Is there any option to Rollback this processed cube and Browse with existing cube?What is require is, If cube process is fail, i want to browse the cube with existing cube after modify the cube also(if fails).
I am trying to create a Adventure Works Tabular cube in my developing machine and im having problems deploying the tabular cube (the DW database was firstly attached without problems)
So I'm having problems processing Adventure Works Tabular Cube, the problems are somewhere between SSAS project configuration and running privileges, i think. I'll describe exactly where i am:
- in SSMS 2012, connections, tables and roles and roles are empty, i only have the full name of the database when I try to process i get an error "the table with id of "product category 92583829......", Name of Product Category refereced by the model Cube, does not exist. An error ocurred when loading the model cube, '?c:program filesmicrosoft sql serverMSAS11.MSSQLSERVEROLAPDataAdventureWorks Tabular model SQL 2012.0.dbModel.21.cub.xml'. (Microsoft AnalysisServices.)"
- in SQL Server configuration Manager, I have Sql Server (MSSQLSERVER) running with account NT ServiceMSSQLSERVER Sql Server Analysis Services (MSSQLSERVER) running with account .L.ABCDE , which is my account, i've changed because i've read somewhere that i could have problems with privileges.
(Note: it's strange because SSAS is tabular, i can see a blue rectangle (sort of) in SSMS but in configurator manager seems to be a cube.)
- in SSAS Project i unzip a clean copy of the project and the first error is that BIM file can't be found, it try's to find server TOSHIBA_S50_BTAB, but as i have only tabular SSAS running, i replace for TOSHIBA_S50... And some time ago it worked, now it doesn't even found TOSHIBA_S50_B?!!!!
We have set up an IS package to process an AS 2005 database (comprising cube & dimensions, etc) daily, via a SQL Server Agent job on both development and production systems. This has been working fine for months.
A new dimension was added to the cube on the development system - automatic processing via the IS package continued without issue. However, when the new dimension was added to the production system the IS package no longer processes the cube correctly. Although all appears ok (and all is present and correct in the logs), no data updates to the cube are made. Only when the cube is manually processed does the cube get updated.
Anyone got any ideas about how to get around this issue? We have created a new IS package, with a single Analysis Services Processing Task, and tried this but get the same outcome.
I have been trying to get the ValueR column of the following query through the MDX but instead getting ValueW as output of the MDX
select exp(Log(sum(MTMROR)+ 1 ))-1 as ValueW, exp(sum(Log(MTMROR + 1)))-1 as ValueR from Temp_Performance where Rundate in ('2015-03-01','2015-03-02')
MDX written for the above query is
With Member [Measures].[LogValuePre] as ([Measures].[MTMROR] + 1) Member [Measures].[LogValuePre1] as VBA
[Code] ...
[MTMROR] measure has the aggregate function Sum. What i can get from this behavior is MDX is first aggregating the result and default aggregation function is Sum. When i try to see the value with more granular data by having the date dimensions on the row (un-commenting the date dimension) i get the correct log and exp log values. Its showing the correct value as date dimension is most granular level in the fact table. While trying to get the data at less granular level(Fund level), getting the sum function applied automatically.
If i set AggregateFunction to none in the cube structure, i get null as the output.
How could i apply the log function before the sum function in the [MTMROR] measure?
Scenario: [**tableA**] plus [**tableDim1**] plus [**tableDim2**]. I create a DSW, a cube I deploy the [@@CUBE@@]..so connect the data using an excel file that show the data as [¬¬dashboard¬¬].
It works.
My question is: after three days the [**tableA**] is populated with new rows. In order to allow my colleagues to see the new rows I deploy the [@@CUBE@@] again. My colleagues can see the new rows in the [¬¬dashboard¬¬].
It works, ok. But do I really need to deploy the [@@CUBE@@] every time or it should be update automatically when you, for example, refresh the data in Excel. Do I miss something?
We are receiving the following error when trying to Update the cube (the last line of code)Cannot update the 'Database' object 'DB Cube_Temp', it needs to be part of a connected Server object.
1) As per my project requirement, if the changes in SSAS cube are approved, they should be committed back to the actual SQL Server 2012 tables. Is that possible, if yes how?
2) For rolling back to original data I truncate the relevant writeback table and process the cube.
I am working on SSAS project and now i am looking for a way to generate some documentation from XMLA code ?I have seen that it's possible to do it with XML, so i think it is possible too in XMLA !
I am trying to write a query for the default values for month, and I am stuck in bringing out only the parameter values, the query I have , brings back the month on row and value on column, and I am trying to get the value out. and will ssrs accept it as a valid expression for a default value
WITH MEMBER [MEASURES].[PARAMETERVALUE] AS [Account].[Month Short Name].CURRENTMEMBER.UNIQUENAME
[Code] .....
I tried using the exp
=Parameters!MONTH.Value(0)
so as to return the first parameter in the parameter dataset , but i have the error the expression that ref the parameter MONTH does not exists in the parameter collection.
I hope some of you can point me in the right direction. I am new to all this and working on a project using Cubes.
The cubes need to be generated in a dynamic fashion as the contents (that means field) in the fact tables change change every quarter and are based off of a external application.
The base strucutre stays the same though. So we thought we would export the XMLA representation of the cube and then munge (word for alter) the data in the XMLA file and send it someplace to create the new cube on a SQL 2005 server. This seemed easy enough but I have found no real information on how to run some piece of code that would create the cube based on this XMLA code.
On another note is there a parsing object for the XMLA so i dont have to right one from scratch?
It seems simple but I'm not able to make it works. I've got a fact table with sites (DimImplantation) with allowed users (DimDroit) :
1 user can have access to many sites. I've tried these codes but only one worked and only in case where the user had only one site :
a) =DimImplantation[DimImplantation_ID]= LOOKUPVALUE (DimDroit[DimImplantation_ID]; DimDroit[Utilisateur];USERNAME() ) ==> worked for 1user with 1 site
b) =DimImplantation[DimImplantation_ID]= LOOKUPVALUE (DimDroit[DimImplantation_ID]; DimDroit[Utilisateur];USERNAME(); DimDroit[DimImplantation_ID];DimImplantation[DimImplantation_ID] ) ==> doesn't work and says 'not able to to identifie the value of DimImplantation[DimImplantation_ID] in the current context.
c) CALCULATE(VALUES(DimDroit[DimImplantation_ID], SUMMARIZE('DimDroit',DimDroit[Utilisateur]), DimDroit[Utilisateur]=USERNAME()) ==> version for SQL2012. I've had 'End of input reached'.
I am a R data miner who is new to SQL and SSIS and would appreciate any help.
I wanted to automate the process of creating and processing decision tree models for every county in the Country. I wanted to use the foreach loop for iterate through all the counties. I wanted the foreach enumerator to be used by the XMLA code that creates the model so it would append it to the name of the model and i would get a different model for every county. I am not sure how to have the XMLA code accept foreach loop enumerator.
Any help would be greatly appreciated and if you could direct me to a previously done example that would greatly benefit me.
I have 3 cubes in a single SSAS database and these cubes should be processed using the following schedule
Cube 1 - Every Day Cube 2 - Every Week Cube 3 - Every Month Cube 4 - Every Day
The issue that I face is that these cubes share the dimensions and so I cant do a FULL process of these SHARED Dimensions as it will affec other cubes.
I can expect additions and deletions to my dimension data , but the structure remains the same. It would be great if someone can suggest how to go about processing the dimensions. I am confused with the number of options(Process Incremental, Process Update etc.,) available for processing the dimensions.
I will creating a SSIS package to automate the processing. One more question is say, if Cube 2 fails during a day and Cube 1 has succesfully processed on the same day earlier, how do I revert back to the old state of Cube 2? Does this mean that I need to do a back up of the SSAS database before processing each cube?
When I process a new cube I recieve an error "Error (211): Unknown dimension member ' 8'; Time: 8/11/00 1:09:45 PM". Any ideas about what this error is about ?
I have a cube which can be processed in our development server. However, when it is deployed to the UAT server, the cube processing was hanged. What I have done then:
Checking sys.sysprocesses found that there are 8 threads suspended with waittype=CXPACKET. The threads are all reading the transactional database.
We then searched from web that CXPACKET is related to parallel query processing. So we have done the following:
Select 'Sequential' option in processing -> result is the same with CXPACKET.
Select 'Parallel' with parallel thread set to 1 -> result in one thread with ASYNC IO wait type
Select 'Parallel' with parallel thread set to 2 -> result in CXPACKET.
Change the number of CPUs SQL Server use from 8 to 1 -> result in one thread with ASYNC IO wait type
Change the number of CPUs SQL Server use from 8 to 2 -> result in CXPACKET.
Change 'Maximum parallel query' in SQL server = 4 -> result in CXPACKET. All the trials result in hanged status
Difference between development server and UAT server: 1. CPU : dev = 2, UAT = 8 2. SQL Server Version : dev = 2005 SP1, UAT = 2005 SP2 3. database size : dev > UAT
Anyone has the problem and solution before? Please share. Thanks.
Errors in the high-level relational engine. The data source view does not contain a definition for the 'WeightRecieved' column in the 'dbo_factPurchases' table or view.
The problem is WeightReceived IS defined in my DSV so I don't know what to do about this error.
I'm looking for a possiblity to schedule the processing of a ms olap services cube in a SQL Server Agent job. Has anyone expericiences with that? Are there any alternatives for scheduling the processing?
I have a problem with processing my cube. My fact table (with telephone data) contains about 400,000 records... which is increasing rapidly (400,000 records is about 8 months of data)... I have a few dimensions: Dimension User: about 200 records Dimension Line: about 200 records Dimension Direction: 4 records Dimension Date: 365 records for each year Dimension TimeInterval: with 24 records
So far so good... when I process this dimension I have no problem.... However, when I add a dimension (CalledNumber, with exactly 101 records) the processing hangs as soon as it starts...
The SQL performed when processing the cube looks like this:
SELECT field1, field2,... fieldn FROM table1, table2,.... tablem WHERE (table1.id=table2.table1id) AND (table2.id=table3.table2id) ...
When I execute above SQL in the Query Analyser from SQL Server Enterprise Manager, it ALSO hangs...
I am not really suprised by that, because this SQL first create a huge table of 400,000 x 200 x 200 x 4 x 365 x 24 x 101 records and after that works through the WHERE statements to filter out the appropriate records.
for me it would be more logical to use the following code to process the cube, but that cannot be changed in Analysis Manager:
SELECT field1, field2,... fieldn FROM table1 LEFT JOIN table2 ON (table1.id=table2.table1id) .... LEFT JOIN tablem ON (tablem.id = tablem-1.tablemid)
When I execute above SQL in the Query Analyser from SQL Servel Enterprise Manager, it does NOT hang, but performs the query in about 35 seconds.... But Analysis Manager does not allow me to change the SQL used for processing the cube...
What can I do to add more dimensions to my cube... (It will be more anyway after adding the CalledNumber dimension)?? any suggestions?
Hi, cube processing is taking more time in a new server while same cubes takes less time in another server. the cubes are processed through DTS package can anybody help finding out the possible reasons for this. Regards Naseem
I have a problem with processing my cube. My fact table (with telephone data) contains about 400,000 records... which is increasing rapidly (400,000 records is about 8 months of data)... I have a few dimensions: Dimension User: about 200 records Dimension Line: about 200 records Dimension Direction: 4 records Dimension Date: 365 records for each year Dimension TimeInterval: with 24 records
So far so good... when I process this dimension I have no problem.... However, when I add a dimension (CalledNumber, with exactly 101 records) the processing hangs as soon as it starts...
The SQL performed when processing the cube looks like this:
SELECT field1, field2,... fieldn FROM table1, table2,.... tablem WHERE (table1.id=table2.table1id) AND (table2.id=table3.table2id) ...
When I execute above SQL in the Query Analyser from SQL Server Enterprise Manager, it ALSO hangs...
I am not really suprised by that, because this SQL first create a huge table of 400,000 x 200 x 200 x 4 x 365 x 24 x 101 records and after that works through the WHERE statements to filter out the appropriate records.
for me it would be more logical to use the following code to process the cube, but that cannot be changed in Analysis Manager:
SELECT field1, field2,... fieldn FROM table1 LEFT JOIN table2 ON (table1.id=table2.table1id) .... LEFT JOIN tablem ON (tablem.id = tablem-1.tablemid)
When I execute above SQL in the Query Analyser from SQL Servel Enterprise Manager, it does NOT hang, but performs the query in about 35 seconds.... But Analysis Manager does not allow me to change the SQL used for processing the cube...
What can I do to add more dimensions to my cube... (It will be more anyway after adding the CalledNumber dimension)?? any suggestions?
We have a MS-OLAP cube that has about 11 partitions and I have created a prototype package which processes these partitions conditionally based on expressions that are fed values from a SQL Server control table. It appears that one or more of the partitions seem to fail due to the fact that all of the data for the various partitions come from the same huge fact table. Is there a way to control the level of concurrency within the package itself? If not, I am thinking I should move some of the partitions to process based on other partitions completing their process successfully. Appreciate any help.
