How To Launch MSOLAP Cube Processing In Batch Mode?
Mar 12, 2001
Jonathan Yang Jonathan.Yang@Weyerhaeuser.Com
How to launch MSOLAP cube processing in batch mode?
================================================== =
When one wants to run SQL Statement or Stored procedure in batch mode, OSQL.exe can be used.
Now we have created OLAP cube and want to process(populate) cube. From MSOLAP Analysis manage we can right click a cube name then click “Process”. But how to do this in batch mode? I.e. any equivalent thing of OSQL.exe to help process cube?
I believe a matured tool should work good in both interactive and batch mode. Unfortunately I did not find how to launch cube processing from MS OLAP documents. Looks like we have to write one using DSO supplied by Microsoft. Do you know if there is any tool for us not to write one? Pls help.
Hi,I have just started using MSSQL and the DOS environment at work. I havea lot of experience with Sybase and the UNIX environment, but this is awhole new ball of wax.I'd like to use osql from a batch file to log into the dataserver andrun a fairly long list of SQL and then exit. I don't want to have a bunchof SQL files sitting around that I have to use the -i option to run, andI'd rather not create temporary SQL files like this.echo "exec sp_who2" >tmp.sqlecho "select * from....... " >>tmp.sqlosql -E -S <DATASERVER> -n -w999 -i tmp.sqldel tmp.sqlI can't use the -Q option, of course because as I said, I'll be writingquite a few lines of SQL, and it won't all fit on the one line, or at leastit wouldn't be pretty if I did.In UNIX, I can simply execute the following from either the command lineor ina script.isql -S<DATASERVER> -U<USER> <<-EOFsp_whogoselect * from .....goEOF --EOF is the isql session terminator exits me back to the command line.This behavior does not appear to work with OSQL. There is a -O optionwhich help mentions, and that talks about disabling the EOF terminator forbatch processing, but I wasn't able to find any usage or examples on the netwhere someone is using EOF to terminate their OSQL SQL batch.This is what help listed.[-O use Old ISQL behavior disables the following]<EOF> batch processingAuto console width scalingWide messagesdefault errorlevel is -1 vs 1Any help you could offer would be appreciated. Thanks.Darren
I have 3 cubes in a single SSAS database and these cubes should be processed using the following schedule
Cube 1 - Every Day Cube 2 - Every Week Cube 3 - Every Month Cube 4 - Every Day
The issue that I face is that these cubes share the dimensions and so I cant do a FULL process of these SHARED Dimensions as it will affec other cubes.
I can expect additions and deletions to my dimension data , but the structure remains the same. It would be great if someone can suggest how to go about processing the dimensions. I am confused with the number of options(Process Incremental, Process Update etc.,) available for processing the dimensions.
I will creating a SSIS package to automate the processing. One more question is say, if Cube 2 fails during a day and Cube 1 has succesfully processed on the same day earlier, how do I revert back to the old state of Cube 2? Does this mean that I need to do a back up of the SSAS database before processing each cube?
When I process a new cube I recieve an error "Error (211): Unknown dimension member ' 8'; Time: 8/11/00 1:09:45 PM". Any ideas about what this error is about ?
I have a cube which can be processed in our development server. However, when it is deployed to the UAT server, the cube processing was hanged. What I have done then:
Checking sys.sysprocesses found that there are 8 threads suspended with waittype=CXPACKET. The threads are all reading the transactional database.
We then searched from web that CXPACKET is related to parallel query processing. So we have done the following:
Select 'Sequential' option in processing -> result is the same with CXPACKET.
Select 'Parallel' with parallel thread set to 1 -> result in one thread with ASYNC IO wait type
Select 'Parallel' with parallel thread set to 2 -> result in CXPACKET.
Change the number of CPUs SQL Server use from 8 to 1 -> result in one thread with ASYNC IO wait type
Change the number of CPUs SQL Server use from 8 to 2 -> result in CXPACKET.
Change 'Maximum parallel query' in SQL server = 4 -> result in CXPACKET. All the trials result in hanged status
Difference between development server and UAT server: 1. CPU : dev = 2, UAT = 8 2. SQL Server Version : dev = 2005 SP1, UAT = 2005 SP2 3. database size : dev > UAT
Anyone has the problem and solution before? Please share. Thanks.
Errors in the high-level relational engine. The data source view does not contain a definition for the 'WeightRecieved' column in the 'dbo_factPurchases' table or view.
The problem is WeightReceived IS defined in my DSV so I don't know what to do about this error.
1. Almost in every SB example you will see this sql :
BEGIN TRANSACTION
WAITFOR (
RECEIVE TOP (1)
@MessageType = message_type_name,
@Message = message_body
FROM [Queue1]
WHERE conversation_handle = @ConversationHandle
), timeout 5000;
If this sql in an activated sp do you really have to have the waitfor ? Since the sp will only be fired if there is a message on the queue ?
2. It is reccomended that for high volume SB apps you do not do a top(1) receive but process batches. Exactly what is the best practice to do this. Receive a batch into a table variable and then what ? Process through it with a cursor ? That is not very efficient either, i would just like some insight into batch queue processing as everywhere i have seen uses top (1) from the queue ?
I'm looking for a possiblity to schedule the processing of a ms olap services cube in a SQL Server Agent job. Has anyone expericiences with that? Are there any alternatives for scheduling the processing?
I have a problem with processing my cube. My fact table (with telephone data) contains about 400,000 records... which is increasing rapidly (400,000 records is about 8 months of data)... I have a few dimensions: Dimension User: about 200 records Dimension Line: about 200 records Dimension Direction: 4 records Dimension Date: 365 records for each year Dimension TimeInterval: with 24 records
So far so good... when I process this dimension I have no problem.... However, when I add a dimension (CalledNumber, with exactly 101 records) the processing hangs as soon as it starts...
The SQL performed when processing the cube looks like this:
SELECT field1, field2,... fieldn FROM table1, table2,.... tablem WHERE (table1.id=table2.table1id) AND (table2.id=table3.table2id) ...
When I execute above SQL in the Query Analyser from SQL Server Enterprise Manager, it ALSO hangs...
I am not really suprised by that, because this SQL first create a huge table of 400,000 x 200 x 200 x 4 x 365 x 24 x 101 records and after that works through the WHERE statements to filter out the appropriate records.
for me it would be more logical to use the following code to process the cube, but that cannot be changed in Analysis Manager:
SELECT field1, field2,... fieldn FROM table1 LEFT JOIN table2 ON (table1.id=table2.table1id) .... LEFT JOIN tablem ON (tablem.id = tablem-1.tablemid)
When I execute above SQL in the Query Analyser from SQL Servel Enterprise Manager, it does NOT hang, but performs the query in about 35 seconds.... But Analysis Manager does not allow me to change the SQL used for processing the cube...
What can I do to add more dimensions to my cube... (It will be more anyway after adding the CalledNumber dimension)?? any suggestions?
Hi, cube processing is taking more time in a new server while same cubes takes less time in another server. the cubes are processed through DTS package can anybody help finding out the possible reasons for this. Regards Naseem
I have a problem with processing my cube. My fact table (with telephone data) contains about 400,000 records... which is increasing rapidly (400,000 records is about 8 months of data)... I have a few dimensions: Dimension User: about 200 records Dimension Line: about 200 records Dimension Direction: 4 records Dimension Date: 365 records for each year Dimension TimeInterval: with 24 records
So far so good... when I process this dimension I have no problem.... However, when I add a dimension (CalledNumber, with exactly 101 records) the processing hangs as soon as it starts...
The SQL performed when processing the cube looks like this:
SELECT field1, field2,... fieldn FROM table1, table2,.... tablem WHERE (table1.id=table2.table1id) AND (table2.id=table3.table2id) ...
When I execute above SQL in the Query Analyser from SQL Server Enterprise Manager, it ALSO hangs...
I am not really suprised by that, because this SQL first create a huge table of 400,000 x 200 x 200 x 4 x 365 x 24 x 101 records and after that works through the WHERE statements to filter out the appropriate records.
for me it would be more logical to use the following code to process the cube, but that cannot be changed in Analysis Manager:
SELECT field1, field2,... fieldn FROM table1 LEFT JOIN table2 ON (table1.id=table2.table1id) .... LEFT JOIN tablem ON (tablem.id = tablem-1.tablemid)
When I execute above SQL in the Query Analyser from SQL Servel Enterprise Manager, it does NOT hang, but performs the query in about 35 seconds.... But Analysis Manager does not allow me to change the SQL used for processing the cube...
What can I do to add more dimensions to my cube... (It will be more anyway after adding the CalledNumber dimension)?? any suggestions?
We have a MS-OLAP cube that has about 11 partitions and I have created a prototype package which processes these partitions conditionally based on expressions that are fed values from a SQL Server control table. It appears that one or more of the partitions seem to fail due to the fact that all of the data for the various partitions come from the same huge fact table. Is there a way to control the level of concurrency within the package itself? If not, I am thinking I should move some of the partitions to process based on other partitions completing their process successfully. Appreciate any help.
I am trying to log the processing time details so that we can identify bottlenecks. My SSIS package has a bunch of OLAP processing tasks. In the Event Handler (onPreExecute and onPostExecute events), I am trying to capture the start and end time for each OLAP processing task by using an "Execute SQL task". In the event handler, I have a conditional expression that checks the following:
@SourceName != @[User::Expression1]
where Expression1 is a variable that contains the value of "Execute SQL Task". This expression I thought would be true only for OLAP processing tasks which btw never fire the OnPreExecute or OnPostExecute events. What am I doing wrong?
Hi! I have a problem with cube processing. When i am processing a cube with 5 million rows it will stop responding and the process on the SQL server has been suspended with the waittype ASYNC_NETWORK_IO. The waittime changes both up and down.
However if i change the the view(the cube is getting the data from a view that gets the data from ONE table) to only return up to 35 rows it works. However, 40 and it will go into suspended again. And i can let it run for several hours without it finishing. Other cubes in the same database with more rows works fine.
If i just run the query that is used when processing in Management Studio it works fine.
I have SP2 both on the AS and DB server.
The DB server looks like this:
4GB ram 2 x 2.80 dualcore intel xeon
The AS server:
16 GB ram 2 x 3.00 intel xeon X5450 quadcore
Any ideas?
Edit: And the AS service runs at 13%, in task manager it has one processor at 100 %. SQL Server Profiler shows no activity on the AS server.
i am getting the following error when i am processing the cube in SSAS 2008...Errors in the back-end database access module. The size specified for a binding was too small, resulting in one or more column values being truncated. Errors in the OLAP storage engine: An error occurred while the 'Policy Type' attribute of the 'Policy Type' dimension from the MyDemo' database.i verified the datatype column length for policytype column in the dimension as well as all fact views.
I have designed a cube. It has two fact tables and some dimensions. Fact table to fact table is many to many relationship.
For example
FactMain DataKey(PK), StartDateKey, PostCodeKey, TotalCost FactBridge DataKey(FK), ProductKey(FK), Position  - PrimaryKey on DataKey + ProductKey + Position DimProduct ProductKey(PK), ProductCode
Cube is built successfully, processed successfully.When I try to process the cube from agent job, I am getting error "Attribute key not found: tablename, value..." I have added a job step to run AnalysisServices Command. I have taken the command from cube process script(taken from manually process the cube and take script generated). I used ProcessAffectedObjects = "true" in the script. When I checked the tables, the key does exist. Why am I getting this error?
When I highlight a few partitions and start processing, the process occasionally stops with a message that operation has been canceled, like this:
Response 3 Server: the operation has been cancelled. Response 4
Server: the operation has been cancelled.
Response 5 Server: the operation has been cancelled.
..etc...
(no further error message details are provided) SQL profiler shows that batch was completed (but rowcount shown in process progress log is too small).
Analysis Services profiler shows no messages at that time at all. It just shows messages when it started, and when I restarted the processing.
The Analysis trace appears to be stopped when processing stops with "Server: the operation has been cancelled."This is an occasional error and sometime occurs within 15-20 minutes from starting to process. It could fail on 1st partition in the process list , or on some partition in the middle. Some partitions might run for a few hours and not error out, but sometimes it fails quickly.I was not sure if this is an issue with the underlying fact data, so I broke up the last partition where it failed into 5 smaller partitions, they processed OK separately.I think restarting SQL Server and SSAS helps to process a few more partitions. Some that failed with this problem process OK after restart, but some fail again. (apologies for re-posting, wanted to put under more specific thread title)
I am getting following errors when i processed Cube in Production.An error occurred while the dimension, with the ID of 'DIM_PARTICIPANT', Name of 'DIM_PARTICIPANT' was being processed. End Error
An error occurred while the 'PARTICIPANT NAME' attribute of the 'DIM_PARTICIPANT' dimension from the 'XL_GCS_SelfServices' database was being processed. End Error
An error occurred while the dimension, with the ID of 'v d Transaction', Name of 'DIM_TRANSACTION' was being processed. End Error
An error occurred while the 'INVOICE NUMBER' attribute of the 'DIM_TRANSACTION' dimension from the 'XL_GCS_SelfServices' database was being processed. End Error.
But i implmented same in Dev and QA server, there is no issues found.
which displays data from a Sales Order Header table and a Sales Order Detail table in a ReportViewer control, I am not able to view any detail data. The header information displays just fine. I checked the query used for gathering the detail records by executing a query through SQL Management Studio and it appears to be a valid query which returns the correct detail records.
I'm running the ReportViewer control in a .ASPX page inside Visual Studio 2005 in Local Processing Mode since ultimately I want to use the web page on a hosted Internet site.
Any ideas as to why the detail records won't display? The row numbers of the detail lines do display.
I have an ASP.NET Web Service that accepts a DataSet object passed to it. This DataSet will contain a large number of records in it's table. What I want to do (if possible) is insert all records in a SQL table in a batch mode (one go). Is this doable?
Is there a clever way of reteiving the identity of an INSERT query in SQLServer?
The example below is a the first insert (into the "parent" table with an identity column) of a heirarchical multi-table update.
sqlCommand.CommandText = "INSERT INTO Orders VALUES ( Date, Customer ) ; SELECT @@IDENTITY AS ID" ; sqlCommand.ExecuteNonQuery() ;
The ExecuteNonQuery() method returns the number of rows inserted but I need the ID for the INSERT into the "child" table.
My options based on my current understanding appear to be: a) use command parameters and set ID as an output param b) use a stored procedure, use command parameters and set ID as an output param c) issue the SELECT @@IDENTITY as a subsequent seperate command using ExecuteScalar()
Is there a more elegant way to get the ID using SQLServer's "batch" query mechanism and ADO.NET v1.1? Thanks,
I am working on SQL 2012.We have a SSAS Cube build. On top of it client use Excel to connect to SSAS Cube and See the reports.My Cube process every hour and take almost a 1-2 min to Process.When ever End user/Client See or refresh the Excel report (Which use Cube as Source),WHEN CUBE IS PROCESSING ,they get an error that Source is not available
We Have tried best but cannot bring down the Processing time of the cube to 3-5 Second , so that End user don't face report refresh issue at the moment of cube processingÂ
Requirement :In Case End User see the report while Cube is processing from back end , Instead of Error they should see some customize Msg which we can provide some thing Like "Data is Refreshing , please wait ".
Can I process a cube without error if there is an application that writes data into the source SQL Server database? Do I have to stop that application every time when I want to process the cube?
I Have one cube, First time its Processed successfully and browsed fine. Now i was change in the cube Calculations/somewhere, now again process the process failed. I am unable to browse my cube. Is there any option to Rollback this processed cube and Browse with existing cube?What is require is, If cube process is fail, i want to browse the cube with existing cube after modify the cube also(if fails).
All I get back is an error message of "Analysis Services Processing Task Error: A Connection cannot be made. Ensure the Server is running" The server is running, I can process the cube by connecting to the AS instance and right-click processing it.
I can process the cube by running the SSIS task inside of SSDT Just when I deploy the SSIS package (in Project mode) and then execute it do I get the error message.
SQL Server, SSAS, and SSIS processes are all running under the same account. SSAS is on a separate server from SSIS and SQL if that matters.
I am trying to create a Adventure Works Tabular cube in my developing machine and im having problems deploying the tabular cube (the DW database was firstly attached without problems)
So I'm having problems processing Adventure Works Tabular Cube, the problems are somewhere between SSAS project configuration and running privileges, i think. I'll describe exactly where i am:
- in SSMS 2012, connections, tables and roles and roles are empty, i only have the full name of the database when I try to process i get an error "the table with id of "product category 92583829......", Name of Product Category refereced by the model Cube, does not exist. An error ocurred when loading the model cube, '?c:program filesmicrosoft sql serverMSAS11.MSSQLSERVEROLAPDataAdventureWorks Tabular model SQL 2012.0.dbModel.21.cub.xml'. (Microsoft AnalysisServices.)"
- in SQL Server configuration Manager, I have Sql Server (MSSQLSERVER) running with account NT ServiceMSSQLSERVER Sql Server Analysis Services (MSSQLSERVER) running with account .L.ABCDE , which is my account, i've changed because i've read somewhere that i could have problems with privileges.
(Note: it's strange because SSAS is tabular, i can see a blue rectangle (sort of) in SSMS but in configurator manager seems to be a cube.)
- in SSAS Project i unzip a clean copy of the project and the first error is that BIM file can't be found, it try's to find server TOSHIBA_S50_BTAB, but as i have only tabular SSAS running, i replace for TOSHIBA_S50... And some time ago it worked, now it doesn't even found TOSHIBA_S50_B?!!!!
We have set up an IS package to process an AS 2005 database (comprising cube & dimensions, etc) daily, via a SQL Server Agent job on both development and production systems. This has been working fine for months.
A new dimension was added to the cube on the development system - automatic processing via the IS package continued without issue. However, when the new dimension was added to the production system the IS package no longer processes the cube correctly. Although all appears ok (and all is present and correct in the logs), no data updates to the cube are made. Only when the cube is manually processed does the cube get updated.
Anyone got any ideas about how to get around this issue? We have created a new IS package, with a single Analysis Services Processing Task, and tried this but get the same outcome.
Hi,This is probably a classic scenario with a shared dimension that weneed to use in different cubes, where all fact tables do not offer thesame level of detail. Dimension is snow-flaked.The cube that's causing me troubles was designed by marking the lowestdimension level Diabled and not Visible. This allows me to get rid ofone of the snow-flake tables (the one with the lowest level), thusallowing an INNER JOIN with the remaining table which has a level ofdetail corresponding to the fact table.When processing the cube, I get a 'member with key '[blah]' was foundin the fact table but was not found in the level '[blah]' of thedimension '[blah]'' that seems to indicate that none of my fact foreignkeys exist as primary keys in the dimension table. However if I thenattempt to query the cube, all data seems to be there.Would anybody be in a position (and willing ;-)) to share his/her ownexperience working around a similar issue?Thanks,SRL
I have an SSAS 2005 database "A" and SSIS package "P" which process full "A" olap database. SSAS SERVER connection string is based on a variable read from XML configuration file.
It works well in BIDS, but when i deployed, the package failed at the step connecting SSAS, the message is "a connection cannot be made, please ensure the server is running"
In the connnecting string, i am using server name like servera.xx.com, if I change it to IP address, it works. if I change it to Localhost(happens to be on the same server), it works.
But I need the server name solution as IP may be changed.
*every processing* of a cube with 7 dimensions 5 measure groups (no table has more than 100.000 rows) hangs while trying to process the measure groups. I have to restart the Analysis Service all the time. But even worse every 2nd processing crashes the whole Analysis Services Process.
The only time I could sucessfully deploy a cube was with 1 measure group only.
How can anyone do any serious work with Analysis Services?
The error messages from evenlog are not helpful either.
The description for Event ID ( 22 ) in Source ( MSSQLServerOLAPService ) cannot be found. The local computer may not have the necessary registry information or message DLL files to display messages from a remote computer. You may be able to use the /AUXSOURCE= flag to retrieve this description; see Help and Support for details. The following information is part of the event: Internal error: An unexpected exception occured..
I am trying to incrementally update a Cube to get near real time data for the end users. Currently we have a Sql server agent Job that does a FullProcess on the Cube. The Cube consists of a single Measure group which is simply one named query containing inner joins of all the dimensions and fact tables in the underlying relational database. The end users have a lot to upload during the day and they would like us to refresh the cube (near real time) to ensure the adjustments are loaded so that they could reconcile their daily PnLs. We have a MeasureId added which is an auto increment column in the Measures table.
I am trying to schedule the below XMLA query in Sql server agent Job and schedule it to run every 15mins or even less (if possible). However it seems to be not working and keep throwing all sorts of errors.
DECLARE @LastMeasureId AS INT, @myXMLA nvarchar(max) SELECT @LastMeasureId = "[Measures].[Maximum Measures Id]" FROM OpenRowset( 'MSOLAP', 'DATA SOURCE=L68F728326574; Initial Catalog=GMDR;', 'SELECT NON EMPTY {[Measures].[Maximum Measures Id]} ON COLUMNS FROM [GMDR]');
