I want to display my problem. I have a cube that connected to hive DB through views. There are some changes that apply to some related tables on hive. This changes reflect on the cube so i make full process for the cube. I want to process only partitions that have been changed without full process. I detect changes on this table on another table on the local database.
View 3 RepliesIn the SSIS Analysis Services processing task, I was wondering if
anyone knows why some dimensions do not have the Process Update option
in the list of options for processing them? If there is
only Process Full, Process Data, and Unprocess, I am not sure how
I can do incremental updates without scripting.
Also, will this affect the cubes if a full process is performed?
Any help is much appreciated!
I have 3 partitions using a year grouping. Current year, previous 4 years, older than 5 years. I have two measure groups, one is a distinct count, so I actually have 6 partitions.I also use usage based optimization to build my aggregations. Should each partition have a separate aggregation or should there be one for each measure group?
View 5 Replies View RelatedI have defined a stored procedure with one parameter. With this parameter I'm able to controll which year of the sales amount data should be selected. This works fine.
Now I want to implement this stored procedure as the source of the partitions. But if I do this I get an error. The syntax-check says, that everything is fine. But if I want to calculate the partition with this command: "exec dst.fact_umsatz_year 0" get the following error (in German):
OLE DB-Fehler: OLE DB- oder ODBC-Fehler : Falsche Syntax in der Nähe von ')'.; 42000; Falsche Syntax in der Nähe des exec-Schlüsselworts.; 42000.
Fehler im OLAP-Speichermodul: Fehler beim Verarbeiten der FACT Umsatz Pivot View-Partition der Anzahl Kunden-Measuregruppe für den Vertrieb-Cube aus der OLAP AS-Datenbank.
I have set the slicers on paritions using Script task.
After cube process, the mdx queries are not hitting partitions as per Slicers.
But once i open each partition slicers from SSMS and close it by doing ok. My mdx query is good enough to hit the relative partition only.
But it is not possible to manually refresh each slicer on partition.
Is there any way out to solve this issue. I am using 2008 R2.
I'm considering using horizontal partitions to separate my data by year.
For example, SomeTable_2004, SomeTable_2003, etc. This works well for backups, maintenance, etc. because I'm working with 150+ GB of data. I'll be a partitioned view for queries.
However, I'm new at this and have a few questions. I would also like to do partitioned updates or inserts. But I need to make sure that the tables don't use similar primary keys. Does that make sense? I need to make sure that the primary keys from the first table are not used again in the second table.
SomeTable_2003
primary keys: 1,5,8,9,15
SomeTable_2004
primary keys: 2,3,4,10
I don't really care what keys are used on what table, as long as they are different. I have apps that already use this data, and I don't want to change the application logic.
Thanks,
T
I have 3 columns. I would like to update a table based on job_cd and permit_nbr column. if we have same job_cd and permit_nbr, reference number should be same else it should take max(reference number) from the table +1 for all rows where reference_nbr column is null
job_cd permit_nbr reference_nbr
ABC1 990 100002
ABC1 990 100002
ABC1 991 100003
ABC1 992 100004
ABC1 993 100005
ABC2 880 100006
ABC2 881 100007
ABC2 881 100007
ABC2 882 100008
ABC2 882 100008
I want to perform SPC on a data set.
The type of analysis I want to do is
Mode,Median,Max,Min,Average,standard deviation.
Does Sql server have any in built functions to accomadate this type of analysis.
Is there any information on the net that I could be referred to. I have SQL Server 7.0 Enterprise Edition.
Thanks Pargat
how can I process and update cubes automaticly every night ?
Thanks
I'm building a cube for sales team , to test out I'm trying to process just one dimension called DimCalandar , when I try to process this dimension I get the following error ,
'Either the user abc/def, does not have access to database, or the database does not exist' ...
I am getting partition process error in one of my cubes. I don't have any clue what could be the workaround with this.
View 2 Replies View RelatedI trying to update my Aggregation Design for a partition using BIDS Helper. The current aggregation design contains about 60 aggregations and the new aggregation I am trying to add is across 5 dimension attributes, the product of which is about 500,000 unique values. The fact table is about 13,000,000 rows.
When I deploy the aggregation and run ProcessIndex, I get the follow error:
File system error: The following file is corrupted: Physical file: ?E:Program FilesMicrosoft SQL ServerMSAS10_50.MSSQLSERVEROLAPDataTestDB.14.dbcube_2.607.cub
cubemeasuregroup_2.633.detDefaultPartition_2.579.prt852.agg.flex.data. Logical file.
If I remove the new aggregation, deploy, and run ProcessIndex again, it processes fine.
Is there some file size limitation I am running into? The agg.flex.data file is 7.8 GB before adding the new aggregation, so it isn't subject to the same 4 GB limit as .asstore.
Windows Server 2008 64 bit
SQL Server 2008 R2 (10.50.1746.0) 64 bit
I have started working on SSAS since last week, I need to perform some calculations on the data fetched from the cube based on the parameters. SSRS is used to display the output and SSAS is used as a data source.
How i could perform the operations on the data fetched from the cube in SSAS? Does SSAS provides the storage structure like temp table in stored procedure where we can perform the various operations before sending final data back to the client side tool(SSRS)?
OR Is there any alternative way to perform the operations based on the input provided through the parameters
I am trying to load all the MDX queries that run on a Analysis Server instance into a database for further analysis. A SQL Profiler is setup which captures the MDX queries, and when I am loading the Profiler info to database, some of the queries are not coming up in full length.The TextData field doestn't show full MDX query. When loading to the database, the field is next data type. Is there any workaround to get the complete MDX query?
View 2 Replies View RelatedOur SSAS integration didn't initially use attribute relationships.Now that our system has been running for a few years and we have bigger databases, we think we need to add them to improve performance. So we're in the process of adding them but we found out that, when attribute relationships are added, the full unique name of our members all go from something like:
[DIM].[HIERARCHY].[LEVEL].&[GRANDPARENT].&[PARENT].&[MEMBER]
to something like:
[DIM].[HIERARCHY].[LEVEL].&[MEMBER]
It looks nice and SSAS will accept the longer names fine but it will return the short ones in response to 'discovery' requests and in the XMLA response of MDX queries. This is causing problems in our low level XMLA-based modules that assume the long names in and out. is there any clean way to use attribute relationships and still have SSAS generate the long member names. We fiddled with the various documented dim/attribute properties but to no avail. It also appears that some switches are obsolete.
I am struggling to calculate Full year in my SSAS Cube. Meaning, regardless of what fiscal year hierarchy level I am in; i need a measure aggregating from 01/01/year of current member to 12/31/year of current member.
I want to replicate it using the Year To Date below:
FY-FQ-FM is the fiscal year quarter hieararchy
I am using for built in time intelligence.
Create Member
CurrentCube.[DimTime].[FY-FQ-FM DimTime Calculations].[Year to Date]
As "NA";
/*Year to Date*/
(
[DimTime].[FY-FQ-FM DimTime Calculations].[Year to Date],
[Code] ....
I have been tasked with processing a large tabular cube using SQL AS 2014 (with latest CUs).The three Fact tables having 1.2 billion rows (in each table) have been divided into 30 vertical partitions to aid in parallel processing. So around 40 million rows per partition.
Using SQL Profiler to monitor the Row counts (IntegerData) of records processed seems to max out around 2 million rows per minute, then tapers down to about 200k /minute.
The processing is taking over 14 hours and I need to get it lower if possible. The server has 48 cores (2.66MHz) and over 1TB RAM installed. But I really don't ever see CPU exceed 20% having a maximum of 206 threads running on the instance msmdvr.exe
Available RAM is always at least 30% (or 300GB).
I have increased the Vertipaq MIN/MAX 60%/80%
I have increased the OLAP / Processing / Max Thread Pool Min 500 and Max to 1000.
The connection properties have been increased to allow 100 connections, the majority of the processing consumes about 92 connections for the 90 large partition views for the facts.
What can be done to increased the server resource utilization and decrease processing times?
I have increased both
I have a simple update statement that will update one field, and that field is part of the primary key. During the update process, some of the rows will cause duplicate error. Is there a way to update the table and suppress the error? What I am looking for is a way to update the records that it can and ignore those it cannot. Right now, the entire process is terminated if duplicate error occurs.
View 2 Replies View Related
My SSIS solution has about hundred packages and time to time I have to edit a package. I understand I could use 'Build' command to compile only updated package, as opposed to Rebuild which recomplies all of the packages.
Nevertheless, in both cases SSIS opens all of the packages in design environment before compilation. My packages are saved in SourceSafe and that process takes quite long and I was wondering if there was any other way to compile only updated package where none of the other packages are opened during Build/Rebuild process? For example we could use dtutil to deploy only updated packages without running Package Installation Wizard.
as title.
Thanks.,
Kelvin Jor
hi,
I'm using a OLE DB COMMAND component to perform an update (SQL statement) but the procces takes about 9 hours, so I changed it to a stored procedure but it was the same, I need to update about a a million of rows and the package is very simple.
How can I improve the time, Can I use another component or startegy?
thanks
Greetings,
I have setup a FormView which functions as it should but after the user input is updated, the table record stays unchanged, and when I trap the FormView1_ItemUpdated and look at the SqlDataSource1.UpdateCommand, it shows this:
UPDATE [aspnet_test] SET first_name = '', last_name = '', email = '' WHERE id = @original_ID
Here is most of the code I am using:<asp:FormView ID="FormView1" runat="server" DataSourceID="SqlDataSource1" DataKeyNames="id, first_name, last_name" OnItemUpdating="FormView1_ItemUpdating" OnItemUpdated="FormView1_ItemUpdated" > .. // my ItemEditTempate is here.</asp:FormView>
<EditItemTemplate>First Name: <asp:TextBox Text='<%# Bind("first_name") %>' runat="server" ID="author_name" Columns="20"></asp:TextBox><br />Last Name: <asp:TextBox Text='<%# Bind("last_name") %>' runat="server" ID="TextBox1" Columns="20"></asp:TextBox><br />E-mail: <asp:TextBox Text='<%# Bind("email") %>' runat="server" ID="TextBox2" Columns="20"></asp:TextBox><br /><br /><asp:Button ID="UpdateButton" runat="server" Text="Update" CommandName="Update" /><asp:Button ID="CancelButton" runat="server" Text="Cancel" CommandName="Cancel" /> </EditItemTemplate>
<asp:SqlDataSource ID="SqlDataSource1" runat="server" ConnectionString="<%$ ConnectionStrings:ASPNETDBConnectionString1 %>"
SelectCommand="SELECT id, first_name, last_name, email FROM aspnet_test where id = 1"UpdateCommand="UPDATE [aspnet_test] SET first_name = '<%# first_name %>', last_name = '<%# last_name %>', email = '<%# email %>' WHERE id = @original_ID ">
<UpdateParameters><asp:Parameter Name="original_ID" Type="Int32" /></UpdateParameters></asp:SqlDataSource>
Any idea where the @original_ID is supposed to get its value from, or why does the SQL command shows blank fields?Thanks
Eric.
I have been trying to get the ValueR column of the following query through the MDX but instead getting ValueW as output of the MDX
select
exp(Log(sum(MTMROR)+ 1 ))-1 as ValueW,
exp(sum(Log(MTMROR + 1)))-1 as ValueR
from
Temp_Performance
where Rundate in ('2015-03-01','2015-03-02')
MDX written for the above query is
With
Member [Measures].[LogValuePre]
as ([Measures].[MTMROR] + 1)
Member [Measures].[LogValuePre1]
as VBA
[Code] ...
[MTMROR] measure has the aggregate function Sum. What i can get from this behavior is MDX is first aggregating the result and default aggregation function is Sum. When i try to see the value with more granular data by having the date dimensions on the row (un-commenting the date dimension) i get the correct log and exp log values. Its showing the correct value as date dimension is most granular level in the fact table. While trying to get the data at less granular level(Fund level), getting the sum function applied automatically.
If i set AggregateFunction to none in the cube structure, i get null as the output.
How could i apply the log function before the sum function in the [MTMROR] measure?
Scenario: [**tableA**] plus [**tableDim1**] plus [**tableDim2**]. I create a DSW, a cube I deploy the [@@CUBE@@]..so connect the data using an excel file that show the data as [¬¬dashboard¬¬].
It works.
My question is: after three days the [**tableA**] is populated with new rows. In order to allow my colleagues to see the new rows I deploy the [@@CUBE@@] again. My colleagues can see the new rows in the [¬¬dashboard¬¬].
It works, ok. But do I really need to deploy the [@@CUBE@@] every time or it should be update automatically when you, for example, refresh the data in Excel. Do I miss something?
I've a weird problem in my application. In of the pages, while trying to update the text box "Name", when I enter Linda's test, it gets saved as Linda''s test. I'm not sure if this is a problem due to SQL server. When I look at the stored procedure, I don't anything different. Also, when I update the table directly in SQL Server, the result is displayed in single quote. But if I update the field thro' the application, the returned name is with double quotes instead of single quote. Has any of you faced problems like this? What am I missing? What do I need to do to get the name saved the way I entered (with single quotes) instead of double quotes?
View 1 Replies View RelatedI'd like to be able to update an Analysis Services cube through a stored proc.
Currently I can:
- Make a DTS package that updates the cube
- run xp_cmdshell which runs dtsrun which runs the DTS package.
That is messy, easily broken, and hard to get good error info when an error occurs. Is there a better route?
we are having an existing cube in that we need to update with new measures . The Measure groups are added to the cube as linked object. so when we are updating the measure group it is throwing the exceptions as follows..“Errors in the OLAP storage engine: The metadata for the statically linked measure group, with the name of 'SalesActual', cannot be verified against the source object.”
View 5 Replies View RelatedWe have an application that takes an existing cube, clones it and then updates it in C#.
Database dbTarget = dbSource.Clone();
dbTarget.Name = databaseName_Target;
dbTarget.ID = databaseName_Target;
dbTarget.DataSourceImpersonationInfo = new ImpersonationInfo(ImpersonationMode.ImpersonateServiceAccount);
sSAS_CalculationServer.Databases.Add(dbTarget);
dbTarget.Update(UpdateOptions.ExpandFull);
We are receiving the following error when trying to Update the cube (the last line of code)Cannot update the 'Database' object 'DB Cube_Temp', it needs to be part of a connected Server object.
I am very new to SSAS. I have two queries:
1) As per my project requirement, if the changes in SSAS cube are approved, they should be committed back to the actual SQL Server 2012 tables. Is that possible, if yes how?
2) For rolling back to original data I truncate the relevant writeback table and process the cube.
I am getting following error on "Table Import Wizard" of Tabular model Cannot update the 'Database' object 'Tabular Sample_', it needs to be part of a connected Server object.
View 4 Replies View RelatedI noticed today a session that was executing a FULL SCAN update as follows:
UPDATE STATISTICS [XXXX].[XXXX].[XXXX] [_WA_Sys_00000009_318D45CA] WITH FULLSCAN
When I checked the sys.dm_exec_query_memory_grants DMV for the session I could see the following values:
requested_memory_kb granted_memory_kb used_memory_kb max_used_memory_kb
145,705,216 145,705,216 139,977,336 139,980,408
When I checked the Properties of the Statistic I can see it is on a varchar(3) field when there are only 3 different values in there - all char(1)
The total size of the data in the table according to the Disk Usage By Top Table Report is 199,680,712KB
So my question is this...
For the UPDATE STATS on this one column with FULL SCAN, does SQL Server read the entire table into the Buffer Pool. If so then if the table had 199,680,712KB of data then why did the session request 145,705,216KB.
Or does SQL Server just read the column and ClusteredIndex Key into the Buffer Pool?
is there an easy way I can give a specific user complete insert,update, and delete permissions on all tables,view, and sp in a db without having to set individually for all?
View 5 Replies View RelatedI am trying to incrementally update a Cube to get near real time data for the end users. Currently we have a Sql server agent Job that does a FullProcess on the Cube. The Cube consists of a single Measure group which is simply one named query containing inner joins of all the dimensions and fact tables in the underlying relational database. The end users have a lot to upload during the day and they would like us to refresh the cube (near real time) to ensure the adjustments are loaded so that they could reconcile their daily PnLs. We have a MeasureId added which is an auto increment column in the Measures table.
I am trying to schedule the below XMLA query in Sql server agent Job and schedule it to run every 15mins or even less (if possible). However it seems to be not working and keep throwing all sorts of errors.
DECLARE @LastMeasureId AS INT, @myXMLA nvarchar(max)
SELECT @LastMeasureId = "[Measures].[Maximum Measures Id]" FROM
OpenRowset(
'MSOLAP',
'DATA SOURCE=L68F728326574; Initial Catalog=GMDR;',
'SELECT NON EMPTY {[Measures].[Maximum Measures Id]} ON COLUMNS FROM [GMDR]');
[Code] ....