Calling any OLAP Guru's. Were attempting to improve a vendors BI process which involves running MDX queries against our SSAS cube, then saves the data into their own proprietary database which is taking to long.
We currently run this process on a x16 CPU, 65GB top end server attached to expensive disk subsystem, however a dual-core processor
laptop completely out performs the expensive hardware by around 45% quicker. Both when the configuration is identical or when we ramp up the processes on the expensive hardware it still performs miserably compared to the laptop. Perfmon counters or Profiler don't so much difference when comparing the server and laptop.
Does anyone know why the laptop completely outstrips the expensive top end hardware perfmance wise although the configuration profile is identical?
I have encountred situations like this before, but this onehas me stumped.I have a pretty simple SP that collects information aboutresidential properties from a large database. First step is toquery on the basis of address or location, and collect a temptable of property IDs. Second step is to populate a compositetable of property information by joining the IDs to a table ofcharacterisitics. Third step is to update some fields by findinga single value from multiple candidates in large tables -- onehas 275 million, another 325 million rows -- e.g., the price ofthe most recent sale for a property.As an SP, this takes absolutely forever, and it seems it's doingendless scans of the large tables. So to analyze it, I took thecode and ran it as a script -- turned the parameter definition atthe top into a DECLARE statement, set values for the variablesthat are the input parameters, no other changes, and go. Presto!It runs in no time flat, and the query plans reveal it's usingthe indexes just like it's supposed to. But the SP might takean hour to do the same thing.Any suggestions about what to look for? I believe both versionshave fresh query plans -- I have recompiled (and dropped andrecreated) the SP, and the plain script should have a fresh plan.Maybe it's because the parameter values are known when the scriptruns, but not when the SP is complied? I would really appreciateany pointers, and can provide more information as needed.Thanks,Jim Geissman
CREATE TABLE [dbo].[LOG]([TYPE] [smallint] NULL ,[TIME_STAMP] [datetime],[ID] [varchar] (44))ID is non-unique. I want to select all IDs where the last entry forthat ID is of type 11.Below is the query that I used. Notice that the subquery used is anested (not correlated) subquery meaning that it doesn't use resultsof outer query. This subquery should only be executed once. However,on large number of rows (3 million), this query never returns.I have also attempted to run subquery separately. That takes 1 minute.Then I put the results in temp table and joined that temp table withthe main query. That takes about 2 minutes.Unfortunately, that solution is unacceptable to us since we have tosupport both MSSQL and Oracle with the same queries, and the syntaxfor temp tables or table variables is different in Oracle.Mysterious.Here's the query:-- main queryselect IDfrom logwhere ID in(-- subqueryselect id from log l1where time_stamp =(select max(time_stamp)from log l2where l2.id = l1.idand l2.type = 11))
My application fetches a batch of data through a web service and writes 1000entities per batch to a SQL Server 2000 database. There are 4 tables inevery batch. There are the following number of SQL commands executed peraverage of every batch;Table #1: always 1Table #2: 5Table #3: 5Table #4: 3The problem is that the performance slows down for every batch. Below is anexcerpt from my log file;2004-12-15 12:00:01 Starting job... (RAM usage: 6,38 mb)2004-12-15 12:00:39 data fetch time: 00:00:28 (RAM usage: 23,04 mb)2004-12-15 12:00:39 Total data fetch time: 00:00:37 (RAM usage: 23,04 mb)2004-12-15 12:00:39 Inserting/updating 1000 entities...2004-12-15 12:01:20 Write SQL time: 00:00:402004-12-15 12:01:49 data fetch time: 00:00:24 (RAM usage: 26,87 mb)2004-12-15 12:01:49 Total data fetch time: 00:00:29 (RAM usage: 26,87 mb)2004-12-15 12:01:49 Inserting/updating 1000 entities...2004-12-15 12:02:59 Write SQL time: 00:01:102004-12-15 12:04:06 data fetch time: 00:00:29 (RAM usage: 27,48 mb)2004-12-15 12:04:06 Total data fetch time: 00:01:06 (RAM usage: 27,48 mb)2004-12-15 12:04:06 Inserting/updating 1000 entities...2004-12-15 12:05:30 Write SQL time: 00:01:232004-12-15 12:06:05 data fetch time: 00:00:31 (RAM usage: 27,03 mb)2004-12-15 12:06:05 Total data fetch time: 00:00:35 (RAM usage: 27,03 mb)2004-12-15 12:06:05 Inserting/updating 1000 entities...2004-12-15 12:07:37 Write SQL time: 00:01:32As one can see, the Write SQL time increases per every batch.I would like this time to stay around one minute per batch.There are one trigger per table. There is one parent table which has aprimary-foreign key relationship to the three sub tables.I have 2% automatic file size growth set on both the data and the log file.Thank you in advance to the guru which helps me out with this!
Does Reporting Services have SERIOUS problems performance wise with OLAP data? Reports running directly against our relational datawarehouse run FASTER than reports going against the OLAP cubes. How can this be?
<rant>
At first i thought i was just missing something but the "upgrades" to reporting service have been a HUGE dissapointment, at least in the OLAP field:
- They STILL flatten the result set. How can this be after FIVE years??? After all the hype about UDM and improved integration with analysis services they STILL do this???
- Report builder: Has anyone seen a worse implementation of a reporting tool? Report models?? What about the UDM thats supposed to bridge the gap between analysis and reporting? Integration with SSAS? What integration? If anyone is using this for anything serious (and im not talking tech demos) i would like to know.
Is this what Microsoft defines as "Enterprise Reporting"?
I've created a data mart on a SQL Server 7.0 database containing 1 fact table and 6 dimension tables. I then created an OLAP cube based off this table. I used the MOLAP format and created over 900 aggregations until the performance measure got to be about 15%. It took about 45 minutes for the OLAP manager to get to that point.
The size of the actual data isn't that big. The fact table has about 500,000 records in it and the other tables have at the most 1000 records. The SQL Server and OLAP server both run on the same box and have a gigabyte of RAM installed. I believe I have thoroughly indexed the tables.
Here's the problem. I try to access the cube through Office 2000's web pivot table control and it connects just fine. As soon as I add more than 3 dimensions and 1 measure, though, performance goes down to almost nothing. My computer (and other computers in the company) just crawl. It takes about 20 minutes for the control to refresh after you add a 4th dimension. I tried accessing the cube through Excel 2000 and it completely crashes after I add the 4 dimension. I can query the tables directly from SQL and the results are returned incredibly fast.
I've got to be missing something here. Any suggestions? Any help will be greatly appreciated!
Our OLAP environment involves an ETL/Data Warehouse/Data Mart server and a cube publisher server. We would like to learn how to automate the Archival/Restore of OLAP databases. We are currently doing it manually though OLAP Manager. Any help would be appreciated. Thanks. James.
-- James E. Bothamley Senior Database Administrator Dave & Buster's, Inc. 2481 Manana Dallas, TX 75220
Work Phone (214) 904-2296 email jbothaml@DaveAndBusters.Com
"Once in a while you can get shown the light in the strangest of places if you look at it right"
I have a table Catalog_Item with 365000 rows. I wish to move the contents of the table to an identical database on a different server using bcp. The table has no primary keys, foreign keys, or indexes. I am able to successfully bcp out the data in character format and native format.
I attempt to bcp the data in to the identical table on the other server, but the bcp ALWAYS fails at 19000 or 20000 rows; it does not complete, just hangs there in the command window, with no more info. When I use a small data set of 5000 rows, the same happens at the end of those 5000. I have tried to redo the bcp in native mode, and character mode. I have tried to specify the first line starts at 125000. I have tried to bcp the data into a different database on a different server; that also quits at 19000 or 20000 rows.
BCP of other large data sets into the same database occurs smoothly. Only this combination of table and data set is giving this problem.
I am having trouble connecting to an SQL Server from an ASP.NET page written in C#. I have reduced the code in my page to this: public void Page_Load( object s, EventArgs e ) {
SqlConnection conn = new SqlConnection( "Server=server;User ID=user;Password=pass;" ); conn.Open();
} and I get the following error: System.Data.SqlClient.SqlException: SQL Server does not exist or access denied.
This works for other server/user/pass except the one I am having trouble with. The SQL Server is using SQL & Windows Authentication. I am able to connect from the machine that is running the webserver via osql, ODBC, and MS Access using the same arguments (server, user, and pass), so I know that the SQL server does exist, and the User/Password should give me access. Does anyone have any ideas why this isn't working or what ASP.NET is does differently from the other access methods? Any ideas are greatly appreciated. Thanks, Mark Dane
I have created a maintenance plan which backups to a device which I have created. However the device does not show in explorer and maintenance plan cannot find it when it trys to open the device. Consequently I'm not getting a backup. Even if I delete the device and re create it things don't change.
I was working at a customer site and wanted to verify that backups were indeed being run successfully before I began to make changes.
I noticed a dump device for the production database with the nightly backup in it but I cannot find the task that produces this dump. I did find the task that dumps the log each hour.
Is there another area that some of you DBAs used to schedule dumps other than the task manager that I might be missing? Maybe an AT job?
Hi all,I have a statement that reads as follows:select [Sales Ledger] - gl as differencefrom(SELECT SUM(RPAAP / 100) AS [Sales Ledger] FROM F03B11) Q1join(SELECT SUM(GBAPYC + GBAN01 + GBAN02 + GBAN03 + GBAN04 + GBAN05 +GBAN06+ GBAN07 + GBAN08 + GBAN09 + GBAN10 + GBAN11 + GBAN12)/100AS GLFROM F0902WHERE (GBAID = '00667809') AND (GBFY = 3)) Q2My first nested statement however I keep getting the message:Server: Msg 170, Level 15, State 1, Line 8Line 8: Incorrect syntax near 'Q2'.I just cannot fathom why this is so?Any suggestions would be most helpful.Moby*** Sent via Developersdex http://www.developersdex.com ***Don't just participate in USENET...get rewarded for it!
Hey All, Hope this is the right place for this post.
I set up logshipping between two databases recently. everything looks fine. the backup, copy and restore jobs are all suceeding(from the job log). the problem is no Tlogs are being restored in secondary database. The restore job skips all the tlogs and says it did not find any tlog back up file to restore!! The jobs finishes with this:
007-07-17 14:40:30.59 Could not find a log backup file that could be applied to secondary database 'SaaSNet_dataStore'.2007-07-17 14:40:30.59 The restore operation was successful. Secondary Database: 'SaaSNet_dataStore', Number of log backup files restored: 02007-07-17 14:40:30.59 Deleting old log backup files. Primary Database: 'SaaSNet_DataStore'2007-07-17 14:40:30.59 The restore operation was successful. Secondary ID: '5808a414-2ada-41d2-a8a0-2cf84f85174a'
In general, patch management solutions use an approach very similar to inventory and deployment, although, obviously, implementation details vary. Inventory relies on an external database to establish what is considered to be a recommended patch level and provides criteria for validating whether a particular patch has been installed. Products from Microsoft and Shavlik Technologies (on which HFNetChk, MBSA, and SMS 2.0 Feature Pack are based), keep track of published patches on the Microsoft Web site using the same mechanism, based on an XML formatted file called mssecure.xml. This file, available centrally at predefined locations, serves as a template against which the status of updates on target systems is compared. mssecure.xml can be obtained directly or via its compressed, digitally signed version, mssecure.cab. Both files can be downloaded from:
the Microsoft Web site at https://www.microsoft.com/technet/security/search/mssecure.xml and http://download.microsoft.com/download/xml/security/1.0/nt5/en-us/mssecure.cab the Shavlik.com Web site at https://xml.shavlik.com/mssecure.xml and http://xml.shavlik.com/mssecure.cab
Hi all, I found problem with my database and was wondering if anyone here could shed some light on the issue.
I have two tables, Absences and AbsenceDates. The first one records the absence of an employee and the second one records a record for each day of the occurance. I do a full select on the second table and I see primary keys that do NOT exist in the select of the second table. so I dug further and here is what I found.
Select * from Absences (rowcount in Query Analyser is: 20883) Select * from Absences Order By AbsenceID Desc (rowcount is 443)
The second select contains the data that I am missing in the first select. So, I called a friend and they said to run DBCC CHECKDB and I did. The data came back as follows...
DBCC results for 'Absences '. There are 21337 rows in 243 pages for object 'Absences'. CHECKDB found 0 allocation errors and 0 consistency errors in database 'EmployeeAbsenteeism'. DBCC execution completed. If DBCC printed error messages, contact your system administrator.
now if you add up the rows that the other two selects return it comes to 21326, not 21337. I am assuming that the value that DBCC gets is from sysobjects and that some sort of update would need to be run for it be accurate. This I don't care about too much, what I really need is for my main select statement to return ALL of the data, not just what it feels like returning.
My experience is with programming mainly (6 years in .net) and not DBA, so any help would be greatly appreciated.
Cheers, Brent
Sorry, this was supposed to go to data corruption forum... reposting..
I'm currently running Profiler sessions to track down Lock Timeoutproblems.My Profiler view contains (amongst others) the dbid column.Much of the time, this displays familiar dbids, such as 2 (tempdb) and5 (my main user db). However, it also regularly displays IDs of 0 and132.Using SELECT DB_NAME(), these translate as "master" and "NULL"respectively.Does anyone know:a) why dbid = 0 translates to "master", when the actual id of thisdatabase is 1, andb) why Profiler reports these dbids in the first place?
I was hoping someone could shed some light on wierd situation i'm experiencing. I have the following query:
select count(*) LeadCount from auto_leads al where received > dbo.GetDay(GetDate())
dbo.GetDay simply returns a smalldatetime value of today's date.
Now I recently got thrown into a data mess and for some reason this query takes 8 seconds to run. Now the first thing I did was update the stats on the Received column of this auto_leads table. I re-run the query and I'm still getting 8 seconds. I look at the execution plan I can make figure out why this is happening.
I then change the above query so the filter received > dbo.GetDay(GetDate()) is now just received > '5/31/2006' and the query comes back immediately. This doesn't make sense to me because the GetDay function is really simple and comes back immediately. I then try the following query to confirm it isn't a problem with the GetDay function:
declare @Today DateTime
set @Today = dbo.getday(GetDate())
select count(*) leads from auto_leads al join type_lead_status tls on (tls.type_lead_status_id = al.type_lead_status_id) where received > @Today
Sure enough, the query came back immediately. Next thing to go through my mind is that the query execution plan has been cached by SQL Server using the execution plan from before I updated the stats on the received column. So I executed sp_recompile 'auto_leads' and tryed the original query again. Still taking 8-10 seconds to come back.
So my question, is why when I remove the GetDay function call in my query filter is the query slow, as opposed to me just passing a variable into the query? Thanks!
I have a varchar field with leading alpha, plus 1 or 2 numerics, like 'A2' or 'A20'
And yet,
select cast(substring(T2.ALPHANUM_CODE, 2, 4) as numeric)
from ...
where ...
/* and substring(T2.ALPHANUM_CODE,2,1) not like '[0-9]' */
/* and substring(T2.ALPHANUM_CODE,3,1) not like '[0-9]' */ /* and substring(T2.ALPHANUM_CODE,4,1) not like '[0-9]' */ /* and substring(T2.ALPHANUM_CODE,5,1) not like '[0-9]' */
this generates a cast as numeric conversion error in my result set. However, if I supplement my query with each of the commented WHERE clause add-ons in turn, I get 4 zero-record result sets.
So, how can I have nothing but digits 0 thru 9 in each of 4 positions, but the cast as numeric attempt of those 4 character positions as a string fails?
I am trying to format a background color based on the field's value. But the expression always returns the false result. Here is the expression I am using
Both fill the background color blue when the textbox clearly contains RSB. I'm not sure why it cannot find the value. The field data type is Char(4). The textbox values are all uppercase.
This all started when I was trying to use a report parameter for conditional formatting and I realized that it wasn't working. I broke down the expression by directly trying a value.
I'm trying to set up transactional replication. The publication and subscription wizards completed successfully. When I try a test transaction I got this:
Category:SQLSERVER Source: SAMPLER Number: 2812 Message: Could not find stored procedure 'sp_MSins_Config'.
I can't find 'sp_MSins_Config' in any database, I can't find it with a Google search or on MSFT's. What the heck is this proc?
Every Sunday morning (a different time each week) there is a full backup created for every database on the server. The backup is not a scheduled backup from any maintenance plan or SQL Agent job. The backup set is unrestorable and has a strange name in the format of 'DBNAME_00_12c95fb7_399d_41ce_9a0d_b5728b6a00ba_'
Because this backup is listed as the last full backup and will not restore, it is causing a problem with our disaster recovery plans as nothing will restore from this point forward to the next full backup.
Does anyone know how a backup record like this can get created, and/or how to find the source. The backups are listed in the backupsets table in msdb - are there other system tables that may hold some clues as to the source of these backups?
In simple terms, our system is as such: We have a website. As someone clicks a button on the website, a stored procedure is executed against our database.
Every single day, between 12:15AM and 12:45AM we have a few stored procedures timing out, with the following message, for example:
2007-04-10 00:37:03,268 [3632] ERROR Service - caught exception Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding. at System.Data.SqlClient.SqlConnection.OnError(SqlExc eption exception, Boolean breakConnection)
I checked and saw that although there are jobs running at that time, all of these jobs are running periodically (e.g. every 30 minutes) and would cause timeouts at other times as well, if they were to blame. Other jobs are running at far away times and checking their history I know that their duraion in no way intersects the time-out times.
I also ran profiler during peak hours and know that no stored procedure of ours has a duration anywhere near 30 seconds (which is the currently set timeout period, although all of our sps run within milliseconds).
I am really puzzled as to what exactly is causing these timeouts. Would anyone suggest any approach to identify the problem. For example, I thought about running profiler (server side tracing) between 12AM and 1AM, but am not sure which counters are best to capture. Any suggestion on this?
I'm importing several hundred thousand lines of text data with multiple columns in each record. While SSIS can redirect a line with an error to an error output is there any way it can "tell me" which column or columns caused the error on that row? (After al,l it certainly recognized that a column was bad, so I'm hoping it can just tell me what it found.)
Please, help. Let me start by saying that I have asked several DBA's this question, none of which have been able to provide a solution. I have a very annoying problem. An additional backup of my SQL Server 7 database is recurring every day and it is hogging up space on the server. I only want one recurring backup.
When checking my database backup plan, it seems there is only one backup instance: qarun_1099_release1_db_200207170200.BAK and qarun_1099_release1_tlog_200207170600.TRN. However, when I look in the default backup directory, SQL/Database/Backup, there is an instance in addition to the backup mentioned above with another name: qarun_1099_release1. This "mystery" backup is neither a .BAK or .TRN, but simply a file? I cannot figure out how and where this extra backup is recurring and I want to delete it.
I have tried disabling the jobs in the sysjobsteps and sysjobs tables of the msdb database, but this did not work! The backups are still recurring every day!
So I've been monitoring long-running transactions on a SQL Server that hosts a couple of vendor-supplied databases that look after our factory.Today I noticed a pair that have confused my Excel spreadsheet (that I've been using to analyze these transactions).So here's the weird thing that I spotted. Given this query:
SELECT p.spid, p.login_time, at.transaction_begin_time, datediff(second, p.login_time, at.transaction_begin_time) as [difference] FROM sys.sysprocesses AS p INNER JOIN sys.dm_tran_session_transactions AS st ON st.session_id = p.spid INNER JOIN sys.dm_tran_active_transactions AS at ON st.transaction_id = at.transaction_id
[code]....
I had a look in the event log on the server, which had just been rebooted at around that time. It seems that the clock got changed on boot-up, with the size of it quite surprising. This meant that these processes were able to start their transactions *before* they logged on. Hopefully this doesn't cause any other weird problems.So I've requested an investigation about time synchronization on our virtualization hosts... and in the mean time, have set the SQL Server services to 'delayed start'.
Hello Everyone,I have a very complex performance issue with our production database.Here's the scenario. We have a production webserver server and adevelopment web server. Both are running SQL Server 2000.I encounted various performance issues with the production server with aparticular query. It would take approximately 22 seconds to return 100rows, thats about 0.22 seconds per row. Note: I ran the query in singleuser mode. So I tested the query on the Development server by taking abackup (.dmp) of the database and moving it onto the dev server. I ranthe same query and found that it ran in less than a second.I took a look at the query execution plan and I found that they we'rethe exact same in both cases.Then I took a look at the various index's, and again I found nodifferences in the table indices.If both databases are identical, I'm assumeing that the issue is relatedto some external hardware issue like: disk space, memory etc. Or couldit be OS software related issues, like service packs, SQL Serverconfiguations etc.Here's what I've done to rule out some obvious hardware issues on theprod server:1. Moved all extraneous files to a secondary harddrive to free up spaceon the primary harddrive. There is 55gb's of free space on the disk.2. Applied SQL Server SP4 service packs3. Defragmented the primary harddrive4. Applied all Windows Server 2003 updatesHere is the prod servers system specs:2x Intel Xeon 2.67GHZTotal Physical Memory 2GB, Available Physical Memory 815MBWindows Server 2003 SE /w SP1Here is the dev serers system specs:2x Intel Xeon 2.80GHz2GB DDR2-SDRAMWindows Server 2003 SE /w SP1I'm not sure what else to do, the query performance is an order ofmagnitude difference and I can't explain it. To me its is a hardware oroperating system related issue.Any Ideas would help me greatly!Thanks,Brian T*** Sent via Developersdex http://www.developersdex.com ***
I have function that returns a table of information about properties. Thedata comes from three different tables -- addresses (called PropertyID),property characteristics, and events concerning those properties (sales,appraisals, etc.), plus a table that maps one representation of propertytypes into another. The records are selected on the basis of location(longitude & latitude), property type, event type, and a range ofevent dates (upper and lower date specified). There are tens of millionsof records of all types, and almost any location, property type, event typeand date range will yield records.The heart of it is a cursor that selects records from joins on this basis:SELECT <a bunch of fields>FROM Property dJOIN PropTypeMap ptm ON ptm.PropertyTypeID = d.PropertyTypeIDJOIN PropertyID a ON a.PropID = d.PropIDJOIN Event e1 ON e1.PropID = d.PropIDLEFT OUTER JOIN Event e2 ON e2.PropID = d.PropIDWHEREd.LastSaleDate >= @LoDateAND a.GeoLongitude BETWEEN @LowerLon AND @UpperLonAND a.GeoLatitude BETWEEN @LowerLat AND @UpperLatAND ptm.PropCategory = @PropTypeAND a.GeoMatch <= @MinGeoQualityAND e1.EventTypeID = @SaleEventTypeAND e1.TransactionType = 'R'AND e1.EventDt BETWEEN @LoDate AND @HiDateAND e1.EventAmt > 0AND e2.EventTypeID = @AssessmentEventTypeAND e2.EventDt <= @HiDateAND e2.EventAmt > 0Each property has one PropertyID record, one Property record, and N Eventrecords (average perhaps five).What is the mystery? If @HiDate, which is the upper end of the time window,is 2002-11-08 or earlier, nothing is returned. If it's 2002-11-09 or later,oodles of records are found. I get the same query plan for either one, andbased on the content of the data, they should return almost exactly the sameset of records -- exactly the same set in almost all cases, in fact.Is 2002-11-08/09 some sort of magic dividing point? I have replicated thison the large database and on a smaller test version on another SQL Server.(SQL Server 2000) I dropped the indexes and tried it, andthe same thing happened. This is driving me crazy!
