This is a general question about data modeling. I'm more curious than anything else.
There is much talk about over-training data model, and I'm sure there are under training as well. As a rule of thumb, depending on the algorithm, what is a good ratio of attributes vs data points?
-Young K
It's often said or done that when inserting or updating into a 'large' table that disabling the non-clustered indexes can is needed for performance.
Now I know the obvious way to find out if this is best or not is by testing the different options. I was wondering if there was a rule of thumb to this?
Say you have a table with half a billion rows and 4 non-clustered indexes and are only updating half a million rows then sometimes disabling every night and re-enabling can take way more time than the actual update. Haven't found an articles advising to disable them when a table is over X rows and you are updating Y% of them...
I've been fixing some issues lately where weekly maintenance has been causing logs to grow and filling disks.
Is there any rule of thumb for allocating log space for doing reorgs and rebuilds in a worst case scenario? I'm thinking 3x the largest database size?
I've been watching them run on databases in the range of 50GB where the logs are growing well over that for rebuilds or even reorgs. Once you have a few databases like this on a server, you can suddenly eat through a lot of disk space just for holding logs during maintenance.
Hi ALL,
I am getting 2 errors for the report which has graph.
1.The chart Chart_ID0EDABDA contains a different number of DataPoints per ChartSeries element than the number of StaticCategory elements. If the chart contains no StaticCategory, it must contain one DataPoint per ChartSeries.
2.[rsWrongNumberOfChartDataPoints] The chart Chart_ID0EDABDA has a ChartData element that contains a collection of 6 DataPoint element(s), which is different from the number of expected datapoints (1). The expected number of datapoints in the definition is (Max(number of static categories, 1) * Max (number of static series, 1)).
I am converting crystal reports to SSRS.Can anybody give me some info about Data points requirments and how to get rid of these errors.I am getting these 2 error almost all the SSRS reports having Graphs.
Thanks .
I am working in SQL Server Master Data Services Version 11.0.5058.0 (SP 2).
I have been asked to group all the financial attributes together. When I move one of the attributes up using the arrows, it works good jumping over one attribute at a time. Then I reach a section of attributes where it leap frogs over 24 attributes.
It appears these 24 attributes are in a subgroup but there are no attribute groups and I removed the subscription view from the entity. If I move one of the 24 attributes in the group, it moves it outside of the 24 attributes.
This is under leaf member attributes. There are no collection or consolidated groups.
Processing Association Rules model on SQL 2005 Standard edition produced following error:
"Error (Data mining): The 'WO_3' mining model has 6690 attributes. This number of attributes exceeds the attribute limit of 5000 allowed by the current version of the algorithm associated with the mining model."
How can I limit number of attributes in a model?
Thank you
I'm using a DW from Northwind database to build a cube to do some analitical taks. I already create the cube and now I am "cleaning" the dimensions. I'm having some difficults to understand the logical off this part. The reason is that When I create the Data Source View, I only import the Foreign Keys that connect the Dimensions to Fact_Table. I have to drag the attributes of Dimension from Data Source View to the tab attributes?
Imagine this:
I have the following dimension:
Dim_Customer:
Customer_ID
Name_Customer
Job_Function
Date_of_Birth
Contact
Address
City
Country
When I create the cube only Customer_ID appears in attributes tab, it's normal?
One more question:
I don't want to create a hierarchy like:
Customer ID -> Name_Customer
Customer ID -> Date_of_Birth
Customer ID -> Address
Customer ID -> City
Customer ID -> Country
My idea is to create the following hierarchy:
Name_Customer -> Date_of_Birth -> Address -> City -> Country
But the first hierarchy that I show is always appears to me. Do you know what is happens?
Dear all,
I have a table containing call records, and made a mining model from that table only. The model has 3 columns : calling_number, called_number, and target_operator, using Association Rule algorithm. The key is calling_number, input was operator, and predicted column called_number.
The result shows no rule, but there are results with item-set size of 1 (column) and 2 (column). On the top record of the result, SQL Server says there are 1891 support for called_number = 1891 and operator = 'INDOSAT'.
I queried the table with this query
SELECT DISTINCT calling_number
FROM call_records
WHERE called_number = '07786000815'
AND target_operator = 'INDOSAT';
It returns 2162 records instead of 1891. If I removed the DISTINCT qualifier, SQL Server returns 2159 records. Why is this differences with the result of mining?
Thank you,
Bernaridho
I am currently importing tick data for a stock. Let's say my table structure is like this:
CREATE TABLE tick
(
tickId bigint identity(1,1) primary key
, tickTime datetime
, price money
)
If the stream of data I get resembles:
'4/17/12 2:00:00.000', 10.00
'4/17/12 2:00:02.000', 10.02
'4/17/12 2:00:01.000', 10.01
'4/17/12 2:00:03.000', 10.03
I want my table to look like this:
1, '4/17/12 2:00:00.000', 10.00
2, '4/17/12 2:00:02.000', 10.02
3, '4/17/12 2:00:03.000', 10.03
Essentially ignoring the out of place '4/17/12 2:00:01.000' record. What is the least expensive way to accomplish this?
Hello all-
I have a specification table that has some attributes defined.
SpecId - Id of the specification
Attribute - Attribute of the spec. (Like Color, HP etc)
Value - Is the value of the attribute
Then I have a car table that actually has information about the cars. Intention is to take each specification and match the cars that match the specification. If the car has more attributes than the spec, we ignore the extra attributes for the match. But if the car has less attributes, we don't even consider the car as a match (even if the attributes present, match). To summarize, the car's attributes should be >= spec's attributes.
The code I have below is bad because I am joining the same tables twice. In addition, it fails in the condition "the car's attributes should be >= spec's attributes"
Any help is greatly appreciated.
DECLARE @Specification TABLE
(SpecId VARCHAR(10),
AttributeVARCHAR(100),
ValueVARCHAR(100))
DECLARE @Car TABLE
(CarName VARCHAR(10),
AttributeVARCHAR(100),
ValueVARCHAR(100))
INSERT INTO @Specification VALUES ('S1', 'Type', 'Sedan')
INSERT INTO @Specification VALUES ('S1', 'Transmission', 'Auto')
INSERT INTO @Specification VALUES ('S1', 'HP', '220')
INSERT INTO @Specification VALUES ('S2', 'Type', 'SUV')
INSERT INTO @Specification VALUES ('S2', 'Transmission', 'Manual')
INSERT INTO @Specification VALUES ('S2', 'HP', '300')
INSERT INTO @Car VALUES ('Accord', 'Type', 'Sedan')
INSERT INTO @Car VALUES ('Accord', 'Transmission', 'Auto')
INSERT INTO @Car VALUES ('Accord', 'HP', '220')
INSERT INTO @Car VALUES ('Accord', 'Color', 'Black')
INSERT INTO @Car VALUES ('Escape', 'Type', 'SUV')
INSERT INTO @Car VALUES ('Escape', 'Transmission', 'Manual')
INSERT INTO @Car VALUES ('Escape', 'HP', '300')
INSERT INTO @Car VALUES ('Explorer', 'Type', 'SUV')
INSERT INTO @Car VALUES ('Explorer', 'Transmission', 'Manual')
SELECT DISTINCT Spec.SpecId, Car.CarName
FROM @Specification Spec
INNER JOIN @Car Car
ON Spec.Attribute = Car.Attribute
AND Spec.Value = Car.Value
WHERE Spec.SpecId NOT IN (SELECT Spec.SpecId
FROM @Specification Spec
LEFT OUTER JOIN @Car Car
ON Spec.Attribute = Car.Attribute
AND Spec.Value = Car.Value
WHERE Car.CarName IS NULL)
I have discovered an instance where the Y-AXIS displays a scale incorrectly. The scale values that are displayed do not match the data points shown on the chart itself. I am not using any Minimum or Maximum values in the properties y-axis tab. This does not occur on all charts and seems to be random.
Has anyone encountered this before? Is this a bug in SSRS 2005? Does anyone know of a fix? All sugestions are welcome, thankx!
Greetings everyone, I'm posting this thread with the hope that someone will notice it and might offer me a helping hand regarding one of my problems.I have the database named "DBEXAMPLE" with the table MEMB_INFO that contains two important columns that are named cur_points and points where cur_points column contains the total available points that a member can use/spend on a game and the points column stores the total of the points that a member used so far.So as i plan to wipe all data of the database, i need to keep the member login,password and total points that each member purchased.So i will somehow need to update the cur_points column with the total of current cur_points+points columns and then wipe points column.I've personally asked a friend regarding this and he said that this should be something complicated and it might require php also.Really appreciate if someone could help me regarding this.small schema:Database: DBEXAMPLETable: MEMB_INFOColumn cur_points -> total available points that a member can spendColumn points -> total points that the member has already spentColumn memb___id -> member login aka account idTODO -> update cur_points column with the total of cur_points+points columns for each member(buyer)With best regards.
View 5 Replies View RelatedI'm trying to produce a chart that has both actual data values from a database and matching "line of best fit" plots on the one chart. I have 4 data series, 2 of the actual data values and 2 that represent the values for the "line of best fit". What I want to do is:
Plot the actual values just as data points (joining them with lines in meaningless) and
Plot the "line of best fit" values as a line.
When I edit the data series on the scatter chart, I can see the "Plot data as line" option but it is "greyed" out.
Have I missed something really simple here or is this not possible. I'm using RS2005
Many thanks
Simon
I deleted some records out of an entity, I'd like to keep the Codes as contiguous and incremental, meaning no breaks between the code numbers.I created a business rule and applied it but codes remain the same.
I used the "Default to a generated Value" action, then selected the Code attrib. --Saved.
Then back to the Entity, I applied business rules. But nothing seemed to have happened. As there was no change in codes.
There are no validation errors either.
Hi,
I am using CRM 3.0 and have a requirement to list all the the tables, attributes names and display names and datatypes. Is there any easy way to export this information from the CRM tool or prepare a SQL query that will list the information?
I'm trying to write a stored procedure that will parse XML attributes and populate columns within a DB with the stripped data. I'm a complete novice who prior to this week knew nothing about SQL commands, My understanding at least is that I need to perform a bulk insert.
Example XML file:
<?xml version="1.0" encoding="ISO-8859-1"?>
<!DOCTYPE Asset_Collection SYSTEM "Asset_Collection.dtd">
<Asset_Collection>
<Collection_Metadata
Name="Asset Collection"
Description="Random XML Feed Test"
[Code] ....
Table/Columns which need to be inserting into:
Table:
TABLE_A
Columns:
ProdID
CustomerID
AreaCode
I have a scatter chart in SSRS (SQL Server 2012, Visual Studio 2010) that is producing the following:
There are five data points on there, however the result set I am using has 10 rows (a 'Completed Date' of datetime and a 'Lateness' integer whose values can be positive or negative.
This is the Dataset and the results it produces:
SELECT DISTINCT TOP 10
a.ACTIVITY_NAME As [Activity Name]
, ad.COMPLETED_DATE As [Completed Date]
, ad.DAYS_LATE As [Lateness]
FROM ACTIVITIES a
JOIN ACTIVITY_DATA ad ON ad.ACTIVITY_ID = a.ACTIVITY_ID
[code]....
How can I tell SSRS to show every data point in my chart?
In my SSIS package I have flat files as a source, I have to load numbers of flat files into SQL target table. I am using For each loop container for that. I am doing it correctly. My aim is to validate the source data from all angle before writing it into target sql table. I am using below points to validate the source data , if I found any bad data I am redirecting those data to error output.
To Checking
1. To check whether data type of column.
2. To check whether buisinesskey column null.
Is there any thing which I am missing to validate source data.
Screen shot for reference
Hi All,
I'm looking to mirror 17 servers from geographically dispersed locations around the UK to one central location.
Is there any rule of thumb to take into account server spec, memory requirements, DB Size
E.G. an x64 server can be the mirror for 30 databases provided it has x GB Memory ?
Cheers,
Steve
I have a table with one of the column of xml type. the column contains xml like given below. I want to read this xml from the table and show as below with T-sql query
"EmployeeID" "IndustryDome" "description "
Where Description value comes from the value of AllDome/ITEM/Dome /Description whose Dome equals to IndustryDome value
EX:
1166586 3951LX01 Description10
<GetEmployeeDetails xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<return xmlns="http://applications.apch1.com/webservice/schema/">
<EmployeeID>1166586</EmployeeID>
<BankAccounts>
[Code] ....
Can i have a combination of sources some with Unique Identifiers and some without?
I need to know what happens when I have option “Create Code values automatically” selected for the entity and same time pulling Unique Identifier for other sources in same entity stage table.
When we select option “Create Code values automatically” for the entity we creates, then during load from external source to MDM stage we don’t send any values to MDM stage “Code” field and in next step when we execute the stored procedure to load the data from MDM stage to MDM model, it assigns the Code values to each record in MDM stage and MDM model.
I need to know whether after executing the store procedure, will it assign Code values for only NULL records in MDM stage and not give us any error for the records that already have Code values present in MDM stage.
Data Scenario – Sitemaster entity has two attributes (Region and Country).
Region and Country are both domain-based attributes on separate entities [Region and . The Country entity has a domain based attribute on REGION.
Data relationship looks like this… REGION > COUNTRY
Customer Requirement
Customer would like the MDS UI to filter the Country drop-down options based on the Region selection.
To my knowledge – the only filtering that can be done is based on user security using Hierarchy Membership Permissions.
When trying to process my minding model (using Association Rules) i receive the following errors
Errors and Warnings from Response
Error (Data mining): The 'Items Recommendations' mining model has 64675 attributes. This number of attributes exceeds the attribute limit of 5000 allowed by the current version of the algorithm associated with the mining model.
Errors related to feature availability and configuration: The 'Unlimited data mining attributes.' feature is not included in the 'Standard Edition' SKU.
I searched through the internet and found that a hotfix can be installed to make it work (http://support.microsoft.com/kb/932609). I emailed microsoft and they provided me back with the hotfix and a password.
I tried to install the fix but i couldnt. It needed a prerequisit fix. So i installed SQL Server 2005 Service Pack 2 from (http://support.microsoft.com/?kbid=921896)
Then tried to install the fix for my unlimited attributes problem.
I tried to process my module after installing the fix and restarting my computer, but am still gettin the same error.I created another new model but the problem still exists.
Am i missing something here? Any idea to make it work?
Is there any alternative? I need to use all the elements in my datasource view.
Are there any rules of thumb to apply when entering the Index Fill Factor?
I just want to make sure I'm enter a optimal value.
Using MS SQL 2000.
Thanks!
--PhB
Hi
I came across something like 3-4-5 rule while going through datamining book....but couldn't get from where that rule has been generalized and how it really works....
can anyone explain this rule ?
Thank you
Hi,
How do I extract rules and it's value from a database?
I can extract the rules through view(sys.objects) but where can I get it's content?
Regards
Marcelo Gamba
How can I setup the dbs in sql server so that when I change the data in one table the changes will cascade down to the tables in my other dbs. Therefore, one database would hold a primary key table. If I had 15 other dbs, then I could somehow link them so the data changed in the primary key table of the 1st database would cascade down to the other dbs.
Thanks
I wonder about whether this rule is valid or invalid for nested BEGIN/END statement...
Code:
BEGIN
BEGIN
--Query #1 (blah)...
END
WHILE EXISTS (SELECT TOP 1 * FROM #tmpTblPurchaseRaw)
BEGIN
BEGIN
--Query #2 (blah)...
END
BEGIN
--Query #3 (blah)...
END
END
END
I have no idea if nested BEGIN/END is allowed or not...
I am getting some data from another database and only want to copy sensible data...
How do I write a validation rule in SQL (SQL SERVER 2000) for a fax number, so that it only contains numbers-- i.e digits 0-9
Thanks in advance
I have a query that maps all products to some customer levels. In this case levels 0,5,7 and 8
DELETE FROM ProductCustomerLevel
WHERE CustomerLevelID IN (0, 5, 7, 8)
INSERT ProductCustomerLevel
(
ProductID,
CustomerLevelID
[Code] ....
Basically this maps all products in a database to these customer levels so that they get a discount.
I know need to create a new customer level, example number 9. These will only have 1 or 2 products applied to it.
How can I change the SQL above so that it does not map those products already in Customer Level 9 to levels 0,5,7 and 8
Hi, I have a database which saves data about bus links. I want to provide a information to passenger about price of their journay. The price depends on three factors: starting busstop, ending busstop and type of ticket (full, part - for students and old people, ...).
So I created a table with three foreign key constraints (two for busstops and one for type).
When the busstop is deleted or type of ticket I want all data connected with it to be deleted automatically. I wanted to use cascade deleting.
But I receive a following exception: Introducing FOREIGN KEY constraint 'FK_TicketPrices_BusStops1' on table 'TicketPrices' may cause cycles or multiple cascade paths. Specify ON DELETE NO ACTION or ON UPDATE NO ACTION, or modify other FOREIGN KEY constraints.
How can I achieve my task? Why should it cause cycles or multiple cascade paths?
Hi,I have a table with the following columns:ID INTEGEDR,Name VARCHAR(32),Surname VARCHAR(32),GroupID INTEGER,SubGroupOneID INTEGER,SubGroupTwoID INTEGERHow can I create a rule/default/check which update SubGroupOneID &SubGroupTwoID columns when GroupID for example is equal 15 onMSSQL2000.It is imposible to make changes on client, so I need to checkinserted/updated value of GroupID column and automaticly updateSubGroupOneID & SubGroupTwoID columns.Sincerely,Rustam Bogubaev
View 4 Replies View RelatedI have run into a .. somewhat of a "duh" question. I'm running association rule to run a basket analysis, and I'm trying to get probability of each prediction. I know this is wrong, but how do I go about running PredictProbability on each ProductPurchase prediction?
When I run the below DMX query, I get this error message...
Error (Data mining): the dot expression is not allowed in the context at line 5, column 25. Use sub-SELECT instead.
Thanks in advance...
-Young K
SELECT
t.[AgeGroupName]
, t.[ChildrenStatusName]
, (Predict([Basket Analysis AR].[Training Product], 3)) as [ProductPurchases]
, (PredictProbability([Basket Analysis AR].[Training Product].[ProductName])) as [ProductPurchases]
From
[Basket Analysis AR]
PREDICTION JOIN
OPENQUERY([DM Reports DM],
'SELECT
[AgeGroupName]
, [ChildrenStatusName]
FROM
[dbo].[DM.BasketAnalysis.Contact]
WHERE isTrainingData = 0
') AS t
ON
[Basket Analysis AR].[Age Group Name] = t.[AgeGroupName]
AND [Basket Analysis AR].[Children Status Name] = t.[ChildrenStatusName]