is there a link in this forum that speks about datamodeling or datawarehouse. iam looking for some help regarding the model that i have to build. i am not able to find a relation between different dimensions in terms of time which would be th e key to populate the fact table.
any help appreciated
Can someone please explain this statement: At the logical level where there can be any number of entities in a relationship while physically you define relationships between two tables.
thx,
Kat
Are there any pitfalls when naming SQL Server 7.0 objects with underscores, caps, etc.?
View 1 Replies View RelatedHello everybody,
I'm new to Sql.
Can anyone explain me
Data Modeling.
Thanks.
shiak bee.
Hi,
does anyone know of any data modeling tools that support or intend to support SQL 7.0 ?
I`d appreciate any comments ...
Kris Klasen
Act. Manager Data Warehouse Project
Information Management Branch
Department of Education, Training, Community and Cultural Development
E-mail: Kris.Klasen@Central.Tased.Edu.Au
http://www.tased.edu.au
Tel: 03 6233 6900
Fax: 03 6233 6969
73 Murray Street
2nd Floor
Hobart 7000
Tasmania
I'm facing the next problem:I have a table with two columns (among others) modeling category andsubcategory data for each row. I need to summarize info on this twocolumns, but with the next specs:1.- Some grouping is only on the category column.2.- Some other grouping consider the two columns.The values for the two columns come from external source, i.e. I haveno means to know the precise universe of data (I suppose soon or laterwe'll have a sufficient sample of data, but for now it's not thecase). So, I would like to have a grouping table so it's not necessaryto insert a row for every pair of category and subcategory (althoughit would be the best approach for the sake of design's simplicity). AsI don't know every possible combination, I would prefer something like'this category is a - no matter the subcategory', and 'this othercategory + subcategory is b'. Let's go with a sample:--------------------------------------------------------Create Table B ( -- groupings ----categ char(8),subcateg char(5),what_group char(10))-- All rows with 432 code are cat. A ----Insert B ( '00000432', ' ', 'Category A' )-- All rows with 636 code are cat. C except when subcat is 8552 (cat.B) ----Insert B ( '00000636', '08552', 'Category B' )Insert B ( '00000636', ' ', 'Category C' )-- Some data ----Create Table A ( -- data ----categ char(8),subcateg char(5))Insert A ( '00000432', '01322' )Insert A ( '00000432', '01222' )Insert A ( '00000432', '01100' )Insert A ( '00000432', ' ' )Insert A ( '00000636', '08552' )Insert A ( '00000636', '08552' )Insert A ( '00000636', '01100' )Insert A ( '00000636', ' ' )Insert A ( '00000636', '01111' )-- The query like:Select b.what_group, count(*) as cntFrom aLeft Join bOn /* ? ? ? ? */-- Should give ---what_group cnt-------------- ----------Category A 4Category B 2Category C 3-------------------------------------------------------------------It would be easier knowing all the pairs categ - subcateg. If I don'tknow them, is a good idea to model the grouping table as I've donewith rows in B?TIA,DiegoBcn, Spain
View 1 Replies View RelatedI need a recommendation on a data modeling tool that can be used with a data warehouse. My warehouse is running SQL 2012.
Here is my challenge: Most of the tables in the warehouse do not have primary keys and none of the tables have foreign keys on them. However, there are indexes and unique keys/indexes on the tables. I am looking for a tool that I can create virtual relationships on how the data is related, so it is visually easier for the ETL developers to write the code.
I have looked at both ER/Studio 11 and ERwin 9.6. Neither of them do it exactly the way I want it too. However, ER/Studio is pretty close.
I have a case study requesting to create an ER diagram, with the attributes listed in each entity. The data I have is an Excel Spreadsheet listing:
CustomerName
PurchaseDate
Destination
Airline
Flight#
departDate
DapartTime
ArriveTime
Hotel
CheckIn
CheckOut
Car Rental
Pickup
Return
The case is related to travel agency that specializes in booking interesting vacations for people who are single. Note that the travellers have a variety of travel bookings: some may rent a car and drive to the hotel at their destination, others may be staying with friends or relatives at their destination, while others will need flights, hotel and car rentals in their booking.
creation of the Tables and their attributes to Normalize the model to 3NF.
I am running the Office Professional Plus 2007 RTM with all options enabled and SQL 2005 Developer Edition on my local box. Based on the system requirements listed on the download page for the Office 2007 Data Mining Add-In, I've also verified that I have the correct CTP of SQL 2005 SP2 and that .Net 2.0 Framework is installed. Finally, I've verified that my local instance of SQL Server is configured correctly to allow temporary data mining models.
In Excel, all of the Table Analysis tools seem to work fine, and most of the options on the Data Mining ribbon also work; however, all of the options under "Data Modeling" on the Data Mining ribbon return the following error when I try to use them:
"Could not load file or assembly 'Microsoft.DataWarehouse, Version=9.0.242.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91' or one of its dependents. The system cannot find the file specified."
I've tried uninstalling everything and reinstalling, but I still get this error when I try to use the Data Modeling options.
Right now, I'm only working against the sample data provided when the data mining add-in is installed.
Any ideas on how to resolve this issue?
I am having a problem when modeling a Foreign Key in an "Operations" table. This table holds all information on customers ´s applications and withdrawals.
Here is the structure:
CustomerID int, SourceID int, Value decimal (16,2), OperationDate datetime
Well the problem is that SourceID sometimes might be NULL depending on how the record was inserted. So its kind of cumbersome to define it as an FK, since it can be null...To get things worse, this SourceID might point to more than 1 table (depending on the CustomerType it will point to SourceA table or SourceB table)...
How should this be modeled?
Is there a good approach to modelling many heterogeneous entity typeswith that have some attributes in common?Say I have entities "employees" which share some attibutes (e.g.firstname, lastname, dateofbirth) but some subsets of employees (e.g.physicians, janitors, nurses, ambulance drivers) may have additionalattributes that do not apply to all employees. Physicians may haveattributes specialty and date of board certification, ambulancedrivers may have a drivers license id, janitors may havepreferredbroomtype and so on.There are many employee subtypes and more can be dynamically addedafter the application is deployed so it's obviously no good to keepadding attributes to the employees table because most attributes willbe NULL (since janitors are never doctors at the same time).The only solution I found for this is a generalization hiearchy whereyou have the employee table with all generic attributes and then youadd tables for each new employee subtype as necessary. The subtypetables share the primary key of the employee table. The employee tablehas a "discriminator" field that allows you to figure out whichsubtype table to load for a particular entity.This solution does not seem to scale since for each value of"discriminator" I need to perform a join with a different table. Whatif I need to retrieve 1,000 employees at once?Is that possible to obtain a single ResultSet with one SQL statementSQL?Or do you I need to iterate look at the discriminator and thenperform the appropriate join? If this kind of iteration is necessarythen obviously this generalization hierarchy approach does not work inpracticesince it would be painfully slow.Is there a better approach to modelling these kind of heterogeneousentities with shared attributes that does not involve creating a tablefor each new employee type or having sparce tables (mostly filled withNULLS)I guess another approach would be to use name/value pairs but thatwould make reporting really ugly.Seems like a very common problem. Any ideas? Is this a fundamentallimitation of SQL?Thanks!- robert
View 13 Replies View RelatedGood day.
i'm a new person here and not that familiar with T-SQL...
the question is: is there any buil-in functions or special libraries in T-SQL that can help to generate correlated random variables with non-normal distribution function?
It would be also good if someone could advice if there is any application (statistical programm or non-microsoft developed library) that can deal with MS SQL and has modeling and forecasting capabilities...
thanks in advance
I'm trying to determine the best approach to model tables for a registration form that will be used temporarily and then taken offline once the event is over. I'd like to either model the tables so that they were reusable for other registration forms or perhaps use another method to store the data, maybe using XML or some other method if that is possible?? I'm not sure.
Different registration forms would have different input fields thereby requiring different table structures. It seems inefficient to create tables that will only be used temporarily and then no longer used. So, I would need to remember to delete the tables after they have been used or they would just take up space.
The basic requirement for this registration form is to allow the user to fill out the required fields, submit, get a registration reciept confirming their registration and allow the administrators for the event to pull the data weekly or daily into an excel spreadsheet.
I can create a flat table and a stored procedure that inserts the data. I can also write the dts package that exports the data to an excel file, which would require my intervention. I'd like to have something more automatic without my intervention.
Any suggestions would be appreciated. I'm not sure the best approach for this. Is using tables the best way to go even if the tables aren't re-usable? Requiring my attention to delete afterwards.
What are the options to generate the excel reports without my intervention?
Thanks in advance for any help.
Hello,
I need a tool that will let me model a SQL Server 2005 database and then generate the tables, constraints, etc. from the model. I've never used a modeling tool so my knowledge is quite limited. I don't need to model or reverse engineer an application - my sole concern is on the SQL Server database side. I'm not concerned if the tool integrates with Visual Studio. And, of course, price is one consideration.
Are there any good tools that I should look at?
Thanks,
BCB
Hi all
Probably not the right forum - pointers would be appreciated - but I'll give it a try anyway:
I'm in the process of designing a relational database to be used in a BI scenario - ie. dimension and fact tables. The data will eventually be used to feed cubes in Analysis services, however end users will probably be allowed to run reports aginst views of the relational database.
I'm currently looking at the employee dimensions and my first try would designate AGE as a SCD Type 2 attribute. As a result every employee gets at least one new record every year as AGE increases. Given that BIRTHDATE is specified should I drop AGE from the tables and recreate it as a computed attribute in database views and/or cubes?
Regards, Steen
Hello--
A question has come up around the following situation where a number of analysts will building data mining models in a specific analysis services databases.
- There is one AS DB for each modeling "project" and the analysts assigned to work on the "project" are grouped together in Windows Security Groups.
- The analysts are only allowed to access the AS DB for their project. To support this security model, we've implemented scripts to create the AS DB for the "project" and then a Role is created within the AS DB called "Administrator" and the members of this role are the members of the corresponding Windows Security Group.
The AS DB, role and membership are created by a machine "admin". After the AS DB is created, it appears that the only way an analyst can build models using Visual Studio (Business Intelligence Workbench) in the AS DB while maintaining the security model is to do the following:
- Run Visual Studio (or BI Workbench)
- Select File -> Open -> Analysis Services Database, then specify the database that they have access to.
In this "online" modeling environment, things are working fine. The question is -- is it possible for an analyst to create an Analysis Services Project in Visual Studio and "bind" it to already-created AS DB? This doesn't appear to work, but I may be missing something.
Thanks,
- Paul
Our business model involves a lot of dates and the business owners frequently want reports based on each of these different dates. For example in any given order there are as follows:
- Order created date
- Client due date
- Order first payment date (an order can have multiple payments)
- Order fully paid date
- Date assigned to vendor
- Vendor return date
- Date delivered to client
On top of that we have other areas of the business, the data from which ties into the above. Here we have more dates e.g.
- Date vendor recruited
- Date vendor reviewed
At any given point the manager may want a report based on any of these dates. For example;
- Product type by order creation date (fiscal year / month)
- Product type by first payment date  (fiscal year / month)
- Product type by client due date (fiscal year / month)
and so forth. I have been asked to create a report using all of the above on at least one occasion, many of them far more frequently. At the moment I have created a standard date table and then duplicated that for each type of date that I need however this is becoming excruciating to work with as I have approximately 10-12 date tables in my data model. Is there a better way of doing this now, in Excel 2013? If not, is there an improvement in 2016 that may make life easier?Â
Hi, all here,
Thank you very much for your kind attention.
I am wondering if it is possible to use SSIS to sample data set to training set and test set directly to my data mining models without saving them somewhere as occupying too much space? Really need guidance for that.
Thank you very much in advance for any help.
With best regards,
Yours sincerely,
After testing out the application i write on the local pc. I deploy it to the webserver to test it out. I get this error.
System.Data.SqlClient.SqlException: The conversion of a char data type to a
datetime data type resulted in an out-of-range datetime value.
Notes: all pages that have this error either has a repeater or datagrid which load data when page loading.
At first I thought the problem is with the date, but then I can see
that some other pages that has datagrid ( that has a date field) work
just fine.
anyone having this problem before?? hopefully you guys can help.
Thanks,
I have used both data readers and data adapters(with datasets) in the projects that I have worked on. I am trying to get some clarification on when I should be using which one. I think I am doing this correctly but I want to be sure I am developing good habits.
As the name might suggest, it seems like a datareader is for only reading data. I have read that the data adapter and dataset are for a disconnected architecture. Or, that they can be used for this type of set up. I have been using the data adapter and datasets when writing to a database and the datareader when reading from a database.
Is this how these should be used? Is the data reader the best choice for reading data? Am I doing this the optimal way from a performance stand point?
......................................................thanks in advance
We already integrated different client data to MDS with MS Excel plugin, now we want to push back updated or new added record to source database. is it possible do using MDS? Do we have any background sync process to which automatically sync data to and from subscriber and MDS?
View 4 Replies View RelatedWhen I enter over 4000 chars in any ntext field in my SQL Server 2005 database (directly in the database and through the application) I get an error saying that the data could not be updated because string or binary data would be truncated.Has anyone ever seen this? I cannot figure out what is causing it, ntext should be able to hold a lot more data that this...
View 7 Replies View RelatedI have a requirement to implement CDC for 50+ tables to implement incremental data changes warehouse/reporting rather than exporting the whole table data. The largest table is having more than half a billion records.
The warehouse use a daily copy of OLTP db (daily DB refresh). How can I accomplish this. Is there a downside in implementing CDC just for the sake of taking incremental changes on the tables?
Is there any performance impact if we enable CDC on OLTP db?
Can we make use of the CDC tables on the environment we do daily db refresh so that the queries don't hit OLTP database?
What is the best way to implement CDC to take incremental changes for reporting.
Hi,This is driving me nuts, I have a table that stores notes regarding anoperation in an IMAGE data type field in MS SQL Server 2000.I can read and write no problem using Access using the StrConv function andI can Update the field correctly in T-SQL using:DECLARE @ptrval varbinary(16)SELECT @ptrval = TEXTPTR(BITS_data)FROM mytable_BINARY WHERE ID = 'RB215'WRITETEXT OPERATION_BINARY.BITS @ptrval 'My notes for this operation'However, I just can not seem to be able to convert back to text theinformation once it is stored using T-SQL.My selects keep returning bin data.How to do this! Thanks for your help.SD
View 1 Replies View RelatedI'm using Script Component to load data into Oracle DB due to the poor performance issue. Now, I found it will missing some data during the transmission. Please see the screenshot below:Â
SQL Server：
Oracle:
DDL:
create table Person
(
BusinessEntityID Integer,
FirstName nvarchar2(50),
MiddleName nvarchar2(50),
LastName nvarchar2(50)
);
Result:
I follow up this article:Â [URL] ....
VB Script:Â
Imports System
Imports System.Data
Imports System.Math
Imports Microsoft.SqlServer.Dts.Pipeline.Wrapper
Imports Microsoft.SqlServer.Dts.Runtime.Wrapper
[Code] ..........
Hi all, i got this error:
[DTS.Pipeline] Error: "component "Excel Source" (1)" failed validation and returned validation status "VS_NEEDSNEWMETADATA".
and also this:
[Excel Source [1]] Warning: The external metadata column collection is out of synchronization with the data source columns. The column "Fiscal Week" needs to be updated in the external metadata column collection. The column "Fiscal Year" needs to be updated in the external metadata column collection. The column "1st level" needs to be added to the external metadata column collection. The column "2nd level" needs to be added to the external metadata column collection. The column "3rd level" needs to be added to the external metadata column collection. The "external metadata column "1st Level" (16745)" needs to be removed from the external metadata column collection. The "external metadata column "3rd Level" (16609)" needs to be removed from the external metadata column collection. The "external metadata column "2nd Level" (16272)" needs to be removed from the external metadata column collection.
I tried going data flow->excel connection->advanced editor for excel source-> input and output properties and tried to refresh the columns affected.
It seems that somehow the 3 columns are not read in from the source file?
ans alslo fiscal year, fiscal week is not set up up properly in my data destination?
anyone faced such errors before?
Thanks
When I execute the below stored procedure I get the error that "Arithmetic overflow error converting expression to data type int".
USE [FileSharing]
GO
/****** Object: StoredProcedure [dbo].[xlaAFSsp_reports] Script Date: 24.07.2015 17:04:10 ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
[Code] .....
Msg 8115, Level 16, State 2, Procedure xlaAFSsp_reports, Line 25
Arithmetic overflow error converting expression to data type int.
The statement has been terminated.
(1 row(s) affected)
is there a step by step paper to get there? here is what i need to consider. I Iwill have many customers that will need their own set of records and access pages "branded for their company" each customer will have many clients. I am hosting this application on a windows 2003 server with SQL 2005 server enterprise.
I am using windows authentication, I have created a username in windows, then i added the windows user in SQL management studio in security, granted "DB Read" and "DB write" and again under the database security tab. still from the web authentication fails. i must be nissing a step or two?
I expect to set up a username for each database as i setup new customers.
RE: XML Data source .. Expression? Variable? Connection? Error: unable to read the XML data.
I want my XML Data source to be an expression as i will be looping through a directory of xml files.
I don't see the expression property or the connection property??
I tried setting the XMLData property to @[User::filename], but that results in:
Information: 0x40043006 at Load XML Files, DTS.Pipeline: Prepare for Execute phase is beginning.
Error: 0xC02090D0 at Load XML Files, XML Source [108]: The component "XML Source" (108) was unable to read the XML data.
Error: 0xC0047019 at Load XML Files, DTS.Pipeline: component "XML Source" (108) failed the prepare phase and returned error code 0xC02090D0.
Information: 0x4004300B at Load XML Files, DTS.Pipeline: "component "OLE DB Destination" (341)" wrote 0 rows.
Task failed: Load XML Files
Information: 0xC002F30E at Bad, File System Task: File or directory "d:jcpxmlLoadjcp2.xml.bad" was deleted.
Warning: 0x80019002 at Package: The Execution method succeeded, but the number of errors raised (2) reached the maximum allowed (1); resulting in failure. This occurs when the number of errors reaches the number specified in MaximumErrorCount. Change the MaximumErrorCount or fix the errors.
SSIS package "Package.dtsx" finished: Failure.
The program '[3312] Package.dtsx: DTS' has exited with code 0 (0x0).
Thanks for any help or information.
I setup this package to import data from a Sharepoint list to a SQL Server data table. The primary key of my SQL table is mapped to the Title column of my Sharepoint list. There is a possibility that duplicate values will be entered in the Title field of the Sharepoint list. So when importing data into my table via SSIS, my package always error-out when there it comes across duplicate values. how you others have managed data integrity when importing from a Sharepoint list with the Title column being mapped to the primary key of a table.
View 4 Replies View RelatedHello,
I am wondering what conversion rules apply, when a string, which contains a number, is saved to a SQL Server 2005 into a column of type decimal.
This is the code I€™m using (C++):
CString cValue = "0.75"
_variant_t vtFieldValue;
vtFieldValue = _variant_t(cValue)
pRecordSet->Fields->Item["MyColumn"]->Value = vtFieldValue;
"pRecordSet" is an ADO recordset. The database column "MyColumn" is of type "decimal(19,10)".
The most important question for me is, if the regional settings of the database server or the regional settings of the client PC are considered during the conversion from the string to the decimal value. For example in standard French regional settings the "." would not be recognized as decimal separator.
I am also wondering if the language of the database instance, in which this data is saved, is considered during this conversion or any other settings of this database instance.
So my general question is: Does anybody know exactly what rules apply during the above mentioned conversion?
Thank you for your help.
Regards,
Volker
I've question about how to handle structural datamodel changes in a datasource of PowerPivot. Suppose I'm developing a starmodel in SQL Server and sometimes a datatype changes or a name of a field changes in a table. It seems to me that PowerPivot handle this not gracefully as Analysis MD does (mostly). I received an error because of a wrong fieldname or even no error when a dattype changes in PowerPivot. Is this common or do I something wrong here. Does this mean that every time the datamodel changes the PowerPivot should be recreated? Or am I missing the clue here?
View 6 Replies View RelatedHi everyone,
I have to extract, dayly a list of contacts on a exchange server in a table on our EDW on sql server 2005. Is it possible to get the information directly from a dataflow or i will have to developpe a script task ?
Need help desperatly !!!