Question About Data Transformation Services : How To Define Myself's Query Without DTS Wizard?
Jul 2, 2007
hi there, I have never use DTS before, now I am reading textbook for some special demand with DTS
the textbook not talk very much for the detail of skills. seems the easy way to finish this query is using DTS wizard.
but my requirement seems can't be done by DTS wizard.
here are my requirement below.
[move online Database to offline Database ]
1. the time of data preserve will have to reference separate firm's history data backup time ( for example, A company used to preserve data 6 months, and B company used to preserve data 12 months and so on..)
2. we will have only 2 kind of preserve time one is 6 months another is 12 months
3. The online DB only keeps 6 months data ( for example, when we do the DTS on 11/1 , we will only keep the data which from 5/1~10/31) , all data have to move to off-line DB except the past 6 months data
4. We will have to reference the history data preserve time to delete data after finished data movement
those requirement looks very diffcult for me because I have never use DTS before , can you please give me a simple example or maybe some article I can reference?
My vendor requires data to be sent in Excel format. Some of my tables have rows over 65,536 so I need to use Excel 2007 (Max of 1,048,576). Right now my data sits in SQL 2000. I am using MS SQL Enterprise Manager 8.0 to prepare the data. Is there some kind of add on or selection I am missing to use DTS to export from SQL to Excel 2007?Thanks in advance.
HiI was told that using DTS will allow me to schedule stored procedures to keep an sql database up to date. For example if a user registers but does not activate the registration, his details will be removed by a stored procedure which is scheduled to run every 24 hours. I use to use the global.asax file to fire a update by using a file containing a the date of the last update and then by adding 24 hours to it, it would execute a SP to delete unwanted data.I have tried to install DTS with no success. I am running the followingVisual web studio expressSQL 2005 Express. (From SQLExpr_exe) and I have told it to install all the extra componentsInstalled SQLEXPR_Toolkit.exe with all its optionsInstalled SQLServer2005_DTS.MSI When I go into the sql server using MS SQL Server Management Studio Express. I cannot see the Data transformation services node. I have also just installed server reports which I had no problems installing.Can somebody please help me.
In my package I am using lookup to get new and similar record. I want to filter the rows for Lookup Reference Data Set by using Variable Value.
I have created variable @[User::CustId] with Int32 datatype, having default value 2 when I am trying to evaluate below query I am getting error
"select CustId,PartNm,LocId,LocTyp from loc where CustId= "+ @[User::CustId]
Error. The Data types "DT_WSTR" and "DT_I4" are incompatible for binary operator "+".
The operand types could not be implicitly cast into compatible types for the operation. To perform this operation , one or both operands need to be explicitly cast with the operator.
I'm using DTS package, a tool to transfer data from a txt file to database(Bulk Insert).
The Bulk Insert task provides an efficient way to copy large amounts of data into a SQL Server table or view.It seems that the Bulk Insert task supports only OLE DB connections for the destination database. But I want to use sql server authentication as OLEDB connection requires windows authentication.
So can the bulk insert be done using SQLServer authentication ? if yes then please help me.
I have given the code snippet below.
Code Sample:
Dim oPackage As New DTS.Package2() Dim oConnection As DTS.Connection Dim oStep As DTS.Step2 Dim oTask As DTS.Task Dim oCustomTask As DTS.BulkInsertTask Try oConnection = oPackage.Connections.New("SQLOLEDB") oStep = oPackage.Steps.New oTask = oPackage.Tasks.New("DTSBulkInsertTask") oCustomTask = oTask.CustomTask With oConnection oConnection.Catalog = "pubs" oConnection.DataSource = "(local)" oConnection.ID = 1 oConnection.UseTrustedConnection = True oConnection.UserID = "Tony Patton" oConnection.Password = "Builder" End With oPackage.Connections.Add(oConnection) oConnection = Nothing With oStep .Name = "GenericPkgStep" .ExecuteInMainThread = True End With With oCustomTask .Name = "GenericPkgTask" .DataFile = "c:dtsauthors.txt" .ConnectionID = 1 .DestinationTableName = "pubs..authors" .FieldTerminator = "|" .RowTerminator = " " End With oStep.TaskName = oCustomTask.Name With oPackage .Steps.Add(oStep) .Tasks.Add(oTask) .FailOnError = True End With oPackage.Execute() Catch ex As Exception MsgBox("Error: " & CStr(Err.Number) & vbCrLf_ & Err.Description, vbExclamation, oPackage.Name) Finally oConnection = Nothing oCustomTask = Nothing oTask = Nothing oStep = Nothing If Not (oPackage Is Nothing) Then oPackage.UnInitialize() End If End Try
I am working on a SSIS Package and I have to do a data transformation but this one is a but tricky. For example
7/6/2015 is my date and when I apply SUBSTRING(ColumnName,5,4) + "-" + SUBSTRING(ColumnName,3,1) + "-" + SUBSTRING(ColumnName, 1,1)
I get 2015-6-7 which is good
But then when the date is like 10/12/2015 I would have to modify my code
My question is, Is their anything I can add to the code to make the single digits have a zero before them so I can use the same code throughout. My updated code would then be
From SQL Server 2014, using SQL Server Data Tools for Visual Studio - BI, I'm trying to edit a Script Component within an SSIS Data Flow Task. The 'Edit Script...' button is enabled and turns a nice shade of blue when moused over, but a click has no effect. Perhaps I'm missing a component of VSTA? Everything else seems to work correctly. What might I be missing?
I am importing the values for field Atype from a .csv file as DT_STR, 13 and I need to fit them into a bit type CType field.
When I write the conditional split ((ISNULL(Atype)?"a":Atype)!=(ISNULL(CType)?"9":CType)) it says that the DT_WSTR and DT_I4 types are incompatible and that I need to explicitly cast with a cast operator. I haven't been able to make it work, how to explicitly cast?
My first post here. I have an Analysis Services Cube and I need to export some of its data into a flat file (a semicolon separated value file) with fixed length columns.
What I have in Integration Services now is:
DataReader Source (ADO.Net conection) Data Transformation (NTEXT -> WSTR) Derived Column (WSTR -> STR, and some ISNULL validation) Flat File Destination (Delimited by ";")
the thing is I have all the measures and calculated measures in the cube formated with format strings and they work fine when I do a query through the SQL server Managment Studio, but in Integration Services before the columns are written into the file they lose its format.
for example, I have a calculated member called "Deuda Total Nacional" that returns something like this +0001234,00 and I need to take off the decimal separator so it is written into the file like this +000123400 , I'm doing it this way
(ISNULL([Deuda Total Nacional]) ? "+00000000000" : REPLACE([Deuda Total Nacional],",",""))
Where is a package visible when running the Data Import/Export wizard, choosing to save a package, and choosing "SQL Server" as the location? When I make an SSIS connection in Management Studio I do not see the package under the "MSDB" node.
I have an error message I was wondering if anyone else has seen and resolved. When I open excel and click on the query wizard on the Data Mining Ribbon, I get an error message that says "Input String not in Correct Format"
After I close this dialog box the 'Welcome to Query .... ' wizard will open, but when I select the Advance button my process aborts due to the input string error.
I have only ran association models on the computer.
The Manage Models and Browse buttons work fine.
The error came up on Friday, but it wasn't there on Thursday. There were no changes done to the machine (no installation of new software).
I have tried reinstalling the add-in.
Thank you all for your help,
Davy
I have pasted the SQL syntax below that I get from the error dialog box after the query wizard closes.
See the end of this message for details on invoking just-in-time (JIT) debugging instead of this dialog box.
************** Exception Text ************** System.NullReferenceException: Object reference not set to an instance of an object. at Microsoft.SqlServer.DataMining.Office.Excel.QueryBuilder.QueryBuilderParameters.UpdateNestedColumnName() at Microsoft.SqlServer.DataMining.Office.Excel.XLClientUIManager.DisplayAdvancedQueryBuilder(Object sender, WizardPageEventArgs e) at Microsoft.SqlServer.DataMining.Office.Excel.Wizard.WizardPageBase.OnWizardPageAdvanced(WizardPageEventArgs e) at Microsoft.SqlServer.DataMining.Office.Excel.Wizard.WizardForm.btnAdvanced_Click(Object sender, EventArgs e) at System.Windows.Forms.Control.OnClick(EventArgs e) at System.Windows.Forms.Button.OnClick(EventArgs e) at System.Windows.Forms.Button.WndProc(Message& m) at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m) at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m) at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)
************** JIT Debugging ************** To enable just-in-time (JIT) debugging, the .config file for this application or computer (machine.config) must have the jitDebugging value set in the system.windows.forms section. The application must also be compiled with debugging enabled.
When JIT debugging is enabled, any unhandled exception will be sent to the JIT debugger registered on the computer rather than be handled by this dialog box.
I have a chart that I am grouping data by Year, Month, day then Hour (this is to view server performance data) - however i need the label for the hour grouping to be in the format of HH:MM and not H:M by using expression
I have a report which will one day display some data from an analysis services cube. my first step is to create a drop down parameter enabling the user to choose the date. I'd like to display only dates that have data, and I'd like it to default to today.
So I've created a dataset that will be the datasource for the dropdown displaying the available non-empty dates, which works fine.
SELECT measures.turnover ON COLUMNS,
nonempty([TBL DIM DATE].[DATE_ONLY].[DATE_ONLY].ALLMEMBERS ) ON ROWS
FROM [Itdev1 Hk]
I've also set the report parameter up to be a queried paramter,and to use the above dataset as it source, with [DATE_ONLY] displayed. and [DATE_ONLY] as the value.
Now, how do I get it to default to the last valid member in the list?
Hi, here am i back ..not sure it's possible to solve it in one query ...there are four tables:1) headquarters---------------hqID (primary key)hqname2) department---------------depID (primary key)depnamehqID3) reports----------repID (p. key)depIDuserID4) users--------userID (p.key)usernameI want to get in one query those four fields:hqID, hqname, depID, depname + varchar(number of distinct users that has made a report foreach department) . This last field is a concatenation.The complexity resides in the fact there is need for a COUNT among other fields ...E.g.: table reports may look like this: repID depID userID 1 1 1 2 1 1 3 1 3 4 3 6 5 4 8This gives:for dep 1: 2 distinct usersfor dep 2: 0for dep 3: 1 distinct userfor dep 4: 1 distinct userThanks for helpTartuffe
Hi,i have a dropdownlist connected to a sqldatasource like this: <asp:DropDownList ID="DropDownList1" runat="server" DataSourceID="SqlDataSource1" DataTextField="Name" DataValueField="Name"> </asp:DropDownList>What i want is to put a particular value on the top of the dd. I can't use ORDER BY.e.g. the table which feeds the dd contains this:RDAhow to put value D on top of the dd?ThanksTartuffe
I am running dts in Sql Server 2005 management studio from Management, Legacy and data Transformation Services.
Once the dts has run, I get this error message "Error Source : Microsoft Data Transformation Services (DTS) Package Error Description : Error accessing Windows Event Log."
I have become frustrated and I am not finding the answers I expect.
Here's the gist, we support both Oracle and SQL for our product and we would like to migrate our Clients who are willing/requesting to go from Oracle to SQL. Seems easy enough.
So, I create a Database in SQL 2005, right click and select "Import Data", Source is Microsoft OLE DB Provider for Oracle and I setup my connection. so far so good.
I create my Destination for SQL Native Client to the Database that I plan on importing into. Still good
Next, I select "Copy data from one or more tables or views". I move on to the next screen and select all of the Objects from a Schema. These are Tables that only relate to our application or in other words, nothing Oracle System wise.
When I get to the end it progresses to about 20% and then throws this error about 300 or so times:
Could not connect source component. Warning 0x80202066: Source - AM_ALERTS [1]: Cannot retrieve the column code page info from the OLE DB provider. If the component supports the "DefaultCodePage" property, the code page from that property will be used. Change the value of the property if the current string code page values are incorrect. If the component does not support the property, the code page from the component's locale ID will be used.
So, I'm thinking "Alright, we can search on this error and I'm sure there's an easy fix." I do some checking and indeed find out that there is a property setting called "AlwaysUseDefaultCodePage" in the OLEDB Data Source Properties. Great! I go back and look at the connection in the Import and .... there's nothing with that property!
Back to the drawing board. I Create a new SSIS package and figure out quickly that the AlwaysUseDefaultCodePage is in there. I can transfter information from the Oracle Source Table to the SQL Server 2005 Destination Table, but it appears to be a one to one thing. Programming this, if I get it to work at all, will take me about 150 hours or so.
This make perfect sense if all you are doing is copying a few columns or maybe one or two objects, but I am talking about 600 + objects with upwards of 2 million rows of data in each!!
This generates 2 questions: 1. If the Import Data Wizard cannot handle this operation on the fly, then why can't the AlwaysUseDefaultCodePage property be shown as part of the connection 2. How do I create and SSIS Package that will copy all of the data from Oracle to SQL Server? The source tables have been created and have the same Schema and Object Names as the Source. I don't want to create a Data Flow Task 600 times.
I want to run some sql code from Query Analyzer. The sql code uses parameters.How do I define the parameter before setting it to a value? I don't want to hardcode the input parameteres. Ex: create @Prim -- This is the line I need help on. Instead of "create", what is -- the proper way of setting this value ? set @Prime=13 SELECT Prime.Quaternion, Gaussian.RegionId, Laplace.Primer,
How do I create value from the dropdown list of each parameter within the "Define Query Parameters" box, which appears when I run a query/dataset from the "Data" tab of the SSRS Report Designer ?
It usually list all the "Parameter Name"s I defined, and has another column for "Parameter Value", but it always shows only <Null> or <Blank> even though I have assigned the report parameters with data (from a Query, not hard coded).
The parameter values work fine when in "Preview" mode, but it would be of convenience to have them during design time.
Is there a way to define a query-filter (Example: WHERE column1 > 5 AND column2 = 'value') in a function?
So I can create a query like this:
SELECT * FROM Table WHERE MyFunction()
I know it's a bit of a strange question, but I'm writing a dynamic software that will have the ability to run Stored Procedures on any database to create some data-checks. Through parametrisation, a user can define for a specific Stored Procedure that some results are no longer necessary in the result-set.
Example: ID - Name - State 1 - Jozef De Veuster - Mad 2 - Piet Husentruut - Not Happy 3 - Jeroen Meus - Angry
Is the result of a Stored Procedure "Show_me_unhappy_persons". But we already know that Jozef De Veuster is ALWAYS Mad, so a user can say: Exclude ID = 1, so it won't appear anymore in the result.
I want to handle this by doing:
SELECT * FROM PeopleStates WHERE --Some stuff-- AND CheckUserExclusions(SomeID) And CheckUserExclusions will translate to "NOT (ID = 1)"
I have a subset of data whose source of truth resides in an Oracle Database. Can I define a SQL Server View to retrieve this subset of data and make it available on the SQL Server side for say Reporting and SSRS? I think I also saw where they tend to prefer User-Defined functions as opposed to a view.
If I define the view using an OPENQUERY, I have noticed that it seems to run FOREVER on the SQL Server side and returns pretty timely if I run in Oracle SQL Developer. Is there anything I can tweak to improve this timeliness or lack there of? And the whole reason I'd like to create a view of this Oracle data is so that I can take advantage of all the views and functions that we have created on the SQL Server side.
We have a issue with a MDS server that have been over us for a couple of days, the original error msg from SQL Server Engine is the one "The query processor could not produce a query plan" but the ones we get on the Excel-Addin are "Sequece contains no elements" or "The value cannot be null" T
• Using Microsoft SQL Server 2012 (SP1) - 11.0.3393.0 (X64) for 6months on this server without issues
• Two weeks ago we started to have 2 errors: "Sequence Contains No Elements" | "The Value Cannot Be Null"
• We are using the last version of Excel Add-in
• We try to reinstall the MDS feature
• If I backup/restore MDS database to other server it works
• We updated to SQL 2012 SP2 + CU4 but the error persisted ...
Looking at the MDSTraceLog we are routed to the this msg
SQL Error Debug Info: Number: 8624, Message: Internal Query Processor Error: The query processor could not produce a query plan. For more information, contact Customer Support Services., Server: bbdvsql03inst01, Proc: udpMetadataEntityGetDetailsXML, Line: 28
At line 28 udpMetadataEntityGetDetailsXML is calling udfMetadataEntityGetDetailsXML … and here is where we stopped
** Error found when try to get data from a entity using Excel add-in ** =================================== Sequence contains no elements ------------------------------ Program Location: at Microsoft.MasterDataServices.AsyncEssentials.AsyncResultBase.EndInvoke() at Microsoft.MasterDataServices.ExcelAddInCore.AsyncProviderBase`1.EndOperation(IAsyncResult ar)
I have two calculated values done by BI Wizard: "Year Over Year Growth" and "Year Over Year Growth %" which are part of my Time Dimension. Those calucalations work fine in a cube browser as long as I use the Time hierarchy connected to those two calculated values. However, when I create a Dataset in Reporting Services and run it, the only values I get for "Year Over Year Growth" and "Year Over Year Growth %" is: "infinity". It is ok with the first year becuase it doesn't have a previous year to compare to but the next year should work have some values other than "infinity". Does anyone experienced something like that? Are there any solutions to that?
We are using the cache transformation in our project , while doing the cache transformation our disk space goes to 0 MB free and SSIS package execution not completes even after 3 hr..Initially we have around 34 GB free space on C: drive .Our server configuration is 64 RAM. We are caching the data from table which contains around 21 million records.We changed the path in properties (“BLOPTempStoragePath”,”BufferTempStoragePath”) of Data Flow task of SSIS in which we are using Cache Transformation.
I have the following 2 fields that are sourced from an Excel spreadsheet
DocNumber - a 10 digit number PostingRow - a number between 1 and 999
I would like to produce a new column that is a concatenation of these two fields, but the PostingRow needs to be a 3 digit number eg. 1000256153-001 ....
In SSIS I use the DQS Cleansing transformation component. I've got a knowledge base (KB) in place and this KB holds various domains and my data source has more input columns than would like to use for a particular clean up operation. I want to use some of the input columns to map against some domains in the KB. It is my understanding that it should be possible to select only the required input columns, but all i can do is select all input columns.
Is it possible to parameter the connection of a Lookup Transformation task - specifically the table/view name? I would like to be able to dynamically set the table that the Lookup Transformation is connecting to at runtime.I've looked into the "Use results of an SQL query" on the connection screen (which correlates to the "SqlCommand" property), but I'm unable to pass in a parameter this way.I've also looked into the SqlCommandParam, but that doesn't allow me to use a parameter in the "FROM" clause of the sql syntax.
