Good morning, everyone. Maybe I'm just having a brain fart, but I'm totally new to SSIS (I dabbled very little with DTS in the 2000 days) and cannot for the love of me figure out how to achieve my goal with it:
My company needs to extract data from a variety of sources; tab-delim files, Access databases, other SQL tables and the like. I know how to do this. However, I need to perform data manipulation queries on this data before I place them into SQL tables, as I want to avoid having umpteen temporary tables that I'll need to add checking for. My predecessor did everything in Access, and has a 76-step process (yikes!) that basically will grab all the data, do some minor manipulation, and plop it into a temp table (this is still in Access, not SQL), then repeat the same thing dozens of times.
To give you an example, here's a sample of what I want to do:
- Extract several columns of data from a tab-delimited file on the local drive. This I know how to do already.
- Perform some data cleanup and manipulation functions on this data (specifically, obtain the lowest value out of three columns, with the added caveat that I make sure it's not zero to begin with). I have the SQL code for this already written.
- Store the results of this data somewhere, so I can pull it and apply additional logic to it; for example, take the lowest value I've retrieved, and update the corresponding column in another database table with it.
Basically, is there any way to avoid the use of dozens of temp tables? There's a lot of data which needs to be pulled in, manipulated, and spit back out to be manipulated by something else a little later on, and the way my predecessor did it was, as I said, to use dozens of Access "Make Table" queries for every minor thing. It's not a big deal if I need to do it, just I'm trying to consolidate the steps needed, as the old way is very inefficient. I've been at this job a month and I'm still trying to wade through all of his queries to discover just what they do, and look into combining several of them.
Forgive the slightly newbish question, but as I stated I've not worked with SSIS really. I'm in the process of learning it better, as I'm sure it can fit our needs.
Looking at BOL for temp tables help, I discover that a local temp table (I want to only have life within my stored proc) SHOULD be visible to all (child) stored procs called by the papa stored proc.
However, the following code works just peachy when I use a GLOBAL temp table (i.e., ##MyTempTbl) but fails when I use a local temp table (i.e., #MyTempTable). Through trial and error, and careful weeding efforts, I know that the error I get on the local version is coming from the xp_sendmail call. The error I get is: ODBC error 208 (42S02) Invalid object name '#MyTempTbl'.
Here is the code that works:SET NOCOUNT ON
CREATE TABLE ##MyTempTbl (SeqNo int identity, MyWords varchar(1000)) INSERT ##MyTempTbl values ('Put your long message here.') INSERT ##MyTempTbl values ('Put your second long message here.') INSERT ##MyTempTbl values ('put your really, really LONG message (yeah, every guy says his message is the longest...whatever!') DECLARE @cmd varchar(256) DECLARE @LargestEventSize int DECLARE @Width int, @Msg varchar(128) SELECT @LargestEventSize = Max(Len(MyWords)) FROM ##MyTempTbl
SET @cmd = 'SELECT Cast(MyWords AS varchar(' + CONVERT(varchar(5), @LargestEventSize) + ')) FROM ##MyTempTbl order by SeqNo' SET @Width = @LargestEventSize + 1 SET @Msg = 'Here is the junk you asked about' + CHAR(13) + '----------------------------' EXECUTE Master.dbo.xp_sendmail 'YoMama@WhoKnows.com', @query = @cmd, @no_header= 'TRUE', @width = @Width, @dbuse = 'MyDB', @subject='none of your darn business', @message= @Msg DROP TABLE ##MyTempTbl
The only thing I change to make it fail is the table name, change it from ##MyTempTbl to #MyTempTbl, and it dashes the email hopes of the stored procedure upon the jagged rocks of electronic despair.
Any insight anyone? Or is BOL just full of...well..."stuff"?
I posted a thread here: http://forums.devshed.com/showthread.php?p=1061405 but was told it might be more useful for me to post something here... should have thought about it to begin with!
The picture is in that thread so do please look at it! http://forums.devshed.com/attachmen...tachmentid=7431 should be the URL.
As you see the top 3 records start and finish at the same time, as do the last 3 records. This can then also be repeated for the next day etc etc. What I want to happen is that the top 3 courses to appear on a single line and the 2nd set of 3 to appear on a single line also.
The main SQL to retrieve the records is as follows: Code:
sqlqry = "SELECT * FROM t_sessions a INNER JOIN t_session_times b ON a.session_id = b.session_ID" sqlqry = sqlqry & " WHERE right(left(convert(varchar,a.session_date,113),11),8)='"+displaym +"'" sqlqry = sqlqry & " AND a.session_supplier_id=61" sqlqry = sqlqry & " AND a.session_status ='A'" sqlqry = sqlqry & " ORDER BY a.session_date, b.session_start, b.session_end"
then while this recordset hasn't come to an eof(end of file) I execute: Code:
query = "SELECT course_name,outline_name" query = query & " FROM t_course_ref" query = query & " LEFT JOIN t_it_outlines ON t_course_ref.course_id=t_it_outlines.course_id" query = query & " WHERE t_course_ref.course_id="+cstr(rs.fields(1).value)
I need to grab data from teradata(using odbc connection).. i have no issues if its just bunch of joins and wheres conditions.. but now i have a challenge. simple scenario, i have to create volatile table, dump data into this and then grab data from this volatile table. (Don't want to modify the query in such a way i don't have to use this volatile table.. its a pretty big query and i have no choice but create bunch of volatile tables, above scenarios is just mentioned on simple 1 volatile table ).
So i created a proc and trying to pass this string into teradata, not sure if it works.. what options i have.. (I dont have a leisure to create proc in terdata and get it executed when ever i want and then grab data from the table. )
I have an application that I am working on that uses some small temptables. I am considering moving them to Table Variables - Would thisbe a performance enhancement?Some background information: The system I am working on has numeroustables but for this exercise there are only three that really matter.Claim, Transaction and Parties.A Claim can have 0 or more transactions.A Claim can have 1 or more parties.A Transaction can have 1 or more parties.A party can have 1 or more claim.A party can have 1 or more transactions. Parties are really many tomany back to Claim and transaction tables.I have three stored procsinsertClaiminsertTransactioninsertPartiesFrom an xml point of view the data looks like this<claim><parties><info />insertClaim takes 3 sets of paramters - All the claim levelinformation (as individual parameters), All the parties on a claim (asone xml parameter), All the transactions on a claim(As one xmlparameter with Parties as part of the xml)insertClaim calls insertParties and passes in the parties xml -insertParties returns a recordset of the newly inserted records.insertClaim then uses that table to join the claim to the parties. Itthen calls insertTransaction and passes the transaction xml into thatsproc.insertTransaciton then inserts the transactions in the xml, and alsocalls insertParties, passing in the XML snippet
I am transferring data from oracle and getting below error message.
I using 4 data flow tasks with in a single control flow and all the 4 tasks quueries same table but populates data in to different sql tables based on the where contidion
[OLE DB Source 1 [853]] Error: An OLE DB error has occurred. Error code: 0x80004005. An OLE DB record is available. Source: "Microsoft OLE DB Provider for Oracle" Hresult: 0x80004005 Description: "ORA-01652: unable to extend temp segment by 64 in tablespace TEMP ".
I have 3 Checkbox list panels that query the DB for the items. Panel nº 2 and 3 need to know selection on panel nº 1. Panels have multiple item selection. Multiple users may use this at the same time and I wanted to have a full separation between the application and the DB. The ASP.net application always uses Stored Procedures to access the DB. Whats the best course of action? Using a permanent 'temp' table on the SQL server? Accomplish everything on the client side?
[Web application being built on ASP.net 3.5 (IIS7) connected to SQL Server 2005)
I am attempting to execute a stored procedure as the sql query for a data transformation from sql into an excel file. The stored procedure I am calling uses temp tables (#tempT1, #tempT2, etc.) to gather results from various calculations. When I try to execute this sp, I get 'Error Source: Microsoft OLE DB Provider for SQL Server Error Description: Invalid Object name "#tempT1"'
Is there a way to make a DTS package call a stored procedure that uses temp tables?
I want to check to see if a temporary table exists before I try creating one but I can't seem to find which sys table or schema collection I check. Any ideas?
-- Add Structural data usp_AddStructural @iEstimateID, 1, 'Structural' usp_AddForming @iEstimateID, 2, 'Forming' ... ... ... GO
Now, a couple of problems, after the table is created and populated, I cannot find it in my list of tables, even after "refreshing".
I checked to ensure that it exists using the query analyzer and it does so I know the table is being created.
Also, I cannot see the table using crystal reports, connecting etc...... Can I not access a temporary table from 3rd party applications? I have crystal reports 7.0 professional.
I am in the process of modifying some stored procedures that currently do not use temp tables. For this modification I am required to make the stored procedures use temp tables. There are several UDF's within this stored procedure that will need to use the temp tables, and this is where in lies the problem. Does anyone know of a work around that would allow UDF's to use temp tables, or does anyone know of alternate methods instead of temp tables that wouldn't involve too much change?
I have a called stored procedure which creates a bunch of temporary tables, inserts data into them, indexes the tables and then returns to the main calling SP. In the main stored procedure I then do SELECTs from the temporary tables. My problem is I keep getting invalid object errors on the temporary tables: Invalid object name '#temp_table1'
The stored procedure is in a test environment. In the SELECT I tried a prefix of database owner (my logon) as well as "dbo." but still get the error. Any suggestions as to what I am doing wrong would be much appreciated.
In these two tables im just to bring the data back where the two DesignID's dont match. Im gettin an error
Server: Msg 107, Level 16, State 3, Line 1 The column prefix '#ttTopSellers' does not match with a table name or alias name used in the query.
Declare @CustomerID as VARCHAR(25) Set @CustomerID = 'DELCOZ01-10'
/*Figure the designs that stores carry*/ Select Design.Description, Item.DesignID, CustomerClassificationID, CustomerID, Region.[ID] as RegionID, Region.Name Into #ttDesign From Mas.dbo.Item Item Inner Join MAS.dbo.Style Style on Item.StyleID = Style.[ID] Inner Join MAS.dbo.Line Line on Style.LineID = Line.[ID] Inner Join MAS.dbo.Design Design on Item.DesignID = Design.[ID] Inner Join Mas.dbo.DesignRegionIndex DRI on Design.[ID] = DRI.DesignID Inner Join MAS.dbo.Region Region on DRI.RegionID = Region.[ID] Inner Join MAS.dbo.CustomerClassificationRegionIndex CRI on Region.[ID] = CRI.RegionID Inner Join MAS.dbo.CustomerClassification CC on CRI.CustomerClassificationID = CC.[ID]
Where @CustomerID = CustomerID Group By Design.Description, Item.DesignID, CustomerClassificationID, CustomerID, Region.[ID], Region.Name
/*This finds the top retail sales globally*/ Select Top 10 Sum(Sales) as Sales, DesignID, Design.[Description] Into #ttTopSellers From Reporting.dbo.RetailSales_ByStore_ByCustomer_ByDay_ByItem DI Inner Join Mas.dbo.Item Item on DI.ItemNumber = Item.ItemNumber Inner Join MAS.dbo.Style Style on Item.StyleID = Style.[ID] Inner Join MAS.dbo.Line Line on Style.LineID = Line.[ID] Inner Join MAS.dbo.Design Design on Item.DesignID = Design.[ID] Where [Date] >= Month(getdate())-12 and DesignID <> 0 Group By DesignID, Design.[Description] Order by Sum(Sales) Desc
Select * From #ttDesign Where #ttDesign.DesignID <> #ttTopSellers.DesignID
Why cant I use the same temptable name i a stored procedure after i have droped it?I use the Pubs database for the test case.CREATE PROCEDURE spFulltUttrekk ASSELECT *INTO #tempFROM JobsSELECT *FROM #tempDROP TABLE #tempSELECT *INTO #tempFROM EmployeeSELECT *FROM #temp
I have a stored procedure which contains a temp table called #NamesThis has n rows of values which are peoples names (they are varchars)i.e#NamesRickRobFrankI have a table called tblPeople.tblPeopleid Name Telephone1 Ric 012334213452 Robert 0321120931233 Paul 123 123 123 123 I want to find all the people in tblPeople whose names are like those in the temp table #NamesHow do I do this?
Please tell me there's something I haven't set. I've done several tests now. If your final return in a stored proc is from a temp table (ala #mytable ) the system cannot read the schema - for that matter, it won't run at all, complaining that the object #mytable doesn't exists. It can't possible be that temp tables aren't allowed in procs used by SQLDatasource - please tell me what I am doing wrong. This proc, when fed to a sqldatasource, fails in the designer with #temp does not exists. CREATE PROCEDURE dbo.repTest_TempASBEGIN CREATE TABLE #Temp( [iTestID] uniqueidentifier, [bTest] [bit], [cTest] [varchar] ) INSERT INTO #TempSELECT *FROM tTest SELECT *FROM #Temp END
I have been researching some performance problems in a very large application and I have a couple of questions about temp tables. (SQL 7.0 SP2)
I have one large procedure that I have been using as a test case. Originally this procedure was a cursor with lots of processing steps involving writing to, reading from and deleting in temp tables inside the cursor. I remember reading that temp tables inside a cursor were a potential performance problem, so I rewrote the procedure, replacing the cursor with a While Loop.
Doing this showed no increase in performance. Since Profiler was showing .5 second duration times on statements in the procedure accessing the temp tables I tested some more. I moved all the create statements to the top of the procedure, as I know these statements after processing steps can cause recompiles to happen. Still no performance increase.
Finally I replaced all the temp tables with actual tables, just to see what would happen. With no other changes the performance increased by more than 500%.
Can someone give me some clues as to what is happening here, because if this is a symptom of something I don't understand, the potential performance problems from other places where temp tables are similarly used in the application are enormous.
The problem is we need local tables in Access for SubReports (can't use store procedures as record source for this), therefore trying to create temp table and link to Access for each report instance.
When we create the ## type table in tempdb this cannot be linked from Access (cannot be seen via DSN). The only other option we can find is doing CREATE TABLE tempdb.dbo.[tablename] but this requires the user to have admin permissions and therefore be 'dbo' on tempdb and not 'guest'.
Is there any way to test if a table exists in the temp dB. In a procedure I create table called ##Test but I want to test if it exists before it's created and if it does I want to delete it. (It has to be a ## table because it's in a procedure being called in another procedure depending on an IF statement). The reason I want to do this is that I have multiple users and Occasionally they get the ... There is already an object named '##Test' in the database. error
I am creating an sp, in which I check for the existence of a global temp table (using the exists) statement. If the Exists returns a false, I move on to processing without the temp table. If it returns a true, I utilize the temp table to do some inserts. I create the temp table when my application first starts up. The problem that I am facing is that the check for the temp table`s existence seems to be failing. Is there any other way to check for the existence of a global temp table??
I am creating a dts package which copies data via transform data task from a sql sever connection to an excel spreadsheet connection. The transform data task uses a query that goes like this.
create table #temp1(CName varchar(10))
Insert #temp1 select CName from tbl1
--tbl1 is in a user DB called SALES
create table #temp2(VName varchar(10))
Insert #temp2 select CName from #temp1
select * from #temp2
--I need the resultset of the last select statement into an excel spreadsheet. The problem is I can't even create the dts package..it shows up with an error saying that #temp1 is an invalid object....I am guessing that since the default database for the transform data task has been selected as SALES it is trying to locate the the #temp1 or #temp2 table in that DB instead of looking in the TEMPDB....My question is..is there any workaround for this problem.
I have 4 temporary tables that hold criteria selected through a report wizard. I've created a SQL statement and used the four tables in my WHERE/ AND clauses but the results retuned are not being filtered correctly.
Would somebody be kind enough to help me out please.
To briefly summarise, I have created a SQL statement that returns all rows in my recordset, I now need to implement some additional SQL to filter the recordset using my temporary tables, which contain the filters as follows:
(1) Temp table 1 (##tblTempAssetFilt) is mandatory and will always contain at least one row. (2) Temp table 2 (##tblTempRepairTypeFilter) is optional and may never contain any rows. If this is the case then I have no reason to filter my resultset against this table. (3) Temp table 3 (##tblTempRepairFilter) / Temp table 4 (##tblTempRepairElementFilter) are both optional, only one of these tables will contain data at one time. Again, as an optional filter the tables may never contain rows, and thus need to be ignored.
I have the following SQL, can somebody tell me how I would go about filtering the recordset using the temporary tables. The creation of the temporary tables occurs at the beginning so will always exist even when no rows have been inserted.
SELECT * FROM tblActualWork [ActualWork] JOIN tblRepair [Repair] ON ActualWork.intRepairID = Repair.intRepairID JOIN tblRepairElement [RepairElement] ON Repair.intRepairElementID = RepairElement.intRepairElementID JOIN tblRepairType [RepairType] ON Repair.intRepairTypeID = RepairType.intRepairTypeID JOIN tblAsset [Asset] ON ActualWork.intAssetID = Asset.intAssetID WHERE ActualWork.intAssetID IN (Select intAssetID From ##tblTempAssetFilter) AND Repair.intRepairTypeID IN (Select intRepairTypeID From ##tblTempRepairTypeFilter) AND Repair.intRepairID IN (Select intRepairID From ##tblTempRepairFilter) AND Repair.intRepairElementID IN (Select intRepairElementID From ##tblTempRepairElementFilter)
Any filtering must be based on the recordset filtered by temp table 1, which is a mandatory filter. Rows will always exist in this temp table.
Please help, not having much joy with this. Many thanks.
I have 2 temporary tables from a previous operation, Tab1 and Tab2, with the same dimensions. How do I create a third table Tab3 with the same dimensions containing the the combined rows of the 2 previous tables? TIA!
how to update a temp table with another temp table for example I have #tempdashboard that has info in it.I also have #tempappscount of which I want to insert the info inside it into #tempdashboard...I believe the column in #tempdashboad is DailyAppsInINT,
I did a query that gave me the number of daily apps created. I was told to create #tempappscount and my query to dump into #tempappsaccount. Then dump the info from #tempappsaccount into #tempdashboard.
I have two temporary tables, both contain field 'BranchID'.
First table @TEMPNET returns 10 different BranchIDs, but the second table @TEMPCREDIT returns only 7 of those IDs, so when I run select from both tables 'WHERE N.BranchID = C.BranchID2' the result is blank. If I limit both tables to the IDs that are the same in both sets, it works.
How can I get ALL the results from table1 AND the results from table2 which match table1?
DECLARE @TEMPNET TABLE ( BranchID int , NetSales decimal(18,4) , Cost decimal(18,4) , SaleMonthNbr int , SaleMonth varchar(15))
Hi how can I delete a local temp table? I know a golbal is deleted by if exists(select * from tempdb..sysobjects where name='##MyTemGlobalTable' and type='U')drop table ##MyTemGlobalTable'
--************************************************************* --SET SQL STMT BASED ON SQL TYPE PASSED IN --*************************************************************/
SELECT Distinct a.station_nbr, c.station_name, a.group_code, a.beg_eff_date As beg_eff_date_DGC, a.end_eff_date As End_eff_date_DGC from TIES_Gathering.dbo.station_group a Inner Join TIES_Gathering.dbo.station c on a.station_nbr = c.station_nbr Inner Join TIES_Gathering.dbo.station_type d on a.station_nbr = d.station_nbr Inner Join TIES_Gathering.dbo.group_detail e on a.group_code = e.group_code Where ((e.group_type = 'NPN')or(e.group_type = 'WTF')or(e.group_type = 'NTF'))and ((d.type_code = 'WHM')or (d.type_code = 'CRP')) and (a.pipeline_code = '100000') Order by a.station_nbr, a.group_code
INSERT INTO #TP_DGC(station_nbr,station_name,group_code,beg_eff_date_DGC,end_eff_date_DGC) values (@stationnumber, @stationame,@groupcode,@begdateG,@endeffdateG)
Select Distinct a.station_nbr,b.station_name,a.group_code, a.beg_eff_date As beg_eff_date_SES, a.end_eff_date As End_eff_date_SES From TIES_Marketing.dbo.station_group a Inner Join TIES_Marketing.dbo.station b on a.station_nbr = b.station_nbr Inner Join TIES_Marketing.dbo.station_type c on a.station_nbr = c.station_nbr Inner Join TIES_Marketing.dbo.group_detail e on a.group_code = e.group_code Where ((c.type_code = 'WHM')or (c.type_code = 'CRP')) and (a.pipeline_code = '100001') Order by a.station_nbr, a.group_code
INSERT INTO #TP_SES(station_nbr,station_name,group_code,beg_eff_date_SES,end_eff_date_SES) values (@stationnumber, @stationame,@groupcode,@begdateM,@endeffdateM)
---------------compare both tables-------------------- Begin
Select #TP_SES.station_nbr, #TP_SES.station_name, #TP_SES.station_group, #TP_DGC.beg_eff_date As beg_eff_date_DGC, #TP_DGC.end_eff_date As End_eff_date_DGC, #TP_SES.beg_eff_date As beg_eff_date_SES,#TP_SES.end_eff_date As End_eff_date_SES from #TP_SES left outer join #TP_DGC on station_nbr=station_nb end
