Unique Addresses From Table. SSIS Newbie Please Answer
Apr 11, 2008
Hello ALL,
My source columns are:
Customer Record DESTINATION TABLE
Name Address_line_1
sex city
PID state
legal_street zip_code
legal_state ID
legal_city
mail_street
mail_city
mail_state
mail_zip
My client needs. All Unique Addresses from the Source.
.
That is the duplicates should be removed and uniques should be available.
Please let me know first what he wants then also tell me what to use in the data flow and how to start. I want to understand the logic. Note: Two persons might have the same address or One address might belong to 2 or more people.
This is what my client said.
what I want to achieve is to load a text file that has email addreses from disk and using the email addresses in the text file look it up against the email addresses in the database table then once matched delete all the users in the table whose email address were in the text file.
I also want to update some users using a different text file.
Hi all, I'm having a bit of an issue with this query. I'm not that familiar with MS SQL so please forgive me. I'm trying to select the bad email addresses in a table so I can remove them or correct them but I'm not getting any results that are helpfull. Bellow are some of the queries I've tried
select email from person where email != "*@*" returned all addresses plus null fields.
select email from person where email not like "%@%" (this didn't work at all)
And a few other that are similar Could somebody please kick me in the right direction.
I am new to SSIS. I am trying to install just the SSIS in one machine("SSIS Machine") and just the DB Engine ("SQL Server Machine") in another machine. What I am trying to do is, separating the SSIS service and packages from the Database Engine and trying to run in in another machine. I have few questions on this topic. I searched on this forum but I couldn't find a concrete answer to those questions. Forgive me if it already answered/asked multiple times.
1. When I install SSIS in "SSIS Machine", do I need to install client components also in the same machine? 2. I already established this setup (SSIS with client components in one machine and SQL Server in another) but when I tried to connect to the SSIS thro' Management Studio from Sql Server machine, I keep getting "Access Denied" error. Is it possible to connect to SSIS server from another machine (using Management studio)? I tried the DCOM security permission options I found in the internet(I don't have domain id so I gave "Everyone" full access) but still I get the same error. Any help would be appriciated. 3. Do I need 2 SQL Server Licenses (Enterprise) if I go with this environment? 4. Is it possible to configure SQL Job to run SSIS installed in another machine?
Please forgive my ignorance in advance, as I am just becoming familar SSIS. I am sure this is possible, but I would like a definite answer and perhaps a good tutorial on this. I have data in SQL 2005 that needs to reach a third party Oracle DB. The party doesnt give us direct write access, however they have provided a webservice to insert data. Is this feasible using SSIS... SQL 2005 ----> 3rd party web service -----> Oracle? If so can someone point me to a tutorial that explains this process? Thanks a bunch.
I am tasked with loading the data from one SQL Server 2005 database into another of the same type and schema on the same server. I'd like to use this task to get my feet wet in terms of using SSIS. Can someone suggest a way of doing this?
Following is what I would like to do, so I can keep updating my central SQL Server database with latest updates from the field. I like to use SSIS 2005 to create a package that could do this. Any help to get me started would be appreciated. I need some help soon, pls give me something to get started. APpreciate it. Thanks.
Open connection and read client location table on the local SQL Server database called PODO For each location id in the table do the following: Store locationid/clientid in a variable called CLLOC_ID Construct file name with mdb extension and store in a variable MDB_FILE Establish connection to the data import folder Search for that MDB_FILE in the folder on the file system If there is a file where match = true then do this: 1) Open the access database 2) Read and import the data from the customer experience table 3) Write that data to the SQL Server tables where location = CLLOC_ID 4) Exit process IF there is no match, exit process Keep looping until all the client ids/loc ids are read from the SQL Server client location table.
I'm new to SSIS (and quite new to SQL Server). I have a process which I'd like to automize via SSIS - just don't know how and couldn't figure it out yet by playing around with the program. Shouldn't be too difficult though.
First of all, that's the process as I do it now:
1) Load several flatfile sources (dumps of SQL tables) into an SQL database. 2) Add identifier rows (to some tables), set the primary and foreign keys so the database is "recreated" and I can work on it. 3) Do several simple transformations, aggregations and selects across tables and finally write a new table containing information for reporting stuff.
I succeded in loading flatfiles within the data flow view, doing some transformations and saving the output to a flatfile. What I didn't find out: how can I "recreate" the database enabling me to perform "SELECT/FROM/WHERE" statements across tables? Will I have to write the imported files to tables within a db (how?) or can I avoid this step?
A little guide (newbie friendly) would be great help!
I used the SSIS import/export wizard to create a single data flow of about a dozen source/destination tables. The tables however have some foreign key constraints between them. Executing the wizard-created package causes foreign key violations. Is there a way to control the order of the table load process within a single data flow? Or is the preferred way to set up several data flows that are ordered via the control flow tab?
Hello, I have a file I need to retrieve nightly from a web server, it's not a SOAP request. It's just a wget call to a remote php script that returns a csv file.
I can't seem to find a way to do this, I see ftp, and web service, but no web get type of function. Is this not available in IS as provided from microsoft, and something I'm going to have to write? Seems like this type of thing would be allmost as common as ftp.
If anyone has any suggestions, any help would be be greatly appreciated. Right now, it looks like I'm going to have to use an external wget.exe to get the file.
I've exposed my data (that exists in a proprietary format) with the ADO.NET provider interfaces (IDbConnection, IDataReader, IDbDataAdapter and IDbCommand). I can't seem to find any examples of how to get Integrated Services to hookup to this .NET code in my class library. Is it possible? My goal is for this provider to be both a destination and a source and for others to be able use IS to manipulate the data however they want.
HI All. I'm trying to tweak the Transfer Logins task to exclude Windows Logins that are local to the Server (e.g. servernameusername) which obviously can't be transferred off the server. Annoying that we have a couple of local Logins on this system instead of all Domain Groups, but we're stuck with them due to firewall issues, and a policy excluding SQL Logins.
My idea is to create a text file as part of my Package that lists Logins to be Excluded From the Transfer - I think I then need to create a New File Connection to the Text File as a Connection Manager, then somehow get that data into a Variable, and then use an Expression to populate the 'LoginsList' Collection from syslogins where loginame not equal to logins in my textfilevariable?
Or maybe I'm over complicating this, and there's an easier solution? Lots of info in Books Online about Expressions and Variables, but having trouble finding examples that I can use. As a DBA, this is my first foray into SSIS, and as you can possibly tell, I'm floundering....
The following is a list of questions that I have not been able to obtain concrete answers. I am probably missing something: 1) ReadWriteVariables -- can the updated value for a ReadWriteVariable be accessed within the same data flow? It appears not as I think the PostExecute() fires at the completion of the data flow not the end of the Script Component. Secondarily, the Script Component is a non-blocking transformation so the component does not "see" the end of the pipeline prior to sending data down stream.
2) Record Count -- Because of #1 above, How could you calculate a record count for a data stream? It does not appear that one can calculate the number of records for a data stream within a data flow and then access the count from within the same data flow.
3) FinishOutputs() -- Is the concept of FinishOutputs() applicable to Script Component Destinations? Asked another way, is FinishOutputs() executed at the end of the data stream regardless of whether there are "real" outputs for the component? I can create a "Dummy" output to create FinishOutputs() but is this ok?
4) Script Component -- It appears that the Script Component Source, Transformation or Destination are really defined based on the columns defined in "Inputs and Outputs". Can you convert an Source script component to a transformation script component by simply adding an Output?
Sorry for these basic questions but I am not getting it completely. As you can tell...
I have a transaction table having about 40 crore rows in source. It don't have timestamp and unique key columns. It have only Bill_month and Bill_Year columns. Actually for loading this table into staging I have added a new datetime column by adding default bill_date as 01. Then
* First we delete last 3 month data from staging tables. * Get last 3 months data from source table. * Load that 3 months data from source to staging table.Â
We do this because we only get update for last three months data. Now I have to include this transaction table as Fact table in DW. What will be the best practice for loading the fact table by picking data form staging table. Also we have to look up with dimensions for Foreign Keys.Â
* Should I implement the same method of deleting last 3 months records and loading them again.Â
need help how to archiv table to another table with unique number for all rows once + date time (not the second only day time +minute) i need whan i insert to the another table add 2 more fields (unique number , date_time )
this is the table 1 i select from ID fname new_date val_holiday ----------------------------------------------------
this is the table 2 i insert into ---------------------------------- ID fname new_date val_holiday unique number date_time --------------------------------------------------------------------------------------------------------------------
for evry archiv table to another table (insert) i need to get a unique number + date time (not the second only day time +minute)
next insert ...... ID fname new_date val_holiday unique number date_time --------------------------------------------------------------------------------------------------------------------
next insert ...... ID fname new_date val_holiday unique number date_time --------------------------------------------------------------------------------------------------------------------
Here's a problem description I read on another post but I have the same issue:
I am trying to Import Data from a csv file into a SQL Sever table. The Data being imported has a Unique Identifier in it but it is being considered as DT_WSTR datatype in SSIS
When i directly try to import, it gives an error:
The conversion returned status value 2 and status text "The value could not be converted because of a potential loss of data.".
I Tried to use Data Conversion to convert it to DT_GUID and import into SQL Server table but it again throws the above error. Anyone has a clue?
--Table 1 "Employee" CREATE TABLE [MyCompany].[Employee]( [EmployeeGID] [int] IDENTITY(1,1) NOT NULL, [BranchFID] [int] NOT NULL, [FirstName] [varchar](50) NOT NULL, [MiddleName] [varchar](50) NOT NULL, [LastName] [varchar](50) NOT NULL, CONSTRAINT [PK_Employee] PRIMARY KEY CLUSTERED ( [EmployeeGID] ) GO ALTER TABLE [MyCompany].[Employee] WITH CHECK ADD CONSTRAINT [FK_Employee_BranchFID] FOREIGN KEY([BranchFID]) REFERENCES [myCompany].[Branch] ([BranchGID]) GO ALTER TABLE [MyCompany].[Employee] CHECK CONSTRAINT [FK_Employee_BranchFID]
-- Table 2 "Branch" CREATE TABLE [Mycompany].[Branch]( [BranchGID] [int] IDENTITY(1,1) NOT NULL, [BranchName] [varchar](50) NOT NULL, [City] [varchar](50) NOT NULL, [ManagerFID] [int] NOT NULL, CONSTRAINT [PK_Branch] PRIMARY KEY CLUSTERED ( [BranchGID] ) GO ALTER TABLE [MyCompany].[Branch] WITH CHECK ADD CONSTRAINT [FK_Branch_ManagerFID] FOREIGN KEY([ManagerFID]) REFERENCES [MyCompany].[Employee] ([EmployeeGID]) GO ALTER TABLE [MyCompany].[Branch] CHECK CONSTRAINT [FK_Branch_ManagerFID]
--Foreign IDs = FID --generated IDs = GID Then I try a simple single row DELETE
DELETE FROM MyCompany.Employee WHERE EmployeeGID= 39
Well this might look like a very basic error: I get this Error after trying to delete something from Table €œEmployee€?
The DELETE statement conflicted with the REFERENCE constraint "FK_Branch_ManagerFID". The conflict occurred in database "MyDatabase", table "myCompany.Branch", column 'ManagerFID'.
Yes what I€™ve been doing is to deactivate the foreign key constraint, in both tables when performing these kinds of operations, same thing if I try to delete a €œBranch€? entry, basically each entry in €œbranch€? and €œEmployee€? is child of each other which makes things more complicated.
My question is, is there a simple way to overcome this obstacle without having to deactivate the foreign key constraints every time or a good way to prevent this from happening in the first place? Is this when I have to use €œON DELETE CASCADE€? or something?
I'm trying to connect to a local instance of SQL Server using an IP address and its not working. When I looked up the connect string, it shows only the IP address replacing the Server name, not the instance name, i.e. Server=127.0.0.1 - how does it know which instance to use? Can I use the 127.0.0.1 loopback address for the server (as its on a local machine)? Do I have to use 'Network Library = DBMSSOCN' to tell the client to use TCP/IP? (TCP/IP is showing as an enabled protocol in the Client Network Utility)
I need to allow for many addresses in my database for each client that is entered. I'm looking for suggestions on how I should handle this. I will need to do mailings to the clients and will need a way to select the address record that the client indicates as the mailing place.
A quick question for you all, We will be changing the IP address of our SQL 7's server sometime next week. Does any1 know of any issues/problems that could occur if we do so?
I got got a pile of bad email addresses to update in our SQL database.I know how to do this for individual records using the update command.Is there a way to execute an update using the list of addresses in anexcel spreadsheet or some other form of list? This kind of scriptingis new territory for me.
I want to replicate between 2 servers not on the same network. One has to be addressed by IP. Currently replication does not support IPs - anyone have a workaround. Will this be resolved in the future? Seems this is a pretty common scenario if your db is on a hosted site.
I've worked with informix for a very long time and this is my first aproach to sql server. I have an extremely simple design for a "small" database and at this moment I'm creating the tables, in informix I can assign a first extent and next extent size to the creation of the table so if your volume and growth analisys is good you can basically be sure that you will allways have contigous space on disk for your table. I'm readin BOL to see if I have that feature here but can't seem to find anything similar. Does that mean that my table data will be "fragmented" all over the primary and secondary files every time I load into them? Would it be a good practice to simulate the extents by creating a secondary file for each table with the size I require?
I am very new to db. so pardon me for some stupid questions.
I have 3 tables Table1, Table2, and table3. Table1 and table2 has 5 columns each and table3 has 4 columns. i have 6 or 7 csv files (these number of files can change) uploaded to a upload directory. these csv filename contains name of the table. for example table1_ab.csv, table1_cd.csv, table2_ef.csv, table2_gh.csv, table3.aa.csv, table3_bb.csv.
a person goes to a web page and clicks a button. This is what the button should do. 1) Check how many files are there in the upload directory. 2) use bulk insert to individually insert each csv file into their respective table (the name of the table is in file name). 3) Copy the csv file into backup directory and delete the file from upload directory.
I am using vb.net. any help would me much appriciated.
Sorry guys I know this is easy but I've been looking for about an hour for a straight forward explanation. I want to store a user's wish list while they browse the site, then they can send me an enquiry populated with their choices. Basically, a shopping cart! I thought of using session variables and string manipulations but I am more comfortable with DB queries. a simple 4 column table would cover everything. SQL server and VBScript Thanks M
I apologize if this has been posted/asked before... a search of the Forum for keyword "NULL" doesn't return any result (not even a 0 found).
When I import an XLS file into SQL2000, everything goes fine, except that every column after my data has <NULL> in it. How do I prevent this from happening, or fix it?
Error = [Microsoft][SQL Native Client]An error has occurred while establishing a connection to the server. When connecting to SQL Server 2005, this failure may be caused by the fact that under the default settings SQL Server does not allow remote connections.
This is on a single machine running Widows XP Home.
Hi there I sorry if I have placed this query in the wrong place. I'm getting to grips with ASP.net 2, slowly but surely! When i try to access my site which uses a Sql Server 2005 express DB i am receiving the following error:
Server Error in '/jarebu/site1' Application.
Database 'd:hostingmemberasangaApp_DataASPNETDB.mdf' already exists.Could not attach file 'd:hostingmemberjarebusite1App_DataASPNETDB.MDF' as database 'ASPNETDB'. Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code. Exception Details: System.Data.SqlClient.SqlException: Database 'd:hostingmemberasangaApp_DataASPNETDB.mdf' already exists.Could not attach file 'd:hostingmemberjarebusite1App_DataASPNETDB.MDF' as database 'ASPNETDB'.Source Error:
An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below. Stack Trace:
Version Information: Microsoft .NET Framework Version:2.0.50727.42; ASP.NET Version:2.0.50727.210
This is the connection string that I am using: <connectionStrings> <add name="ConnectionString" connectionString="Data Source=.SQLEXPRESS;AttachDbFilename=|DataDirectory|ASPNETDB.MDF;Integrated Security=True;Initial Catalog=ASPNETDB;User Instance=True" providerName="System.Data.SqlClient"/> </connectionStrings>
The database is definitly in the folder that the error message relates to. What I'm finding confusing is that the connection string seems to be finding "aranga"s database. Is it something daft?
