Import Dedicated Records From A Flatfile To A DB
Jun 29, 2006
Hello all
I got a Problem when I try to store Data from a Flatfile to a DB.
The following Error appears in the Progress Control:
An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80004005 Description: "Violation of PRIMARY KEY constraint 'PK_Products_1'. Cannot insert duplicate key in object 'dbo.Products'.".
I have a Flat File Source, and would like to store the needed records in a DB.
In Column 0 in the Flatfile I have multiple Entries with equal Values.
In the DB this Column is set as Primary Key and can only have one Record with the same Value in this Column.
How can I read out (or store) only one Record with the same Value from the Flatfile to store it in the DB?
How can I check if there is a Record from the Flatfile in the DB with the same value in the Primary Key?
How can I change any of the remaining Columns with different Values in the DB to match with the Flatfile?
Thanks in advance for any answer
Chaepp
View 9 Replies
ADVERTISEMENT
Jan 23, 2008
Hello Folks,
Im trying to use Integrations services to import a file on a daily basis.
My flatfile has to different type of rows that comes from an transaction system, it looks like this:
132,1/1/2008,00,123654,text,1,123.00
132,1/1/2008,00,123652,text,1,23.00
132,1/1/2008,00,123655,text,1,3.00
123,1/1/1/2008,01,149.00
1125,1/1/2008,00,123654,text,1,123.00
1125,1/1/2008,00,123652,text,1,23.00
1125,1/1/2008,00,123655,text,1,3.00
1125,1/1/1/2008,01,149.00;Cash;EUR;01;12
After the date you can see that there is two digits number either 00 or 01. The rows also have a different lengthts.
When ever that columns contains 00 the line should be inserted to a special text file, if the columns contains 01 it should to another file.
How can I solve this in a good way?
One of the problems I have is that when I try to import the rows the flat file connections indicates(erros message) that I have partial row in the file which is true since the the rows with the columns content 01 have more fields then the other.
Thanks for you help.
holtis
View 3 Replies
View Related
Dec 28, 2007
The first is the
With every keystroke in the flat file connection manager editor on the filename
the bids system goes out and trys to find the file
this is stupidly slow when using \sqldevelopc$zipcode.txt
The second is creating an import from a flat file to database file using data flow task
when you run the task I get the following error trying to open a c:demozipcode.txt file
[AdventureWorks [30]] Error: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80040E14. An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80040E14 Description: "Could not bulk load because SSIS file mapping object 'GlobalDTSQLIMPORT ' could not be opened. Operating system error code 2(The system cannot find the file specified.). Make sure you are accessing a local server via Windows security.".
I can see there is an issue with BIDS that when you developing it is using the local drive C drive and running it using the servers C drive
I have tried unc paths also and it doesnt work either
Everything in the connection looks fine, i can see the data, the columns etc
I can get this to work using bulk load task and the exact same connection, but not in the data flow task
View 8 Replies
View Related
Jan 23, 2008
Hello Folks,
Im trying to use Integrations services to import a file on a daily basis.
My flatfile has to different type of rows that comes from an transaction system, it looks like this:
132,1/1/2008,00,123654,text,1,123.00
132,1/1/2008,00,123652,text,1,23.00
132,1/1/2008,00,123655,text,1,3.00
123,1/1/1/2008,01,149.00
1125,1/1/2008,00,123654,text,1,123.00
1125,1/1/2008,00,123652,text,1,23.00
1125,1/1/2008,00,123655,text,1,3.00
1125,1/1/1/2008,01,149.00;Cash;EUR;01;12
After the date you can see that there is two digits number either 00 or 01. The rows also have a different lengthts.
When ever that columns contains 00 the line should be inserted to a special text file, if the columns contains 01 it should to another file.
How can I solve this in a good way?
One of the problems I have is that when I try to import the rows the flat file connections indicates(erros message) that I have partial row in the file which is true since the the rows with the columns content 01 have more fields then the other.
Thanks for you help.
holtis
View 8 Replies
View Related
Jul 23, 2005
Hi,I am having trouble importing data from an excel spreadsheet into MSSQL Server 2000 using DTS Wizard. The DTS import process issuccessfull, no errors, but only 50 rows of approx. 1500 rows of dataare imported. I tried to remove 20 rows in the excel spreadsheet inthe interval row 0-50. When i later ran the import, only 30 rows wereimported. I deleted almost every row in the interval 0-50, with theresult of the import having 0 rows imported (but job ransuccessfully). I decided to delete rows 0-100 in the spreadsheet inorder to see if the resolved the problem, but it didn't. As Isuspected something in the excel file to be the cause, I exported theexcel spreadsheeet to a tab delimited textfile, with only one row. ADTS import resulted in importing approx 100 rows, double the amount ofthe textfile, but the other 1400 rows were not imported. The data inthe column is containing numeric values only.Please help me! What could possibly be the cause of DTS skipping rowslike that. DTS doesn't feel reliable at all :/Regards,Björn
View 3 Replies
View Related
Mar 14, 2007
I have a server that i would like to use as a dedicated DB Server. It is running Win Server 2k3 and SQL Server 2005. The typical upstream bandwidth is anywhere from 384Kb/s to 700Kb/s. Is this enough to host multiple SQL Server 2005 DB's?
The DB's will be used for database driven websites.
Please let me know what your thoughts are.
Thanks,
View 1 Replies
View Related
Oct 27, 2006
We are using the following configuration:
2 stand alone Windows 2003 servers 64 bit.
Each have two Gig Network cards configured with different IP addresses.
Each running multiple instances of SQL Server 64.
I am trying to set up a mirror where mirroring traffic between servers will be dedicated to a secondary IP address on the second NIC.
I am also trying to avoid Windows authentication.
Interesting enough: Security Configuration screen suggesting you to use fully qualified TCP addresses and, at the same time, does not give you such an option...
Would someone please point me in the right direction?
Thank you.
View 11 Replies
View Related
Nov 8, 2007
Can anyone point me to MS Best Practices that advise you against installing non-SQL-related applications on a SQL DB server? I feel it to be common sense but I have been asked to justify this with a direct statement from Microsoft. Help?
View 6 Replies
View Related
Nov 18, 2003
Hello All,
I need to import records from one SQL table to another SQL table. Both tables are in the same database. Does anyone know how?
Thank you,
Tina
View 2 Replies
View Related
Oct 23, 2007
How can i import records into SQL Compact
I have the information in Access and also the same information in Excel spreadsheets
I dont have a full version of SQL
What are my choices, is there perhaps someone that makes a import program
View 3 Replies
View Related
May 17, 2007
For single instance installations I try to follow a standard configuration where our log files go to E, data files to F, tempdb to H and backups to G. With a new project we have decided to combine our Test and Dev environments on one server with Test being a default instance of SQL Server and Dev being a named instance. My plan was to stick with the above convention, but I was asked if it would be better for each instance to have dedicated data, log and tempdb drives. The problem is we would not have a dedicated controller for each drive. We would have one controller for the default and named instance data drives, one for the log drives and one for tempdb. The other issue is having to modify some of my admin scripts/procedures to work with different drive letters. That's now a real big deal, but does add more work to my plate.
Do you think it's worth having separate drives for a Test/QA server or is it typically sufficeint to use shared?
Thanks, Dave
View 7 Replies
View Related
Jun 28, 2007
Hi all:
Give me some feedback.
In my company, we have 10+ sql servers and many sql jobs running on every SQl server.
What about the idea that running all jobs on a dedicated sql servers?
at least 2 advantages
1.easy to managment
2.if agent server down, we restart the server not to impact the downtime of the production server.
any suggestion is welcome.
View 1 Replies
View Related
Apr 13, 2007
Hi! need your help.
I have 2 dedicated sql servers xx.xx.x.155 and xx.xx.x.165 (Hosted Environment)
Both SQL Server 2005.
I need to replicate (snapshot daily) data from 155 (Apollo1) to 165 (Apollo2).
and I have many questions as this is the 1st time that I work on this environment.
I created 3 windows accounts and was able to create the publications (Apollo1).
Apollo1ReplSnapAgent
Apollo1ReplDistAgent
Apollo1PublisherUser
However (the distributor) is not visible to Apollo2 so I am unable to subscribe.
Do you have any idea what I need to do to make it visible?
I setup an ftp site to drop the publication (as I was unable to create an UNC Share) and I was hoping to be able to subscribe to it.
Probably it is a matter of permissions...maybe a firewall? should I ask my provider to open a port? do I need to create the same 3 accounts in the Subscriber?
something like Apollo2SubscriberUser?
Please help!
I am lost!
View 5 Replies
View Related
Jul 15, 1999
I am importing records from a text file into the database. The DTS pacakage is dropping some of the records for some reason.
There are no contraints on the table I am importing the records into.
Has anyone experienced this? Any help will be greatly appreciated.
View 1 Replies
View Related
Nov 1, 2004
I want to modify a current DTS job I have. It simply copies records from one table to another after a given date. The catch is I only have read priviliges on the source table. I currently delete all the records from the destination table but have to use Query Analyzer to do it and then run the current package that is just a simple select statement. The statement in the DTS job is something like:
SELECT *
FROM tablename
WHERE date_field > '2004-09-30'
I have been running this multiple times per month since every week or so I want to get more recent data to run a couple of reports. The source table is not ours, it is in a different department, so I can only read off of it. The destination server is ours so I have full priviliges there. I assume the fact that the source server allows me limited access that I cannot run a "delete from tablename" first even though it is on our table? Is there another way to set this up? Thanks again. You guys are the best.
ddave
View 4 Replies
View Related
Oct 15, 2007
All,
I was wondering if anyone could tell me why I could not connect to my server with the DAC when the server CPU utilization for all CPU's was greater than 90%.
BACKGROUND: I have successfully established DAC connections to this server in other periods when the CPU utilization was not pinned so high. I have done so by RDP'ing into the server, opening SSMS and cancelling the request to connect Object Explorer. Finally, I start a new query adding the "Admin" prefix to my server.
As I mentioned, when the server has CPU resource available, this works fine. This indicates that all the groundwork to setting up and using the DAC is laid correctly (i.e. DAC is enabled, I am connected locally, I am logged in with a user role with proper permissions, etc.).
This leads to my question. Why can I not connect to SQL Server 2005 Standard Edition using the DAC when all the (8-way) CPU utilization is at 90% or higher?
I need to make this connection to analyze why the server is so busy.
Any help will be greatly appreciated!!
Thanks,
Tom
View 14 Replies
View Related
Sep 5, 2006
I'm planning an SQL 2005 deployment and would like to know if an SQL server should be kept as dedicated as possible. Should apps that access SQL be kept on other servers? What are the do's and don't's along these lines? The plan here is to install everything that accesses SQL on the same server with the database, but this seems like a bad move to me. Am I right?
Thanks!
Gregg
View 2 Replies
View Related
Mar 11, 2008
I'm thinking of using SQL Server Agent Service for my PDA app. But, I want to use different accounts for SQL Server and SQL Server Agent Service. How can we do this in SQL Server 2005? Do we do this when installing it? Thanks
View 3 Replies
View Related
Jun 7, 2007
Hello,
I am attempting to import a fixed width flat file into a SQL Server table. When I import the file, 704 records don't make it into the table. I know this because if I do the import with MS Access 2003 into an Access table, all of the records from the flat file make it into the table. The flat files have a .txt extension.
The only possible problem that I can see is that some of the rows in the flat file do not contain the full set of characters. When I do the import into SQL Server and create a table on the fly, I still end up 704 records short. There are no error messages during or after the import.
I suppose I could isolate some of the missing records, put them into a different file and try to import them to see what would happen. Other than that, how do I begin to troubleshoot this problem? Are there known issues where records can be dropped from a fixed width file?
Thank you for your help!
cdun2
View 4 Replies
View Related
Feb 6, 2008
Hi,
I have a flat file that contains 2 types of records - Dev and production. The Dev will be noted with an D and the Production with a P. These records are different - The dev records are in a different order and contain different info then the Production. I need to use SSIS to import the data into 2 different SQL Tables. How to do this?
Can any one help me
Thanks in advance
View 3 Replies
View Related
Jul 20, 2005
I need to import an ASCII tab delimited file that has roughly 5,000 recordsonce a week into a SQL Server table. I have researched BCP and it seemslike the way to go. Am I headed in the right direction?Thanks in advance,James
View 3 Replies
View Related
Oct 3, 2005
Hello all,
I hope someone can help with my current issue as i have now spent the last 8
hours trying to solve it! Here goes...
I am currently moving all my websites from our in house set up to a FASTHOSTS
dedicated server. Our in house set up consists of separate server for web
and data and works well even with my recent foray into .Net sites.
I have just moved the most recent of my .Net sites
(http://www.alfadealerads.co.uk/home.aspx) to the FASTHOSTS server and cannot
get the site to connect to the database (this has also been installed on the
FASTHOSTS server). One of the pages that connects to the DB is
http://www.alfadealerads.co.uk/links.aspx, where i am getting the error..
System.Data.SqlClient.SqlException: Cannot open database requested in login
'MasseratiChauffeurDrive'. Login fails.
Login failed for user 'lateral'.
This DB has been restored from back up from the original DB. I have
recreated the user name on the new SQL server and added it to the new DB
also. I am using a webconfig to connect to the DB, the exact same one
that works on my current live server
(http://www.maseratichauffeurdrive.com/home.aspx) only with the obvious change
to the SQL server name.
The only difference i can see in the 2 setups is that the new one has both the
webserver and the SQL server on the one machine and that machine uses domains.
My current in house setup does not use domains and as mentioned before web and
date are 2 separate servers.
Can anyone help? Or even understand what the hell i have written? lol
Thanks in advance
View 3 Replies
View Related
Sep 28, 2007
Hi,Currently, I am developing an application for my client. The issue is that the client has the office and warehouse which are in different locations. He wants the application run and updated all the data in to a database (called warehouse database). Every month, all records in warehouse database will be synchronized to another database which is identical with the warehouse database but it is run in his laptop at the office. Since networking is not an option, I have to create two identical database like this. Does any one know what is the good solution to this? Is there any synchronize feature in SQL Server 2005?Many thanks,
View 1 Replies
View Related
Oct 23, 2014
I get error reports in simple text files like the one below in relatively the same format. The only thing that varies is the number error reasons as there can be any number of error reasons for a file. Usually there is only one but there can be a handful. What is the best way to capture the error description and count of errors no matter how many there are? I want to take these items and update a table I have in sql server 2008r.
Report Message example:
************************************************** *****
Original File Name: some.file.YYYYMMDD.d.incr.02of02.1.dat
Source File ID: file02YYYYMMDD
File Receipt Date: 10/17/2014
Total records received: 1331136
Total records loaded: 1329987
ERROR REASONS
Error code: EBBW002 Error desc: Duplicate Record Total records: 1146
Error code: EABC001 Error desc: Invalid Length Record Total records: 1
Error code: ERRCM10 Error desc: Missing First Name Total records: 2
Total number of Errors encountered during the ODS update processing: 1149
************************************************** *****
View 2 Replies
View Related
Jun 10, 2015
We have a database in SQL Server 2008 R2 with mirroring and want that replication is done by dedicated network.We stop the endpoint and when we try to run the following command, syntax error occurred:
Msg 102, Level 15, State 1, Line 1
Incorrect syntax near '192.168.1.14'.
What is the correct syntax of the command line below?
ALTER ENDPOINT Endpoint_Mirroring AS TCP (LISTENER_IP = '192.168.1.14')
View 2 Replies
View Related
Aug 3, 2004
I currently have a flatfile with a seperate COBOL copybook. I need to be able to import all of it correctly into a db in mssql. Are there any *free programs that will turn the flatfile into some form for insertion into the db? What about csv format into the db?
View 14 Replies
View Related
Jun 30, 2006
Hello again,
As a beginner I would like to replace records in a Table with Records from a Flatfile.
ILet's say, I got a custnum from the Flatfile and I like to replace the whole record of custnum (Primary Key) in a Table.
Can someone give me a hint how to do?
I'm looking forward to an early answer.
Regards
Chaepp
View 3 Replies
View Related
Jul 18, 2007
What is the true difference between FlatFile and File in terms of ConnectionManagerType? If FlatFile is just a subset, why does SSIS architechture need it? What is the reasoning behind?
View 1 Replies
View Related
Mar 29, 2006
The CreatePackage sample provided with SQL Server programmatically creates a package that has a source type of OLEDB to a flat file destination. I am building exactly the opposite, source=flatfile, destination=SQL Server. I expect that will be a more common scenario is using SSIS.
The problem I have is populating the source columns in the FlatFileSource connection manager programmatically. I know it can be done because it happens when you build a package in Visual Studio. What I'd like to know is how to do it programmatically in the object model. How can I interrogate the datasource through the connection manager to find out what columns it has? If I know, I can add the columns to the connection manager. My sample below does this, but it doesn't know the number of columns in the source so that value is hardcoded. I'm guessing there is a better way to do this than what I've got below.
How can I find the number of columns in my source so I can add the columns to the connection manager?
Thanks.
Private Sub AddColumnsToFlatFileConnectionManager()
Dim ff As wrap.IDTSConnectionManagerFlatFile90 = Nothing
For Each cm As ConnectionManager In _Package.Connections
If cm.Name.Equals(_ExternalConnectionID) Then
ff = TryCast(cm.InnerObject, wrap.IDTSConnectionManagerFlatFile90)
DtsConvert.ToConnectionManager90(cm)
End If
Next
If Not ff Is Nothing Then
Dim col As wrap.IDTSConnectionManagerFlatFileColumn90
Dim name As wrap.IDTSName90
Dim Min As Int32 = 0
Dim Max As Int32 = Min + 3 ' *** HARDCODED LIMIT ***
For cols As Integer = Min To Max
col = ff.Columns.Add()
If cols = Max Then
col.ColumnDelimiter = vbCrLf
Else
col.ColumnDelimiter = ","
End If
Dim width As Int32 = 50
Dim DataType As wrap.DataType = wrap.DataType.DT_STR
col.ColumnType = "Delimited"
col.DataType = DataType
col.MaximumWidth = width
col.DataPrecision = 0
col.DataScale = 0
col.ColumnWidth = width
name = TryCast(col, wrap.IDTSName90)
name.Name = "Column " & cols.ToString
Next
End If
End Sub
View 4 Replies
View Related
Jan 28, 2008
Hi,
Iam migration data from a table into a comma delimited flatfile,but i need to specify all the columns within [ " ] in the flatfile
for example
i have a column [Name] the values are John,Mani,Raghu.....
The flat file should be outputted as
"John"
"Mani"
"Raghu"
Is there anyway to do this.Pls help.
Thanks,
SVGP
View 3 Replies
View Related
Mar 27, 2007
Hi all,
my package has a lot of lookups of fact table fields against dimension tables.
i did a redirect row for each lookup error to a flat file.
i wanted to put all lookup mismatch on the same file, but i couldn't do it because there'll be error saying that
The process cannot access the file because it is being used by another process.
i don't think it's practical to have flat files for each lookups. how is this normally done? please help...
thanks!
View 13 Replies
View Related
Jan 3, 2007
This seems so obvious, but is causing me a lot of trouble:
The flat files input to my SSIS packages may contain an empty line between each row of data, or may be "continuous" without any extra empty lines. Here are small examples of the data I'm working with:
With the extra line:
"M","SFH","Single Family Mortgage Loans","Single Fam Mtg"
"M","MFH","Multifamily Mortgage Loans","Multi-Fam Mtg"
"R","MIX","Mixed/Various/Unknown","Mixed/Various"
Without the extra line:
"M","SFH","Single Family Mortgage Loans","Single Fam Mtg"
"M","MFH","Multifamily Mortgage Loans","Multi-Fam Mtg"
"R","MIX","Mixed/Various/Unknown","Mixed/Various"
Seems to me it should be possible to set up a single FlatFile connection to handle both these formats, but it's not easy.
I can easily manually edit the FlatFile connection and change the "RowDelimiter" to either "{CR}{LF}" or "{CR}{LF}{CR}{LF}" and the package runs against the corresponding flat file.
However, when I try to use an expression (@[User::StrRowDelimiter]) to set the RowDelimiter to the value of a package variable, the expression is completely ignored.
I can only change the RowDelimiter property on the FlatFile connection by manually opening the editor on the connection and changing it to match the incoming file.
Why isn't this connection seeing the expression I've set up for its RowDelimiter property?
Thanks for any help!
View 8 Replies
View Related
Sep 10, 2007
Hi,
I have a Conditional Split to FlatFile Destination.
How can I put the result, that goes in the FlatFile Destination, in a variable also (like in Recordset Destination).
Do I have to runs this thing twise (and put the first time in FlatFile Destination and the second time in Recordset Destination)?
Thank you.
View 3 Replies
View Related
