Importing Csv Files To SQL Server Express
Nov 22, 2007
Hello,
I receive 4 .csv file downloads periodically (3 times per day) via email from our corporate database. I open each file in Excel, save as excel files, import to Access, replace the previous tables...run action queries, generate reports combined with production data from CSRs and supervisors. ALL DONE MANUALLY!!!
Here's where I am now:
-I've recently switched over to SQL Server Express.
-Used SSMA to bring tables over from previous Access database.
-Exercised the option to LINK these tables to the original Access database.(I'll explain why in a moment)
-Created ADPs for front end data entry use.
-Imported old Access database forms into the new ADPs previously used in Access.
-Connected to the new Server Express.
-I've eliminated my concurrent user problems by doing this.
HOWEVER, I am still bound to using old Access/Jet database to generate reports based on periodic downloads from corporate .csv files.
Here's the question:
What is the best way to import the csv files being sent to me via email into SQL Server Express? I've tried DTS. Seems to me you can't save AND actually use the packages later since it's the Express edition..... Importing manually 4 files, 3 times per day is a very tedious option I'd like to avoid. Any ideas?
Oh, by the way. Corporate has told me they would be willing to post the files to an FTP site instead of emailing the files. That's about as much help as I'm going to get from them. Can SQL Server Express be set up to run stored procedures (triggers) on a hot folder?
Thanks for your help.
David
View 3 Replies
ADVERTISEMENT
Jul 19, 2006
Hi everyone,
I have to write a program in VB to receive the read data from a RFID reader for my graduation project.The problem is I am not a computer science student so I have only general info on programming.
I created my DB in VB express but I couldn't find out how to send the read data (that will be either in XML or ASCII format) to my database...The read data will be transferred to my computer by the RFID reader's software but after that I don'T know how to transfer it to my DB.As I know I have to use commands like read.xml etc,but no idea how write the complete program.
I checked the forum and couldn't find the answer,sorry if someone already answered my question and I missed it.
Thanks...
Can
View 2 Replies
View Related
Sep 20, 2004
Hi,
Can anyone help? Need to upload a text file to a sql database but keep getting errors.
I'm creating a page that will allow users to to bulk import and update to a MsSql database. The users provide a text file every so often with new/update information. So i want to use a DTS package to transform the infomation, and create a table in the database, then check against existing/non existing records, if the record exist, update it, if not insert it. I'm using Visual Studio.Net, ASP.Net and coding in VB.Net.
Anyone know where i can find documentation/code regarding the above?
I will be greatful for any help.
View 5 Replies
View Related
Jul 23, 2005
HI!I am importing .txt files. How can i check the errors? I have created alog file, but the problem is that i lose some characters.I import for example:CodeABCFZHJHNfrom a text file, but sometimes Code can be 4 caracters longI import this 3 characters long now. When i add the same structuredtext file with some rows lenght 4, it skips the last character, but iget nothing in the log file.please helpxgirl
View 4 Replies
View Related
Sep 26, 2005
I work for a company that makes heat transfers for the imprinted apparel market. We're developing a database of merchandise images for all of our non-design inventory. Using Access we're going to be inserting thumbnails of psd (photoshop) files. We're wondering if there is any way to import multiple psd's into the sql server database into matching records like matching a column named "filename" and the actual filename of the file without having to upload each file individually. We want to be able to dump the files from the database of the matching records, also. This way, once our catalog designer has found which designs they need to put into the new catalog, it will dump the psd's for us. The same for our staffer who does color separations.
Any suggestions out there? If you need me to post further of what we're trying here, I will. This is for the bossman.
View 4 Replies
View Related
Mar 22, 2007
Hi all,
Sorry if this is in the wrong forum but i didn't know if i should put it in the Data Mining forum.
My problem is two fold.
Scenario:
I have 11 text files with standard data format. They range in size from 20megs to 300+ megs (containing 100's of thousands of lines of data)
I build a simple DTS Package to automate the importing of the text files into a database table with the same format as the text files.
Running the files through my DTS produced errors on all files. The error was :
Too many columns found in the current row; non-whitespace characters were found after the last defined column's data.
The error also gave an approximate position of the problem row. So i opened up a text editor that can handle large files and took out about 5000 records before and after the problem area. So i now had a new test file that had 5000 records and contained what "should" be the problem row. I ran this file and it imported correctly. It did not produce the error that i got previously. However it did import things incorrectly. Going through only one of the files i found several rows (over two dozen) that had half the data on one row, then the other half on the other row. Obviouslly there is a problem here with the way the data was exported. I wanted to know if anyone knows of any programs i can get that will analyze these text files for inconsistencies. I have considered writing one in C# but want to see if there is something already out there that can help me.
Basically i need to look at these text files and find Rows that are "incomplete"
the files come from a unix based system, use LF for row delims and Chr(29) for Column delimiters.
any ideas?
thanks,
mcm
View 4 Replies
View Related
May 12, 2007
Greetings,
I'm having a tough time importing some of my legacy database into sql.
I have a number of dbase (IV) files I need to get into SQL. I have tried building a SSIS package with either an foxpro oledb connection or a jet 4.0 one, none of them work bec. of inconsistencies in the data format in my tables (e.g. date fields, etc).
I have tried to save the .dbfs as excel 2007 files, taking advantage of the larger space that comes with '07. Problem is you can't use the import/export wizard with 2007 for some reason and I haven't been able to create a package with the access 12 oledb as I have read.
I have to get some crucial data out of that old system and into the new one and I can't seem to be able to import them properly.
Any hints on what I should do ? (maybe I'm doing something awfully wrong)
Thank you for taking the time to answer my question,
Val
View 12 Replies
View Related
Feb 15, 2008
An error has occurred while establishing a connection to the server. When connecting to SQL Server 2005, this failure may be caused by the fact that under the default settings SQL Server does not allow remote connections. (provider: SQL Network Interfaces, error: 26 - Error Locating Server/Instance Specified)
Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code. Exception Details: System.Data.SqlClient.SqlException: An error has occurred while establishing a connection to the server. When connecting to SQL Server 2005, this failure may be caused by the fact that under the default settings SQL Server does not allow remote connections. (provider: SQL Network Interfaces, error: 26 - Error Locating Server/Instance Specified)Source Error:
here is my web.config file:<?xml version="1.0"?><!--
Note: As an alternative to hand editing this file you can use the
web admin tool to configure settings for your application. Use
the Website->Asp.Net Configuration option in Visual Studio.
A full list of settings and comments can be found in
machine.config.comments usually located in
WindowsMicrosoft.NetFrameworkv2.xConfig
--><configuration xmlns="http://schemas.microsoft.com/.NetConfiguration/v2.0">
<appSettings><add key="EmailFrom" value="webmaster@reaganpower.com"/>
<add key="EmailSubject" value="File Ready for Download!"/><add key="SmtpServer" value=""/>
<add key="MailUser" value=""/><add key="MailPassword" value=""/>
<add key="MailPort" value="25"/><add key="EmailFormatSelected" value="Text"/>
<add key="PageTitle" value="Send It Now!"/><add key="ShareLocalFolderPath" value="H:MIS DepartmentIntranetSendItNowFileStorage"/>
<add key="httpDownloadPath" value="http://misfs/SendItNow/ContentFiles/"/>
<!--
<add key="CurrentTheme" value="CleanBlue" />-->
<add key="CurrentTheme" value="CleanOrange"/></appSettings>
<connectionStrings>
<add name="ConnectionString" connectionString="Data Source=(local)SqlExpress;AttachDbFilename=|DataDirectory|FileShareDB.mdf;Integrated Security=True;User Instance=True"providerName="System.Data.SqlClient"/>
</connectionStrings><system.web><!--
Set compilation debug="true" to insert debugging
symbols into the compiled page. Because this
affects performance, set this value to true only
during development.
Visual Basic options:
Set strict="true" to disallow all data type conversions
where data loss can occur.
Set explicit="true" to force declaration of all variables.-->
<identity impersonate="false"/>
<roleManager enabled="true"/><compilation debug="true" strict="false" explicit="true">
<assemblies><add assembly="System.Design, Version=2.0.0.0, Culture=neutral, PublicKeyToken=B03F5F7F11D50A3A"/></assemblies></compilation>
<pages><namespaces>
<clear/><add namespace="System"/>
<add namespace="System.Collections"/><add namespace="System.Collections.Specialized"/>
<add namespace="System.Configuration"/><add namespace="System.Text"/>
<add namespace="System.Text.RegularExpressions"/><add namespace="System.Web"/>
<add namespace="System.Web.Caching"/><add namespace="System.Web.SessionState"/>
<add namespace="System.Web.Security"/><add namespace="System.Web.Profile"/>
<add namespace="System.Web.UI"/><add namespace="System.Web.UI.WebControls"/>
<add namespace="System.Web.UI.WebControls.WebParts"/><add namespace="System.Web.UI.HtmlControls"/>
</namespaces></pages>
<!--
The <authentication> section enables configuration
of the security authentication mode used by
ASP.NET to identify an incoming user. -->
<authentication mode="Forms"/>
<!--
The <customErrors> section enables configuration
of what to do if/when an unhandled error occurs
during the execution of a request. Specifically,
it enables developers to configure html error pages
to be displayed in place of a error stack trace.
<customErrors mode="RemoteOnly" defaultRedirect="GenericErrorPage.htm">
<error statusCode="403" redirect="NoAccess.htm" />
<error statusCode="404" redirect="FileNotFound.htm" />
</customErrors>-->
</system.web>
</configuration>
View 2 Replies
View Related
Aug 23, 2006
Really confused ... I created 2 tables in a VB Express program (which went well). Problem is that I can't see/find those tables with MSME. I don't really understand how to connect the MSME to where the tables are. I created the tables under My DocumentsData and they are DBapp.mdf + DBapp_log.LDF and the tables are Items and Preferences. The VB application can "see" the tables OK.
Do I need to move the tables to C:Program FilesMicrosoft SQL ServerMSQL.1MSQLDATA (like the Adventureworks installation which I haven't done) or what?
Please help a newbie
Thanks
George
View 3 Replies
View Related
Apr 1, 2006
I am having trouble importing a database script into SQL Server Express. The script is from the Web Applications book from the Microsoft MCSD exams. My installation of SQL Server Express appears to be fine as I can login and create/delete databases through it. When I attempt to import the database script I get a variety of errors as follows:Msg 911, Level 16, State 1, Server ORTHANCSQLEXPRESS, Line 1Could not locate entry in sysdatabases for database 'Contacts'. No entry found with that name. Make sure that the name is entered correctly.Msg 15401, Level 16, State 1, Server ORTHANCSQLEXPRESS, Line 1Windows NT user or group 'ORTHANCSQLEXPRESSASPNET' not found. Check the name again.Msg 15410, Level 11, State 1, Server ORTHANCSQLEXPRESS, Procedure sp_addrolemember, Line 80User or role 'aspnet' does not exist in this database.Msg 15410, Level 11, State 1, Server ORTHANCSQLEXPRESS, Procedure sp_addrolemember, Line 80User or role 'aspnet' does not exist in this database.I have added access permissions for the ASPNET user account on the SQL Server Express folder but that doesn't appear to have helped. I'm not sure if the error about contacts.mdf means that I should have a contacts.mdf created as a result of the script or whether it should be there to begin with. I found this thread http://forums.asp.net/thread/433540.aspx but it didn't solve my problem.Apologies for the length of the post. Any help at all is greatly appreciated!
View 4 Replies
View Related
Oct 29, 2006
hi. this is a beginner question as i am new to databases :)so, I have an access database and I want to take 2 tables from it and copy it to sql server in my ASP.NET application (.MDF file)I have visual studio 2005 installed on my computer and got no sql server 2005 installed on it. what is the way copy 2 tables to the ASP.NET database application?thanks alot guys.
View 2 Replies
View Related
Feb 15, 2006
[Note: originally posted this on microsoft.public.sqlserver.programmingbut had no luck there...sorry if you're seeing this twice]Some facts:1) I have an Access database (.mdb file) sitting on my harddrive.2) I have Visual Studio 2005, Sql Server Express, and Sql ServerManagement Studio Express.3) I do *not* have Microsoft Access.What I'm trying to do:I simply want to import the Access database into Sql Server Express. Inother words, I want to end up with a Sql Server Express database thathas all the same tables, keys, and relationships as the Access databaseas well as all the data from it. I can live without the queries storedin the Access database, but those would be nice too.What I've tried so far:I'm able to connect to the Access database using the "Linked Servers"features in Management Studio Express. From there, I was able to writesome simple Transact-SQL queries to find out what tables are in theAccess database and copy them, one at a time, into a Sql Server Expressdatabase.This is definitely a good start, but it doesn't take care of theprimary keys or foreign keys. There appear to be procedures for thoseas well (sp_primarykeys, sp_foreignkeys), but I keep thinking theremust be an easier way.Which brings me to...Questions:Without having to buy additional software/tools, can I import thisAccess database without a lot of programming? If so, how?Thanks in advance,-Dan
View 1 Replies
View Related
Jan 18, 2006
I'm trying to import a database. It seems my most likely route is via Excel, so I've moved my tables in there.
Then I tried setting up Excel as a linked server as described in http://support.microsoft.com/default.aspx?scid=kb;en-us;Q321686
My linked server gets created apparently with no errors, but I can't see any contents when I click on it in the object explorer.
Anyone with more clues than me?
thanks
View 6 Replies
View Related
May 8, 2007
also i was wondering, that i have been reading that with this sp2 for sql server 2005 express you are able to use the dts wizard, is this true? i have a few databases in access that i would like to import into sql, would i have to create the tables first in sql then import or will the import also bring the tables & structure in?
thank you to all
View 1 Replies
View Related
Apr 7, 2007
I have software that uses SQL Server Express as it's database. I am only able to import so many records until it stops and fails to allow me to import anymore.
I'm very new at this but, is there some type of limitation on Table size that's preventing me from importing anymore data?
I'd really appreciate it if someone could help me,
Thanks... Bill
View 10 Replies
View Related
Nov 29, 2006
hi, i have an excel spreadsheet with data that i want to place into a sql database.is there any easy way of doing this with sql server 2005 express?
View 3 Replies
View Related
Sep 5, 2007
Hi,Could someone give a pointer how to import couple of hundred imagesinto Sql Server 2005 Express Edition database?Is there a tool for it? Can it be done with Sql Management Studio oris it just a matter of writing own piece of software (a little helperapp) to do it?-timonardo
View 1 Replies
View Related
Jan 11, 2006
Hello,
I'm trying to import some tables from another sql server to my laptop. I just downloaded Microsoft SQL Server Management Studio Express and i'm not sure how to import things. Could anyone help me with that?
Thanks a bunch
j
View 9 Replies
View Related
Feb 16, 2007
With DTS it was easy to import a DBF file, but its a huge pain with SSIS. Is there any way on the horizon to import DBF files as easily as DTS did?
View 1 Replies
View Related
Jul 23, 2005
Hello,I need to import a bunch of .csv files. The problem I am having is the"non data" information in the files creating bogus rows and columndefinitions. Here is an example of the csv file.CBOT - End-of-Day Futures Bulk Download 2001.2 Year U.S. Treasury Notes FuturesDateSymbolMonth CodeYear CodeOpen20010103ZTH2001102.0937520010104ZTH2001102.0312520010105ZTH2001102.28125In this case, there are bogues rows created with the text at thebeginning of the file, and also the column names get placed into a rowas well. My question is; how do you import the file and strip out the"non-data" data? so that only the actual data gets inserted into the db?Thanks,TGru*** Sent via Developersdex http://www.developersdex.com ***Don't just participate in USENET...get rewarded for it!
View 4 Replies
View Related
Jul 20, 2005
What is the best way to import a structured .prt export file from anaccounting package into SQl Server??Tabs, Fixed col length???
View 1 Replies
View Related
Sep 14, 2006
I have the original copies of the .MDF and .LBF of another database that has a bunch of data in that I'm trying to important into MSSQL Express for a project I'm working on. I no longer have a running version of this database just the .MDF and .LBF files and I have no clue what version of MSSQL they were originally created with. If anyone knows a way to import this database please help.
View 1 Replies
View Related
Oct 16, 2006
I am developing using visual web dev and sql server express. When this project goes live its going on another server running sql server(full version) this is my first asp.net app and my question is...Do I just move the mdf file from the dev machine running sql server express to the new server running sql server full version to retain all the tables and data?I know its been asked before but I want to be sure if this is the proper procedure.Thanks in advance!
View 5 Replies
View Related
May 2, 2007
I am trying to import a CSV file into an SQL Server table with the OleDbDataReader and SqlBulkCopy objects, like this: using (OleDbConnection dconn = new OleDbConnection("Provider=Microsoft.Jet.OLEDB.4.0;Data Source=c:\mystuff\;Extended Properties="text;HDR=No;FMT=Delimited""))
{
using (OleDbCommand dcmd = new OleDbCommand("select * from mytable.csv", dconn))
{
try
{
dconn.Open();
using (OleDbDataReader dreader = dcmd.ExecuteReader())
{
try
{
using (SqlConnection dconn2 = new SqlConnection(@"data source=MyDBServer;initial catalog=MyDB;user id=mydbid;password=mydbpwd"))
{
using (SqlBulkCopy bc = new SqlBulkCopy(dconn2))
{
try
{
dconn2.Open();
bc.DestinationTableName = "dbo.mytable";
bc.WriteToServer(dreader);
}
finally
{
dconn2.Close();
}
}
}
}
finally
{
dreader.Close();
}
}
}
finally
{
dconn.Close();
}
}
}
A couple of the columns for the destination table use a bit datatype. The CSV files uses the strings "1" and "0" to represent these.When I run this code, it throws this exception:Unhandled Exception: System.InvalidOperationException: The given value of type String from the data source cannot be converted to type bit of the specified target column. ---> System.FormatException: Failed to convert parameter value from a String to a Boolean. ---> System.FormatException: String was not recognized asa valid Boolean. at System.Boolean.Parse(String value) at System.String.System.IConvertible.ToBoolean(IFormatProvider provider) at System.Convert.ChangeType(Object value, Type conversionType, IFormatProvider provider) at System.Data.SqlClient.SqlParameter.CoerceValue(Object value, MetaType destinationType) --- End of inner exception stack trace --- at System.Data.SqlClient.SqlParameter.CoerceValue(Object value, MetaType destinationType) at System.Data.SqlClient.SqlBulkCopy.ConvertValue(Object value, _SqlMetaDatametadata) --- End of inner exception stack trace --- at System.Data.SqlClient.SqlBulkCopy.ConvertValue(Object value, _SqlMetaDatametadata) at System.Data.SqlClient.SqlBulkCopy.WriteToServerInternal() at System.Data.SqlClient.SqlBulkCopy.WriteRowSourceToServer(Int32 columnCount) at System.Data.SqlClient.SqlBulkCopy.WriteToServer(IDataReader reader) at MyClass.Main()It appears not to accept "1" and "0" as valid strings to convert to booleans. The System.Convert.ToBoolean method appears to work the same way. Is there any way to change this behavior? I discovered if you change the "1" to "true" and "0" to "false" in the CSV file it will accept them.
View 3 Replies
View Related
Sep 14, 2007
Hi,
I got one more problem with importing csv files using .net. The problem is that the csv file contains double-quotation marks (""). For example, the record looks like:
...,Bearing Double "D" Flange,...
And the result is: ... | Bearing Double | null (all following columns are null)
The code is as following:
string strCsvConn = @"Provider=Microsoft.Jet.OLEDB.4.0;Data Source=;Extended Properties='text;HDR=Yes;FMT=Delimited(,)';";
using (OleDbConnection cn = new OleDbConnection(strCsvConn))
{
string strSQL = "SELECT * FROM " + strFileName;
OleDbCommand cmd = new OleDbCommand(strSQL, cn);
cn.Open();
using (OleDbDataReader dr = cmd.ExecuteReader())
{while (dr.Read())
{
string str = Convert.ToString(dr[8]);
}
// Bulk Copy to SQL Server
//using (SqlBulkCopy bulkCopy = new SqlBulkCopy(strSqlConn))
//{
// bulkCopy.DestinationTableName = strSqlTable;
// bulkCopy.WriteToServer(dr);
//}
}
}
Any idea is highly appreciated.
shz
View 2 Replies
View Related
Apr 10, 2001
Hi,
I want to import multiple text files into a single table. I know I have to use BCP or DTS. But, I want import all files at once, instead of one at a time. And the file names are in sequence, viz. file1, file2, file3 etc.
Can anybody tell me, How I can achieve this.
Thanks
View 4 Replies
View Related
Jul 7, 2000
Has anyone written a SQL script to import ASCII comma or space delimited?
I am trying to load a PROGRESS exported file (Unix) into SQL SERVER 6.5 (NT).
Much appreciated,
Nickd
View 1 Replies
View Related
Sep 19, 2001
I have to import 18000 text files into a sql database.
Each file contains 10 fields and around 5000 records.
I am currently doing this with DTS.
What I am wondering is this: Is DTS the most efficient i.e. quickest way to import all this data. Bearing in mind there is about 90 million records to import in all.
I would appreciate the benefit of somebody elses experience when dealing with this type of thing.
Cheers,
Brookesy
View 2 Replies
View Related
May 10, 2004
Hello,
I've been trying to import a TXT file into an SQL database and I'm having trouble making it work correctly. It is a ASCII text file with over 100,000 records. The fields vary by the number of characters. This can be 2 characters up to 40 (STATE would be 2 characters, CITY is 32 characters, etc.)
I can import the file with DTS. I go in and select exactly where I want the field breaks to be. Then it imports everything as Characters with column headers of Col001, Col002, Col003, etc. My problem is that I don't want everything as Characters or Col001 etc. I want different column names and columns of data to be INT, NUMERIC(x,x), etc. instead of characters every time. If I change these values to anything than the default in DTS it won't import the data correctly.
Also, I have an SQL script that I wrote for a table where I can create the field lengths, data type etc. the way I want it to look, FWIW. This seems to be going nowhere fast.
What am I doing wrong? Should I be using something else than DTS?
Any suggestions are greatly appreciated.
Thanks,
JB
View 5 Replies
View Related
Apr 25, 2008
Hi i've 6 files with the same name part but all have a different ext. The 17 always changes to the current week number
xbouns.A17
xbouns.B17
xbouns.C17
I want to import these files into a database table. So I've create a foreach loop and select the foreach file enumerater but am not sure if this is the right way to go about it some help woth be great thanks.
View 7 Replies
View Related
Aug 5, 2005
I have over three hundred text files that I need to import to SQLServer.Each is in the exact same format.I want to import tham as seperate tables.Is there any way to do it in one process?Regards,Ciarán
View 5 Replies
View Related
Apr 28, 2008
Hi i've 6 files with the same name part but all have a different ext. The 17 always changes to the current week number
xbouns.A17
xbouns.B17
xbouns.C17
I'd like to do this within a SSIS package some help in getting this to work would be great.
Thanks
View 12 Replies
View Related
Jan 24, 2008
Hello Everyone.
I am a bit new to SQL Server but not to DBA or programming per se. I am having difficulties getting either an Excel or Text flat file to import properly.
I guess it would be best to ask, using either SSIS or BULK INSERT, what options need to be entered for a typical excel flat file?
View 2 Replies
View Related
