I have to upload about a million records from a text file using DTS packages. The file has a predefined format which map to the fields in the database. Some of the fields width exceed the width in the database. Does that mean that it will truncate these fields, or will it fail the whole bulk upload?
Any help will be appreciated. Thanks.
i am loading xml files present in a folder to sql server. i am doing this by means of ActiveX script (VB). i am passing every xml file and its schema definition file as parameter. later this script automatically create a table in script and loads the the data to database. for every xml in that foleder there is only one xsd.--------this was my intention.
but when i run the DTS script error is coming like "Error source: schema mapping, unable to upload xsd file".
is that always necessary to add
(xlmns:sql "urn:schemas-microsoft-com:mapping-schema") in its schema definition file.
i have tried to move upload of files from SQL2000 to SQl2005.
in SQL2005 i have created a JOB in which has 2 steps,each one of them activates the same batch files i used in SQL2000 (where i have change the server name and so on),for example :
for %%f in ( %BULK_UPLOAD_BASE%Temp%1*.txt) do bcp "db1.dbo.Test_TargetTable" in %%f -S SERVER-REP -U n145 -h "TABLOCK" -e %BULK_UPLOAD_BASE%ErrorErr10.txt -o %BULK_UPLOAD_BASE%LogsOutput10.txt -P n145 -C RAW -f %BULK_UPLOAD_BASE%format_with_RawData_4test.fmt
when irun this code in a bat file, the data is loaded with no problem.
when i run the JOB i see in the LOG that everything was ok, but actually the data fromthe files was not loaded - any idea why?
thnaks in advance
peleg
Israel -the best place to live in aftr heaven 9but no one wan't to go there so fast -:)
Hi all,
I have a problem during the bulk insertion. SQL truncated each line of the file.
The format file is:
9.0
1
1 SQLCHAR 0 8000 "
" 1 if1 ""
Table:
CREATE TABLE #if1 (
[if1] [varchar] (8000) NULL
)
Query:
SET @sqlstr = 'BULK INSERT #if1 FROM ''' + @inputfilepath + '.if1'' WITH (FORMATFILE = ''' + @ifformat + '.fmt1'')'
EXEC(@SQLSTR)
Everything is fine. The only problem is the lines are truncated.
Any ideas?
Hi,
I am trying to do a bulk insert using a text file that is comma delimited.
In the text file there are fields which are surrounded by quotes and contain commas; these commans should not be considered delimiters.
Is there a way to tell the BULK INSERT statement not to treat these as delimiters?
thanks,
sivilian
Hi folks....
I am able to import a CSV file into a temporary table as long as I know the number of fields in the CSV file. Here is what I would like to do:
I would like to have a CSV file which has UP to 6 entries per row. I would like to insert each row into a table; if the there three fields, then I want to insert them into the first three columns to the temporary table. If there are four, then insert into the first four fields. Is this possible?
Does anyone have any suggestions?
Thanks!
Forch
I have imported a whole bunch of tables. Most of them have an ID (int) column. Is there a way to set the ID columns across all tables to auto increment Primary Keys in bulk?
View 11 Replies View RelatedI am rewritting our DTS that upload tables in FoxPro to SQL Server using SSIS. I am finding the that SSIS is way slower. My uploads using DTS take just 7 minutes where doing the same thing in SSIS is taking 1 hour 15 minutes.
View 2 Replies View RelatedI have a record in an Excel format (Excel 2010) and I would like to bulk import that into SQL Server 2008 and also while importing, SQL Server will automatically create a new table based on the header fields or row of the source file.
I am not sure if SQL Server 2008 has this capabilities.
With a database size of almost 2 GB, I run the 'truncate table eventlog command' which completes successfully, but the database size only decreases by about 10 MB so stays too large - indeed the number of rows in the eventlog table is minimal, but the otehr tables in this database don't show such an amount of tables large enough to cause the size issue either. What could be the reason and how can I reduce it (possibly truncating another table but then which one, how could I determine which is too large and needs truncating?).
View 3 Replies View RelatedHi all...
Please forgive the elementary nature of my question, but could someone please explain the differences between these two database backup types:
1. Log backup
2. Log backup no truncate
From what I understand and have read, the "no truncate" backup method keeps the entire transaction log indefinitely. Using the truncation method, the transaction log is either 1) compressed or 2) cleaned up so that any completed transactions are removed from the log. Which one of these is true?
And, for the big question: is it better to run a backup of the transaction log with truncation or not? Our current backup scheme is similar to the following:
Full backup every 24 hours
transaction log backup every hour with no truncation
Should we insert a truncation backup somewhere in here? What is the danger of removing (or compressing) parts of the transaction log? Will this affect the restore process?
Thanks in advance!
Im having some issues with bulk insert.
This is the table:
CREATE TABLE [dbo].[tmp_GA_status](
[GA_recno] [int] NOT NULL,
[GA_desc] [varchar](40) NULL
)
This is the file (unicode):
1|"test1"
2|"test2"
3|"test3"
4|"test4"
5|"test5"
6|"test6"
7|"test7"
8|"test8"
and this is the sql:
bulk insert tmp_GA_status from 'C: empTextDumpGA_status.dta'
with (CODEPAGE='RAW', FIELDTERMINATOR='|', ROWTERMINATOR='
', DATAFILETYPE='widechar')
so yeah, pretty simple. But whatever I do I get this;
Msg 4864, Level 16, State 1, Line 1
Bulk load data conversion error (type mismatch or invalid character for the specified codepage) for row 1, column 2 (GA_desc).
So what am I doing wrong ?
I have to update a field within a table of 60 records or so. Each record has a different field value. it's type varchar. i was given an excel file with the field values and was thinking of a bulk update like bulk insert, but i don't recall that it's possible that way.
Is the only way to create a table, bulk insert, then merge the two tables together with UPDATE?
Just wanted to see if there was an easier way to do it, otherwise i'll take the latter route. Thanks!
Hi, folks:
View 18 Replies View RelatedI have a table containing 8 million records.
I need to replace 2 million of these records with
a scaled down query that goes something like:
SELECT 1, ShareholderID, Assets1
FROM MyTable (Yields appx. 200,000 recods)
SELECT 2, ShareholderID, Assets2
FROM MyTable (Yields appx. 200,000 recods)
.
.
.
SELECT 10, ShareholderID, Assets1 + Assest2 + Assets3 + ... + Assets9
FROM MyTable (Yields appx. 200,000 recods)
Updates and cursors just seem to be too slow.
So far I have done the following, but was wondering if anyone could think of a better way.
SELECT 6 million records that don't need to be deleted into a #TempTable
Use statements above to select into same #TempTable
DROP and recreate Original Table
SELECT 6 + 2 million records INTO original table.
This seems rather convoluted. Is there a better approach? Would it be worth while to dump data to a file and use bcp / Bulk Insert
Any comments are appreciated,
-Marc
I have just started using SQL Server reporting services and am stuck with creating subreports.
I have a added a sub report to the main report. When I right click on the sub report, go to properties -> Parameters, and click on the dropdown for Parameter Value, I see all Sum and Count fields but not the data fields.
For example, In the dropdownlist for the Parameter value, I see Sum(Fields!TASK_ID.Value, "AppTest"), Count(Fields!TASK_NAME.Value, "CammpTest") but not Fields!TASK_NAME.Value, Fields!TASK_ID.Value which are the fields retrieved from the dataset assigned to the subreport.
When I manually change the parameter value to Fields!TASK_ID.Value, and try to preview the report, I get Error: Subreport could not be shown. I have no idea what the underlying issue is but am guessing that it's because the field - Fields!TASK_ID.Value is not in the dropdown but am trying to link the main report and sub report with this field.
Am I missing something here? Any help is appreciated.
Thanks,
Sirisha
How can I create a Table whose one field will be 'tableid INT IDENTITY(1,1)' and other fields will be the fields from the table "ashu".
can this be possible in SQL Server without explicitly writing the"ashu" table's fields name.
sir
I have got this error message to establish connction with recordset vb .net, Can you please rectify this
Too many arguments to 'Public Overridable ReadOnly Default Property Fields() As ADODB.Fields'
my code like this
rs = New ADODB.Recordset
rs.Open("Select * from UserLogin where userid='" & txtUserName.Text & "'", gstrDB, DB.CursorTypeEnum.adOpenStatic)
If txtUserName.Text = rs.Fields.Append(userid) Then
MsgBox("OK", MsgBoxStyle.OKOnly, "Confirmation")
End If
thanks
I'm trying to use Bulk insert for the first time and getting the following error. I think it might have something to do with my Format File and from the error msg there's a conversion error for the first column. In my database the Field is nvarchar(6) so my best guess is to use SQLNChar for the first column. I've checked the end of each line is CR LF therefore the is correct for line 7 right?
Msg 4863, Level 16, State 1, Line 1
Bulk load data conversion error (truncation) for row 1, column 1 (ASXCode).
Msg 7399, Level 16, State 1, Line 1
The OLE DB provider "BULK" for linked server "(null)" reported an error. The provider did not give any information about the error.
Msg 7330, Level 16, State 2, Line 1
Cannot fetch a row from OLE DB provider "BULK" for linked server "(null)".
BULK
INSERTtbl_ASX_Data_temp
FROM
'M:DataASXImportTest.txt'
WITH
(FORMATFILE='M:DataASXSQLFormatImport.Fmt')
[code]...
Hello !I'm trying to update one table field with another table searched firstdate record.getting some problem.If anyone have experience similar thing or have any idea about it,please guide.Sample case is given below.Thanks in adv.T.S.Negi--Sample caseDROP TABLE TEST1DROP TABLE TEST2CREATE TABLE TEST1(CUST_CD VARCHAR(10),BOOKING_DATE DATETIME,BOOKPHONE_NO VARCHAR(10))CREATE TABLE TEST2(CUST_CD VARCHAR(10),ENTRY_DATE DATETIME,FIRSTPHONE_NO VARCHAR(10))DELETE FROM TEST1INSERT INTO TEST1 VALUES('C1',GETDATE()+5,'11111111')INSERT INTO TEST1 VALUES('C1',GETDATE()+10,'22222222')INSERT INTO TEST1 VALUES('C1',GETDATE()+15,'44444444')INSERT INTO TEST1 VALUES('C1',GETDATE()+16,'33333333')DELETE FROM TEST2INSERT INTO TEST2 VALUES('C1',GETDATE(),'')INSERT INTO TEST2 VALUES('C1',GETDATE()+2,'')INSERT INTO TEST2 VALUES('C1',GETDATE()+11,'')INSERT INTO TEST2 VALUES('C1',GETDATE()+12,'')--SELECT * FROM TEST1--SELECT * FROM TEST2/*Sample dataTEST1CUST_CD BOOKING_DATE BOOKPHONE_NOC12005-04-08 21:46:47.78011111111C12005-04-13 21:46:47.78022222222C12005-04-18 21:46:47.78044444444C12005-04-19 21:46:47.78033333333TEST2CUST_CD ENTRY_DATE FIRSTPHONE_NOC12005-04-03 21:46:47.800C12005-04-05 21:46:47.800C12005-04-14 21:46:47.800C12005-04-15 21:46:47.800DESIRED RESULTCUST_CD ENTRY_DATE FIRSTPHONE_NOC12005-04-03 21:46:47.80011111111C12005-04-05 21:46:47.80011111111C12005-04-14 21:46:47.80044444444C12005-04-15 21:46:47.80044444444*/
View 3 Replies View RelatedI am using MS SQL 2012. I have a table that contains all the data that I need, but I need to summarize the data and also add up decimal fields while at it. Then I need a total of those added decimal fields. My data is like this:
I have Providers, a unique ID that Providers will have multiples of, and then decimal fields. Here are my fields:
ID, provider_name, uniq_id, total_spent, total_earned
Here is sample data:
1, Harbor, A07B8, 500.00, 1200.00
2, Harbor, A07B8, 400.00, 800.00
3, Harbor, B01C8, 600.00, 700.00
4, Harbor, B01C8, 300.00, 1100,00
5, LifeLine, L01D8, 700.00, 1300.00
6, LifeLine, L01D8, 200.00, 800.00
I need the results to be just 3 lines:
Harbor, A07B8, 900.00, 2000.00
Harbor, B01C8, 900.00, 1800.00
LifeLine, L01D8, 900.00, 2100.00
But then I would need the totals for the Provider, so:
Harbor, 1800.00, 3800.00
Hi~,
Before implementing memory based bulk copy insert with IRowsetFastLoad interface of SQL Server 2005 OLE DB provider, I want to know some considerations.
- performance : compared with T-SQL's "BULK INSERT ..." and bcp utility
- SQL Server's resource usage : when running memory based bulk copy, server resource's influence
- server side action(behavior) : when server is busy, delayed-update means IRowsetFastLoad::Commit(true) method can insert right after?
- row-count : The rowcount limitation can be inserted by IRowsetFastLoad::InsertRow() method before IRowsetFastLoad::Commit
- any other guide lines
I would like to search a table for a phrase, or for a partial phrase,eg on table product - for name or description, or name + descprition.How does one say select * from product where name + description like%phrase%or contains phraseCurrently I can get where name, or where descriotion like %phrase%,eg, where name like krups, or where description like coffee makerBut if I search for where name like %krups coffee maker% i get noresults. krups is in the name field, coffee maker is in thedescription field.Thanks,-M
View 1 Replies View Related
Hello,
I'm just learning SSIS and I've hit my first bump. I am doing a bulk import from a tab delimited text file to an empty sql table that has a Idendity column defined. How do I tell the bulk insert task to skip that column when inserting from the text file. If I remove the identity column it imports the data fine, but I want to create the indentity column in the table too.
Thanks.
Pardon me if this question is too elementary. I am trying to create a trigger that will cause certain datafields to be updated with values from other data fields in the same row when a certain column, created specifically to fire the trigger, is updated. The purpose of this is to reduce data entry by field personnel.I think I have the create trigger statement correct, but I'm a little confused on the update statement.
In a nutshell, how can I write something like:
UPDATE "TABLENAME"
SET DATAFIELD1 = DATAFIELD2
WHERE RECORDNUMBER = (THE SAME RECORD NUMBER)
I do know that I have to ensure that sp_dboption Recursive Triggers value is set to false, thanks.
I have a list of items in one table and a field (pageName) in another table that may contain one of the aforementioned items somewhere within that field. There is no fixed position within the field where the itemNo may be so I can't just use SUBSTRING(pageName,2,5) in(select itemNo from tblItem).
Logically, it's like I need to combine IN and LIKE: select pageName where pageName LIKE IN %select itemNo from tblitemNo%..LIKE can only handle one comparison string.
Hi Guys,
I have one question, must be simple for you guys. I want to upload .pdf document into the SQL server database.
I have one question, that which data type I should use. Should I use TEXT or IMAGE.
Could you please tell me which data type I should use and why?
Thanks for your help in advance.
how to upload data from excel sheet to table in sql server is there any command for that
View 1 Replies View RelatedI used upload image feature and below is the code that I used. The image is stored in a folder. I want to save the image in sql database. I create the database which is “database�, and I create a table which is “user�. User table has auto “id� field and “image� field. I want to save the image in the image field. I need your help to do the other codes.
I am using asp.net with VB
This is what I did so far:
============================
Partial Class upload
Inherits System.Web.UI.Page
Dim strFileName As String
Protected Sub btnupload_Click(ByVal sender As Object, ByVal e As System.EventArgs) Handles btnupload.Click
UploadImage()
End Sub
Sub UploadImage()
If Not (fileupload1.PostedFile Is Nothing) Then 'Check to make sure we actually have a file to upload
Dim strLongFilePath As String = fileupload1.PostedFile.FileName
Dim intFileNameLength As Integer = InStr(1, StrReverse(strLongFilePath), "")
strFileName = Mid(strLongFilePath, (Len(strLongFilePath) - intFileNameLength) + 2)
Select Case fileupload1.PostedFile.ContentType
Case "image/pjpeg", "image/jpeg" 'Make sure we are getting a valid JPG image
FileUpload1.PostedFile.SaveAs(Server.MapPath(" estimages") & strFileName)
lbstatus.Text = strFileName & " was uploaded successfully to: " & Server.MapPath(" estimages") & strFileName
Case Else
'Not a valid jpeg image
lbstatus.Text = "Not a valid jpeg image"
End Select
InsertToDatabase()
End If
End Sub
End class
I am using the article, "
Storing Binary Files Directly in the Database Using ASP.NET 2.0 By Scott Mitchell "
http://aspnet.4guysfromrolla.com/articles/120606-1.aspx
I have tried to modify the code and my datatable to replicate the function, but I am getting an error, "MIMEType" is not a SQL Parameter
An SqlParameter with ParameterName 'Type' is not contained by this SqlParameterCollection.
Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code. Exception Details: System.IndexOutOfRangeException: An SqlParameter with ParameterName 'Type' is not contained by this SqlParameterCollection.Source Error:
Line 53:
Line 54: 'Specify the values for the MIMEType and ImageData parameters
Line 55: e.Command.Parameters("Type").Value = MIMEType
Line 56:
Line 57: 'Load FileUpload's InputStream into Byte arraySource File: E:homeDefaultetances.ushtdocshomeadminaddNews.aspx Line: 55
Code: Protected Sub AddNewsBut_Click(ByVal sender As Object, ByVal e As System.EventArgs)
If Page.IsValid Then
InsertDs.Insert()
Response.Redirect("~/news/default.aspx")
End If
End Sub
Protected Sub InsertDs_Inserting(ByVal sender As Object, ByVal e As System.Web.UI.WebControls.SqlDataSourceCommandEventArgs)
'Reference the FileUpload control
'Make sure a file has been successfully uploaded
If UploadedFile.PostedFile Is Nothing OrElse String.IsNullOrEmpty(UploadedFile.PostedFile.FileName) OrElse UploadedFile.PostedFile.InputStream Is Nothing Then
Label1.Text = "If there is no photo of this event, please use a stock photo if you have none available. Else, please upload a photo of the event."
e.Cancel = True
Exit Sub
End If
'Make sure we are dealing with a JPG or GIF file
Dim extension As String = Path.GetExtension(UploadedFile.PostedFile.FileName).ToLower()
Dim MIMEType As String = Nothing
Select Case extension
Case ".gif"
MIMEType = "image/gif"
Case ".jpg", ".jpeg", ".jpe"
MIMEType = "image/jpeg"
Case ".png"
MIMEType = "image/png"
Case Else
'Invalid file type uploaded
Label1.Text = "Please use the correct file type: .jpg, .jpe, .gif, or .png."
e.Cancel = True
Exit Sub
End Select
'Specify the values for the MIMEType and ImageData parameters
e.Command.Parameters("Type").Value = MIMEType
'Load FileUpload's InputStream into Byte array
Dim imageBytes(UploadedFile.PostedFile.InputStream.Length) As Byte
UploadedFile.PostedFile.InputStream.Read(imageBytes, 0, imageBytes.Length)
e.Command.Parameters("Photo").Value = imageBytes
End Sub Please help.
I created a website a year ago. I remember I need to download something from Microsoft (some kind of sql admin thing), which allows me to convert my database in the Express Visual Web developer to sql file, then I can copy those sql commands into my web hosting company's sql admin window to upload the database online. I forgot what things I should download from Microsoft? Please help.
View 1 Replies View RelatedI am building this project and for all the data i put in to the DB Title,Author, etc.... ineed to also add a picture of the book. I terms on space what is the best way to do this? In the DB? Or ref a folder? If so does anyone have sample code for each.
ThanksMike
Hi, I am trying to upload a file to database.
I have used the following code, every loads good to the database apart from the image, does it go anywhere?
I have created a column in the table called 'FileUploadAdvert' and made it an image?
Any ideas where I'm going wrong?
ThanksGordon
Protected Sub btnAdvertSubmit_Click(ByVal sender As Object, ByVal e As System.EventArgs) Handles btnAdvertSubmit.ClickDim dashDataSource As New SqlDataSource()dashDataSource.ConnectionString = ConfigurationManager.ConnectionStrings("DashConnectionString1").ToString()
dashDataSource.InsertCommandType = SqlDataSourceCommandType.Text
dashDataSource.InsertCommand = "INSERT INTO tblAdvert (AdvertOwner, AdvertName, TopLeftH, TopLeftV, Height, Width, ToolTip, WebLink, AcceptTerms, DateTimeStamp, IPAddress) VALUES (@AdvertOwner, @AdvertName, @TopLeftH, @TopLeftV, @Height, @Width, @ToolTip, @WebLink, @AcceptTerms, @DateTimeStamp, @IPAddress)"dashDataSource.InsertParameters.Add("AdvertOwner", txtName.Text)
dashDataSource.InsertParameters.Add("AdvertName", txtCompName.Text)dashDataSource.InsertParameters.Add("TopLeftH", DropDownAccross.Text)
dashDataSource.InsertParameters.Add("TopLeftV", DropDownDown.Text)dashDataSource.InsertParameters.Add("Height", DropDownHeight.Text)
dashDataSource.InsertParameters.Add("Width", DropDownWidth.Text)dashDataSource.InsertParameters.Add("ToolTip", txtOver.Text)
dashDataSource.InsertParameters.Add("Weblink", txtURL.Text)dashDataSource.InsertParameters.Add("AcceptTerms", CheckBoxAgree.Checked)
dashDataSource.InsertParameters.Add("IPAddress", Request.UserHostAddress.ToString)dashDataSource.InsertParameters.Add("DateTimeStamp", DateTime.Now)
If Not FileUploadAdvert.PostedFile Is Nothing ThenDim filepath As String = FileUploadAdvert.PostedFile.FileName
Dim pat As String = "\(?:.+)\(.+).(.+)"Dim r As Regex = New Regex(pat)
'runDim m As Match = r.Match(filepath)
Dim file_ext As String = m.Groups(2).Captures(0).ToString()Dim filename As String = m.Groups(1).Captures(0).ToString()
Dim file As String = filename & "." & file_ext
'save the file to the server FileUploadAdvert.PostedFile.SaveAs(Server.MapPath(".") & file)
lblStatus.Text = "File Saved to: " & Server.MapPath(".") & fileEnd If
Dim rowsAffected As Integer = 0
Try
rowsAffected = dashDataSource.Insert()Catch ex As Exception
Server.Transfer("Register_Fail.aspx")
Finally
dashDataSource = Nothing
End Try
If rowsAffected <> 1 ThenServer.Transfer("Register_Fail.aspx")
ElseServer.Transfer("Register_Complete.aspx")
End If
End Sub