Simple Question About Altering Datafile Contents Prior To Or During Import
Oct 25, 2007
Hey everyone, I truly hope this post is in the right place. Apologies if that's not the case!
I know there has to be a simple solution to this, I just haven't managed to find it yet. Essentially what I'm trying to do is merge the contents of a datafile (for sake of argument let's say it contains three columns: ID, RefID and Content) with an existing table, but I need to alter the ID values contained within the datafile prior to doing the import.
The existing table in every client has a baseline of records ending at ID=100. The datafile contains a collection of records where ID>100 (let's say 3 rows worth). Clients for this "replication" model may have created custom content within this table, so the MAX(ID) value on client database tables can be greater than or equal to 100 (anything above 100 being custom content that has to be preserved).
Additionally, the RefID column can refer to any existing ID within this table.
So...
What I want to do is take the MAX(ID) value of the table I intend to import into, subtract 100 from it to establish my custom content differential, and then increment the ID values within the datafile by that differential. I'll also increase the RefID values by the differential, provided the RefID is greater than 100 (I don't want to change RefIDs that refer to baseline records).
I hope I explained that sufficiently! Any ideas about the smartest way to go about doing this? I'm just much more of a web/dbe than a dba/dbe lol...
Thanks in advance for any help!
View 3 Replies
ADVERTISEMENT
Jun 25, 2007
Hi All,I have come up against a wall which i cannot get over.I have an sql db where the date column is set as a varchar (i know, should have used datetime but this was done before my time and i've got to work with what is there). The majority of values are in the format dd/mm/yyyy. However, some values contain the word 'various'.I'm attempting to compare the date chosen on a c# .net page with the values in the db and also return all the 'various' values as well.I have accomplished casting the varchar to a datetime and then comparing to the selected date on the .net page. However, it errors when it comes across the 'various' entrant.Is there anyway to carry out a select statement comparing the start_date values in the db to the selected date on the .net page and also pull out all 'various' entrants at the same time without it erroring? i thought about replacing the 'various' to a date like '01/01/2010' so it doesn't stumble over the none recognised format, but am unsure of how to do it.This is how far i have got: casting the varchar column to datetime and comparing. SELECT * FROM table1 WHERE Cast(SUBSTRING(Start_Date,4,2) + '/' + SUBSTRING(Start_Date,1,2) + '/' +SUBSTRING(Start_Date,7,4) as datetime) '" + date + "'"Many thanks in advance!
View 7 Replies
View Related
Apr 23, 2007
Hello. Here a two different problems that occur one and a while when I try to import a textfile to SQL Server 2005.
I have a flat file connection to a csv-file that originally is a export from an AS4000 DB2 database
This csv-file is defined as a variable length file
Why do SSIS automatically interpret the length of each column as varchar(50)? It does not matter if a define the same file as a fixed lenght file. The problem is that I will get a warning that information in columns will be truncated. I would like to do a direct export to the SQL Server 2005 table with shorter varchar fields. I can solve this by using the task for transforming data types but this only works on the text fields. This task cannot transform a string to a decimal or an integer column in the SQL Server 2005 table. Is there no other way than having a staging table between the text file and the SSIS-data pipe?
I also get a lot of collation or code page errors even if we set the receiving columns to nvarchar and nchar. Is there any good article on this subject? Code page errors
Last question. Is there parameter support in the data reader source connection?
I appreciate all help.
Regards
Thomas Ivarsson
View 3 Replies
View Related
Jul 26, 2006
Boy, do I need HELP! Have a simple csv file that I need to import. Worked fine in sql2000; I put it into dts to execute on a monthly basis. Makes connection, db connection, table creation fine, but stops at validation of flat file?
Basically, I want to go out and get a flat file, drop the existing table, and create the table, and import the information from the flat file. Not a complicated table of about 30,000 records.
Create table [db].[dbo].[tblPatient] (
[patientID] into not null, [chartID] varChar(15) null, [doctorID] int null, [birthdate] datetime null, [sex] varchar(1) null, [raceID] int null, [city] varchar(100) null, [state] varchar(2) null, [zip9] varchar(9) null, [patientTypeID] int null, [patName] varchar(100) null)
Below is the error report that tells me NOTHING!
Operation stopped...
- Initializing Data Flow Task (Success)
- Initializing Connections (Success)
- Setting SQL Command (Success)
- Setting Source Connection (Success)
- Setting Destination Connection (Success)
- Validating (Error)
Messages
* Error 0xc00470fe: Data Flow Task:
The product level is insufficient for component "Source - pmPatientInfo_csv" (1).
(SQL Server Import and Export Wizard)
* Error 0xc00470fe: Data Flow Task:
The product level is insufficient for component "Data Conversion 1" (71).
(SQL Server Import and Export Wizard)
- Prepare for Execute (Stopped)
- Pre-execute (Stopped)
- Executing (Success)
- Copying to [fhc].[dbo].[tblpatient3] (Stopped)
- Post-execute (Stopped)
- Cleanup (Stopped)
View 12 Replies
View Related
Mar 17, 2006
I am trying to copy data from a SQL Server 2000 DB to SQL Server 2005 DB using the import/export wizard in SQL Server Management Studio. The two databases are not identical with different table names and different columns but I thought I had set up all the right mappings and had set the 'Enable identity insert' option. When I ran the wizard it errored at the Pre-execute phase. I simplified the wizard down to one table and only as couple of varchar columns and this also errored in the same way. The error report is detailed below.
For reference the SQL2000(ent. edn) DB is on a windows 2000 server and the SQL2005(dev. edn) DB and the management studio are both on my WinXPSP2 workstation.
Could somebody explain why these errors have occured and more importantly how to rectify the problem?
Many thanks,
Michael.
Operation stopped...
- Initializing Data Flow Task (Success)
- Initializing Connections (Success)
- Setting SQL Command (Success)
- Setting Source Connection (Success)
- Setting Destination Connection (Success)
- Validating (Warning)
Messages
Warning 0x80047076: Data Flow Task: The output column "DateAdd" (23) on output "OLE DB Source Output" (11) and component "Source - tccNewsArticles" (1) is not subsequently used in the Data Flow task. Removing this unused output column can increase Data Flow task performance.
(SQL Server Import and Export Wizard)
Warning 0x80047076: Data Flow Task: The output column "DateChg" (26) on output "OLE DB Source Output" (11) and component "Source - tccNewsArticles" (1) is not subsequently used in the Data Flow task. Removing this unused output column can increase Data Flow task performance.
(SQL Server Import and Export Wizard)
... NOTE: I have removed the rest of the warnings as they were the same as above (many of them).
- Pre-execute (Error)
Messages
Error 0xc0202009: Data Flow Task: An OLE DB error has occurred. Error code: 0x80040E21.
An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80040E21 Description: "Multiple-step OLE DB operation generated errors. Check each OLE DB status value, if available. No work was done.".
(SQL Server Import and Export Wizard)
Error 0xc0202025: Data Flow Task: Cannot create an OLE DB accessor. Verify that the column metadata is valid.
(SQL Server Import and Export Wizard)
Error 0xc004701a: Data Flow Task: component "Destination - Nrs_NewsArticles" (112) failed the pre-execute phase and returned error code 0xC0202025.
(SQL Server Import and Export Wizard)
- Executing (Success)
- Copying to [NereusV2_1].[dbo].[Nrs_NewsArticles] (Stopped)
- Post-execute (Stopped)
- Cleanup (Success)
Messages
Information 0x4004300b: Data Flow Task: "component "Destination - Nrs_NewsArticles" (112)" wrote 0 rows.
(SQL Server Import and Export Wizard)
View 1 Replies
View Related
Feb 1, 2005
1) I write a select statemnet
select a.columname1, b.columname1
from table1 a,table2 b
where a.columname2 = b. columname3
How do i compare
a.columname2 <--> int type column
while b. columname3 <--->varchar type
how should i use convert function
2) whats the best way to import 1.5 million rows ?
How about text file
View 2 Replies
View Related
Dec 21, 2006
We have a sql 2005 x64 database (datawarehouse related), essentially a work area for us, that we truncate and re-populate via BCP weekly. (We don't backup the database at all) . From the perspective of data-import speed what is the best recovery model to use: Bulk-Logged or Simple? (I have read sql 2005 BOL and don't find it partcularly clear on this point.)
Barkingdog
P.S. Anyone know of an article listing "best practices" for high-speed data import?
View 1 Replies
View Related
Nov 7, 2007
I lost my datafile , that's corrupted and I don't have database backup. Any idea ?:(
View 1 Replies
View Related
Nov 3, 2004
If a database consists of more than one datafile, how does SQL Server use the space in these datafiles ?, does it fill up the first one then move to the next and so forth, or does it use up pages across all the files evenly ?.
View 3 Replies
View Related
May 23, 2007
Hi need to something like this:
"alter database modify file (name=<name>, ONLINE)",
but it isn't possible.
Here what I did:
alter database bdj add file (name ='bdjfg1', filename='d:db
djfg1.ndf' )
to filegroup bdjfg;
alter database bdj modify file (name='bdjfg1', OFFLINE);
alter database bdj modify file (name ='bdjfg1', filename='d:db
ewdestdjfg1.ndf' );
--Msg 5056, Level 16, State 4, Line 1
--Cannot add, remove, or modify a file in filegroup 'bdjfg' because
the filegroup is offline.
alter database bdj modify filegroup bdjfg READWRITE;
--Msg 5056, Level 16, State 3, Line 1
--Cannot add, remove, or modify a file in filegroup 'bdjfg' because
the filegroup is offline.
Yes, yes and should have read the cautions section saying:
"Use this option only when the file is corrupted and can be restored. A file set to OFFLINE can only
be set online by restoring the file from backup. For more information about restoring a single file,
see RESTORE (Transact-SQL)."
But I have not an backup of the datafile, but I have the datafile itself!
What can I do to get it online again, the old location could be fine, but it would be better on an new location (thats is the reason for all the trouble, the original drive has not much space left, so I wanted to move the datafile)
What can I do, HELP
/Bjorn
View 1 Replies
View Related
Mar 12, 2008
Hi,
If I run the package from BIDS, it works fine. If I run the package inside Management Studio it works when I run it as a package.
It does NOT run when I schedule the job.
Error: 2008-03-12 10:51:56.16
Code: 0xC020200E
Source: Data Flow Task Flat File Destination [194]
Description: Cannot open the datafile "D:old_timesheet_reposTimeSheetfilesdate.txt". End Error
Error: 2008-03-12 10:51:56.16
Code: 0xC004701A
Source: Data Flow Task DTS.Pipeline
Description: component "Flat File Destination" (194) failed the pre-execute phase and returned error code 0xC020200E. End Error
DTExec: The package execution returned DTSER_FAILURE (1).
Started: 10:51:55 AM Finished: 10:51:56 AM Elapsed: 0.344 seconds.
The package execution failed. The step failed.,00:00:01,0,0,,,,0
Any idea?
Thanks in advance.
Eric.
View 6 Replies
View Related
Aug 3, 2001
Hi,
Is there any way to change the location for the datafile. I need to change the drive from say c to d because it is filling up. Is there anyway to do this or do I have to recreate the database from scracth. I have a whole lot of data in the database already.
Thank you all.
View 1 Replies
View Related
Dec 14, 2004
Hi All,
I have a database -- MDB -- with datafile for data and transaction log under the folder d:mssqldata . Now i want to move the data file from d: to e:, say e:mssqldata . Can someone let me know if this is possible under SQL server v7.0 and if so, how
View 2 Replies
View Related
Nov 2, 2005
Hi,
i try to shrink and remove one datafile. But i always get following error:
Server: Msg 5042, Level 16, State 1, Line 1
The file 'M1Pdata15' cannot be removed because it is not empty.
***********************************************
use M1K
go
dbcc shrinkfile (M1Pdata15,emptyfile)
go
use master
go
Alter database M1K
remove file M1Pdata15
go
***********************************************
Microsoft SQL Server 2000 - 8.00.760 (Intel X86)
After Shrinking the file has 1.94 MB.
please help.
thx
DocHolliday1108
View 5 Replies
View Related
Nov 10, 2005
Hi,
In SQL Server 2000 how to move a secondary data file to different filegroup without removing and re-creating the secondary data file.
Thanks in advance.
-S
View 1 Replies
View Related
Aug 17, 2006
I obviously did not search the archives on the right terms so what isthe easiest and fastest way to move a 3G database from a nearly full Cdrive to the nearly empty D drive that should have been used.I could back it up, drop it, recreate it using the D drive, and restoreit but it seems like there should be a way to just move the datafileand use if from the new location.I am thinking that detatch/attach is the best method, but I would likeconfirmation or suggestions on how to proceed or things to be aware ofwhen using this method.-- Mark D Powell --
View 8 Replies
View Related
May 19, 2007
Hi,
I want to insert the contents of a data file into a SQL Server table with XML datatype column using SSIS. I am fairly new to SSIS and script tasks.
Example:
Salary Data file:
EmpID, Name, Salary
100, John, 100000
200, Scott, 200000
Database Table:
create table xmlTable (id int identity(1,1), xmlColumn XML)
The contents of the file needs to be written to xmlTable. For each row in the Salary Data file there will be a corresponding row in the xmlTable.
xmlTable (Result):
1, <EmpID 100...>
2,<EmpID 200...>
Any help with details is greatly appreciated.
Thanks,
Rob
View 4 Replies
View Related
Jan 4, 2007
Hi there,
I have written a program that loads a package (SomePackage.dtsx) from the physical drive and executes that. The package does nothing but imports data from a csv file to the Sql server 2005. But I can see that the package is failing continuously. I meant the package.Execute() method is returning a DTSExecResult.Failure. I investigated the Package.Errors property that contains the error collection and found that there are two DTSError objects into the collection.
The first one€™s description says that
Cannot open the datafile "D:SOME.csv".
And the later one€™s is
component "SOURCE FLAT FILE COMPONENT" (1) failed the pre-execute phase and returned error code 0xC020200E.
But the most interesting thing is if I execute the package through the Execute package Utility (double clicking onto the SomePackage.dtsx file) ships with Sql server 2005 then it executes fine and works as expected. I have checked the permission of the csv file and it has everyone€™s full access.
Can anyone help me on this?
I will appreciate all kind of suggestions.
Thanks
Moim
View 14 Replies
View Related
Dec 2, 2003
Hai All.
I want to know ,is there any way to modify a table's field like adding of new field to a table.
If any one have idea plz enlighten me.
Bye
Regards,
Karthik.A
View 1 Replies
View Related
Jul 20, 2005
Hello!I have a problem that I really could use some help to solve. I have a tablewhich looks like this:id1, id2, id3, rate, ratenrExamples of a select * from this table would be:1047336399 21000 1 617 11047336399 21000 1 624B 11047336399 21000 1 621D 11047336399 21000 2 624B 11047336399 21000 2 612A 11047336399 21000 2 621D 11047336399 21000 3 617 11047336399 21000 3 624B 11047336399 21000 3 621D 1I would like to transform this table into something like this:1047336399 21000 1 617 1 624B 1 621D 11047336399 21000 2 624B 1 612A 1 621D 11047336399 21000 3 617 1 624B 1 621D 1the three first columns should be the primary key.Any help is appreciatedGunnar
View 1 Replies
View Related
Nov 28, 2005
Is it possible to convert a SQL2K datafile to SQL2K5? I have a 2K database that I need to easily convert to 2K5, I apprecaite any insight on this issue.
View 1 Replies
View Related
Dec 22, 1999
In order to perform an automatic way
to link a software using .txt database
to our SQL Database, I need some tips.....
I think about the following solution
- maybe using a "data-base extractor" (Access)
and convert the result into a .txt file,
Which can be automaticly refresh using a
.bat file which will open the .txt file created
3 times a day to refesh the data.
If you have some solution less complicated,
Please send it to
atoarlindo@yahoo.com.hk
Arlindo
View 1 Replies
View Related
Jul 16, 2001
Is it possible to move (detach and attach) the physical data and log file (.mdf &.ldf) to another location?
View 1 Replies
View Related
Apr 6, 2005
hi,
I'm trying to calculate how much unused space i have on one datafile. My main goal is to determine the max space i can save by doing a dbcc shrink.
Any help is greatly appreciated.
ThanKs in advance.
Al
View 1 Replies
View Related
Dec 8, 2004
Is there any limit to the maximum size of a datafile or transaction log you can have with SQL Server 2000 on Windows 2000. Also is there a maximum size that should be adhered to for performance and admin reasons ?.
View 4 Replies
View Related
Jan 30, 2004
Hi need lots of help here.
I have a dts package scheduled to run hourly as job since recent November. (Win2000 server, MSSQL2000 standard)
Its been running fine, except last few days, SQL Agent shows it attempted to run it but fails
Checking the history for the job i got this error message each time:
================================================== ======
DTSRun: Executing... DTSRun OnStart: DTSStep_DTSDataPumpTask_2
DTSRun OnError: DTSStep_DTSDataPumpTask_2,
Error = -2147467259 (80004005) Error string: Error creating datafile
mapping: The volume for a file has been externally altered so that the
opened file is no longer valid.
Error source: Microsoft Data Transformation Services Flat File Rowset
Provider Help file: DTSFFile.hlp Help context: 0 Error
Detail Records:
Error: 1006 (3EE); Provider Error: 1006 (3EE) Error string: Error
creating datafile mapping: The volume for a file has been externally altered
so that the opened file is no longer valid.
Error source: Microsoft Data Transformation Services Flat File Rowset
Provider Help file: DTSFFile.hlp Help context: 0 DTSRun
OnFinish: DTSStep_DTSDataPumpTask_2 DTSRun: Package execution complete.
Process Exit Code 1. The step failed.
=================================================
The key information i think would be these few lines:
Error creating datafile mapping: The volume for a file has been externally altered so that the opened file is no longer valid.
I don't think its SQL Agent as its scheduling is running other jobs fine(backup)
The DTS package runs fine manually.
Such i suspect dtsrun.exe itself. But where do i go from here?
Thks.
View 1 Replies
View Related
Nov 20, 2007
i am Hemanth
i have written a query for bulk insert , it is of
BULK INSERT
DTSTEST..FBM
FROM 'c:FBM1.TXT'
WITH
(
BATCHSIZE = 100000,
CHECK_CONSTRAINTS,
DATAFILETYPE = 'char',
KEEPIDENTITY,
KEEPNULLS,
ROWTERMINATOR = '',
FIELDTERMINATOR = '|'
)
this query is throwing an error msg
"Msg 4860, Level 16, State 1, Line 1
Cannot bulk load. The file "c:FBM1.TXT" does not exist."
but the file is present in c drive.
please help me in resolving this issue.
Thanks & Regards
Hemanth
View 4 Replies
View Related
Jul 23, 2005
Hi Faculties,Is is possible to assign space quotas on filegroup(s)/files todatabase users?*** Sent via Developersdex http://www.developersdex.com ***
View 1 Replies
View Related
Jan 28, 2008
Hi all
I've found this problem working with a VLDB, six month ago when I install the DBMS (Win2k3 x64+sp2, SQL 2k5 x64 +sp2, 4 dual core processor and 12 GbRAM) I've got 10 disk (actually ten LUN from a Storage Area Network), each 50Gb.
I've put TempDB and Transaction Log on two separate 50 Gb disk and put the database on 8 different data file on the 8 disk; I've created each datafile with a size of 50 Gb (autogrowth disable), so my DB has 400Gb space in it's datafile.
After a while the datafile began to fill and we decide to add a couple more 50Gb disk where I decide to put to new datafile; now my db is around 430 Gb and I've got this strange situation:
The first 8 datafile now are almost full of data, and obviously they can't growth since they already occupy the whole disk.
The two additional datafile are relatively empty (about 15 Gb each).
As far as I understand now each time that SQL should write to the databse it writes only on the 2 new datafile, and I fear that this can affect performance.
I'd like to reorganize the space in order to have 10 datafile, each with 43Gb of data, but I didn't find any instruction/tool able to move data between datafile.
Anyone has any hint ?
Thank you in advance for any suggestion
Stefano
View 5 Replies
View Related
Jul 27, 2006
Can I do this with a sql 2000 database
http://www.cryer.co.uk/brian/sqlserver/howto_movedatabasefile.htm
then attach sql express to this mdf ?
View 8 Replies
View Related
Oct 9, 2007
I need to move a datafile on my secondary database which is in standby mode. I have attempted to use the Restore command with the move and standby parameters
use master
RESTORE LOG BWP FROM
DISK='L: rans_bkpBWP_20071009080001.trn'
WITH MOVE 'BWPDATA3'
TO 'N:BWPDATA3BWPDATA3.ndf',
standby='L:TRANS_BKPBWP_20071009130001.tuf'
But I get the following error message
Msg 3174, Level 16, State 1, Line 1
The file 'BWPDATA3' cannot be moved by this RESTORE operation.
Msg 3119, Level 16, State 1, Line 1
Problems were identified while planning for the RESTORE statement. Previous messages provide details.
Msg 3013, Level 16, State 1, Line 1
RESTORE LOG is terminating abnormally.
View 20 Replies
View Related
Oct 18, 2001
I have an environment where I would like to grant the ability to perform certain functions to specific users for all databases on one of my SQL Servers. Specifically, I want to allow someone to alter table structures on the server, without having the ability to modify data within any databases on the server and without having the ability to create new databases on the server. Is this possible, and if so, how?
View 2 Replies
View Related
Oct 24, 2001
Pros,
I would like to change the logical filenames for a database on my test server. I know this is done with ALTER DATABASE, but I'm not sure of the exact syntax. Can anyone help?
rb
View 3 Replies
View Related
