I have a file I'm pulling from another type of database into an Excel spreadsheet and then using my dtsx package to import the spreadsheet into my SQL database. The problem I'm having is that one of the fields coming out of the database to the spreadsheet has the thousands seperator in the field that I want to use as a numeric field without the ",". Right now I have a macro that I run on the spreadsheet to reset the field to straight numbers without commas before importing it, but would like to configure my Integration package to do it automatically.
I have a table with 201 columns . I am importing 200 columns from a file using DFT. I want update the 201th column with the fileId of the file that just imported. I am storing the fileId of the file in varFileID .
How do I go back and update the 201th column ( column name sfileId) with the varFileID value?
I can use Execute SQL Task but how will I know it's the records of the files that I just imported not other rows.
I have a record in an Excel format (Excel 2010) and I would like to bulk import that into SQL Server 2008 and also while importing, SQL Server will automatically create a new table based on the header fields or row of the source file.
I am not sure if SQL Server 2008 has this capabilities.
I am working with an .xml file that I want to break up into various tables. A couple of fields in my xml file include html tags (<p> tags specifically). The generated xsd file thinks these are nested xml tags and creates a "P" table for the information contained. Is there a way I can modify the generated xsd within the designer? Or is the only way to fix this is to manually modify the xsd?
There is also an <id> tag within the xml. However, SSIS is not using that <id> as the primary key as it adds data to the various tables that I've specified. Instead it appears to be creating its own primary key - calling it "Id". Is there some way to specify that SSIS use the <id> tag contained within the xml instead of creating its own primary key?
Related to my last questions on SSIS work i'm doing, Is there a way to pad 0 on my generated flat file dynamically. I'm getting the data from 1 table and then generating the file. The file i need to generate would have data at the desired location as the file is being used by another system. Depedning on the data I want to put the padding of "0" and "3" inc certain fields. How am i suppose to do it. Apart from this I would need to megre 2 or more column and before the merge do an airthmatic operation. What would be the best component to use script component or derived column?
I noticed in the last two days the presence of 2 bigs journal files (about 2Go each, transfered from primary to secondary ).
I want to know if there is a way to see what was the operation that lead to have this to big 2 log files (sql statement or transaction... : with Oracle for instance, if we have chance, we can find this kind of info in a dynamic views by the name of : v$sqlarea...)...
SELECT '5' AS 'value/@version', 'database' AS 'value/@type', 'master' AS 'value/name', LTRIM(RTRIM(( [Server Name] ))) AS 'value/server', 'True' AS 'value/integratedSecurity', 15 AS 'value/connectionTimeout', 4096 AS 'value/packetSize', 'False' AS 'value/encrypted', 'True' AS 'value/selected', LTRIM(RTRIM(( [Server Name] ))) AS 'value/cserver' FROM dbo.RedGateServerList FOR XML PATH(''), ELEMENTS
I need to add some header information to the beginning of the query:
<?xml version="1.0" encoding="utf-16" standalone="yes"?><!-- SQL Multi Script 1 SQL Multi Script Version:1.1.0.34--><multiScriptApplication version="2" type="multiScriptApplication"><databaseLists type="List_databaseList" version="1">
Everything I have tried ends up as a failure, usually compile issues. My goal here is to be able to automare a configuration file for multiscript so I can keep my server list up to date.
I've created a stored procedure that creates a script to create a number of objects within the database (based on what existing objects are in the database). From Management Studio, this works fine, and the output is exactly as I want it.
I'm now trying to create a job that will execute this stored procedure, and deposit the results into a file somewhere on the server. When the job runs, the script is created in the correct place and is essentially ok.
However, there are a couple of questions I'd like to ask.
Why does SQL Server Agent put a header at the top of the output file? I was hoping to be able to use that output file 'as is' and execute it automatically to recreate my objects when required. (Obviously, I can manually remove the header, but this is an inconvenience in this situation). How do I stop it?
Also, when executed from SSMS, the output is correctly line-spaced. But, the output from the scheduled job adds an extra line between each line of text, which is, again, inconvenient. Why does it do this, and how can I prevent this (again, without manually editting the output)?
I am creating a SSRS report which would be executed by User manually through ReportServer URL.User would be generating the SSRS report for different Customer ID based on ad-hoc basis.
I am passing CustomerID as input parameter to the report. Is there any way to get the manually generated SSRS report name as 'Report_CustomerID_TodayDate.xls'.
E.g.If User is generating report for Customer ID 123 today then report name should be 'Report_123_07092013.xls'
I tried the Beta 1 of the service pack 1 to .net 3.5. If I try to add an entity (and try to save this), I get the Exception "No support for server-generated keys and server-generated values".
How can I add entities to my Sqlce- database?
I tried to give the id- column (primary key) in the database an identity, another time without identity, only primary key --> none of them worked. I always get the same error.
What do I have to change to make successfully a SaveChanges()?
Hi, I want to create a text file and write to text it by calling its assembly from Stored Procedure. Full Detail is given below
I write a code in class to create a text file and write text in it. 1) I creat a class in Visual Basic.Net 2005, whose code is given below: Imports System Imports System.IO Imports Microsoft.VisualBasic Imports System.Diagnostics Public Class WLog Public Shared Sub LogToTextFile(ByVal LogName As String, ByVal newMessage As String) Dim w As StreamWriter = File.AppendText(LogName) LogIt(newMessage, w) w.Close() End Sub Public Shared Sub LogIt(ByVal logMessage As String, ByVal wr As StreamWriter) wr.Write(ControlChars.CrLf & "Log Entry:") wr.WriteLine("(0) {1}", DateTime.Now.ToLongTimeString(), DateTime.Now.ToLongDateString()) wr.WriteLine(" :") wr.WriteLine(" :{0}", logMessage) wr.WriteLine("---------------------------") wr.Flush() End Sub Public Shared Sub LotToEventLog(ByVal errorMessage As String) Dim log As System.Diagnostics.EventLog = New System.Diagnostics.EventLog log.Source = "My Application" log.WriteEntry(errorMessage) End Sub End Class
2) Make & register its assembly, in SQL Server 2005. 3)Create Stored Procedure as given below:
CREATE PROCEDURE dbo.SP_LogTextFile ( @LogName nvarchar(255), @NewMessage nvarchar(255) ) AS EXTERNAL NAME [asmLog].[WriteLog.WLog].[LogToTextFile]
4) When i execute this stored procedure as Execute SP_LogTextFile 'C:Test.txt','Message1'
5) Then i got the following error Msg 6522, Level 16, State 1, Procedure SP_LogTextFile, Line 0 A .NET Framework error occurred during execution of user defined routine or aggregate 'SP_LogTextFile': System.UnauthorizedAccessException: Access to the path 'C:Test.txt' is denied. System.UnauthorizedAccessException: at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath) at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy) at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, ileOptions options) at System.IO.StreamWriter.CreateFile(String path, Boolean append) at System.IO.StreamWriter..ctor(String path, Boolean append, Encoding encoding, Int32 bufferSize) at System.IO.StreamWriter..ctor(String path, Boolean append) at System.IO.File.AppendText(String path) at WriteLog.WLog.LogToTextFile(String LogName, String newMessage)
I have a trade data tables (about 10) and I need to retrieve information based on input parameters. Each table has about 3-4 million rows.
The table has columns like Commodity, Unit, Quantity, Value, Month, Country
A typical query I use to select data is "Select top 10 commodity , sum(value), sum(quantity) , column4, column5, column6 from table where month=xx and country=xxxx"
The column4 = (column2)/(total sum of value) and column 5=(column3)/(total sum of quantity). Column6=column5/column4.
It takes about 3-4 minutes for the query to complete and its a lot of time specially since I need to pull this information from a webpage.
I wanted to know if there is an alternate way to pull the data from server ?
I mean can I write a script that creates tables for all the input combinations i.e month x country (12x228) and save them in table (subtable-table) with a naming convention so from the web I can just pull the table with input parameters mapped to name convention and not running any runtime queries on database ??
OR
Can I write a script that creates a html files for each table for all input combinations save them ?
Similar to a previous post (http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=244646&SiteID=1), I am trying to import data into a SQL Table.
I am trying to program a small application that will import product data obtained through suppliers via CD-ROM. One supplier in particular uses Fixed width colums, and data looks like this:
Example of Data
0124015Apple Crate 32.12
0124016Bananna Box 12.56
0124017Mango Carton 15.98
0124018Seedless Watermelon 42.98 My Table would then have: ProductID as int Name as text Cost as money
How would I go about extracting the data with an XML Format file? I am stumbling over how to tell it where to start picking up data for a specific column. Is there any way that I could trim the Name column (i.e.: "Mango Carton " --> "Mango Carton")?
I don't know if it makes any difference, but I've been calling SQL from my code by doing this:
Code in C# Form
SqlConnection SqlConnection = new SqlConnection(global::SQLClients.Properties.Settings.Default.ClientPhonebookConnectionString); SqlCommand cmd = new SqlCommand();
SqlConnection.Open(); cmd.ExecuteNonQuery(); SqlConnection.Close(); RefreshData(); I am running Visual Studio C# Express 2005 and SQL Server Express 2005.
What is the easiest way to accomplish this task with SSIS?
Basically I have a stored procedure that unions multiple queries between databases. I need to be able to export this to a text file on a daily basis and add a total records: row to the end of the text file.
Hello Experts, I am createing one task (user control) in SSIS. I have property grid in my GUI and 2 buttons (OK & Cancle). PropertyGrid has Properties like SourceConnection, OutputConnection etc....right now I am able to populate Connections in list box next to Source and Output Property.
Now my question to you guys is depending on Source Connection it should read that text file associated with connection manager. After validation it should pick header (first line of text file bases on record type) and write it into new file when task is executed. I have following code for your reference. Please let me know I am going in right direction or not.. What should go here ? ->Under Class A
Hi, I have imported 3 DTS from SQL 2000 to the SQL 2005 server. The wizard went fine, everything is ok. When I close the wizard window, I cannot see any of them. If I reimport it, it asks confirmation to override it. Where are the 3 DTS in the management Studio if they are not under Legacy DTS? Thanks
I am using the following SQl statement in a DTS vb file. The data is coming from an Excel spreadsheet and being put into a SQl server table.
oCustomTask1.SourceSQLStatement = "select `Order No`,`Country`,`Desc`,`Amount` from `Sheet1$` WHERE `Order No` = 1 "
I want to validate the data being put into the table to ensure no duplicates records are entered UNLESS the record has changed in any way. E.g. If record 1 had a column called "NumberOne" which changed to "Number1" change this information and move the original value to another table.
Hi. I am working on a dynamic website and backed up the data from our database (SQL Server) into a .bak file from our server. I want to develop the site locally on another machine (doesn't have SQL Server, but MSDE) using the database so I restored the database backup using RESTORE DATABASE FROM... (in the DOS prompt). I found a tutorial how to do this at
It did the restore successfully (or so it said) but all that it restored was the database structure (table structure, user information, etc.) but not the actual data in the database. I am using Specify (www.specifysoftware.org) as a database manager.
Could anyone help me out? Can I not go from SQL Server to MSDE? This doesn't seem to be the case because the structure restored okay. What should I do to correct this problem?
I am developing a DB with others in my group. When I import tables created on other servers to my server, the primary key and other properties do not import with the tables. Can anyone explain why this is happening? Is there a setting I have over looked?
I am in the process of importing data that is in a text format to thesql server, I want to add couple of fields and insert general data inthe fields added, this data is going to be similar to all the recordsimported. Your help in this regard will be greatly appreciated.
I am making web application using Asp.net C#(Visual Studio2005). And Sql server 2000 as a back End.
Actually I am fetching data from other database(named as retailexcel but in same server). I have wriiten a view (in retailexcel database) to get required data from four table of other database. My problem is when I fetch those data(about 40000 to 50000 record) It taking to much time. I tried catch feature of asp.net2.0. Time reduced but not significantly. So I am trying to import those tables to my database (named Inventory) from that retailexcel.
Now problem is every day or two, record of those tables changes. How can I keep update my tables (which are importing from retailexcel)
We have an OLTP application with multiple one to many tables relationships that are edited client side. For Example the primary table has descriptive components. One of the multiple table have things like parts and quantity ordered linked by a record number back to the primary table.
When our client transmits their transaction, they actally running a batch job which sends all the client side tables to the SQL Server and then we are trying to issue a stored procedure server side to push this data into our 'dbo' owned server side master tables.
Our problem is generating the fully qualified table name for the from clause.
We have tried declaring a variable but the stored procedure won't compile. Tried user, user_name().
HiI have a script that uses bcp to import data from an ascii text fileinto SQL tables. The french characters are not copied properly. Theyare converted to letters of the alphabet. I tried to change all thefields to nvarchar instead of varchar and nchar instead of char, but Igot Greek characters instead.How can I fix this?Here is some code:--------------------------CREATE TABLE [dbo].[1_HLGT_HLT_COMP_f9.0] ([hlgt_code] [int] NOT NULL ,[hlt_code] [int] NOT NULL) ON [PRIMARY]GOPRINT 'HLGT_HLT'DECLARE @s as nvarchar(300)SET @s='bcp MedDRA..[1_hlgt_hlt_comp_f9.0] in ' + char(34) +'F:MedDRA9.0FrenchMedAsciihlgt_hlt.asc' + char(34) + ' -c -t' +char(34) + '$' + char(34) + ' -r$ -e' + char(34) +'F:MedDRA9.0Frenchlogshlgt_hlt.err.txt' + char(34) + ' -b250 -m50-SDEV -Usa -Ppassword -h' + char(34) + 'TABLOCK' + char(34)EXEC master..xp_cmdshell @s
Hi All, I am importing data from an oracle db and one of the columns is a nclob. I would like to truncate the value if it is more than a certain length. I want to use a derived column transformation. I cannot use the len and substring functions because it is not a string. I could convert it to a unicode string and use those functions, but I am afraid there is a size limitation (4000 characters) on it. Is there smth I am missing? Can anybody please suggest any ideas? Thanks a lot.
I have used the wizard to import a DTS package from a SQL 2000 server to our new SQL 2005 server and need to make edits to reference the new server and database. I am able to see the package within the SQL Management studio under Integration Services, but I cannot find it in the Development studio?
I used a DTS package to import data from one database to a new database (different servers) and it seemed to work fine with the exception that it lost all the passwords. I got one error telling me that for security reasons, it left all passwords NULL. Sure enough - they are now all blank.
Ideas - any good articles with quick fixes ???
Thanks a bunch - I am just getting into a lot of unusual problems this week. Nancy
By the way - what happened to the SQL*Pass message boards ??? Nobody visits there now....
hi, I am importing data daily to many tables, I want to keep track of the #of rows for each import process. I already have created a trigger as follow: CREATE TRIGGER tr_bcp_log ON dbo.A FOR INSERT AS
declare @name varchar(30), @row_count int
select @name=name , @row_count= @@rowcount from inserted
insert into bcp_tracks (name,row_count) values(@name,@row_count) GO
The problem is that I am getting a row for each inserted row in table A.for instance if I have 500 rows in table A, I will get 500 rows in the log table like this table_name,#of rows A 1 A 1 A 1 etc up to 500 rows for table A
This is not what I want, I want to capture the num of rows for every bcp process , so in the log table I want to see the following : table_name, #of rows A 500 B 600 C 450 A 250 etc
I have a stored procedure which will run automatically. I've got try...catch code in the procedure, but I found a bug with the code where if there are any import errors, it doesn't recognize that that there was an error and it runs through the try code as through there was no problems. (I reported the bug). I added some code using @@rowcount to check if there were rows imported, and if not, it moves the data file to a error folder so I know there was a problem with the import. But this only checks if at least one row was imported, not if all the rows in the datafile have been imported. (i.e. if the first row imported correctly, and the second did not, it still sees it as successful). The problem is some of the data files have only one row to import and some have multiple rows. Is there a way to count the number of rows in the datafile, then count the number of rows imported, to verify they are the same number imported? Thanks, Laura
Today I tried out Integration Services and after a couple of hours of confusion I am impressed by the power of this product. I was wondering if I might get some help on the 'best practice' for the following requirements:
I have a foreach container that scans a directory and calls a data flow task for each file in the directory. The files import to the database fine, but I want to modify the procedure so it only imports files that have not yet been imported. There are a couple of scenarios:
- a file is created in the directory - a file is modified in the directory
In both cases I want to insert (or reinsert) the file. How can I modify my package to accomodate this behaviour? Storing the filename is an option, but I am not sure how to also bring in the file creation/modified dates.
