External Scripts / Imports / Updates
May 12, 2004
Looking for suggestions on this one. What I want to do have have a text file that may have any number of rows and cols (with a predefined format) that a user can update or insert into a table. The definition of the row/cols and data mapping etc, has been done, it is the mechanics of actually doing the below I would appreciate help and advice on.
As the user is an 'end-user' (and has no SQL knowledge at all) the text file to import from will be placed in a predefined location and then a small script will be executed from their PC (as it happens, it's a Mac that runs an app that can exec an SQL command on the currently open database) that will in turn run a stored proc which is then reads in (imports or updates) the appropriate tables witht he contents of the external text file.
Sorry the explanation is a bit long winded but if anyone had any practical suggestions and examples, it would be greatly appreciated.
FYI, they are running SQL 2000 on both XP Pro and W2K3 server.
Thanks
Starb
View 2 Replies
ADVERTISEMENT
Apr 28, 2008
I got anywhere from a couple hundred to a hundred thousand records that need to be updated or inserted into their SQL Server 2005 end destination. What are some of the best ways to accomplish this? Right now we are doing it manually through line by line updates and inserts. Would I use BC or some other bulk import tool?
View 5 Replies
View Related
Nov 9, 2015
I have created a package that will insert new rows into destination1 if the AnID does not exist in Source1.
This uses a data flow task which contains a oledb source, lookup and oledb destination.
Source1
Field Name AnID Acol1 Acol2 Destination1 Field Name AnID Acol1 Acol2
I want to be able to update the destination1. Acol1 and destination1. Acol2 if the Source1.Acol1 or Source1.Acol2 have changed.
To do this, would I need to create separate data flow task that includes a source, lookup and destination.
Or is this possible to build into my insert new records data flow task...
View 3 Replies
View Related
Jun 12, 2006
MS-SQL 2005, C# 2.0
I will have many client databases that will be updated. When they are updated I need to transfer some of that data to a central Server database somewhere on the Internet. Note, schemas do not necessarily match.
Transfering the data (Web Services, Remoting) is not a problem.
What I am looking for is a cool, correct, advisable way, for the client database to notify me of an Insert, Update or Delete. I can then initiate the connection to the Server and 'push' the data (maybe pull some back from the Server too).
Obviously Triggers may be a place to start... But I need to know 'external to the database' that an update has happened or can I send the data from within the SQL Server assembly?
Anyone have any ideas or a technique, maybe something new in SQL Server 2005 (all databases will be 2005), for acheiving this? Just so I don't go down the 'wrong' path...
There are some BLOB's involved (1-2 page PDF's), if that makes a difference.
I envisage that the process transfering the data will be a Windows Service running on each client. The connection may not always be available, so some kind of 'to do' list of outstanding data to be transfered is required.
I'm just starting on this, so any pointers would be great, I'm sure it's all been done before ;)
Thanks
Rob.
View 1 Replies
View Related
Oct 12, 2015
In SSIS 2008R2, I have a dataflow with an xlsx source and the destination is a SQL Server 2008R2 table. The files are delivered from a location where staff members 'work with' the source files. The files are produced monthly.
The dataflow that contains the file breaks upon the attempt to process subsequent monthly xlsx files with a message similar to the following:
--*************
[TNUQQ [16]] Warning: The external columns for component "TNUQQ" (16) are out of synchronization with the data source columns. The column "F12" needs to be added to the external columns.
The external column "county_taxable_sale_amount" needs to be updated.
The external column "city_taxable_sale_amount" needs to be updated.
The external column "district_taxable_sale_amount" needs to be updated.
The external column "QTY" (62) needs to be removed from the external columns.
--*************
I've noticed that some columns in the file ship with no data. A column with no data can be typed as datetime one month, and then float another month. I've tried to load xlsx to raw to table, but that does not work around this issue.
I've tried to set 'ValidateExternalMetadata' to 'False' on the Excel source, but that does not work either. Aside from going back to the folks who ship the file to us, is there anything that can be done in SSIS to work around this issue, and still wind up with valid data?
View 5 Replies
View Related
Feb 29, 2008
I'm created a package to import XML the are PGP encrypted from a FTP Site. What tools can be used to decode the encryption through SSIS?
Is this possible? If not what can be used to automate this process?
View 6 Replies
View Related
Dec 19, 2007
I have a project that consists of a SQL db with an Access front end as the user interface. Here is the structure of the table on which this question is based:
Code Block
create table #IncomeAndExpenseData (
recordID nvarchar(5)NOT NULL,
itemID int NOT NULL,
itemvalue decimal(18, 2) NULL,
monthitemvalue decimal(18, 2) NULL
)
The itemvalue field is where the user enters his/her numbers via Access. There is an IncomeAndExpenseCodes table as well which holds item information, including the itemID and entry unit of measure. Some itemIDs have an entry unit of measure of $/mo, while others are entered in terms of $/yr, others in %/yr.
For itemvalues of itemIDs with entry units of measure that are not $/mo a stored procedure performs calculations which converts them into numbers that has a unit of measure of $/mo and updates IncomeAndExpenseData putting these numbers in the monthitemvalue field. This stored procedure is written to only calculate values for monthitemvalue fields which are null in order to avoid recalculating every single row in the table.
If the user edits the itemvalue field there is a trigger on IncomeAndExpenseData which sets the monthitemvalue to null so the stored procedure recalculates the monthitemvalue for the changed rows. However, it appears this trigger is also setting monthitemvalue to null after the stored procedure updates the IncomeAndExpenseData table with the recalculated monthitemvalues, thus wiping out the answers.
How do I write a trigger that sets the monthitemvalue to null only when the user edits the itemvalue field, not when the stored procedure puts the recalculated monthitemvalue into the IncomeAndExpenseData table?
View 4 Replies
View Related
Mar 31, 2003
Hi there,
I use Asp.Net Application to upload a Excel file and then a DTS to import data from the file to the SQL2000 and finally to display the read data on the screen.
The DTS starts with setting some variables with the help of Dynamic properties.
On Succes.
DTS rum 2 simultaneous Transform Data task importing data from excel to SQL2000.
This works fine for "most" of the time, but then there are the other times.
One of the TDT(Transform Data task) reads nothing from the excel file. but it can read the data if i upload the same file again right after.
Any kind of input is welcome
Thanx
/bs26
View 1 Replies
View Related
Jul 20, 2005
Hello Specialists !Please help me - i need advice in importing textual data to SQL Server.I am using DTS with a simple process : Text(source)->Connection.I want to increase speed of importing because i have to import 4GB (1000char lines lenght) of data.Do you have any tips for me ?Best regards,Manu
View 1 Replies
View Related
May 14, 2007
I am trying to add an Imports statement to the beggining of the custome code section of my report but I recieve this error. There is an error on line 0 of custom code: [BC30465] 'Imports' statements must precede any declarations.
My syntax looks like this:
Imports SB = System.Text.StringBuilder
Imports System.Enum
public function Test ...
end function
Does anyone know if this is even possible and why or why not?
In relation to that, can anyone answear as to why there is a 32K limit on the code section in reporting services?
View 4 Replies
View Related
Mar 26, 2007
hours wasted what else is new with Micro --- crap try to import an excel into a table, longest field (via vba macro report) is 278 receiving DB field is 4000 get "Truncation error, I must stop I am a piece of s*** program " the only thing you can find on the web is make sure you have SQL server 2005 SP2 install it same thing and g** only knows what other problems I have just created by installing another piece of Microsoft magic generated in India or China by the best technologists making at least $5 and hour
so MVPs when is this rediculous situation going to be fixed ?? oh that's right your answer will be "go to VISTA" which won't fix my problem but will probably help your stock situation
MS owes me (conservatively) 100K ... my current plan is to install Linux get up on MySQL and NEVER deal with SHoddy half built pieces of garbage again
so where is the fix ???
when will we see it (short of buying another bloated piece of ... oh i mean Vista)
an addendum
i am trying to import 25,000 rows with approx 20 columns from Excel
i added a first column with an ID .... if I import whole table it dies on row 1852 for truncation problems
if I empty that column all goes in make other exel get rid of all rows except ID and bad row (contains web addresses eg http://www. blah blah)
import that excel it dies on row 2395 in other words the first offending cell was no problem if it was the 2cnd row instead of the 15th row
try to tell me this is not a random bug .... long live MYSQL
View 3 Replies
View Related
May 18, 2004
Hi.
Is there anyway to export the table structures : data type,length,NULLABLE,Description into an Excel file using MS SQL Server?
Or I need to do it manually?
Thank you in advanced.
Sincerely
Agustina
View 2 Replies
View Related
Aug 4, 2004
Hi guys,
I need to be able to send a text file with data seperated by tabs to a stored procedure that populates my table.
I am new to SQL and SPROCs that I am not sure how to even start. If you guys have any ideas I would really like to hear them or maybe any on-line docs and examples.
Thanks in advance!
View 2 Replies
View Related
Feb 4, 2008
Hi there,
I was wondering about the stability of SSIS when it comes to importing data on a real-time basis. Let's say you have a scenario where flat files, for instance, will be dropped at random intervals ranging from 1 second to 10 seconds apart and the importer has to import these files immediately.
I would imagine that this is done with a package which runs a loop sniffing the directory forever but I stand corrected on the best ways of doing it.
I'm not too sure whether SSIS is a good idea for this as lots of people have had bad comments on SSIS in real-time in my company but they cannot elaborate on why enough to convince me. I have done some pretty cool stuff and must admit that I am a fan of the technology but dont want to defend it into a corner
Please give me your thoughts on this
Regards
Michael
View 13 Replies
View Related
Mar 12, 2007
I'm pretty new to using SQL 2005 Management Studio. Generally speaking, it works pretty much the way I'm used to (using Enterprise Manager) as far as moving data around and designing databases is concerned. But I've been trying to import some data to my local SQL server from an online SQL database and I am getting the most bizarre results.
Basically, it appears to work perfectly, but when the import is finished, there is only one new row in the destination table.
I have tried this with two completely different online databases and I have tried importing using a query and just downloading the table as-is, but whatever happens, I just get one row! All of the databases I'm exporting from or importing into are SQL 2000 - I just happen to be using the SQL 2005 client software.
Have you heard of anything like this?
I'm appending the report to the end of this email. As you can see, it says "success, success" all the way down and it clearly states "71236 rows transferred", but when I get done, there's just one.
Any thoughts?
One explanation is that every new row is overwriting the last new row somehow, but I don't think so because the row that actually gets copied is always the first one in the record set, not the last. Unless they're being transferred in reverse order, I suppose.
I'm pretty stumped and I haven't found any useful blogs or help on the web.
Hope you can think of something because I don't have Enterprise Manager on my computer any more and it's going to be a pain to install it.
Jon
Here's the report:
The execution was successful
- Initializing Data Flow Task (Success)
- Initializing Connections (Success)
- Setting SQL Command (Success)
- Setting Source Connection (Success)
- Setting Destination Connection (Success)
- Validating (Success)
- Prepare for Execute (Success)
- Pre-execute (Success)
- Executing (Success)
- Copying to [inframes].[dbo].[sum_shop_clicks] (Success)
* 71236 rows transferred
Messages
* Information 0x402090df: Data Flow Task: The final commit for the data insertion has started.
(SQL Server Import and Export Wizard)
* Information 0x402090e0: Data Flow Task: The final commit for the data insertion has ended.
(SQL Server Import and Export Wizard)
- Post-execute (Success)
- Cleanup (Success)
Messages
* Information 0x4004300b: Data Flow Task: "component "Destination - sum_shop_clicks" (37)" wrote 71236 rows.
(SQL Server Import and Export Wizard)
View 12 Replies
View Related
Sep 5, 2007
I have been trying to follow/implement the examples in the following help topics (thanks to Jamie for these links).
Building Packages Programmatically
(http://msdn2.microsoft.com/en-us/library/ms345167.aspx)
Connecting Data Flow Components Programmatically
(http://msdn2.microsoft.com/en-us/library/ms136086.aspx)
The problem I am having is that MainPipe is not recognized as a valid type in my Script task, even though I have the imports statements that are listed in the example. I get the message "Error 30002: Type 'MainPipe' is not defined". The other and related problem is that when I type "imports microsoft.sqlserver.dts", the intellisense offers only two choices: {}Runtime and {}Tasks. I don't see any choice for Pipeline. Can anyone tell what I am missing? It seems to be some kind of configuration/installation issue, but I have no idea how to resolve it. I have tried this on 3 different machines, with both the RTM SQL 2005 standard edition, and with SP2 installed, all with the same result. Any help is appreciated
Here is my code:
' Microsoft SQL Server Integration Services Script Task
' Write scripts using Microsoft Visual Basic
' The ScriptMain class is the entry point of the Script Task.
Imports System
Imports System.Data
Imports System.Math
Imports Microsoft.SqlServer.Dts.Runtime
Imports Microsoft.SqlServer.Dts.Pipeline
Imports Microsoft.SqlServer.Dts.Pipeline.wrapper
Imports Microsoft.SqlServer.Dts.
Public Class ScriptMain
Public Sub Main()
'
Dim package As Microsoft.SqlServer.Dts.Runtime.Package = _
New Microsoft.SqlServer.Dts.Runtime.Package()
Dim e As Executable = package.Executables.Add("DTS.Pipeline.1")
Dim thMainPipe As Microsoft.SqlServer.Dts.Runtime.TaskHost = _
CType(e, Microsoft.SqlServer.Dts.Runtime.TaskHost)
Dim dataFlowTask As MainPipe = CType(thMainPipe.InnerObject, MainPipe)
Dts.TaskResult = Dts.Results.Success
End Sub
End Class
View 3 Replies
View Related
Jul 2, 2014
I've been reading about the "table lock on bulk load" option and TABLOCK hint.
So my understanding is by default only row locks are taken out and other queries can read/write data while the bulk load is going on. However if you were doing parallel bulk loads with overlapping keys from a clustered index then they may block each other.
But if the option is enabled, you can do the parallel bulk loads without blocking because a table lock is taken out, however, other processes couldn't read/write the data until they're all done.
Is that the gist of it? I think I got confused by some misinformation. Don't all those row locks eventually likely escalate to a table lock anyway though?
View 1 Replies
View Related
Sep 12, 2007
Hello,
I have a problem with the Import of an Excel file and hope one of you can help me out.
There is a column with mixed data (format is TEXT) in an excel file and I want to import it as Text (DT_WSTR (255)).
So far everything works fine but some fields like "9760020" imports "9.76002e+006".
My settings so far are:
Provider=Microsoft.Jet.OLEDB.4.0;Data Source=<FileName>;Extended Properties="EXCEL 8.0;HDR=NO;IMEX=1"
In addition I altered the registry entry
TypeGuessRows to 0 (ImportMixedType = Text)
Has someone got a solution?
Thankx
View 3 Replies
View Related
Oct 30, 2007
Hi...
I have data that i am getting through a dbf file. and i am dumping that data to a sql server... and then taking the data from the sql server after scrubing it i put it into the production database.. right my stored procedure handles a single plan only... but now there may be two or more plans together in the same sql server database which i need to scrub and then update that particular plan already exists or inserts if they dont...
this is my sproc...
ALTER PROCEDURE [dbo].[usp_Import_Plan]
@ClientId int,
@UserId int = NULL,
@HistoryId int,
@ShowStatus bit = 0-- Indicates whether status messages should be returned during the import.
AS
SET NOCOUNT ON
DECLARE
@Count int,
@Sproc varchar(50),
@Status varchar(200),
@TotalCount int
SET @Sproc = OBJECT_NAME(@@ProcId)
SET @Status = 'Updating plan information in Plan table.'
UPDATE
Statements..Plan
SET
PlanName = PlanName1,
Description = PlanName2
FROM
Statements..Plan cp
JOIN (
SELECT DISTINCT
PlanId,
PlanName1,
PlanName2
FROM
Census
) c
ON cp.CPlanId = c.PlanId
WHERE
cp.ClientId = @ClientId
AND
(
IsNull(cp.PlanName,'') <> IsNull(c.PlanName1,'')
OR
IsNull(cp.Description,'') <> IsNull(c.PlanName2,'')
)
SET @Count = @@ROWCOUNT
IF @Count > 0
BEGIN
SET @Status = 'Updated ' + Cast(@Count AS varchar(10)) + ' record(s) in ClientPlan.'
END
ELSE
BEGIN
SET @Status = 'No records were updated in Plan.'
END
SET @Status = 'Adding plan information to Plan table.'
INSERT INTO Statements..Plan (
ClientId,
ClientPlanId,
UserId,
PlanName,
Description
)
SELECT DISTINCT
@ClientId,
CPlanId,
@UserId,
PlanName1,
PlanName2
FROM
Census
WHERE
PlanId NOT IN (
SELECT DISTINCT
CPlanId
FROM
Statements..Plan
WHERE
ClientId = @ClientId
AND
ClientPlanId IS NOT NULL
)
SET @Count = @@ROWCOUNT
IF @Count > 0
BEGIN
SET @Status = 'Added ' + Cast(@Count AS varchar(10)) + ' record(s) to Plan.'
END
ELSE
BEGIN
SET @Status = 'No information was added Plan.'
END
SET NOCOUNT OFF
So how do i do multiple inserts and updates using this stored procedure...
Regards
Karen
View 5 Replies
View Related
May 6, 2008
Dear all,,
I need your help,,I'm work in website project using ASP.NET,,I have to register the users of this site,, the users are over 200,,so,,I'm thinking in away to save my time,,All the information of these users are stored in Excel file,,What I want to do is to imports these data from the excel file into a table in my database(SQLserver database),,Could you help in coding by VB.NETThanks in advance,,
View 9 Replies
View Related
Sep 13, 2007
Hello,
I have a problem with the Import of an Excel file with SSIS and hope one of you can help me out.
There is a column with mixed data (format is TEXT) in an excel file and I want to import it as Text (DT_WSTR (255)).
So far everything works fine but some fields like "9760020" imports "9.76002e+006".
My settings so far are:
Provider=Microsoft.Jet.OLEDB.4.0;Data Source=<FileName>;Extended Properties="EXCEL 8.0;HDR=NO;IMEX=1"
In addition I altered the registry entry from Microsoft.Jet.engine.excel
TypeGuessRows to 0 (ImportMixedType = Text)
Has someone got a solution?
Thankx
View 7 Replies
View Related
Sep 23, 2007
I want to write a batch file that will do just that. The problem is bcp or bulkcopy never works with my code, it doesn't recognize it. Maybe I'm going the wrong way to do this, but I could use some help. Ideally I'll make the batch to do the importing via bulkcopy or something like it of a text file into a table and use windows scheduler to automate it. Be as specific as you can please, I'm very new to sql server.
View 6 Replies
View Related
Sep 29, 2006
Hi,
Our company wants to allow our customers the ability to import employee data. Each customer's employee data changes depending on things like organisation structure etc. so the format of the data to be imported needs to change. We can do a lot of this with dynamic SQL, but are looking at moving it to SSIS as we think it will save us a lot of pain later on and for other features in SSIS.
We're stuck at a fairly early part of the process as we don't know how to dynamically import our spreadsheet. So far our best idea is to connect to the excel connection as the only step in the data task and then to use a script task on the control task, write code to connect to the excel source (set the excel source to not treat the first row as headers so we can do a 'select top 1 *...' ) then create a dynamic sql command to create the table.
Once this is done we would then have another data task that actually puts the data into the newly created table. This all sounds very difficult though. What are the options for doing what we want to do... have a feeling that we're missing something basic.
Any help appreciated,
TIA, Anthony.
View 4 Replies
View Related
May 28, 2015
I'm in the process of importing a series of flat files into SQL Server. I'm using a ~ to separate the columns and the row delimiter is {CR}{LF}. One of the files has a field that contains the CRLF combination in a few places so that field is split over several rows. This is readily visible when I look at the flat file. However, when I'm importing the file, the Import and Export wizard seems to ignore them and import the files as they should with one row per record.
View 0 Replies
View Related
Jan 10, 2008
I'm trying to develop an application for a friend, who has his own server and SQL 2005 setup on it Is it possible to add a connection to an externally hosted SQL Database using VWD2008? I have a login and password for it, I just don't know where I set the IP address to connect to and the name of the DB etc...?Any help greatly appreciated.
View 3 Replies
View Related
Oct 17, 2005
hi,am working with my friend so we need to use the same DB ..how can i get him DB and insert it in mine to use it.i 've try to copy it and paste in C:Program FilesMicrosoft SQL ServerMSSQLData then try to connect it by New connection bt it doesn't workNote:am working n Web Matrix ProjectRegards,
View 3 Replies
View Related
Mar 25, 2001
For every one
Visual basic permit use shell command to execute any file
shell "c:programs Filesword.exe
How could I make it in SQLServer 7.0
Sincerely.
Fain Boris.
View 2 Replies
View Related
Feb 20, 2007
have setup SQL 6.5 on a machine. I have verified that port 1433 is opened for SQL. I am also using a router dell 2300. I am running Windows XP.
I have set up my machine to port forward and have setup a static ip 192.168.2.136 nothing I do seems to work.
I can connect to the server by name
ex:
MyStr = "Provider=sqloledb;Data Source=MYSQLPC,1433;Network Library=DBMSSOCN;Initial Catalog=master;User ID=myusername;Password=password;"
Conn.open MyStr
I can also connect using internal IP address
ex:
MyStr = "Provider=sqloledb;Data Source=192.168.2.136,1433;Network Library=DBMSSOCN;Initial Catalog=master;User ID=username;Password=password;"
Conn.open MyStr
but when I try to connect to the external IP 12.xxx.xxx.xxx
I get the dreaded sql does not exist error.
I can register the server by external ip. When I try to start the server I get an (RPC server unavailable) error.
I can ping the external ip.
I can telnet 192.168.2.136 1433 without error
I can connect though security manager using 12.xxx.xx.xx
I can't telnet 12.xxx.xx.xx.
I think its router but I do not know what I'm missing. I have static ip 192.68.136 set up.
Router Port Forward Settings
Service Enabled In Port Dest Addy Dest Port Type
MSSQLSER Y 1433-1434 192.168.2.136 1433-1433 TCP
MSSQLSER Y 1434-1434 192.168.2.136 1434-1434 UDP
I have also read that some ISP's block port 1433 due to the worm problem. So I changed the listining port from 1433 to 1971. I am still able to connect using this new port with local ip or local name. I still cannot connect to external IP. (I did change router setting to reflect new port forward to 1971)
I am not sure what I missing.
Thanks in advance for any help
Chris
View 1 Replies
View Related
Jul 23, 2005
Hello,I want to access my database server from the internet. I've created a ruleon my router to redirect port 1433 to the desired machine, but still get aServer does not exist, or access denied message. When I change the(external) ip to the local IP address, connection succeeds. Is thereanything i need to change within SQL Server to allow external (internet)access?
View 2 Replies
View Related
Jan 15, 2008
Can I include an external file (html) in report's definition? Like a subreport, and expecting show the file content...
thanks
View 5 Replies
View Related
Oct 15, 2007
Hi Experts,
I am looking for some guidance on the following CE based project. The project would mean placing a program on multiple appliances (either PDA mrtphone) which would then periodically connect onto a SQL 2005 server and both read and right to a table on the server. This would mean multiple connections to the server.
I have read various articles which discuss the use of SQL_ce to achieve this but SQL_ce is a single user application. Is it possible to achieve this objective by publishing a share of the SQL 2005 database tables to an internet share folder to allow multiple external connections? Also if it is possible can I test the application on a windows XP platform?
Thanks in advance,
Joe
View 1 Replies
View Related
Sep 12, 2005
I know that the BI studio can build a nice flow document that is associated with the package. Is there an easy way to build a series fo documents that can be shared with other users without having them go into Visual Studio?
View 15 Replies
View Related
Jul 13, 2007
Hello,First, I tried to find the answet to this question, but no luck - so I decided to post it.
When I was creating my applications in asp.net - for the first time I decided to start using external DataSets. I think they are great and work very nice!But once I pushed the application to GoDaddy, I got an error of this nature:I was trying to access the exterbal dataSet like this:Dim productsAdapter As New NorthwindTableAdapters.ProductsTableAdapter()
Dim products as Northwind.ProductsDataTable...And got an error saying that it couldn't find this type (but it worked fine on my local machine in visual studio).So did I miss something that prevented this application working on GoDaddy? or there are some limitations on GoDaddy? ...or something else.Thank you for looking into this for me.Valera
View 7 Replies
View Related
