Moving Data To And From SQL Server 2005, With Manipulation In C#
Oct 9, 2006
Hi
I am wishing to
* take a table of data into a C# CLR procedure and store it in an array
* take a second table of data into this procedure row by row, and return a row (into a third table) for each row (and this returned row is based on the data in the array and the data in this row).
I am new to CLR programming, and pulling my hair out at the moment. I€™m sure that what I€™m trying to do is simple, but I am not making any progress with the on-line help and error messages L. I have now described what I am trying to do in more detail, firstly in English, and then in a T-SQL implementation that works (for this simple example). I€™m looking for the C# CLR code to be returned €“ containing preferably two parts:
* the C# code and
* the CLR code to €˜make it live€™
Since I am not sure where my coding is going wrong. (I think it should be possible to read in the one table, and then loop through thte second table, calculating and returning the necessary output as you do the calculations...
Problem in English
Consider a situation where there are three tables: DATATABLE, PARAMETERTABLE and RESULTSTABLE.
For each row in PARAMETERTABLE, I will calculate a row for RESULTSTABLE based on calculations involving all the entries form DATATABLE.
I am wishing to do this in a C# CLR for performance reasons, and because the functions I will be using will be significantly more complex, and recursively built up.
The values of the results table are calculated as the
sum (for i = 1 to numberofrows)
of
(Parameter1+i*parameterb-datavalue)^2
Which leads to the values shown.
T-SQL Implementation
-- Set up database and tables to use in example
USE master
GO
CREATE DATABASE QuestionDatabase
GO
sp_configure 'clr enabled', 1
GO
USE QuestionDatabase
GO
RECONFIGURE
GO
CREATE TABLE DataTable (i INT NOT NULL, DataValue REAL NOT NULL)
GO
CREATE TABLE ParameterTable (ParameterNumber INT NOT NULL, ParameterA REAL NOT NULL, ParameterB REAL NOT NULL)
GO
CREATE TABLE ResultsTable (ParameterNumber INT NOT NULL, ResultsValue REAL NOT NULL)
GO
--Initialise the Tables
INSERT INTO DataTable (i, DataValue) VALUES (1,12.5)
INSERT INTO DataTable (i, DataValue) VALUES (2,10)
INSERT INTO DataTable (i, DataValue) VALUES (3,14)
INSERT INTO DataTable (i, DataValue) VALUES (4,17.5)
INSERT INTO ParameterTable (ParameterNumber, ParameterA, ParameterB) VALUES (1, 10, 2)
INSERT INTO ParameterTable (ParameterNumber, ParameterA, ParameterB) VALUES (2, 11.7, 1.1)
-- The TSQL to be rewritten in C#, to produce the Output as hoped
INSERT INTO ResultsTable (ParameterNumber, ResultsValue)
SELECT ParameterNumber, SUM((parametera+i*parameterb-datavalue)*(parametera+i*parameterb-datavalue)) AS b
FROM DataTable
CROSS JOIN
ParameterTable
GROUP BY ParameterNumber
-- Output as hoped
SELECT * FROM DataTable
SELECT * FROM ParameterTable
SELECT * FROM ResultsTable
-- which produced
1 12.5
2 10
3 14
4 17.5
1 10 2
2 11.7 1.1
1 20.5
2 18.26
-- but I hope to do the same with something like:
CREATE ASSEMBLY *** FROM 'C:***.dll'
CREATE PROCEDURE GenerateResultsInCLR(@i int, @r1 real, @r2 real)
RETURNS TABLE (i int, r real)
EXTERNAL NAME ***.***.***
EXEC GenerateResultsInCLR
This is a simple example, that can be easily written in T-SQL. I am looking to develop things that are recursive in nature, which makes them unsuited to T-SQL, unless one is using cursors, but this becomes very slow when the parameter table has 1m records, and the data table 100k records. This is why the datatable must be read in once, and manipulated many times, and the manipulation will need to be in the form of a loop.
Hi All, We have a Progress DB in our Company. We are trying to move all the data to SQL Server 2005. When I try to run the Import Data task from SQL Server 2005 the radio button called "Copy data from one or more tables or views" is getting disabled and it is asking me to write a SQL script to copy all data from 120 tables.
I am using a .NET Framework Data Provider for ODBC when I run the Import Data task from SQL Server 2005.
I am using the DataDirect 4.1 32-Bit Progress SQL92 v9.1D ODBC Driver to connect to Progress DB. I am getting a connection but the copy feature is getting disabled.
Can anyone please help me to resolve this issue, or even if you provide me with some pointers that will be really helpful.
I am trying to load data from an Excel spreadsheet file into SQL Server 2005 Express. I understand that DTS is the best tool for doing this but from my research it appears that DTS is not available with the Express edition and the import wizard that does come with Express is not well suited for this type of conversion.
Does anyone have any suggestions for how to achieve this objective? Thanks for any help you can provide.
Problem: Moving data from mysql to sql server 2005
I am trying to pull data over from mysql to sql server. First the import wizard greys out so I have to put in 1 query at a time which is pain. and second it does not even work! it takes me through the end of the wizard for me to click finish and then says oops it does not work. there was an error!
Anyway i tried going through the ssis route cuz its going to be a nightly job. i used the ado.net odbc connection. It worked but the performance is really not acceptable. it took 5 mins to import 24000 rows where as dts was taking 1 sec to do this. i wish i could use the native mysql odbc 3.51 connector and import. can some one give me step by step instructions on how to do that ?
I hear someone mentioned of using excute sql task which can use mysql odbc 3.51 driver. but since i am new how do i get it to work. say for example in the excute sql task i run a statement like select * from addr. then what?
cuz eventually i want the result to be saved in a sql server table called addr. How can i get the result from that excute sql task and put it inside of an addr table in sql server. should i save the result to a variable of type object. but then how do i get the data from object and tell sql server in the designer that the result contains these columns and it needs to map to these columns in the addr table of sql server.
Very confused. i wish the first option would have given me results which an enterprise ETL gives. but apparently it is too slow that it wont be acceptable in a production envrioment. when i will have millions of rows coming in .
I'm a newbie on SSIS and am trying to grasp my way through this.
I am trying to copy data from a Sql Server 2000 database to a simplified table in Sql Server 2005 database.
What I want is to move the data to a staging table, then drop the main table and rename the staging table to the main table, to minimize the down-time of the data. I can't get the workflow to work, because the staging table has to exist when I run the package. I thought I could use an "Execute SQL" task to generate the table before I would run the task, but that doesn't work. Am I going about this the wrong way? Is there an optimal solution to this problem so my data can be accessible as much as possible.
Hi,I recently installed SQL Server 2005 Enterprise Edition (9.0.1399) andI have problems on queries concerning system file manipulation...For example, when I try to increase the Log File of MSDB database :ALTER DATABASE msdbMODIFY FILE(NAME = 'MSDBLog',SIZE = 50MB)This query never terminates...All SQL queries for database creation never terminateCREATE DATABASE EASYSHAREON PRIMARY(NAME = EASYSHAREData,FILENAME = 'D:AQSACOMDATAEasyshareEasyshareData.mdf',SIZE = 200MB,MAXSIZE = 1000MB,FILEGROWTH = 100MB)LOG ON(NAME = EASYSHARELog,FILENAME = 'E:AQSALOGEasyshareEasyshareLog.ldf',SIZE = 100MB,MAXSIZE = 300MB,FILEGROWTH = 50MB)Would someone have an idea of what happens...I'm running on Windows 2000 Server...ThanksPatrick
We need to upgrade our HR system from SQL 2000 to SQL 2005. Support for SQL 2000 ends 12/31/2007.
We have a third party vender that do not currently support SQL 2005. Employee's data is transfered from our HR database to the third party database for the purpose of yearly performance evaluation on a daily basis.
Both databases are currenlty on a SQL 2000 server.
Can data be transferred from a SQL 2005 database to a SQL 2000 database?
How to move some tables with data & procedures etc from 1 database to another in sql server 2005 express edition. i did by scripting but i transfer tables and procedures and not data data is the problem. tnx
I'm moving data between identical tables and have to use a flat file as an intermediary. I thought: "No problem, SSIS can do a quick export to a file, then move the file to another server, then use SSIS to import the data to the new server."
Seems simple, right?
I'm hitting all sorts of surprising data conversion errors. I used the export wizard to create the export package. This works fine. However using the same flat file definition, the import package fails -- even when I have no destination. That is I have just one data flow task that contains only one control: the Flat File source. When I run the package the flat file definition fails with data type conversion and truncation errors. One of the obvious errors is for boolean types. The SQL field is a bit, SSIS defined the column as DT_BOOL, the output of the data are literal text values "TRUE" and "FALSE". So SSIS converts a sql datatype of bit to "TRUE" and "FALSE" on export, but can't make the reverse conversion on import?
Does anyone else find this surprising? I would expect that what SSIS exports, it can import given all the same table and flat file definitions. Is SSIS the wrong tool to do such simple bulk copies? I'd like to avoid using BCP because this process will need to run automatically within SQL Agent so we can leverage all the error tracking and system monitoring.
The logic is like the min(effdt) as effdt, min(effdt) - 1 as expdt of the next id2 from the source. Nut the problem comes when the same ID2 comes later after a different id2, so just a group by on id1, id2 does not suffice. If it is the last record then the expdt is that month's end date. Can someone throw me some light on what kind of logic I should be using to accomplish this.
Is it possible to INSERT, UPDATE, DELETE data in this type of relationship? If so, how in VS2005? Not having issues with SELECT, even without joins. Thanks
Hello Everyone and thanks for your help in advance. I am setting up a new testing and development machine. My previous machine had VS 2003 and SQL Server 2000 installed on it. The new machine is going to be VS 2005 and SQL Server 2005 Standard. I have approximately 12 databases on the old machine in SQL 2000 that I want to bring over onto the new machine and convert to 2005. What is the best way to go aobut this. I'm sure I'm not the first to do this, but I'm having trouble finding documentation on this topic. Any help would be greatly appreciated. Thanks.
I'm new to SQL server. Basically, I am trying to make my Access database accessible on line to my users, so I am transfering it into SQL Server and then writing a web site to use the connection to the database.
My questions are as follows:
1) How can I easily move the structure of the tables (relationships too, but content is not important) from Access into SQL?
2) I have the following query in Access which I don't believe will work in SQL because of the Sum funtion... How can I change this to work in SQL?:
SELECT DISTINCTROW Player.TeamID, Player.PlayerID, Player.SFD, Sum(Payments.Amount) AS Payment, [SFD]-[Payment] AS Due FROM Player INNER JOIN Payments ON Player.PlayerID = Payments.PlayerID GROUP BY Player.TeamID, Player.PlayerID, Player.SFD;
All help will be greatfully received... thanks
"In the face of adversity, I stand on the shoulders of giants..."
Our current sql server 7 is overloaded and I have been asked to move some data to a new box which is to be acquired. My query is as follows:
1what should be the ideal spec of a sql server to allow for windows 2000 in future. And how do I go about moving data from one to the other. I am new to sql and this is my first assignment.
I have a server that will be going off-line in a couple of weeks, and in the meantime, I need to backup everything and move it to a new location. This involves a lot of data and databases.
What I have is 33 databases in MS SQL Server 2005. The target machine is MS SQL Server 2008. Each of these databases has a myriad of stored procedures and compiled functions that will need to be moved as well. Essentially the task is to completely replicate the entire MS SQL environment.
Is there a shortcut to making this happen or am I relegated to backing up each database individually and then restoring them individually?
Access 2k -> SQL Server 2kMy client has an app that is A2k FE with A2k BE. They have asked meto move the BE to SQL Server.I have a bit of experience with SQL Server, and I'm happy withscripting the database etc.However, when it comes time to move the data itself, I have a teensylittle concern.Let's take our typical Customers -> Orders relation, where CustomerIDis the Foreign Key in the Orders table.Let's say I have 4 customers, but I used to have 6. The CustomerID isan AutoNum column, and they are 1, 2, 4, 6. However, when I insertthese records into the matching SQL Server table with an Identitycolumn, they will (presumably) be 1, 2, 3, 4.So what can I do about the matching orders? Is there any alternativeto this, which seeems very long winded.1. Insert the records as above, but copy the OLD Access AutoNumcolumn (called OldCustomerID) into a temporary column in the new SQLServer table for both the Customers and Orders tables.2. Insert the Orders records into the new SQL Server table.3. Run an update query thus:UPDATE OSET CustomerID = C.CustomerIDFROM Orders OINNER JOIN Customers C ON O.OldCustomerID = C.OldCustomerIDCan I do this any easier? For example, is there any way in SQL Serverto maintain the values of the AutoNum field in the Insert but havethem still be Identity fields? I know that I can import the AutoNumdata into a simple int column, and once the import is complete, turnON the identity, but is this reliable? Might I lose any dataintegrity?TIAEdward--The reading group's reading group:http://www.bookgroup.org.uk
I'm trying to use SQL for data warehousing using dates for manipulating data. As SQL doesnt have date, just timedate, and if I want to compare 2 dates to obtain someones age ( ie date - date of birth = age ) I have to use a TIMEDIFF or equivelent. This is REALLY annoying. Is there anyway to use just dates without having to code extra?
Does SQL ignore the time part of a datetime column - ie can I just do a date manipulation without concerning myself with the time ( assuming the time is NOT default at 00:00:00 for every entry ) or am I stuck with treating the field as a complete datetime? It may seem a dumbass question, but as I have to do a lot of date manipulations, it is essential to know.
I have data in a trace file, and I need to extract some info such as phone number.The problem is the phone number could be varying lengths, and various positions in the row.
For example:
@City='New York', @Phone='2035551212' (10 characters, no dashes) or @City='San Francisco', @Phone='918-555-1212' (12 characters, with dashes) or @City+'Berlin', @Phone='55-123456-7890' (14 characters, with dashes)
I can use CHARINDEX to search & find @Phone=' so I know where the phone number starts, but stuck on a programatic way to find the data between the quotes since it can vary.
My old database server runs SQL 2000 and has the CRM 3.0 app and database, our corporate web site (SharePoint Services 3.0) database, and Project Server 2007 database. The corporate site and project apps and web sites reside on our file server.
I want to move, upgrade or migrate the databases, corporate site and Project site to this new SQL 2005 server. I am a little unsure the best way to do this.
Should I move all the database first, then move the apps?
Should I do a restore of everything on the new server, and after I know it works just turn off the other server?
I have done a bit of searching around and cant find a clear answer to this question.
Current Setup Desktop application (c#) that connects to a SQL Server 2005 express database on the same local network as the application (currently 3 users)
It is only a very small company and has just taken on their first remote worker, but expects to take on another 6-8 over the next few months. They have asked for the database to be moved online.
The application was written in such a way that everything has been done using no stored procs, or views, it is all native SQL.
This will be my first DB hosted online and before I go ahead and do anything I just wanted to make sure what I have to do is correct, sorry if this is a very basic question, although I have been programming for a long time, I have never had the chance to do any online databases before.
Will this work. 1.Find a SQL Server 2005 Hosting company. 2.Move the database to the server. 3.Setup the users permissions. 3.Alter the connection string in the application to point to the new location.
So the only thing that would change would be a new connection string in the application preferences?
Or am I living in a dream world, because nothing is ever that simple.
One thing I am worried about is the security/visiblity of the database and data as it travels from the server to the client and back.
I need to write stored procedure in SQL Server 2000 that moves data from table in SQL Server 2000 to same table in Access. .mdb file is located on the same computer and I know its location (path).
as mentioned in title i'm trying to move data from excel into a table in a sql server 2000 database. We have spreadsheets that produce data in excel using querytables.
we want a system the other way, by typing data into a spreadsheet, pressing a button which pushes the data into a sql server table.
I have been programming an application with VC++ 2005 and SQL Server 2005. I have converted an old 16-bit database to 32-bit managed code and SQL server and the application seems to be good. Now I want to deploy the application to another server for testing.
I have installed XP SP2, Windows Installer 3.1, Net framework 2.0 and SQL Server 2005 express to the test server. I have transferred the application with WI 3.1 and the program works well in the test server till the first SQL command. I have made a back up of the database and restored it in the test server. In the test server I can log in the database with Server Management studio and I can read the data there correctly. I have enabled both named pipes and TCP/IP for the database in the test server. With Surface Area Configuration I have enabled Local and Remote Connections Using both TCP/IP and named pipes. I only need Windows authentication at this time.
After all this when I come to the first SQL command in the application on the test server I receive the error message: An error has occurred while establishing a connection to the server. When connecting to SQL Server 2005, this failure may be caused by the fact that under the default settings SQL Server does not allow remote connections. (provider: Named Pipes Provider, error: 40 €“ Could not open a connection to SQL Server).
My connection string to the database is: 'connection->ConnectionString = "Persist Security Info=False; Integrated Security =SSPI;" "Data Source=TESTSERVER; Initial Catalog=TESTDATABASE;";'
When I use "Data Source=DevelopmentServer", the application works well on the development server.
Can't understand what is still wrong. Can you possibly have an answer for me?
I have about 50,000 data entries to move from MS Access to SQL Server 2005 Express. There is no DTS in the tools. I already have the tables, just need to move the data. Appreciate any and all help.
Hello, I have been having a bit of trouble finding help on the safestway to move data files to a different disk on the same server. Mosthelp is about moving data files to a different sqlserver. I just wantto move the files to a different drive on the same server. Any helpwould be appreciated.Thanks,David
Followed this article to move my model and msdb databes but I think I messed up. http://msdn.microsoft.com/en-us/library/ms345408.aspx The instance will not start because it says one of the files does not match it's primary file (not sure if it's model or msdb)
I can bring the instance up in single user mode (NET START mssqlserver /f /t3608) but when I try to do a query to re-do my alter statements, it says i can't attach because only one seesion is allowed. in 2000 i used to be able to start query analyzer without starting enterprise manager..
When i try and start with a minimal config (sqlservr -c -m) it won't start. I get an error saying that one of my files does not match it's primary file (either model or msbd don't know which) any ideas????????
I am a new in .Net Environment. I am moving from VB to VB.Net and Access DB to SQL Server 2005. Please reply me the following questions bellow.
Access Works
SQL SERVER (SS)?
When I create .MSI file it include ADO library in that executable file and my client install software and don€™t need any kind of file to install and wherever my program install it can be accessed by using following connection string. Con.ConnectionString = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & App.Path & "dbPAYROLL_DB.mdb;Persist Security Info=False" Con.Open
What file needed on client€™s PC to access SS on server. What about connection string change dynamically in client sides
I create relation on Access Relationship Diagram.
Where to create these diagrams either on VB.Net Server Explorer or on Management Studio? And how?
Please answers me this basic questions further I have more question in mind but please first answers me these questions€¦