NEWBIE Needs Help Automating Data From Another Server
Sep 7, 2007
Hi,
I am able to do these actions interactively from SQL 2005 (not developers nor enterprise edition, just using SQL 2005 Mgmt Studio) and want to "script/batch" them so I can have them automatically run at a pre selected time.
First: I am able to delete the table by performing a right click on the table, then click Delete from Mgmt Studio SQL 2005. I verify the table is completely gone with a refresh. (I pulled the code that did this ..... DROP TABLE etc. to Notepad)
2nd: I am able to import the table (again from Mgmt Studio SQL 2005) and have saved this action as a SSIS. Execute the script and "waLa" I have all 17K rows of data. I pulled this create table code into notepad also.
Now I put the code of both of the above actions together (drop table and create table) into one SQL query and execute it. This does not give me the same results of above, instead my table is blank now.
Maybe there is a better way. The business problem I am attempting to solve: I am refreshing the data in a as/400 table weekly. I want that refreshed data to be available in the SQL2005 database without my having to press buttons first thing Monday morning. Can any one help? Thanks in advance.
Below is the Code:
USE [400kas]
GO
/****** Object: Table [dbo].[navar100] Script Date: 09/07/2007 16:09:04 ******/
DROP TABLE [dbo].[navar100]
GO
USE [400kas]
GO
/****** Object: Table [dbo].[Query] Script Date: 09/07/2007 16:12:31 ******/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE TABLE [dbo].[navar100](
[CMPNO] [decimal](3, 0) NOT NULL,
[ARTDT] [datetime] NOT NULL,
[AUDDT] [datetime] NOT NULL,
[ARDDT] [datetime] NOT NULL,
[CCUS#] [decimal](6, 0) NOT NULL,
[CCNAM] [nvarchar](25) NOT NULL,
[CUSNO] [decimal](6, 0) NOT NULL,
[CNAME] [nvarchar](25) NOT NULL,
[SHPNO] [decimal](4, 0) NOT NULL,
[ARRCD] [nvarchar](1) NOT NULL,
[AUDUS] [nvarchar](10) NOT NULL,
[INVNO] [decimal](6, 0) NOT NULL,
[CUSPO] [nvarchar](15) NOT NULL,
[REFNO] [decimal](6, 0) NOT NULL,
[COMNT] [nvarchar](10) NOT NULL,
[SHPPO] [nvarchar](15) NOT NULL,
[AMONT] [decimal](13, 2) NOT NULL,
[AMOUNT] [decimal](24, 8) NOT NULL,
[REMAN] [decimal](13, 2) NOT NULL,
[INREG] [decimal](3, 0) NOT NULL,
[INSAL] [decimal](3, 0) NOT NULL,
[TMCOD] [nvarchar](2) NOT NULL,
[CRHLD] [nvarchar](1) NOT NULL,
[CRLIM] [decimal](13, 0) NOT NULL,
[CRDAY] [decimal](3, 0) NOT NULL,
[TCRCD] [nvarchar](3) NOT NULL,
[TEXRT] [decimal](11, 6) NOT NULL,
[R1RGL] [decimal](13, 2) NOT NULL,
[TAXAM] [decimal](13, 2) NOT NULL,
[TFRTX] [decimal](13, 3) NOT NULL,
[TFRGT] [decimal](13, 2) NOT NULL,
[TSPCH] [decimal](13, 2) NOT NULL,
[SPCST] [decimal](13, 2) NOT NULL,
[IRPFT] [decimal](13, 2) NOT NULL
) ON [PRIMARY]
I would like to know recommendations on automate the following:
1. I want to know how to take data from various sql server 2000 data queries and load the data onto excel spreadsheets in an automated method. Some of the data loaded into the excel spreadsheets load detail data and some load data into pivot tables.
Is there any way that the data can be taken from sql server to the excel spreadsheets using odbc connections?
2. Some of the data taken from sql server 2000 database is loaded into pdf files. Is there any way this process can be automated?
question... i heard a bit about Control Macro Generator but have not found much reading material on it. here at our company we have employees fill out a weekely timesheet and typically have clerks enter the information into the TM.PTA.00 (Project Timesheet with Rate/Amount Entry) and submit it that way. say we have this timesheet in a nice little excel sheet with columns that match the input boxes on the form, how can we get it to automatically input the data instead of having a person sit there and type it all in?
I have written a Data Processing extension for my application and can deploy it on my development machine no problems. My question is: what is the "correct" way of deploying an extension to an end user's machine? Do I have to write a special program to find and modify the Reporting Server config files and copy the extension over. Surely many developers have the same need so there must be a generic solution to this problem, however, I haven't managed to find one.
I first came across this issue in SQL2000 and I thought/hoped it would be rectified with SQL2005 but it appears not to be (unless I'm missing something).
I have a sql query that I would like to run every morning and output into a csv file.Unfortunately, I can't seem to find simple instructions on how to use SQL Server Agent to run that.
Hi. I just set up my first sql server database and I've managed to connect to it via ASP as a test. I'm not sure how to add data to my tables. In MS Access, you can edit the table and add records. How do I do that in SQL Server? I'm using the Enterprise manager tool to create tables.... does it have something i can use?
We have the following scenario: We receive CSV files every month for which SSIS packages were built to process the data. The following problems occur from time to time:
1. The structure of the CSV file changed (e.g. column added or removed) 2. There were no footers in the data, but now footers started to appear 3. Date format changed (e.g. used to be mm/dd/yyyy, but became mm.dd.yyyy) 4. Number format changed (e.g. from 2000 to 2,000)
Currently we have person who manually opens each file, and using our "validation document" validates to ensure none of these or similar problems occur. We would like to move away from this manual process if possible. I understand that items 3. and 4. could be caught by loading data into a staging table with VARCHAR data types, and performing validation before moving it any further.
Item 2 is a bit questionable (meaning depending on the footer size SSIS load could fail or not).
Item 1, however, is a sure fail of the SSIS package that directly loads the data into a table.
Thus I feel the two possible options are:
1. Create a custom script that will run through the file, row by row, apply all the necessary validations and report an error or continue if all checks out
2. Use some 3rd party tool to validate the files (semi-manually) before kicking off the SSIS processing.
Hi, hope someone can help or point me down the right track.
I have to load 50+ tables (all have different file layouts) using a data reader (The source only allows an OBDC connection) into a SQL Server 2005 databases, rather than create 50+ ETL packages that will be identical in process terms is it possible to create a single package that will dynamically re-map the source destination joins. I know I can set the Source and Destination using expressions however how do I ensure that the mappings are updated and for the errors can I automate so that they re-direct? Is this possible using a script task? Or by some other means?
Can someone help me to understand something about SQL Server? I will use Access for comparison.
Suppose you have a SQL Server database and you want users to be able to enter data into the fields just like you could with an Access database and forms. How would you set that up with SQL Server? I have noticed that you can convert an Access database into SQL Server, but you still need to use Access forms to enter data into the tables stored in SQL Server.
What if you wanted to create an SQL Server database from the ground up and set it up for user entry?
The basic question What is the best way to export data from an excel spreadsheet into a sql server table? My Application Getting data indicating hours worked from employee timesheets into a centralised DB, then running analysis reports on it. The columns and datatypes in the excel sheet are as follows: Week (int) | EmployeeID (int) | JobNum (int) | ActivityNum (int) | Hours (int)
There will be a new excel file each week that, once the employee has filled out the data, would need to be saved and exported to the sql table. The columns in the sql table are exactly the same as the excel table with the addition of a RecordID primary key column.
Can I create a macro button that they can push when they have completed their timesheet OR would it be better to tell the employees to save copies of their timesheets in a certain folder on the company network and then run a batch on all the files in the folder at the end of the day?
Or is there another more efficient solution? Would I use SSIS for this or something else?
I've never used SSIS before and am a newbie at SQL Server too.
I will be getting data in either Excel or Access form on a daily basis. I would like to automate the process of converting this (excel or access) data to a table in an existing SQL database. Since this conversion needs to performed on a daily basis, note that I need to update the table that contains data from the day before.
Is it possible to do this and if it is possible, can someone tell me how to do it.
I'm a newbie on SSIS and am trying to grasp my way through this.
I am trying to copy data from a Sql Server 2000 database to a simplified table in Sql Server 2005 database.
What I want is to move the data to a staging table, then drop the main table and rename the staging table to the main table, to minimize the down-time of the data. I can't get the workflow to work, because the staging table has to exist when I run the package. I thought I could use an "Execute SQL" task to generate the table before I would run the task, but that doesn't work. Am I going about this the wrong way? Is there an optimal solution to this problem so my data can be accessible as much as possible.
We're trying to put a view of data maintained in desktop Access databases online and into SQL Server.The desktop Access system uses separate databases instead of tables within one database, It's a strange design, but it can't be changed.We have been importing all of the separate databases into a single, new Access database, then upsizing the new databse to SQL Server, then uploading it.This is not going to work long term, because we are stuck with a 250 mB Access database to upsize and upload, when we never need to update more than 2 or 3 of the tables and upload more than 2 mB.We'd like to be able to upload only the tables -- preferably the Access *.mdb's -- that have changed, and then replace the SQL Server tables with the new information. And we'd like to automate it as much as possible, without upsizing Wizardy.I don't know where to even begin looking for information about how this might be done.Any suggestions would be deeply appreciated.- Tinker
I am able to import - export manually, how can you schedule this import export in a job? I want the table to drop and be rebuilt each time it runs. Since linking servers isn't recommended I want go with that approach.
Is there anyway to automate the configuring Reporting Server after is installed. I have many servers that I need to install SQL 2005 with Analysis servcies and Reporting Server and it will be good if I could run a tool to configure reporting server with some default values.
I figure this should not be a complex one. I know how to manually pull in data from Oracle 9i into SQL Server 2000 using DTS. However this is my issue.....
I simply want to automate the pulling in of data from 1 table in my ORACLE 9i database into another table in my Sql Server 200. I was hoping I could simple write a stored procedure that would sort of utilize a dblink like in ORACLE and then schedule that procedure. Is this feasible in Sql Server, and how would one go about setting this automated import up????
Is there any Best Practice, 3rd Party Tool for Automating SQL Server Patching for around 10000+ SQL instances.
It is very difficult to do the same manually because each server will consume atleast 1-2 hours and if this is completely done manually then n number of people might be required for accomplishing this task which doesn't looks feasible at all.
Hello All, I am migrating data from one database to another. I am using Multicast to seperate (legal street,legal mail and legal city) and (mail_street,mail_state,mail_zip,mail_city) also later after UNION of the above I am doing two lookups as I had to get contact ID and Customer ID from other two tables. In UNION i am matching (Mail street legal street) and so on.
I am getting double the data in the output. my input data is 1000000 and im gettin 2000000.
CREATE TABLE [Sales].[Test_inmem] ( [c1] [int] NOT NULL, [c2] [nvarchar](20) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL, [ModifiedDate] [datetime2](7) NOT NULL CONSTRAINT [IMDF_Test_ModifiedDate] DEFAULT (sysdatetime()),
[Code] ....
I have to generate 1000000 random records into it. I tried various ways to insert records, but not being a developer could not do it. I hope to make the C1 as a serial number, C2 can be anything, C3 I want to be the timestamp.
Hi everyone... sorry, but I'm really new at this! I'm trying to make a table out of an imported Excel file (which is automatically generated as an all-text data type). I wrote a query that takes this excel file and puts it into the format we need (i.e., appropriate column headers, removing "garbage" characters and spaces). Is there any way to format the data going into the table by a specified datatype?
Sorry if this is simplistic or wasting your time, but I'm really stumped :o. However, keep in mind that I started learning SQL this morning and I've already gotten this far :).
I am just starting out with MSSQL, but have previous experience with MySQL and PGSQL. What I'm trying to do is do a nightly dump of data from a proprietary DB into MSSQL. Access to the prop DB is through an ODBC driver.
I already have a DTS script that will dump a specific table, but every time it runs it just appends the data to the end of the table.
What I am looking for is a way to download a list of tables, upload them into the MSSQL DB, and if something failed, rollback to the state before the data transfer and somehow alert of the fact.
Hello everyone, I am new to mssql so please excuse all my silly questions.
I can create databases and tables. This is fairly easy, now I am having trouble with inputting data into the database using the data entry browser (or whatever it is called to enter data directly into a table).
I usually get an error when trying to submit the entry, firstly I have a "id" field which I think I have set to auto increase (Identity is ticked, seed and increment is set to 1).
when I go to add data all fields have "Null" in them and from what I understand it allows the fields to be empty, except the primary key which is the id field in this case and I expect that to be filled in automatically like with mysql but isn’t.
Can some kind person instruct or direct me somewhere on how to enter data into a table without it failing.
It was suggested to me to try and post this here in the sql express forum.
I am trying to query a specific field from my sse database based on two different variables. And use this information to fill a textbox with an integer.
The first variable is a combobox that is filled with data in one of the other tables in the sse database. This table has a PKcolumn that is an integer and the identity of the column. The only other column is a nvarchar(30) string (which is the actual data in the combobox)
The second variable is a numeric up down that isn't actualy using info from the database but will be a reference. It has a max value written in to the code:
if NumericUpDown1.Value > 20 Then
MessageBox.Show("This form only supports characters up to level 20")
NumericUpDown1.Value = 20
The texbox value will be found in a table that has several columns, the first column is an identity column (also the PK) then it has a column that holds an integer value (the filter criteria for the numeric up down) the next column has the value that will fill the text box, then several other columns that are irrelevant for this, and then the final column is a foreign key that is linked to the primary key of information found in the table that feeds the combo box.
I will post a picture of the database diagram to show all of this information for a better understanding of this post.
I am not looking for someone to write the code for me, but I am totally stumped as to what I need to do, If someone can help me out with this I would greatly appreciate it, thanks in advanced.
http://static.flickr.com/75/154867997_a55d3b0b9e.jpg - this is the table diagram
Hi there. i have the following excel spreadsheet that needs to be moved into a database:
ADDRESS STREET SURNAME GIVEN PHONE CITY POSTAL NOTES TERR
40 Goodless Crt You M (416) 123-1234 SC M1B 1A1
SC-123-12
59 Bellevue Ave Doe Jon (416) 123-1234 SC M1B 1A3
SC-123-13
Most of these fields are going to be created as reference tables... and the main contact list table will only include a fk to the actual value. eg) the contact table for the above will look like:
ADDRESS STREET SURNAME GIVEN PHONE CITY POSTAL NOTES TERR
40 1 You M (416) 123-1234 1 M1B 1A1
5
59 2 Doe Jon (416) 123-1234 1 M1B 1A3
6
My question is, what is the best way to move all this data into SQL Server Express? I've been playing around with the LinkedServers section in sql and have managed to create a connection to my excel workbook. I am able to select all worksheets as tables... and i've also been able to select all records from the main contacts sheet. not knowing any better, my initial thoughts are to create separate sql statements for each reference table. eg) select street from excellink...contact$ and then combine it somehow with an insert statement into the streets table in my database. Is this the right approach? if it is, can you help with the syntax of the sql statement? Please and thanks.
Hi there I sorry if I have placed this query in the wrong place. I'm getting to grips with ASP.net 2, slowly but surely! When i try to access my site which uses a Sql Server 2005 express DB i am receiving the following error:
Server Error in '/jarebu/site1' Application.
Database 'd:hostingmemberasangaApp_DataASPNETDB.mdf' already exists.Could not attach file 'd:hostingmemberjarebusite1App_DataASPNETDB.MDF' as database 'ASPNETDB'. Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code. Exception Details: System.Data.SqlClient.SqlException: Database 'd:hostingmemberasangaApp_DataASPNETDB.mdf' already exists.Could not attach file 'd:hostingmemberjarebusite1App_DataASPNETDB.MDF' as database 'ASPNETDB'.Source Error:
An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below. Stack Trace:
Version Information: Microsoft .NET Framework Version:2.0.50727.42; ASP.NET Version:2.0.50727.210
This is the connection string that I am using: <connectionStrings> <add name="ConnectionString" connectionString="Data Source=.SQLEXPRESS;AttachDbFilename=|DataDirectory|ASPNETDB.MDF;Integrated Security=True;Initial Catalog=ASPNETDB;User Instance=True" providerName="System.Data.SqlClient"/> </connectionStrings>
The database is definitly in the folder that the error message relates to. What I'm finding confusing is that the connection string seems to be finding "aranga"s database. Is it something daft?
We currently have about 900 stored procedures which have logic to group healthcare claims into different 'Edit Groups' depending on the logic within each 'Edit' stored procedure.
Examples of the logic for the Edit stored procedures would be something like:
Edit1: Find all claims from same patient and same provider (matching on SubsriberID and ProviderID) which has a procedure code in (P1, P2, P3....P345) and a diagnosis code in (D1, D2,...D123) and does NOT have a modifer code in (M1, M2, M3)
Edit2: Find all claims from same patient and same provider (matching on SubsriberID and ProviderID) which has a procedure code in (P7, P8, P9....Pxxx) and a diagnosis code in (D1, D2,...Dyyy) and has a modifer code in (M3, M4, M7), which are dated within 120 days of each other.
Do you think one of the SQL Server 2005 Data mining algorithms (Clustering or Classification or Association Rules) could play some part in this? Most of the 900 stored procs can be grouped based on logic, I mean the logic is similar for each group and only the parameters (in brackets above) vary for each stored proc within the same group.
We're totally new to data mining, although we do have some moderately complex cubes running. Which algorithm (if any) would be the most appropriate for our needs?
Completely new to the world of databases. I'm a designer who works primarily in Flash. In any case, I'm trying to manage an application that uses MS SQL and learn about the wonderful world of databases.
Ok, I modified a table (e.g. I added a column called "Rate") that had associated views (created by another developer). Noticed that my application went a little wonky as some of my variables within my app took on the value of the data in the "Rate" column. I checked one of the views and noticed that a column within the view (e.g. TutorID) was assuming the values in the "Rate" column. Note: The column TutorID had been blank before the change to the table. I'm completely lost as to why this is happening. Do I need to rebuild the view? Can I just reset the original view?
Thanks.
Oh yeah, I'm using SQL4X Manager J from Mac Guru (if that helps).
I've exposed my data (that exists in a proprietary format) with the ADO.NET provider interfaces (IDbConnection, IDataReader, IDbDataAdapter and IDbCommand). I can't seem to find any examples of how to get Integrated Services to hookup to this .NET code in my class library. Is it possible? My goal is for this provider to be both a destination and a source and for others to be able use IS to manipulate the data however they want.
The following is a list of questions that I have not been able to obtain concrete answers. I am probably missing something: 1) ReadWriteVariables -- can the updated value for a ReadWriteVariable be accessed within the same data flow? It appears not as I think the PostExecute() fires at the completion of the data flow not the end of the Script Component. Secondarily, the Script Component is a non-blocking transformation so the component does not "see" the end of the pipeline prior to sending data down stream.
2) Record Count -- Because of #1 above, How could you calculate a record count for a data stream? It does not appear that one can calculate the number of records for a data stream within a data flow and then access the count from within the same data flow.
3) FinishOutputs() -- Is the concept of FinishOutputs() applicable to Script Component Destinations? Asked another way, is FinishOutputs() executed at the end of the data stream regardless of whether there are "real" outputs for the component? I can create a "Dummy" output to create FinishOutputs() but is this ok?
4) Script Component -- It appears that the Script Component Source, Transformation or Destination are really defined based on the columns defined in "Inputs and Outputs". Can you convert an Source script component to a transformation script component by simply adding an Output?
Sorry for these basic questions but I am not getting it completely. As you can tell...
I have the membership stuff up and running. I've added a field to the membership table called custnmbr. Once a user logs in, I want store his custnbmr in the session and use that to lookup data in another db. ie: Joe logs in and his custnumbr is 001, he goes to the login success page and sees his list of service calls which is: select top 10 * from svc00200 where custnmbr = 001 (the membership.custnmbr for the logged in user) I know how to do this in old ASP using session variables....but I have no idea where to even start with .Net. Many thanks
We are post-deployment with a serious reporting issue that's causing us to rethink our reporting solution. We're considering moving from what we have to SQL Server Reporting Services (client side).
I have spent the past couple of days getting up to speed on this feature and seeing if we can easily migrate it into our existing application. However, I've hit a couple of stumbling blocks and was hoping perhaps someone here could either tell me the solution or point me in the right direction.
I'm not having any problems creating basic reports (e.g. flat data).
However, I am having problems creating reports where there are related tables in a dataset.
The way it works with our existing solution is that I get a dataset (which contains several data tables) and point it to the report's datasource. That report expects those tables and I have defined table-relationships in the report which process and display the information correctly.
I'm not having as much luck with RDLC.
I can go into futher detail about how I'm creating the report, but let me just ask these general questions first: 1. Can I set a dataset containing multiple datatables equal to a property on an RDLC report and that RDLC report know how to treat and display the data? 2. Is there a better/smarter than this to get a field selection from my datasource (remember, this information is coming from a stored procedure so connecting directly to the database is not an option): a. In code, populate dataset b. In code, write dataset schema to xml (e.g. an xsd file) c. In Visual Studio, add the XSD file to project d. Use fields from XSD file to drag and drop fields on report
A fancy example would be nice too. I've googled like crazy the past couple of days and downloaded as many samples as I can find (including the ones on ftponline.com, gotreportviewer.com, "Tudor's WebLog", and several others). However, I have yet to find one that uses grouping and related datatables.
Thank you so much (if, for nothing else, reading this post )
