Data Collection From Many To One Servers
Mar 7, 2008
I need to feed head office sql server with the data from regional servers. Servers are spread through all continents
Data input done locally on Head office server as well and plus need to ship data from other servers.
So clarify this - Head office server is not standby one. Mirroring is out of the picture, I think..
Initially, I thought ship a log every 15 min and restore on Head office server but is this going to create an issue for the local data processing?
Any bright ideas welcome!
View 2 Replies
ADVERTISEMENT
Sep 20, 2005
What is the better table design for a data collection application.1. Vertical model (pk, attributeName, AttributeValue)2. Custom columns (pk, custom1, custom2, custom3...custom50)Since the data elements collected may change year over year, whichmodel better takes of this column dynamicness
View 7 Replies
View Related
May 30, 2014
i am trying to configure data collector on my server. so i configured data collector on server A and setup on server B. but the "Query statistics collection set" do not show me any data.
i right click and select "collect and upload now " item and get success result for this. but in the report i cant see any data...
also in the log page of data collection i see so many errors with messages like this:
"Failed to create kernel event for collection set: {2DC02BD6-E230-4C05-8516-4E8C0EF21F95}. Inner Error ------------------> Cannot create a file when that file already exists."
i tried some solution like disabling and enabling again, re-configuring, removing and configuring again .... but none of them work right.
View 2 Replies
View Related
Mar 30, 2014
It is possible that Data Collection can cause massive increasing MB/sec to tempdb ? I cannot find connection with tempdb and I set cash file, but on same disk.
Or it can be something different? Last two weeks what I checked was Read/Write MB/s to tempdb increasing progressively.
One time it was about 20MB/sec
After it was reseting and again 1MB/sec..
What I checked , External company which install SQL Server made one file for tempdb, next week or during breaktime(it will be possible), I would like make 8files next weekend work.
Now I saw that TempDB mdf was still increased, but using was just 8-10%
View 2 Replies
View Related
May 19, 2008
hi,
this is sanjeev,
i have SSIS package, using my c# program i want to add one execute package task to this package's sequence container.
it is creating the new package with out any probelm. but when i opened the package and try to move the newly created exeute package task it is giving the following error.
the element cannot be found in a collection. this error happens when you try to retrieve an element from a collection on a container during the execution of the package
this is my code
{
Package pkg = new Package();
string str = (string)entry.Key;
pkg.Name = str;
alEntity = (ArrayList)entry.Value;
ConnectionManager conMgr;
Executable chPackage;
TaskHost executePackageTask;
Microsoft.SqlServer.Dts.Runtime.Application app = new Microsoft.SqlServer.Dts.Runtime.Application();
//string PackagePath = @"c:Genesis.dtsx";
//p = app.LoadPackage(PackagePath, null);
p = new Package();
p.LoadFromXML(parentPackageBody, null);
p.Name = str;
//Sequence seqContainer;
IDTSSequence seqContainer;
//seqContainer = (Sequence)p.Executables["Extract Genesis Data"];
seqContainer = ((Sequence)p.Executables[0]);
string packageLocation = @"Geneva Packages";
conMgr = p.Connections["SQLChildPackagesConnectionString"];
foreach (string val in alEntity)
{
if (seqContainer.Executables.Contains("Load_" + val) == false)
{
chPackage = seqContainer.Executables.Add("STOCK:ExecutePackageTask");
executePackageTask = (TaskHost)chPackage;
executePackageTask.Name = "Load_" + val;
executePackageTask.Description = "Execute Package Task";
executePackageTask.Properties["Connection"].SetValue(executePackageTask, conMgr.Name);
executePackageTask.Properties["PackageName"].SetValue(executePackageTask, packageLocation + ddlApplication.SelectedItem.Text + @"" + executePackageTask.Name);
}
}
app.SaveToXml(Server.MapPath("../SynchronizeScript/Packages/" + ddlApplication.SelectedItem.Text + @"") + str + ".dtsx", p, null);
}
please let me know what is the wrong in my code.
thanks in advance.
regards
sanjeev bolllina
sanjay.bollina@gmail.com
View 14 Replies
View Related
Feb 6, 2000
I have a product where we feed an SQL 7 DB data collected from Manufacturing. Presently, the Data transport Program is in charge of getting prepared data from machines and inserting into the DB. This design assumes SQL7 is always ready and able - which is not true due to customer queries or backups or etc. consuming resources. There is a low level buffer in system at manufacturing level if transport dies, but transport is ignorant of SQL distress, so keeps hammering DB's frontdoor. I'm looking for help in putting SQL server in charge of allowing data in - when resources are adequate. Seems I need a function that can determine server stress QUICKLY to forestall transport program and a buffer for records at the transport layer. Anyone know / done a system where SQL server CHECKS for waiting records or OK's an external program to send until told to stop? What indicates (reliably) low server resources? Anyone ever used MSMQ?
"Black Holes are proof SOMEBODY, SOMEWHERE really did have a particularly bad Y2K problem!"
View 7 Replies
View Related
Apr 17, 2014
I Enabled Data Collection on one of the server and planned to make it as Centralised Management Data Warehouse I configured data collection on it and can view reports. Next, I went to other server and configured "Set up data collection" to use my first instance as the centralised Database. But the issue is I can only see reports of first server. Am I missing something here.
I did exactly as explained in this video [URL] .....
View 9 Replies
View Related
May 6, 2008
Dear all,
I have an aplpication whih collect different types of history data from an SQL database. On type of those data are hitory log information where amount of records can be really big.
What i would like to implement is a kind of paging mecanism for colelcting those data.
fro example the first call will retrun the first 100 rows, then the next call would get rows 101 to 200 . etc....
How to perform thsi behaviour from SQL side ?
regrds
serge
View 2 Replies
View Related
Jun 19, 2015
I created Data Collection in wrong DB, how can I change the DB or return to default(as it came with clean version of SS) ?
View 3 Replies
View Related
Jun 5, 2014
I have a two node SQL 2012 AlwaysOn HADR cluster (v11.0.3412) with 4 availability groups configured. The AG groups are set to synchronous mode and the secondary is not readable (we do not want the synchronous replica readable so we do not risk any reads causing contention so we maintain fast performance).
On the secondary we are getting a persistent failure with the Data Collector job called Collection_Set_3_Upload. The failure occurs within the second job step. That job step is executing the following command:
dcexec -u -s 3 -i "$(ESCAPE_DQUOTE(MACH))$(ESCAPE_DQUOTE(INST))"
The error message is as follows:
Log Job History (collection_set_3_upload) Step ID 2 Server CLUSTERNODE2
Job Name collection_set_3_upload
Step Name collection_set_3_upload_upload
Duration 00:00:07
[Code] ....
I know I can prevent this error message by enabling readable secondaries, but we do not want this.
I have tried stopping the data collection jobs and purging the cache directory but to no avail. It will succeed the first time then persistently fail again with the same message every time after that.
In addition, if I set the one failing AG group to readable secondary the job succeeds. So that means that 3/4 work fine, only this one is having an issue.
View 0 Replies
View Related
Oct 26, 2015
Last weekend many of our severs had a failed job "collection_set_3_upload". The error that occured is: "Violation of PRIMARY KEY constraint 'PK_ active_ sessions_ and_requests'. Cannot insert duplicate key in object 'snapshots.active_sessions_and_requests'. The duplicate key value is (2824333, 2015-10-25 02:54:49.7630000 +02:00, 1)."Last weekend we happened to go from summer time to winter time. i.e. the clock passed 02:00 - 3:00 two times during this night.
I.e. there is a bug in the Data Collector component that collects data for the Management Data Warehouse: it uses local time instead of UTC. I've created a Connect item to report it to Microsoft.URL...how do you get your process running again? the job will no longer run because it will every 5 minutes keep on trying to upload the conflicting data for the 2nd 2:00 - 3:00 period. I've only found one solution: get rid of all data collected but not yet uploaded.
You do this by stopping the Collection set (in SSMS go to Object Explorer -> <the server you want to fix> -> Management -> Data Collection -> System Data Collection Sets. Right click "Query Statistics" and select "Stop Data Collection Set").Then you delete the cached results from the sql server machine's harddisk. These cached results are in files located in a Temp folder on the sql machine itself, inside the AppData folder for the service account SQL Server Agent is running under. Usually it will be something like: c:Users<sql agent service account>AppDataLocalTemp.
Inside this folder delete all files that have 'QueryActivity' in their name. You'll loose all data collected since the start of wintertime, but at least your data collection process will work again.After this you can start the Collection set again by right clicking it and select "Start Data Collection Set". Every 5 minutes the data will be summarised and uploaded into your management data warehouse.
Posting Data Etiquette - Jeff Moden
Posting Performance Based Questions - Gail Shaw
Hidden RBAR - Jeff Moden
Cross Tabs and Pivots - Jeff Moden
Catch-all queries - Gail Shaw
View 0 Replies
View Related
Mar 18, 2015
I setup data collection on a production server to capture growth rates.
When I run the dis usage report, it shows a daily growth rate of over 500 megs. This seems excessive to me.
As a troubleshooting step I then ran sp_space used and got these results:
database_namedatabase_sizeunallocated space
rgc_prod 273442.63 MB3648.48 MB
reserved data index_size unused
265345488 KB164385384 KB99826072 KB1134032 KB
What should my next steps be to try and determine why there is so much growth? And isn't the index size rather large?
View 1 Replies
View Related
Apr 17, 2007
Hello,
I have a SSIS package with a Data Flow task. This task transfers the data from SQL Server 2000 to a table in SQL Server 2005.
I deployed and tested this package on the Test Server. Then put this package in a job and executed it - Works fine.
On the production server- If I execute the package through DTEXECUI, it works fine. But when I try executing it through a job- the job fails and it gives me following error:
Description: The external metadata column collection is out of synchronization with the data source columns. The "external metadata column "T_FieldName" (82)" needs to be removed from the external metadata column collection....
What I don't understand is, why are there no errors displayed when I execute the package through DTEXECUI.
Can anyone help me to resolve this issue.
Thanks.
View 3 Replies
View Related
May 16, 2008
Hi,
How do I retrieve the connections (connection managers) collections from Custom Data Flow destination? ComponentMetadata.RuntimeConnectionCollection is empty. I would like to be able to access all the connections defined in the package from the custom data flow task.
I came across code in which it was possible to access the Connections collection using the IDtsConnectionService for custom task (destination). The custom task has access to serviceProvider, whcih can be used to get access to the IDtsConnectionService interface but not the custom data flow task.
Any help appreciated.
Thanks
Naveen
View 5 Replies
View Related
May 14, 2015
I'm trying to edit the Expressions of a Data Flow task. This seems to happen when I rename some of the Data Flow components but not always. The error I get is:
Element "[ADO Net Source].[SqlCommand]" does not exist in the collection "Properties"
However, if you look at the XML, this property does exist. So I'm not sure why this should occur.
I'm using SSIS 2008 R2 with Visual Studio 2008 V 9.0.30729.4462 QFE.
<component id="1" name="ADO Net Source" componentClassID="{2E42D45B-F83C-400F-8D77-61DDE6A7DF29}" description="Extracts data from a relational database by using a .NET provider." localeId="-1" usesDispositions="true" validateExternalMetadata="True" version="4" pipelineVersion="0" contactInfo="Extracts data from a relational database by using a .NET provider.;
[Code] ....
View 3 Replies
View Related
Mar 12, 2008
Hi all,
From the "How to Call a Parameterized Stored Procedure by Using ADO.NET and Visual Basic.NET" in http://support.microsft.com/kb/308049, I copied the following code to a project "pubsTestProc1.vb" of my VB 2005 Express Windows Application:
Imports System.Data
Imports System.Data.SqlClient
Imports System.Data.SqlDbType
Public Class Form1
Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
Dim PubsConn As SqlConnection = New SqlConnection("Data Source=.SQLEXPRESS;integrated security=sspi;" & "initial Catalog=pubs;")
Dim testCMD As SqlCommand = New SqlCommand("TestProcedure", PubsConn)
testCMD.CommandType = CommandType.StoredProcedure
Dim RetValue As SqlParameter = testCMD.Parameters.Add("RetValue", SqlDbType.Int)
RetValue.Direction = ParameterDirection.ReturnValue
Dim auIDIN As SqlParameter = testCMD.Parameters.Add("@au_idIN", SqlDbType.VarChar, 11)
auIDIN.Direction = ParameterDirection.Input
Dim NumTitles As SqlParameter = testCMD.Parameters.Add("@numtitlesout", SqlDbType.Int)
NumTitles.Direction = ParameterDirection.Output
auIDIN.Value = "213-46-8915"
PubsConn.Open()
Dim myReader As SqlDataReader = testCMD.ExecuteReader()
Console.WriteLine("Book Titles for this Author:")
Do While myReader.Read
Console.WriteLine("{0}", myReader.GetString(2))
Loop
myReader.Close()
Console.WriteLine("Return Value: " & (RetValue.Value))
Console.WriteLine("Number of Records: " & (NumTitles.Value))
End Sub
End Class
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
The original article uses the code statements in pink for the Console Applcation of VB.NET. I do not know how to print out the output of ("Book Titles for this Author:"), ("{0}", myReader.GetString(2)), ("Return Value: " & (RetValue.Value)) and ("Number of Records: " & (NumTitles.Value)) in the Windows Application Form1 of my VB 2005 Express. Please help and advise.
Thanks in advance,
Scott Chang
View 29 Replies
View Related
Dec 8, 2006
Hello,
I want to transfer data in between two or more sql servers. Actually One server is on the web while others are in small networks in diffrent locations.
I want to update the main server with the changes in local servers. My problem is if the connection goes failed in between process of data transfer then how will it be recovered that how much of data is uploaded and how much is left.
Please I need help on this isue.
Thanks in advance
pronov
View 1 Replies
View Related
Jun 20, 2007
am an authenticated user of a remote data base
I can log in using sql server management studio and add tables and
even I can back up the database .
Good.
But I want to get a copy of the
database to my local computer
Is there any way to do
this?The problem is that i don't have access to the directory of sql sever pleases tel me a way that is possible just by using sql server or any add on on it
View 7 Replies
View Related
Aug 27, 1999
Hello,
I have a developer here who wants to be able to access two databases on different servers with the same query statement.
Both servers have SQL Server 7.0 ??? Any suggesstions?
Joanna
View 1 Replies
View Related
Aug 25, 2000
hi,
I have a question. how do query data between the two servers.
thanks
kumar
View 2 Replies
View Related
Jun 17, 1999
We have a database that I would like to replicate on another server but am unable to use regular replication via publish/subscribe due to the fact that the production database has no primary keys on tables, only clustered indexes. The backup db needs to be as close to real-time synchronized as possible and will be in fairly active use most of the time. Has anyone had success in developing such a system? How did you do it and what are the pitfalls? Any advice would be greatly appreciated. Thank you.
W.
View 1 Replies
View Related
Jun 22, 1999
Hi,
I need to access data across my sql servers . All the servers are 6.5. Think MS DTC is the solution. But how to implement the same. Can somebody give me step by step instructions.
Thanks in Advance.
Cheers
View 1 Replies
View Related
Apr 1, 2008
Hi,
Not sure thisis the right forum for my question.
I have to insert data from one table into another. The problem is that the 2 tables are on different servers and computers for that matter. I only need data that is in one specific table.
What is the easiest and most efficient way to do that.
If I access the 2 computers via remote desktop from my computer and actually go and copy the data from the table in computer 1 and paste it inside the table in computer 2 it does paste the data.However, for some reason it doesn't paste all the data. The column datatype is ntext.
I mean if the entry is for example 'hello world how are you today? blah bla blah'
It will only paste 'hello world how are'
I will appreciate your help.
Many thanks
Whisky-my beloved dog who died suddenly on the 29/06/06-I miss u so much.
View 4 Replies
View Related
Apr 6, 2006
Guys,
How do I create a view that will pull data from dbases on 2 different servers ?
I have 2 server names....
svr10mbr01
svr11mbr02
2 Databases...
PWBstaff on svr10mbr01
PWBdata on svr10mbr02
I need to create a view in PWBdata that will do something like....
Select * from PWBstaff.dbo.staff
Is this possible, if so what is the syntax.
Ta
Bob
"I dislilke 7am. If &am were a person, I would kick 7am in the biscuits." - Paul Ryan, dailyramblings.com
View 3 Replies
View Related
Sep 20, 2006
Hi,
I've got a quick question here. I'm still quite new to SQL Server. My question is how to transfer file between SQL Servers. For example, i have been working on the database - updating or editing some tables in the SQL Server 2000 at my end. Then, i need to pass whichever tables got updated on to my client who also uses the SQL Server 2000. So we have two separate SQL Servers 2000. i wonder how to transfer data like tables between the two SQL Servers.
Thank you in advance
View 2 Replies
View Related
May 14, 2006
I need to append data from a database on one server to a table in a databaseon a different server. Both servers are running SQL 7. How can that be done?Thanks.
View 2 Replies
View Related
Jul 20, 2005
Ok..here is my problem. I have two SQL servers that I need to haveidentical, at least semi-real time data on. One is in a public DMZwith full access to and from the internet, and one is behind acorporate firewall and the box can get out to the internet but cannotbe hit directly from the internet in. The two are on seperatenetworks and cannot be connected via lan. Is there a way for me tohave semi-real time data transfers between the two to keep bothservers identical? I have thought that the server behind the firewallcould initiate an XML session with my dmz sql server and processupdates, but I am unfamiliar with how it would know what has changedon the remote server and how the remote server would know what haschanged on the server behind the corporate firewall. Any good ideas?ThanksDan Hirsch
View 3 Replies
View Related
Mar 29, 2006
What is the easiest way to synchronize all data and database objects between my development machine and hosting server? Is there any SSIS free script I can use?
View 7 Replies
View Related
May 29, 2007
Dear all
I have tow server on the same intranet. One server has a sql server 2000 database and the other one has SQL server 2005 databse.
The sql 2000 database has a table called employee. When ever a new employee is inserted in the database i would like the same values to be sent to the sql 2005 database. But this cant be done on the application level. It has to be done in the database. The application level can not be changed.
I was thinking a trigger but how to achieve the writing from one database to another. If they were on the same server then it would be easier but because they are on different servers i dont know how to do it.
Has anyone had similar issue before?
Any help is apreciated.
Sincerely
Dan
View 2 Replies
View Related
Oct 27, 1999
Hi there,
I was just wondering is it possible to select data from two different databases on two different servers?
ie Select * from
Server1.databasename..table, Server2.databasename..table
If anyone has any suggestions they would be appreciated,
Thanks,
Fin
View 2 Replies
View Related
Nov 12, 2004
Hello to all,
my problem is about the slow execution of a query.
I have two database located in two different servers. The servers are linked with the linked server connection made with the enterprice manager.
The databases structure is the same and I copy some data from a database to the other with the following query:
INSERT INTO DATASERVER.RESULTS.dbo.Tab_Tests SELECT * FROM LINESERVER.WIP.dbo.Tab_Tests WHERE ID_Result = '{76271FC1-9470-4EF7-A403-000CF75C2215}' OR ID_Result = '{08C8EEB9-CD22-4CF9-8269-8C1CC58C752B}' OR ID_Result = '{B7B7EED1-36E4-48FA-9636-ABBE62AAE04D}' OR ID_Result = '{8F7BA69E-D2CE-4EB2-BBC4-AFFA165CE0CE}' OR ID_Result = '{273CB805-945F-4EFA-AE20-BC4E0898A19F}'
Everything works till the database grows up more than 400 MB.
Also if I select one result, that consist of some records, the query is very slow and the CPU is working over the 75 %.
I ask you if my working way is corret or if you have some solution or some explanation for me.
Thank you
View 1 Replies
View Related
Sep 26, 2005
Hi, I'm looking for an artical or Someone who can explain me what is the best way to transfer data between two remot database?
I'll Explane:
two business have local SqlServer in their office.
1 of business need to get Specific data from other. each database is protected. how can I read ake information of it?
Thanks.
View 5 Replies
View Related
Jul 7, 2007
I am preparing a new website that will contain names, address and phone numbers of people in my region of Costa Rica. Will I be able to post a database prepared in Visual Basic 2005 Express / SQL Server 2005 Express on the website's MySQL Server and utilize the data to populate webpages?
If not, what should I use to prepare the database and drive the webpage.
New to SQL and VB
Don Johnson
Latitude 8 Lodge
Costa Rica
View 2 Replies
View Related
