I split a source into two. I get an Id from one after doing couple of lookups and maxId from other after doing a lookup and and an Aggregate. I want to compare Id and max Id in the same data flow. How can I do that please?
View 10 RepliesHello, I am using a multicast with 5 outputs, I attempting to pass about four thousand rows to different destinations. However, upon execution only two of the outputs send the rows to the destination. After deleting the multicast and reinserting it, different destinations received the data. The number of destinations is also not consistant, sometimes 3 work and sometimes only 1 works.
Thank you.
Hello,
I have a data-flow containing 150000 rows that is taking a long-g-g-g- time to run.
The bottleneck seems to be a SORT component. The SORT is immediately preceded by a MULTICAST.
Now, all 150000 rows have gone down the other output from my MULTICAST all the way to the destinaton, admittedly very slowly. My SORT is still caching the 150000 rows.
I'm wondering if the MULTICAST is causing a problem. Does each output fro the MULTICAST have its own buffer?
Can an asynch component that caches all the data (e.g. SORT) cause backpressure on the MULTICAST that might cause problems.
As I write, the package has been running 20 mins and the SORT component still has not cached all the rows.
Any clues would be welcome.
cheers
Jamie
I have a dataflow where each row passes through about 20 validations. This data flow could contain 200,000 to 20 million rows. Initially I set it up with a multicast component sending data each of the validations.
This got to be pretty messy.
Then I thought about using several multicast to subdivide the validations and make the data flow easier to follow. I tested using two multicasts to see if it would affect the performance of the dataflow. Results were inconclusive.
Before I continue adding and testing different multicast configurations, I thought I would check here to see if anyone else has tried and measured this?
Thanks for any insight you can provide.
I have to do various controls on a dataset - I created a multicast. After performing controls (one control per copy), I merge my (7) multicasted datasets using a Union All transformation. The problem I'm having are the duplicate rows created by merging the multicast copies.
How do I get rid of the duplicates? Is the Sort Transformation the solution by setting the option Remove rows with duplicate sort values to True? I have a unique key by which I'm able to discard the duplicates correctly. Are there any other ways (at a Union All level)? Is there sth like Union and Union All like in SQL?
I'm working on my 1st integration serv. project and it seems that more I work more questions I have. Shoudn't be the opposite? Thank you for the help.
Wondering how this is handled by Multicast component. If multiple copies are created in the memory and the size of the dataset is large, this could cause some performance problems. Any thoughts? TIA
View 8 Replies View RelatedHi,
Does anyone know if it is possible to point data that underwent the "merge join" transformation (in one data flow) to the following data flow? I don't want to recreate all that merging, sorting and calling the same sources again in the following data flow if the data that I am using exists in the previous data flow. The merged data is simply too big to export to an excel file, so does anyone have any ideas? Thanks!
I am using the SP below to try to copy some data from one table to another and I am getting an error. Does anyone have any idea what would be causing it.
Alter Procedure usp_ImportData
As
truncate table tblLineItems
set identity_insert tblLineItems on
INSERT INTO
tblLineItems (ReportPreparer, ReportDate, REV, CalDate, Limits, RMAReport, FailureCause, DispositionofProduct, ReturntoStock, Trash)
SELECT
a.[Report Preparer], a.[Report Date], a.[REV], a.[Cal Date], a.[Limits], a.[RMA Report], a.[Failure Cause], a.[Disposition of Product], a.[Return to Stock], a.[Trash]
FROM
tblRMA a
INNER JOIN
tblLineItems b ON (a.[RMA Number]= b.RMANumber)
set identity_insert tblLineItems off
The error I am getting is "Explicit Value must be specified for identity column in table 'tblLineItems' when IDENTITY_INSERT is set to ON"
My data is look like this.
EvS_VoucherCode--------Name----TotalAmount
ACT----------------------------1------------------3
ACT----------------------------1------------------3
DEF----------------------------2------------------4
DEF----------------------------2------------------4
DEF----------------------------2------------------4
I want data looks like this.
EvS_VoucherCode--------Name---------TotalAmount
ACT-------------------------------2-----------------6
DEF-------------------------------6------------------12
I have a query that pulled data in the following format:
First_Name Last_Name Drug Reason
Jim Smith Aspirin
Jim Smith Headache
Here's what I would like for it to display:
Jim Smith Aspirin Headache
Our fields and are weird. I had to use a case statement to get drugs and reason.
Would putting a max on them put everything on one row?
I am currently writing a query to show the quote number and its brand names inside that quote which should be separated with "/" if there are different brands in a quote. see below.
QuoteDetail Table:
QuoteNum | Brand
10047 | NISSAN
[Code]....
find the correct query. I tried some functions which I found in the internet but it didn't work.
Hi Guys,
Can anyone please explain and show an example of how to merge data from one row to another using in SQL 2000
thanks in advance
I have some data that I am inputing and if the record already exists i would like to add data to the fields that are not populated ONLY if they are not populated
for example
Already in the table
address= "123 this place"
city= ""
State="MI"
Zip="48462"
FirstName=""
LastName=""
being added
address= "123 this place"
city= "Ontate"
State=""
Zip="48462"
FirstName="Person"
LastName="Guy"
once i find that this record already exists because of the address and zip (which i already have complete)
I would like it to update the City first name and last name in the data that is already in the table. thank you for your help
I am using Merge Statement. Here is my requirement, I don't want to Insert data if Client State is NY, but I want to update all data
When Not Matched
and State not in ('NY')
THEN INSERT
the problem is sometime data NY data is inserted and sometime don't.
Hi!
I've the following replication topology:
SrvA (Publisher/Distributor) -> (Transactional Publication/Push Synchronization) -> SrvB (Subscriber, Publisher/Distributor) -> (Merge Publication/Pull Synchronization via HTTPS for remote clients) -> Client (Anonymous Subscriber)
To test synchronization with this tier I do the following:
1. Update data on SrvA
2. Push changes to SrvB (there are transactions that replicated)
3. Run RMO Merge Agent (.NET application) on Client and get messsage "No data needed to be merged", although there were changes made on publisher SrvA and SrvB
I saw the thread where similar problem have been discussed (http://forums.microsoft.com/TechNet/ShowPost.aspx?PostID=664153&SiteID=17). The sugestion there was to create subscription using wizard rather then T-SQL script. But all my clients are remote and have only HTTPS connection to the IIS over 443. And in the MSDN there was the script that I used to create pull subscription for web synchronization.
Guys, HELP! I almost gave up - do not see the reason for the case in the documentation, settings and code.
Thank you.
Alexander.
thanks 4 ur rpl..well i have a simple doubt if u can guide me it will b gr8 help..well i am developing an application of location based services, in which i am going to store my data on SQLserver 2005 and will fetch the data from it when the user logs in to the user's PDA device so that a user can work offline on the data when there is no connection and when he gets the connection he can synchronize the data with SQLserver 2005. so now i m confuse whether i should use the merge replication or ado.net synchronization framework or RDA?what is the difference between merge replication and synchronization and which is better?
Dear ppl,
I have a question for you. One of our client has asked us how is the data secured over the air during Replication?
I have read that Merge Replication uses TLS (Transport Layer Security) protocol to secure the data over the air. But I was wondering if it is all done automatically ? Or do we need to install certificates like for SSL.
We have a Windows Mobile 5.0 application using SQL Mobile that runs over 50 devices and they all synchronise with a single Publisher SQL Server 2005 using GPRS connection. We haven't got any SSL certificates installed on the server.
Now how can i make use of TLS in my application to secure my data over the air (using TLS)?
Regards
Nabeel Farid
I am designing a mobile application.I am having database in sql server 2005 and i want to synchronize this database with sql server ce database on pda.I am using merge replication method for synchronization.Please give step by step procedure for data synchronization using merge replication between sql server 2005 and sql server ce 2.0,also let me know whether sql server enterprise manager is needed or will it work without it,if needed give me the link from where to download it as it is not available in sql server 2005.
Reply soon.
Thanks in advance.
We had merge replication setup between 2 tables, Table A and Table B using SQL 2000. This was working 100%. The users asked to disable updated/deletes to both these tables if data existed on 2 other tables. Table AA and Table BB. We implemented it as follows:
1) Created Insert/Update/Delete triggers for Table A & B. It basically check for Table A is there a record in Table AA, if it exists, raise an error and don€™t commit.
2) Removed all foreign constraints from Table AA and BB
3) Added Table AA and BB to the current replication.
Then all hell broke loose, we got conflicts all other the place saying that Table AA cannot be updated because records does not exist in Table X. To our surprise we found triggers generated by Erwin in 1998 €“ that check for €śforeign contsraints€? and removed them immediately.
We continued to get conflicts but could see from the error messages it was generated by the triggers in point 1. We added the NOT FOR REPLICATION clause and everything has been running smoothly or so we thought€¦..
After 2 months we got a call that data is missing. It€™s random data and the only explanation I have is that replication caused that. My biggest reason for saying this is tracking the application audit trail I€™ve found that all the data missing was added during the period we had all the conflicts.
I need a solid explanation for this and can anyone confirm that this is possible?
Merging table :
--------Dummy TABLE
create table #Tbl1 (date1 date,WSH varchar(10),ITN int,Executions int)
insert into #Tbl1 (date1 , WSH , ITN , Executions)
select '20130202' ,'ABC', 1 , 100
union all
select '20130203' ,'DEF', 1 , 200
[Code] .....
I want Result like this :
date1 WSH ITNExecutionsMCGPositions
2013-02-02 ABC 1 100 2 500
2013-02-03 DEF1 200 NULL NULL
2013-02-05 NULLNULL NULL 2 600
I have table of standardized data that I'm merging into from an Import table.
This import table may have the [ContactName]. I one row in the import table has the contact name then every row would have the contact name.
What I'd like to be able to do is something like this in the UPDATE portion of the Merge (I don't care about the insert phase, since I can insert a null [ContactName] since I'm not overwritting an existing [ContactName]
MERGE INTO [dbo].[Standardized] AS [target]
USING [dbo].[ImportDestination]
AS [Source]
ON [Source].[Key] = [Target].[Key]
WHEN MATCHED
THEN UPDATE
SET[ContactName] = CASE WHEN source.[ContactName] IS NOT NULL THEN source.[ContactName] else target.[ContactName];
Is anything like that possible and if the code above works it would only work on a row by row bases. Is it possible to figure out if any [ContactName] has data and if so possibly overwrite an old [ContactName] with a null?
I m extracting data from Excel-sheet and inserting into SQL Server 2005 Database using SSIS(SQL Server Integration services).
Having 2 columns in excel-sheet and want to insert those into 1 column of SQL Table.
Can anyone help me about, how can I insert data from 2 columns into 1 column in SSIS. Is Export/ Import column or Copy Column or Merge will work..
Plz reply soon..
Thanks...
Hi gang,
We have a database that's using merge replication between two servers, and we need to insert a lot (about 1GB) of data into it.
The servers, however, are separated by a 192k WAN connection, so it's impractical to rely on the merge replication to send the data across to the subscriber.
Is there a way to insert the data at both ends? I can get the data out there on a DVD or a laptop easily enough. Can I load the data into both copies of the database and tell the merge agent that it's not to be replicated?
Thanks,
Matt
Hello,
When I reinitialized a subscription from SQL2005 db (publisher) to a SQL 2000 DB (Subscriber) and had the Upload changes before reinitialization turned on, the data from the SQL 2000 db didn't get uploaded. This resulted in a loss of 2 days worth of data.
Does anyone have any idea if there's anywhere that data is kept. (unfortunately a series of errors caused our backups to not be running on the 2000 db either).
Thank You
Bob
I'm having an error with merge replication my suscriber are downloading old data, all this start happen after I install SP4.
let explaint this with more details ,
if a made an update to a row in a table from the publisher , the agent history show that 1 change have downloaded to the suscriber, a put an audit in the suscriber and yes one update was downloaded but the row still the same, look like the suscriber downloaded old data instead of the curent change that i made.
If i delete the row, the rows gets delete at the suscriber, if a insert the row the row get insterted ok.
The problem only happen with rows inserted at the suscriber, that later on get update at the publisher. My suscriber has MSDE 2000 SP3 an my Server has SQL Enterprice 2000 SP4.
Any help would be apreciate. thxs.
Hi,
We have configured one-way merge replication in our topology. That is data flows from Subscriber to Publisher only. We have a publisher and a subscriber. There are 3 publications in this category and each publication has a subscription. We use SQL Server 2005 SP1 in both the servers. The retention period is the 14 days (default). After this period, I get the following error in the subscription in Replication Monitor. The Error message is
Error messages:
The Merge Agent failed after detecting that retention-based metadata cleanup has deleted metadata at the Publisher for changes not yet sent to the Subscriber. You must reinitialize the subscription (without upload). (Source: MSSQL_REPL, Error number: MSSQL_REPL-2147199402)
I read the post http://forums.microsoft.com/MSDN/ShowPost.aspxPostID=372790&SiteID=1
which said that this error might be solved in the SP2. We have not yet applied SP2, but even after applying SP2, will this error be solved for One Way Merge Replication since the data from publisher will not go to the subscriber always in this type of topology??.
Kindly get back to us regarding this as soon as possible. Thanks in advance.
Regards,
Swapna.B.
Hi all.
I 'm trying to set up merge replication between two servers that have the same schema databases. The two database have the majority of there data the same but as well as data inserted at a later time independently on the two servers. (The two servers were connected in a merge replication scheme that failed at some later time and replication was paused, but users continout to insert data indepentedly at the two servers.)
I need to get them up and running.
I cleaned replication at both servers, I recreated the publication at the puplisher distributor and all is fine.
When I create a push subscription to the subscriper I get the error invalid column name 'rowguidcol' .
I so far managed to have merge replication running on two identical databases (schema and data).
Just some thoughts. After some reading I found that it might be related to identities and identity range or indexes. I set the identity seed and increment at 2, 2 at publisher and at 1,2 at subscriper. (On the same tables at puplisher and subscriber.) Is that ok? is that the way to do it?
Digging a bit more Using the SQL Profiler I can locate the error to happen when sp_MSaddmergetriggers executes.
Thanks allot for any help
Version.
Microsoft SQL Server 2000 - 8.00.2039 (Intel X86) May 3 2005 23:18:38 Copyright (c) 1988-2003 Microsoft Corporation Standard Edition on Windows NT 5.0 (Build 2195: Service Pack 4)
I've got a problem:
I want to transfer data between an sql server 2005 and my pocket pc. So i've installed a sql server CE on it.
I know that i can use the merge replication but, i don't have any IIS. So, i've got to transmit data though email that have got a file attached. I Know how to email from sql server 2005. When the pocket receive that email, i don't know how to insert the data in sql ce from outlook compact. Doi have tu use SSIS , in that case how can i run the package?
thanks
I have the merge replication (Push)Â for SQL Server 2008.The right-click option for Start and Stop Synchronizing in subscriptions has been disabled. How can I stop the merge replication and start again ?
This error happened 1 week after I created the merge replication. The merge data cannot sync to subscriber and only solution I can use is drop and re-create the merge replication again but error would be re-occur a week later. The merge replication work fine before and start from middle on May 2015, but it keep happen right now.
I have database on SQL Server 2000 set up with a merge publication.This publication is configured with a number of dynamic filters toreduce the amount of data sent to each client. Each client has ananonymous pull subscription. The merge process can be triggered by thewindows sync manager and my application.To improve performance I have created some helper tables to hold themapping between user login and primary keys of selected entities.For the replicated data to be correct the contents of the helper tablesneeds to be up to date.I need to fire off a stored procedure on the publisher beforereplication starts to verify that this data is up to date. I can notsee any documented way of doing this however I have been experimentingwith some unorthodox systems.Firstly has anyone any ideas?I have been considering adding a trigger to some of the tables used bythe Microsoft replication code - yes I know this is very nasty.My problems arise because executing this stored procedure will causesome data to be updated. In updating data we could create a newgeneration in the database. I must therefore run my stored procedurebefore any the Microsoft code makes any generation checks / updates.Anyone done anything similar, Anyone have any better ideas?Any comments would be gratefully received.
View 1 Replies View RelatedHi all,
there are 2 datagrids i have taken,
1st for displaying data before inserting the value into table
2nd for displaying data after inserting the value into table
just see the screeenshot which i have taken
actually the problem is, we are not getting any type of errors even though I have try/catch blocks.
I have colored that particular statement in RED color, where it is getting stuck, while executing, if we sit 40-45 minutes then also the execution pointer is not going on next step.
please look into this, and if u have any idea/person by which/whom we can solve this then please tell us.......
-chaukse rahul
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Text;
using System.Windows.Forms;
using System.Reflection;
using System.Data.SqlServerCe;
using System.Data.Common;
namespace ProjectSQLMobile2
{
public partial class Form1 : Form
{
private DataSet dsMemberList;
public Form1()
{
InitializeComponent();
}
private void Form1_Load(object sender, EventArgs e)
{
Sync();
}
private void Sync()
{
SqlCeReplication repl = new SqlCeReplication();
repl.InternetUrl = @"http://169.254.25.129/WebSQLMobile/sqlcesa30.dll";
repl.Publisher = @"RAHU";
repl.PublisherDatabase = @"SQLMobile";
repl.PublisherSecurityMode = SecurityType.DBAuthentication;
repl.PublisherLogin = @"sa";
repl.PublisherPassword = @"sa12345";
repl.Publication = @"PubSQLMobile";
repl.Subscriber = @"SubSQLMobile";
repl.SubscriberConnectionString = @"Data Source=TestLast.sdf;Max Database Size=128;Default Lock Escalation =100;";
try
{
if (!System.IO.File.Exists(@"TestLast.sdf"))
{
repl.AddSubscription(AddOption.CreateDatabase);
}
repl.Synchronize();
}
catch (SqlCeException ex)
{
DisplaySQLCEErrors(ex);
}
finally
{
repl.Dispose();
}
/*
*
* To Display Data Into DataGrid1 Before Insert
*
*/
SqlCeConnection cn = new SqlCeConnection(@"Data Source=TestLast.sdf");
SqlCeDataAdapter daMemberList = new SqlCeDataAdapter("SELECT MemberID, MemberName FROM MembershipData", cn);
if (dsMemberList == null)
{
dsMemberList = new DataSet();
}
try
{
dsMemberList.Clear();
daMemberList.Fill(dsMemberList, "MembershipData");
dataGrid1.DataSource = dsMemberList.Tables["MembershipData"];
}
catch (SqlCeException ex)
{
DisplaySQLCEErrors(ex);
}
}
private void Merge()
{
SqlCeReplication repl = new SqlCeReplication();
repl.InternetUrl = @"http://169.254.25.129/WebSQLMobile/sqlcesa30.dll";
repl.Publisher = @"RAHU";
repl.PublisherDatabase = @"SQLMobile";
repl.PublisherSecurityMode = SecurityType.DBAuthentication;
repl.PublisherLogin = @"sa";
repl.PublisherPassword = @"sa12345";
repl.Publication = @"PubSQLMobile";
repl.Subscriber = @"SubSQLMobile";
repl.SubscriberConnectionString = @"Data Source=TestLast.sdf;Max Database Size=128;Default Lock Escalation =100;";
try
{
if (!System.IO.File.Exists(@"TestLast.sdf"))
{
repl.AddSubscription(AddOption.CreateDatabase);
}
repl.Synchronize();
}
catch (SqlCeException ex)
{
DisplaySQLCEErrors(ex);
}
finally
{
repl.Dispose();
}
/*
*
* To Display Data Into DataGrid2 After Insert
*
*/
SqlCeConnection cn = new SqlCeConnection(@"Data Source=TestLast.sdf");
SqlCeDataAdapter daMemberList = new SqlCeDataAdapter("SELECT MemberID, MemberName FROM MembershipData", cn);
if (dsMemberList == null)
{
dsMemberList = new DataSet();
}
try
{
dsMemberList.Clear();
daMemberList.Fill(dsMemberList, "MembershipData");
dataGrid2.DataSource = dsMemberList.Tables["MembershipData"];
}
catch (SqlCeException ex)
{
DisplaySQLCEErrors(ex);
}
}
private void DisplaySQLCEErrors(SqlCeException ex)
{
for (int i = 0; i < ex.Errors.Count; i++)
{
MessageBox.Show("Index #" + i.ToString() + ""
+ ex.Errors.Source + ""
+ "Error: " + ex.Errors.Message,
"Error No. " + ex.Errors.NativeError.ToString());
}
}
private void buttonInsertData_Click(object sender, EventArgs e)
{
//add directly into ce database sample:
SqlCeConnection cn = new SqlCeConnection(@"Data Source=""TestLast.sdf""");
string SQL = "INSERT INTO MembershipData(MemberName) VALUES('" + textBox1.Text + "')";
cn.Open();
SqlCeCommand cmd = new SqlCeCommand(SQL, cn);
cmd.CommandType = CommandType.Text;
try
{
/*
* This statement is taking too much time to be executed * what should we do??????? */ cmd.ExecuteNonQuery(); }
catch (SqlCeException ex)
{
DisplaySQLCEErrors(ex);
}
finally
{
cn.Close();
}
Merge();
}
}
}
please help,
ur help will be appriciates
NOTE:- I am getting output in 1st datagrid, but in 2nd dtagrid there is no output
Hi All!
Microsoft SQL Server 2005 - 9.00.3159.00 (Intel X86)
Mar 23 2007 16:15:11
Copyright (c) 1988-2005 Microsoft Corporation
Enterprise Edition on Windows NT 5.2 (Build 3790: Service Pack 2)
40 subsribers.
Adding new article.
On any aubscribe need regenerate snapshot agent agent.
The tables are very big. With filters.
Regenerating process anew copy data in in ...ReplDataame.bcp, during 30-40 minutes for any subscriber!
Full work to small correct of replication flow during 2-3 days....
Is it possible to initialize subscription without copying bcp data?
SQL 2008R2
Request is to merge or join or case stmt or union or... from up to four unique columns all in separate tables to new combined table (matrix) of results from said.
What is example of best method to do this ?