Sql Server Destination In Memory (not The Usual Question)

Mar 15, 2007

All,

I've used the SQL Server Destination, and I know it is faster than Ole Db Destinations in some setups. I have read that this is due to the in memory transfer of data that happens from SSIS to the destination database. I haven't heard it explained in finer detail, and I am wondering if someone can confirm how this works.

My guess is that the SSIS server process handles loading the data into memory from the Source destination (I assume this does not have to be local). I'm guessing there must be some interprocess communication going on between the SQL Server process and the SSIS server process to do the transfer of data (please confirm this.) I know that the SSIS process and the SQL Server process must be loaded on the same machine. And in testing see that the process errors out with permission-like errors when SSIS is not on the same server as the destination. (Which I expect based on the documentation - this is not the source of my question.) Is there any sort of local disk activity that SSIS uses to cache the data or anything like that? Or is all of the resulting disk activity writes that occur in the SQL Server user destinaation database or temp database?

Any information you can provide on how the transfer of data occurs from the SSIS process to the SQL Server process would be very interesting to me. I use SSIS a lot and would just like to understand the mechanics there.

Thanks much (and Microsoft I would _really_ appreciate an answer from you guys if possible!)

dcb99

View 4 Replies


ADVERTISEMENT

The Usual Date Problems!

Aug 16, 2006

Hi All,
I have a database table with a date range in it. Ie. a start date and an end date. I get the data with a stored procedure to which I pass a 'from date' and a 'to date'. During debug the dates pased to the SP seem correct but it doesnt return any results. When I run the query manualy from SQL server I get the same problem but when I swap the month and day in the query it works. At some point the month and day are being swapped arround or are not correct in the first place. How do I work this out.
The where clause for the SP is created with the following code. calAvailability is a standard calendar control on the form. Please note if I uncomment the commented line and comment the line above it, it gets the results expected.
Dim ST As Date = "01/" & calAvailability.VisibleDate.Month & "/" & calAvailability.VisibleDate.Year
Dim Where As String = "StartDate >= '" & ST & "' AND EndDate < '" & DateAdd(DateInterval.Month, 1, CType(ST, Date)) & "'"
'Dim Where As String = "StartDate >= '08/01/2006' AND EndDate < '09/01/2006'"
Danny

View 8 Replies View Related

Warning Users Of Longer Than Usual Render Times

Feb 8, 2008

assuming we already know how to estimate longer than usual run times for reports based on wider than usual date range parameter selections by the user, what tricks are available or being used for displaying this estimate either before the user hits the "view report" button, as an additional step after the user hits the button a first of 2 times, or even on the same display as the revolving green arrow that indicates report is running.

View 7 Replies View Related

Performance Issues Total Server Memory Vs Target Server Memory

Aug 2, 2006

Hi

I did a load testing and found the following observations:

1. The Memory:Pages/sec was crossing the limit beyond 20.

2. The Target Server Memory was always greater than Total Server Memory

Seeing the above data it seems to be memory pressure. But I found that AvailableMemory was always above 200 MB. Also Buffer Cache HitRatio was close to 99.99. What could be the reason for the above behavior?

View 1 Replies View Related

Sql Server 2000 Using Less Memory After More Memory Added

Aug 22, 2007

sql server 2000 is running on windows server 2003 ... 4gb of memory on server .... 2003 was allocated 2.3gb nd sql server was allocated (and using all of it) 1.6gb for total of approx 4gb based on idera monitor software ... all memory allocated betweeen the OS and sql server .... then 4 more gb of memory added for total now of 8g ... now idera monitor shows 1.7gb for OS and 1.0 gb for sql server ..... 'system' info shows 8gb memory with PAE ... so I assume that the full 8gb can now be addressed .... why are less resources being used now with more total memory .... especially sql server ..... i thought about specifying a minimum memmry for sql server but i amnot convinced that would even work since it seems that this 1gb limit is artificial .... it it used 1.6 gb before why would it not use at least that much now ??

thank you

View 4 Replies View Related

OLE DB DESTINATION And SQL Server Destination

Jul 4, 2007

Hey All:



I was totally confused.

When designing the SSIS dataflow part, firstly , i tried SQL Server Destination because my target server is a sql server.

then execute the task with failure.

Then i tried to use OLE DB DESTINATION instead of SQL Server Destination.

This Dataflow worked.



i can not figour out why.

By the way , i used the connection is OLE DB.And i choosed OLE DB source as the datasource cuz i can not find SQL server datasource.



Who can tell me some reasons for this?



View 9 Replies View Related

SQL In-Memory :: Table Memory Optimization Advisor Validation Passed But Cannot Migrate

Jul 13, 2015

I am looking to test this feature - and the "Transaction Performance Collector" has recommended me a table to port to In-Memory OLTP. 

I have now tried the "Table Memory Optimization Advisor" tool.

After a couple of tweaks to the table design - the tool is now passing validation but the tool is not allowing to progress to the next step:

Could it be down to not having enough memory? But would this not show in the advisor?

View 4 Replies View Related

Attempted To Read Or Write Protected Memory. This Is Often An Indication That Other Memory Is Corrupt. (Microsoft Visual Studio)

Sep 28, 2007

Hello. I have received the follwoing error upon an attempt to Browse the Cube. All other tabs are functional, including the Calculations tab. We are running Windows Server 2003 SP2 and SQL Server 2005 SP2. Any suggestions would be greatly appreciated!

**EDIT** - Have confirmed SP1 for VS2005 is installed both locally and on server, also.


Attempted to read or write protected memory. This is often an indication that other memory is corrupt. (Microsoft Visual Studio)

------------------------------
Program Location:

at Microsoft.Office.Interop.Owc11.PivotView.get_FieldSets()
at Microsoft.AnalysisServices.Controls.PivotTableFontAdjustor.TransformFonts(Font font)
at Microsoft.AnalysisServices.Browse.CubeBrowser.UpdatePivotTable(Boolean translate)
at Microsoft.AnalysisServices.Browse.CubeBrowser.UpdateAll(Boolean translate)
at Microsoft.AnalysisServices.Browse.CubeBrowser.InitialUpdate()
at Microsoft.AnalysisServices.Browse.CubeBrowser.SupportFunctionWhichCanFail(FunctionWhichCanFail function)

View 4 Replies View Related

AWE/Lock Page In Memory Drawbacks? Min/Max Settings, Memory Issues

Oct 11, 2007

I've been researching AWE to determine if we should enable this for our environment.

Currently we have a quad core box with 4 gb of RAM (VMware). OS: Windows 2003 std, SQL Server 2005 std. 3GB is not set but will be as soon as we can perform maintenance on the server.

I have read mixed feedback on AWE, either it works great or grinds you to a hault. I would assume that the grinding to a hault is due to not setting the min/max values correctly or not enabling the lock page in memory setting.

We only have one instance of SQL on the server and this box won't be used for anything else aside from hosting SQL services. We do plan on running SSRS off of this server as well.

1. Will running SSRS and enabling AWE cause me problems? Will I have to reduce the max setting by the SSRS memory usage or will it share and play nice?

2. How do I go about setting the Max value? Should it be less than the physical RAM in the box? Right now its set to the default of 214748364, even if I don't enable AWE should this default value be changed?

3. It seems that even at idle the SQL server holds a lot of memory and the page file grows. If I restart the process in the morning, memory usage in taskmon is at 600mb or so. By the end of the day, its up around 2gb. How can I track down whats causing this, should this even concern me?

4. The lock Page in memory setting worries me. Everything I've read on this seems to give a warning about serious OS and other program support degradation. In some cases to the point where they have to restore the settings on the server before they can bring it back up. What are your thoughts on this.

View 3 Replies View Related

SQL In-Memory :: How To Reduce Memory Usage Without Killing Any Process

Aug 28, 2015

I have a Windows sever 2012 with sql server 2012 enterprise. Ram size is 22GB. Sometimes SQL sever takes 95% memory.My question, How to reduce memory size without killing any process because it's production server.So there are many background process is running. And,Is there any guides to learn why Memory is raise d so high and how to reduce it.

View 10 Replies View Related

Error At Transfer Objects Task: The Source Server Can Not Be The Same As The Destination Server.

Jul 4, 2005

SQL Server 2005 - June CTP

View 7 Replies View Related

Would Max Memory Including SSIS And SSAS Memory

Mar 27, 2008

Hello, I understand that we should use SSMS -> Server Properties -> Memory to put a cap on the SQL server memory usage, therefore it gives some space memory for OS, this is based on the fact if the max memory is not specified, SQL will use whatever available memory and eventually crash the system.

My question is that when a server has SSIS and SSAS services installed along with the SQL service. Would the max memory setting covers the SSIS and SSAS memory usage, or the SSIS and SSAS has to shared the memory with OS?

Thanks,
fshguo.

View 1 Replies View Related

Memory Issues, SSIS Package Out Of Memory Help

Dec 6, 2006

I am running Visual Studio 2005. I have an SSIS Package which is consuming a huge amount of memory. During the execution of the package the memory keeps increasing. Until finally i get an Out of Memory exception. I have run this package using dtexec, and in the BIDS. No difference. I do have some script components and have added some code to get the assemblies in the current appdomain. I do see that one particular assembly is increasing on every loop. VBAssembly every time it hits the script component is increasing by 6, and along with it the memory is climbing. What is this VBAssembly being used for is there an update to SQL Server Integration Services that I need?

Thanks! Aaron B.

View 6 Replies View Related

SQL In-Memory :: How To Find Memory Usage By Index

Oct 4, 2015

i want to create a lot of index for my database for performance.but i need find memory usage by indexes.

How to find memory usage by index in sql server?

View 9 Replies View Related

SQL In-Memory :: Remove Memory Optimized Filegroup

Jun 15, 2015

I've a database with a memory optimized filegroup on it. How can I remove it?I have removed the memory optimized table I had on it, but when I try to remove the filegroup I receive an error.

View 12 Replies View Related

SQL Server Destination [16]

Jan 15, 2006

[SQL Server Destination [16]] Error: An OLE DB error has occurred. Error code: 0x80040E14. An OLE DB record is available.  Source: "Microsoft SQL Native Client"  Hresult: 0x80040E14  Description: "Could not bulk load because SSIS file mapping object 'GlobalDTSQLIMPORT              ' could not be opened. Operating system error code 2(The system cannot find the file specified.). Make sure you are accessing a local server via Windows security.".

I'm trying to do a simple upload to a production server from an Access mdb of the kind I used to do routinely in DTS.  Any ideas?

 

 

View 10 Replies View Related

SQL Server Destination

Sep 11, 2007



Hi,
I have created a data flow with 2 target components ( SQL Server destination )
I ran the package and everything went fine.
Unfortunatelly when I changed TransactionOption in the package properties to Required. It got stuck in the very beggining.
The service DTC is working.
Does someone know why I can't load 2 targets in one transaction ?

Thanks ahead
Eran

View 1 Replies View Related

DB Engine :: Copy Files From Source Server On Destination Server

Sep 25, 2015

I want to schedule a job which pulls files from a non SQL server (Sybase) which later needs to have a step 2 kicking the ssis package. Problem is that, on the source a batch file will run every 4 hours and outputs total of 10 text files. (takes 5 minutes complete). Now, on destination i want to pull these files via SQL job but while scheduling;

1. I don't see any option saying like 4 hours 10 minutes or so
2. If its out there, then i believe this might be a problem as this time would be an increment one e.g next run would be 4 hours 20 minutes in that case.

How should i achieve pulling these files up because we have an SSIS package on destination that needs those text files to be used as soon as they arrive on SQL server(destination)

View 2 Replies View Related

Copying Table Data From SQL Server 2005 To SQL Server 2000 - Very Slow When Using OLEDB Source And Destination Sources?

May 8, 2006

An SSIS package to transfer data from a DB instance on SQL Server 2005 to SQL Server 2000 is extremely slow. The package uses an OLEDB Source to OLEDB Destination for data transfer which is basically one table from sql server 2005 to sql server 2000. The job takes 5 minutes to transfer about 400 rows at night when there is very little activity on the server. During the day the job almost always times out.

On SQL Server 200 instances the job ran in minutes in the old 2000 package.

Is there an alternative to this. Tranfer Objects task does not work as there is apparently a defect according to Microsoft. Please let me know if there is any other option other than using a Execute 2000 package task or using an ActiveX Script to read records from one source and to insert them into the destination source, which I am not certain how long it might take and how viable will that be?

Any inputs will be much appreciated.

Thanks,

MShah

View 5 Replies View Related

Memory Configurations For SQL Server 2005 EE X64 With SQL Server 2000 EE X32 On Windows Server EE X64 Server

Apr 20, 2007

My server is a dual AMD x64 2.19 GHz with 8 GB RAM running under Windows Server 2003 Enterprise Edition with service pack 1 installed. We have SQL 2000 32-bit Enterprise installed in the default instance. AWE is enabled using Dynamically configured SQL Server memory with 6215 MB minimum memory and 6656 maximum memory settings.

I have now installed, side-by-side, SQL Server 2005 Enterprise Edition in a separate named instance. Everything is running fine but I believe SQL Server2005 could run faster and need to ensure I am giving it plenty of resources. I realize AWE is not needed with SQL Server 2005 and I have seen suggestions to grant the SQL Server account the 'lock pages in memory' rights. This box only runs the SQL 2000 and SQL 2005 server databases and I would like to ensure, if possible, that each is splitting the available memory equally, at least until we can retire SQL Server 2000 next year. Any suggestions?

View 8 Replies View Related

Transfer Logins: The Source Server Can Not Be The Same As The Destination Server

Nov 30, 2005

I am replacing a server so I need to migrate everything. The old server is running SQL2000 and the new server is running SQL2005. I am trying to write an SSIS solution to migrate everything for me and I can't even get started because I get the error "The source server can not be the same as the destination server". At the same time I am changing the name of the Domain so the two machines arenot even members of the same Domain. I am doing this over the Internet so the machines are not even on the same subnet. The only thing I can think of is that the machine names are the same so even though the domains are different therefore the full names are different, the NetBIOS names are the same. Could that be the problem?

View 1 Replies View Related

Transfer Login: Source Server Can Not Be Same As Destination Server

Jul 3, 2007

Hi!



I have two servers. Server A and Server B Both the servers are in different domains and both have different names.



Server A runs a clustered sql server (sql 2005 standard sp1)



Server B also runs a clustered sql server (sql 2005 enterprise sp1)



Both server A and Server B has same virtual server name X.



So, when I try to copy all the logins from server A to Server B using Transfer Login task in ssis, I gert the following error-



"Source Server Can not be same as destination server"



I am using IP addresses to connect to both the servers.



How do i solve this issue now?



Thanks

View 1 Replies View Related

Getting Error In SQL Server Destination

Aug 18, 2005

Hi

View 26 Replies View Related

Error Using SQL Server Destination

Dec 19, 2007



Hey,

I get an error when I try to use a SQL Server Destination.
I want to use this because its performance, before I've used OLE DB Destination.
The idea is that I put data from a table or SQL statement and that I put that in a table using SQL Server Destination.
Now the SQL server stands on different server then where my SSIS is saved. I'm not using my windows credentials ( safety reasons ).
The message I get is standing below:

[SQL Server Destination [905]] Error: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80040E14. An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80040E14 Description: "Could not bulk load because SSIS file mapping object 'GlobalDTSQLIMPORT ' could not be opened. Operating system error code 2(The system cannot find the file specified.). Make sure you are accessing a local server via Windows security.".


Thanx for the info.

View 5 Replies View Related

I Get This Error Using Sql Server Destination

Sep 5, 2007



Does anyone have any idea?

I was using OLE DB DESTINATION and after going to Sql Server Destination task:



[SQL Server Destination [11259]] Error: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80040E14. An OLE DB record is available. Source: "Microsoft SQL Native Client" Hresult: 0x80040E14 Description: "Could not bulk load because SSIS file mapping object 'GlobalDTSQLIMPORT ' could not be opened. Operating system error code 2(The system cannot find the file specified.). Make sure you are accessing a local server via Windows security.".



??

TIA

View 5 Replies View Related

SQL Server Destination [733] Error

Jan 15, 2006

I am getting the following error on a FuzzyLookup transformation.  I have 1,000 rows in the source table and 331,580 rows in the reference table.  Both tables are in the same SQL2005 db on my local machine.  Any ideas?

[SQL Server Destination [733]] Error: An OLE DB error has occurred. Error code: 0x80040E14. An OLE DB record is available.  Source: "Microsoft SQL Native Client"  Hresult: 0x80040E14  Description: "Cannot fetch a row from OLE DB provider "BULK" for linked server "(null)".". An OLE DB record is available.  Source: "Microsoft SQL Native Client"  Hresult: 0x80040E14  Description: "The OLE DB provider "BULK" for linked server "(null)" reported an error. The provider did not give any information about the error.". An OLE DB record is available.  Source: "Microsoft SQL Native Client"  Hresult: 0x80040E14  Description: "Reading from DTS buffer timed out.".

View 2 Replies View Related

SQL Server Destination Using Variables?

Jul 27, 2007



Im using the SQL Server destination object to load some data (about 10gb) into a table. However I would like to use a variable to define what table I load into. All the tables have the same structure. I dont see a way to bind a configuration to it, or to use a variable as the loading table name.

Thanks.

View 4 Replies View Related

Dynamic SQL Server Destination???

Dec 23, 2005

Hi,

Is there anyway to dynamically change the Sql Server Destination? Let's imagine i want the data to go to a table based on a specific parameter like year for example...

Imagine tables like table1_2005, table1_2006 and i want to dinamically change the SQL Server Destination so it would insert 2005 or 2006 and so on....

I can't seem to find that option... It was really a good idea for microsoft to implement Expressions in such components... Dynamics it's all about it :)

 

Best Regards,

View 8 Replies View Related

MaxInsertCommitSize - SQL Server Destination

Mar 26, 2007

Installing SP1 for SQL Server 2005 should get this property ? I am not able to get the same even after installing SP1

Can Anybody guide me here ?

View 4 Replies View Related

SQL Server Destination Not Loading All Rows

Jan 23, 2008

I have a SSIS package that transfers data from three SQL Server 7 servers to a SQL Server 2005 database. This package has about 30 different tables it copies. The table structures in the source database and destination tables are identical. About 25 different tables load without any issues. I have about 5 tables that load some nights without a problem. On other nights, the data transfers seem to randomly (though usually the most recent records) ignore some of the data. I have logging turned on and receive no errors. It just appears to stop loading data.

I should also mention that I truncate each destination table before begining and each table is loaded from data from each of the 13 source database (I am combining data from 13 regional database for reporting purposes). This is done using a Foreach Loop Container that updates the Server/Region connection string for each region. I am using the OLE DB Source connection to the SQL Server Destination. I have tried as well with the OLE DB Destination with the same result (and no error). I do not do any manipulation to the data on the transfer, but added a "RowCount" transformation between the source and destination and it gives the correct number of rows, but not all the rows get loaded.

View 6 Replies View Related

SQL Server XML Destination - Nested XML File

Apr 15, 2008

I am creating an XML file from 'n' number of tables using Dataset.Writexml()

Now, I join tables using INNER JOIN and fill the resultset into a dataset (There is a foreign key called ID in each of the tables).

The resultant XML doesn't shows the nested nodes, It shows the nodes sequentially.

The Result I got was,

<Root>
<Name>
<FName>Fxxx</FName>
</Name>
<Name1>
<LName>Lxxx</LName>
</Name1>
</Root>

The FName and LName resides in different tables.


<Root>
<Name>
<FName>Fxxx</FName>
<Name1>
<LName>Lxxx</LName>
</Name1>
</Name>
</Root>


I have tried the DataTable Relations and Dataset merge.

Any work-around / Straight approaches ?

View 7 Replies View Related

SSIS - Add A New SQL Server Mobile Destination

Nov 14, 2005

Hi, I try to add a new SQLMOBILE Connection.

View 6 Replies View Related

MaxInsertCommitSize Not Found In SQL Server Destination

Aug 29, 2007

Hi,

I have installed SQL Server 2005 Service Pack 1 installed. I could see that from running @@Version command.

i am not able to find the MaxInsertCommitSize custom property in my sql server destination in SSIS packages.

is there anything specific i need to do to see that.

--

Mahadevan H

View 9 Replies View Related







Copyrights 2005-15 www.BigResource.com, All rights reserved