Usage Of Raw File In SSIS
May 31, 2006What is raw file in SSIS, could you tell me samples for usage of raw file
View 4 Replies
ADVERTISEMENT
SQL 2012 :: TempDB Log File Usage Constantly Rising And File Keeps Growing?
Jun 16, 2014The TEMPDB transaction log file keeps growing.The database server is new and the transaction log was presized to 1 GB on installation. After installing a number of databases, the log file grew over a day to 38GB. Issuing a manual checkpoint was the only way to free some space to allow it to be shrunk back to a usable size. The usage of the file is still going up.
I am struggling to find what process is causing the log to be used so heavily. Looking at the log reuse wait desc for tempdb returns "Nothing" and tempdb itself isn't being used very much or growing in size.
CPU Usage By SSIS
Nov 6, 2007In my quest of writing a generic SSIS package programatically, I have another block.
When I execute the exe seperately it consumes more than half of the CPU, some times its more than 90%. Is there any way where I can say how much of CPU to be used?
Thanks & Regards,
Karunakaran
High CPU Usage SSIS Package
May 2, 2006Anyone aware of anything I can look at to improve performance of my SSIS packages? i basically have two master packages that call about 15 child package tasks each. All the package tasks are called sequentially and all they do is just load flat files into temporary tables and execute a simple stored proc. Each package tasks check to see if a file is there and if so it processes it else it just ends. The packages are scheduled to run every minute to check for files. If I watch the CPU during execution i can see the two instances of DTSEXEC.exe running and consuming about 25-30% of the CPU. The box that this runs on is a Quad processor with 4 GB of ram.
Any ideas on how to improve the performance of this? I have tried playing with the transaction level, but that didn't seem to do much performance wise.
Thanks for the help!
SSIS Performance And Memory Usage
Sep 24, 2007Hi -
I am facing 2 problems :
PROBLEM 1 :
We have a few packages that run pretty fast on a desktop server with 2 Gig RAM, Dual processor (approx 4-5 hours). But the same packages run very very slow on the another server containing 8 CPU and 12 Gig RAM (ran for 24 hours without completing).
PROBLEM 2 :
The CPU% ranges from 40-80% and the PF usage is stagnant at 2GB on desktop server for the same package. But in the 8CPU server, the CPU % ranges from 0-10% but the PF Usage raises from 750 MB to 8 GB.
This has become critical to our application.
TIA,
Shabs
Log File Space Usage
Feb 24, 2000On SQL*Server7, is there a system table to find out the transaction log space used? DBCC SQLPERF (logspace) must be using a undocumented system table.
Thanks.
Page File Usage
Jun 17, 2007Hi,
I have SQL Server 2005 running on Windows Server 2003 Standard Edition SP2.
The server has 3GB of physical memory and its PF size is fixed with a 2GB min and 4 GB max.
This server has only 1 database with maybe a max of about 10 concurrent db connections.
Everytime after a reboot or databse restart, the PF usage slowly grows and grows until the server locks up. It appears as though the server is not releasing the RAM after is uses it...
For example, right now, it is early in the morning and there is only 1 db connection but the PF usage is sitting at 1.87GB and slowly growing.
Has anyone ever had this issue before or does someone know how I can resolve this issue?
I would appreciate any help
Cheers
SQL Server Page File Usage;
Jun 22, 2007Hi all,
We have our SQL Server 2005 installed in a server with below specifications.
MS Windows Server 2003 R2
Standard x64 Edition
Service Pack 2
Intel(R)Xeon(R)CPU
5160 @ 3.00GHz
3.00 GHz, 8.00 GB of RAM
The issue is ,
When a query window is opened and a procedure is executed or a DBCC command is executed , the PF usage keeps on increasing and reaches the maximum 8.81 GB and remains there forever even if the query is stopped and the session is closed.
The CPU usage also shoots up and stays between 90 - 100 %.
Now, it is not possible to open a new query or do any operation in the SQL server and a Timeout Expired error is thrown.
My questions are ,
1. Is that good for the SQL server to occupy the total RAM ?
2. Is the performance related to the RAM space occupied by SQL server ?
3. What should we do in order free the PF usage after the a process is complete ?
4. Have anyone faced similar scenario ? Any solutions ?
Thanks for your time & any help should be apprecitated.
DBLearner
High Page File Usage
May 3, 2007Just built a new DB server. Using SQL2K5Standard 64bit on W2K3Server 64bit. Hardware includes 16GBs memory/dual 4-core procs and many spindles, however we only have a 2GB page file on C:. SQL is set to to usa a maximum of 12GB memory which is way more than it should need. Problem is, we are occasionally seeing very high page file usage. If Task Manager is correct, (which it really can't) I am using 10 and 13.8GB (as listed in PF Usage) But I only have a 2GB page file! Occasionally when simply copying a file from one partition to another, PerfMon shows Pages/sec jump to 10,000. Also see some very high Disk Queueing.This system isn't even in production yet. My main questions are two: 1) some documentation says with SQL2K5-64 bit running on W2K3-64bit I may not even need a page file. Is this correct? 2) What would be causing high usage of the swap file with 16GBs of installed memory? And help would be appreciated. Thanks in advance!
MikeE
Usage Question - Running SSIS Packages Remotely
Jan 29, 2007Hello,
I currently use DTSRun to run packages on my scheduler box. I would like to use the same set up to run SSIS packages.
Is there any way to duplicate this without installing SSIS on my scheduler box? Can I specify the server on which it should run?
Just don't want topurchase another license, if possible.
- Dave-
Usage Of Dtutil And Dtexec In SQL Server 2005 SSIS
Nov 22, 20061) We are migrating data from SQL Server 2000 to 2005 DB.
2) We are developing SSIS packages in client, build, deploy and install and copying to SQL Server 2005 server to execute for migration.
3) For this we encrypt all packages with password, then decrypt all while executing in a batch file using dtexec. This works fine.
4) What does not work is executing a master execute task in which are individual execute tasks. This is so since the other packages are still encrypted.
5) We decrypted them separately first using dtutil using Decrypt and Copy/Move option but while executing error is thrown that these other packages are still protected by password.
6) Do we need to decrypt, then encrypt using default package protection level and then decrypt again.
7) How exactly to just execute a master package when packages are developed on client.
SQL Server Performance Monitor Log File Usage ??
Nov 4, 1999Hi,
I logged some of the parameters of my SQL server using the Performance monitor into a log file - smn5.log & the log settings in smn5.pml. I started the log and the log file (smn5.log) started growing in size indicating that it was collecting data.
I then went to Options button and said Save. After this in the File menu, I selected Export Log and saved it in a .CSV file, expecting it to contain the Logged data. However, it contains only the Log settings as shown below :
--------------------------------
Reported on L&T1362
Log File C:MSSQLLOGsmn5
Interval: 15.000 seconds
Object,Computer
SQLServer-Log,ERMINTRUDE
SQLServer-Procedure Cache,ERMINTRUDE
SQLServer-Locks,ERMINTRUDE
SQLServer,ERMINTRUDE
SQLServer-Log,FLORENCE
SQLServer-Procedure Cache,FLORENCE
SQLServer-Locks,FLORENCE
SQLServer,FLORENCE
---------------------------------
Could some one please tell me how to gather and view the logged information ? smn5.log contains 10MB of data - the perf. monitor shows that.
Thanks
Satish
SQL 2012 :: Tracking Page File Usage - Values Appear Incorrect
Jul 31, 2015I'm looking for monitoring page file usage for SQL Server. Ultimately, I'm trying to prove that there is not a problem with a server and that it is not struggling with the workload, but someone has looked at page file usage and suggested there is a problem.
I've set up performance counters running from a separate server, but the counters relating to page file usage seem wildly inaccurate. E.g., I've got (_Total\% Usage), and (Total)\%Usage Peak), with Total Usage showing a constant value of 64, and the peak showing 92. I've also got Process (sqlservr)Page File Bytes and Page File Bytes Peak, which are showing values like 61,275,688,960 and hardly dropping below this.
If I look at Performance Monitor directly the values shown in here correlate with this, as shown in my first screenshot. However, if I look at resource monitor and look at disk activity, you can see that there is basically nothing going on (screenshot 2).
The server is more than equipped to deal with the workload, and looking at other counters for SQL there is no indication of problems, e.g. page life expectancy has a minimum value showing of 576,258, there is never less than 6GB of free RAM, SQL compilations are at a maximum of 5% of batch requests.
It is a physical server with 2x 8 core multi threaded CPUs, 64GB RAM with 58GB as the SQL server maximum. Hopefully this is enough info, but I'm happy to check anything else.
I/O Congestion Troubleshooting: Memory And Swap File Usage Question
Jul 20, 2005We are hosting a 140 GB database on SQL Server Version 7 and Windows2000 Advanced Server on an 8-cpu box connected to a 15K rpm RAID 5SAN, with 4 GB of RAM (only 2 GB of which seem to be visible to theOS) and a 4 GB swap file. (The PeopleSoft CIS application will notpermit us to upgrade to SQL 2K.) We recently upgraded the server from4 to 8 cpus and the SAN disks from 10K to 15K drives. But we stillhave heavy SAN disk usage, sometimes at 100%, and read queues oftenaveraging 4 and peaking at 12.The CPUs are loaded at only 20-50%. (The politics are such that it iseasier to throw hardware at the problems.)We are looking into archiving, converting from RAID 5 to RAID 10, andat splitting the mdf file into several file groups in an attempt toget more disk heads into play. (We are also looking at rewriting theapplication to reduce the read volume and frequency.) Does anyone haveany other ideas?Incidentally, does swapfile get used when the physical memory equalsthe OS maximum? If the OS can only see 2 GB and we have 2 GB (actually4 GB) of memory, is the 4GB local swap file on the C drive unused?Thanks in advance for any assistance.
View 1 Replies View RelatedComputing The CPU Usage ,memory Usage For An Inserted Record
Nov 2, 2007
I have a client program that writes to sql server database 10 records per second . i want to compute the CPU usage and the memory usage for the whole program or CPU usage,memory usage for the insert statement in the program .
Can anybody help me with this?
CPU Usage(%), Logical IO Performed (%) Usage For Adhoc Queries Is 90%
Sep 7, 2007
Hello, When I am seeing SQL Server 2005 Management studio Server Dashboard> I am seeing my(USERS) databases and msdb database usage is very small % of in CPU Usage(%), Logical IO Performed (%) Usage pie chart.
90% of Total cpu usage is showing for Adhoc Queries. what excatly this means in Dashboard? if application uses more than it would have shown in Database level or not?
sicerely this dashboard is good, if any one is watching daily, please advice their experiences here.
Thanks in advance. Hail SQL Server!
SQL Server 2012 :: Query To Get CPU Usage / Memory Usage Details Of Server?
Jan 30, 2014providing a query for fetching the data for CPU Usage, Memory usage, blocking and all details ...
I want to create a job which will run on a Node every 15 min and store data in a table for each instance...
DMV is not giving more stuff and xtended events not sure if i can store that data into a table?
Read Text File From Flat File Connection Manager SSIS
May 13, 2008Hello Experts,
I am createing one task (user control) in SSIS. I have property grid in my GUI and 2 buttons (OK & Cancle).
PropertyGrid has Properties like SourceConnection, OutputConnection etc....right now I am able to populate Connections in list box next to Source and Output Property.
Now my question to you guys is depending on Source Connection it should read that text file associated with connection manager. After validation it should pick header (first line of text file bases on record type) and write it into new file when task is executed. I have following code for your reference. Please let me know I am going in right direction or not..
What should go here ?
->Under Class A
public override DTSExecResult Execute(Connections connections, VariableDispenser variableDispenser, IDTSComponentEvents componentEvents, IDTSLogging log, object transaction)
{
//Some code to read file and write it into new file
return DTSExecResult.Success;
}
public const string Property_Task = "CustomErrorControl";
public const string Property_SourceConnection = "SourceConnection";
public void LoadFromXML(XmlElement node, IDTSInfoEvents infoEvents)
{
if (node.Name != Property_Task)
{
throw new Exception(String.Format("Invalid task element '{0}' in LoadFromXML.", node.Name));
}
else
{
try
{
_sourceConnectionId = node.Attributes.GetNamedItem(Property_SourceConnection).Value;
}
catch (Exception ex)
{
infoEvents.FireError(0, "LoadFromXML", ex.Message, "", 0);
}
}
}
public void SaveToXML(XmlDocument doc, IDTSInfoEvents infoEvents)
{
try
{
// // Create Task Element
XmlElement taskElement = doc.CreateElement("", Property_Task, "");
doc.AppendChild(taskElement);
// // Save source FileConnection
XmlAttribute sourcefileAttribute = doc.CreateAttribute(Property_SourceConnection);
sourcefileAttribute.Value = _sourceConnectionId;
taskElement.Attributes.Append(sourcefileAttribute);
}
catch (Exception ex)
{
infoEvents.FireError(0, "SaveXML", ex.Message, "", 0);
}
}
In UI Class there is OK Click event.
private void btnOK_Click(object sender, EventArgs e)
{
try
{
_taskHost.Properties[CustomErrorControl.Property_SourceConnection].SetValue(_taskHost, propertyGrid1.Text);
btnOK.DialogResult = DialogResult.OK;
}
catch (Exception ex)
{
Console.WriteLine(ex);
}
#endregion
}
How To Specify Multiple File Types In SSIS Foreach Loop (File Enumerator)?
May 2, 2007I want to enumerate all *.xls and *.csv file. How to fill the Files box? I tried
*.xls, *.csv
*.xls *.csv
*.(xls|cvs)
all doesn't work
SSIS - Data Flow To Flat File - Insert At Start Of File
Oct 24, 2007Hi all,
In a foreachloop, I am inserting records into a flat file which is working fine. But the thing is that as the file grows, it takes longer for it to locate the EOF(End of File) of the flat file so as to insert the records.
I have around 70-100 lines written to the file at each loop and there are more than 20k records to be looped. wihich means that at the end I should be having 1400k - 20000k line in the text file.
One solution would be to insert the records at the start of the file itself so that it does not has to lookup the EOF each time before writting.
Another would be to generate separate files and then merge it.
Any idea how can this can be done?
Beside this I have to zip the file and then SFTP to a given address.
Any suggestion or help would be welcome.
Rdgs
David
SQL Server 2008 :: SSIS Copy File From SharePoint Library Using File System Task Permissions?
Jun 19, 2015Historically I've always written a VB script to copy a file from a sharepoint library. I don't like this method because I have to input a username & password in the script and maintain a config file.
Yesterday I was playing around with using a file system task. The sharepoint file has a UNC path so why not? I created a simple test package with a single file system task that copies the sharepoint file (addressed via UNC) to another network location. Package runs fine locally.
I try running on our utility server but am getting a "The file name [SHAREPOINT UNC PATH] specified in the connection was not valid" error. Package is running with a proxy on the server and the proxy account has the same permissions to the sharepoint site (so far as I can tell) as me.
Integration Services :: Reading Data File Present In A File System From A Package Deployed In SSIS DB?
Dec 4, 2014I am trying to create and later read a data file from a package deployed in SSISDB, but it is not reading it while it is successfully creating the file. The same package when run from the file system package, runs successfully. Generating ispac and deploying in SSISDB is running for infinite time. Is it a permission issue?
View 7 Replies View RelatedUse Of SSIS In Reading And XSL File. Should I Use SSIS Or Just Straight C#
Aug 1, 2007I need to build an asp.net/C# application to read values from an Excel spreadsheet. Once the values are read from the spreadsheet, the C# code will do some elementary statistics on the values read. Then the values read and their computations will be written to a sql server database.
My manager suggested that SSIS might be a good candidate technology for doing this type of work. Does that sound correct? My only hesitation with using SSIS is that I want to keep the application as simple as possible, so that the code can be more portable. Maybe might argument is not a good one, but maybe someone can help me out here.
Ralph
SSIS - Get ServerName From A XML Or TXT File Into SSIS Package
Jan 4, 2008Dear Friends,
I store several configurations in the main database of my SSIS packages. I need to get the servername from a xml or txt file in order to get those configurations stored in my database.
How you think is the better way to do that?
Using a FlatFileSource to read the file and a script to save the value into a SSIS variable?
Using the package configuration I cant do that... maybe I dont know, but I can save the SSIS variale in the configuration file, but what I need is to do the inverse, read the configuration file and save the value in the SSIS variable.
How the best way you suggest?!
Regards!!
Thanks.
How Can Unzip File Text File Using SSIS
Oct 1, 2006
Hi,
I am pulling text files in gzip format from UNIX system. I want to unzip these files and then import data from these files into database using SSIS.
SQL 2012 :: Creating Dynamic SSIS File Format - Dynamic CSV File As Output
Mar 2, 2014I am trying to create an ssis package with dynamic csv file as output. and out format contains query output.
sample file name:
Unique identifier + query output + systemdate();
The expression is looking like this.
@[User::FilePath] + @[User::FileName] + ".CSV"
-- user filepath is a variable from ssis package. File name is the output from SQL query. using script task i have assigned the values to @[User::FileName] .
When I debugged the script task the value getting properly but same variable am using for Flafile destination. but its not working.
Integration Services :: Send Excel File From SSIS Using Send Mail Task Without Saving Excel File Locally?
Jul 14, 2015Is there anyway to send excel file from ssis using send mail task without saving the excel file locally. I need to automate the process which involves loading the excel file from the database and send it to some people.
View 6 Replies View RelatedSSIS Log File Name
Jan 30, 2008Originally in my DTS, the logfilename property is assign by a variable in a dynamic properties task.
After migrate, it appear commented in the script task. So may i know how can i assign the logfilename with the variable value?
Please advice, thank you.
Using .ds File In SSIS
May 29, 2007Hello,
Can we keep a database connection information in .ds (datasource) files. Is it possible to use these instead of the standard approach of putting the connection string in the configuration file.
Furthermore, when they run the SSIS package from the command line (using DTEXEC), is it possible to pass the .ds file as a parameter?
SSIS And XML File
Apr 22, 2008Hi,
I am using an XML source component to integrate it into an SQL server data base.My problem is that when I try to open my data flow containing the XML source component, it take much time to open it, so I don't know why beacause I have a very good machine with 2GO of RAM. is it a problem of SSIS with XML files?
thank you in advance
How To Rename A File In SSIS
Sep 26, 2006I want to rename a file that is a variable to a fixed file name in a specific folder in SQL Server System Integration Services using the File System Task Editor. Example: File1.txt to Users.txt or File2 to Users.txt. The source file is a variable the destination file is not. Here is an example of the File System Task EditorHOW DO I SPECIFY A FIXED OR VARIABLE FOLDER NAME WITH A SPECIFIC FILE NAME?????????File System Task EditorProperties ValueDestination ConnectionIsDestinationPathVariable TrueDestinationVariable User::ArchivePathOverWriteDestination TrueGeneralName Rename a fileDescription File System TaskOperationOperation Rename fileSource ConnectionIsSourcePathVariable TrueSourceVariable User::ImportFilePath Thanks
View 1 Replies View Related[help] SSIS File Configuration (XML)
Apr 3, 2007We are deploying our SSIS packages into different folders. For example:
Test1
est2
production
Test1 points to the Test1 database, etc.
So, I configured my SSIS package to use a database connection called dbMAIN.
I then setup the SSIS File Configuration (XML) so that "dbMain" points to Test1. This xml file, called Global.dtsConfig, sits in C: est1, the same place as my ssis.dtsx file.
I want to copy my dtsx file and my dtsconfig file into est2 and production. The problem is that the the location of dtsconfig is HARDCODED in the ssis package!
How do I dynamically change it?
I tried to hardcode the location to ".Global.dtsconfig", but that did not work! Please advise!
URL File Download From SSIS
May 28, 2013 have a URL that I get from a partner by calling another URL and feeding it some query parameters. That call returns to me a URL which is what I need to then go get the file I'm downloading. I have no problem getting that URL into a variable in my SSIS package. URL....there are several key-value pairs after that question mark which denote the login information, the report ID that it's picking up, etc.
If I manually take that URL and dump it into IE and hit enter, I get the standard "Open/Save/Cancel" prompt. If I open it, I see the XML that should be in my file in the browser. If I save it, I get the standard save dialoge box so that I can save the xml file.How do I do this in SSIS? I can create a file connection for where I want this XML file to be saved, but I don't see an option in the XML task to simply save the file from a URL.