Reporting Services And Huge Data Extracts Causes IIS To Use A Lot Of Memory
Dec 30, 2007
Hi.
I am working on a serial tracking application using Sql Server 2005 and .Net. One of the requirments is to have an ad-hoc file export utility in which users can drag-n-drop fields from a set of tables and export the results to CSV. It all sounds ok and Sql Server Reporting Services' Report Builder seem to be just the right tool for it, but there is one problem :
The report size is big, about 7K - 8K pages and 4 - 5 columns wide; while rendering the report, IIS memory usage shoots up to about 2GB and remains at about 2GB.
Any idea if something can be done to mitigate this problem? Note that I dont need the HTML rendering at all. All I need is to have the CSV at the end of the day, while users are able to chose columns in an ad-hoc manner.
We have an engineering software which was not designed for SQL use. Basically what they have are a bunch of tables, and in those table values are comma delimted lists. They then created views off these crappy tables and created a "key" in the view.. I'm not sure you already guessed that this sucks for writing reports and things of that nature, so, I decided to create my own database and create extracts. I know how to get new data into the tables which I'll outline one below, but the part I'm not sure of how to do is how to see if a value is changed. They don't have timestamps tracking a update otherwise I'd just check against that and I'd rather prefer so just update data that needs to be versus doing a complete wipe of the data. Some of the data (such as sales orders) the employees requested to be extracted live however I was able to negotiate with them down to every 15 minutes. Here's what my table looks like for customers.. CustomerId int NOT NULL, CustomerNumber nvarchar(16), CustomerName nvarchar(100), BAddress1 nvarchar(100), BAddress2 nvarchar(100), BCity nvarchar(50), BState nvarchar(10), BZip nvarchar(15), SAddress1 nvarchar(100), SAddress2 nvarchar(100), SCity nvarchar(50), SState nvarchar(10), SZip nvarchar(15), Phone nvarchar(30), Fax nvarchar(30)
And this is how i'm inserting new data.INSERT INTO Intranet.dbo.FS_Customer SELECT c.CustomerKey, c.CustomerID, c.CustomerName, c.BillToAddress1, c.BillToAddress2, c.BillToCity, c.BillToState, c.BillToZip, c.CustomerAddress1, c.CustomerAddress2, c.CustomerCity, c.CustomerState, c.CustomerZip, c.CustomerContactPhone, c.CustomerContactFax FROM FSDBKL.dbo.FS_Customer AS c WHERE c.CustomerKey NOT IN (SELECT cu.CustomerId FROM intranet.dbo.FS_Customer AS cu)
FSDBKL.dbo.FS_Customer is the old table, and intranet.dbo.FS_Customer is the new one I created. If I have to I can update each record but right now the customers table alone has 9,000 records and I think doing a update on each record will be time consuming.
Hello, I have a very big T-SQL script (~24mb) and I need the SQL Server on my hosted site to execute it.
Right now, I have a web page that uses Sql.Connection.ExecuteNonQuery () to do it, but the .NET process runs out of memory when I load the file into a C# string.
How would I go about executing this T-SQL script on the server? Is there such a command: EXECUTE SCRIPT "myscript.sql" FROM DISC ?
Does anybody know why BCP on v6.5 grabs so much memory for SQL Server? I have a few table imports where the BCP process will consume over 460MB of RAM during the imports.
The BCP cmd file is executed via an xp_cmdshell call. The server has 2+GB of RAM, but the BCP process effectively flushes large amounts of data from the buffer. It takes quite along time for the cache to recover from this, and after this, the rest of the nightly processes run much slower, as they end up having to hit the drives to retrieve information that should already be in cache.
If anyone can shed some light on this it would be much appreciated.
I know I should probably be posting this in the RS section but I have a Win 2008 R2 server and RS 2012 along with SSIS and Database server installed. I also have SQL 2005 instance with Sql 2005 SSIS running on the server.
I saw that Reporting services was consuming 9gb of ram a few days ago with no published reports. It's just a default install.
So I investigated the settings and the document: [URL]......
I added this section to the reportserver.config file to restrict memory usage
windows is sql server 2012 64 bit edition and sql server is 2012 64 bit edition.
RAM installed on both server is around 65 gb of which 49 gb is max server memory allocated for sql services on both servers.
database related to reporting services are also in always on group .
We have also configured for reporting services and both are running on their respecting server.
Issue is on primary server reporting service is using almost 7 gb while on secondary it is using 10 gb even when there are 5 reports and its used within offices .
what issue and how to check why ssrs is using high memory..
any query , perfmon counters
reports are randomly used at client side
i have checked memory utilization through task manger..
Getting old favourite message 'Out of Memory' when running a horizontal clustered chart with Category: 200 items Series: 200 items per Category.URL... 'By default, the report server sets WorkingSetMaximum to the amount of available memory on the computer. This value is detected when the service starts.'
The report has been tried on 3 large servers with 30GB & 60GB memory. The report runs on a 2008 install but not a 2008R2 install. We've created a test report that simply generates a series of data 1-n for Category and 1-n for Series where n can be set by parameter - so the issue is not to do with the volume of data or the processing required by SSMS. The report runs if the data is output to a table rather than a chart - so the issue appears to be with rendering the chart.
The chart is rendered when the Category has 200 items with a Series of 150 per Category. The report fails with Category 200 items and Series 200 per Category.
I gave up on the ScriptTasks, but desided to use Custom tasks instead. Problem again. My code opens a 400M file and reads it line by line using StreamReader. Each line is approximately the same length. For each line, there is some processing, and then the line is written into another file using StreamWriter. I am watching the DtsDebugHost process with TaskManager open and here is what happens:
Initially its all good. Then when it read through first 150M+ of the input file, the memory usage of the DtsDebugHost shoots up dramatically - about 1Gig (both virtual and physical memory). Then the task fails with OutOfMemoryException. I thought the problem is with my code, but it still happens even if I only read a line from one file and write it to another, without any processing!
When I invoke the same code from Execute Process task, its all good - no problems at all whatsoever.
I work in a big project and we will begin in using reporting services as the base technique for reports and I will be responsible for this part. but I have a problem I will discuss in the following:
Currently: We use currently devexpress reports and we have 2 languages(Arabic and English). the data in tables saved in two ways (Arabic and English). when the end user change the language of the web site the report data language changed when run it.
Example:
we have table with (ID, NAME_AR, NAME_EN, JobTitle_AR, JobTitle_EN). designed report will display(ID, NAME_EN, JobTitle_EN) . but the end user change the language of the system the report will
I have worked in other ETL tools. So, i am trying to figure out how to do thefile decryption and process the data in memory using SSIS.I am using SSIS on Azure VM and my source files are on Azure storage. The files are encrypted and we are trying to use Phython script to decrypt the files and pass it to SSIS. I found out that Execute Process task can call the Phython script. However, i would like to get the decrypted data from the file and pass it to the next task (control flow) in SSIS without saving it as a file (in-memory). I found that execute process task output can be stored as a Standard Output Variable or to an object. Will this work or do I need to follow any other methods (since we need the entire file to be sent for additional processing).
My requirement for the parameter is multivalue parameter with a text box. for example when user enters aa15 it need to include product aa15. when the user enters aa15, aa16, zz15 than it needs to include all the three products. the last case is when the user enters AA** than i need to inclued all the products start with AA. when i use default multivalue parameter with data source analytical services than i am getting a drop down box. I dont want that. I need a text box where user can enter the value. 1. In sql we have a like key word to query . for example select * from product where product like "AA%". what is equavalent mdx query to get such results ? 2.How to impliment the multivalue parameters without using dropdown box?
I am trying to find a reference for a client that lists the fields available to be substituted into a data driven subscription from the query, along with the expected data types. For example, the field on whether or not to include a link to the report seems to be expecting a bit data type.I have searched and can't seem to find anything. I guess I could walk through the interface and try different data types, but if a list exists, that would be better.
I have 4 Tablix and 2 of the Tablix get data from Server 1 and other 2 get the data from Server 2.I have set NoRowsMessage "=Data Not Available for the Selected Values" for all the 4 Tablix.Now if data is not available from Server 1 then I must show "Data Not Available for the Selected Values" only once in the outputbut now its appearing twice in the output because of the 2 tablix that had no rows.Similarly if data not available from Server 2 then it should show "Data Not Available for the Selected Values" only once in my output.If Data not avilable from all the Tablix then also i t should show only once as "Data Not Available for the Selected Values" in the report output.
I am using SQL Server report 2008/2012 (SSRS) and my report viewer contains body content with 3 Row groups. While printing the report, data print with blank space and move to continue data to next page.
Departure flight : 70 rows First Page : 42 rows printed Second Page : 23 rows printed [ Supposed to be print 28 , if the total count of records more than 23 and less than 42 then the page print only 23 records ] Third Page : 5 rows printed
Departure flight : 42 rows First Page : 42 rows printed [Report max. record allowed to print 42 rows so if total record is 42 then print perfectly ]
Departure flight : 26 rows First Page : 23 rows printed [Supposed to be print 26, if the total count of records more than 23 and less than 42 then the page print only 23 records ] Second Page : 3 rows printed
I have a report where in I have a combination of matrix ,table data regions.
The problem what I am facing is that the data tables don't remain fixed in their position and they tend to move down.
E.g. table 1 and table 2 are on the same page in design time side by side (right and left)however during the runtime the table1 is pushed down and table2 is at its position .
Now how can I keep them all fixed in their same position. Most of the tables have fixed size rows and some who have high size of rows have been put at the end . What settings we can set?
I need to periodically import a (HUGE) table of data from an external data source (not SQL Server) into SQL Server, with the following scenarios: Some of the records in the external data source may not exist in SQL.Some of the records in the external data source may have a different value at different imports, but this records are identified univocally by the same primary key in the external datasource and in SQL Server.Some of the records in the external data source may be the same in SQL.
Due to the massive volume of the import, I would like to import only the records which are different from what I have in SQL Server (cases 1 and 2 above). In fact case 2 is the most critical.
I thought of making a query with a left outer join between the data in the external data source table (SOURCE) and the data in the SQL Server table (DESTIN). The join is done on the respective primary keys (composed keys of up to 10 columns) and one of the WHERE conditions will be that the value in SOURCE is different from the value in DESTIN.
The result of this query would be exactly what I need to import. How to do this in SSIS??? I couldn't figure out how to join tables in different data sources yet.
In fact I cannot write a stored procedure to do that, since one of the sources is in a datasources not SQL Server. I have seen the Lookup transformation in this article http://www.sqlis.com/default.aspx?311 but this is not exacltly what I want to do. Another possibility is to use the merge join, but due to the sorting I believe its performances would be terrible!
We have a daily process, which copies millions of rows of data from one DB to another over Linked Server. Just checking on the best practise, are there more efficient ways than the Linked server to copy millions of rows of data from one DB to another? I checked bulk insert but that transfers the data from the file to DB not DB to DB.
I created a report in Reporting Services which was working. Then I filled in the "No Rows" section in the properties of a tablix in the report. Now it shows the No Rows message even if there is data. If I open the Shared Dataset that is used in the report and open Query Designer and run the query I get data.
I am trying to migrate my reports from SQL server 2000 reporting services 32bit to 2005 64bit. I am following the migration steps that MS specified. Restored my Reportserver and ReportserverTempDB databases Then I was using the configure Report services to upgrade these databases but I always end up getting the follwoing exception when I run the upgrade on the "Database Setup" configuration for 'ReportServerTempDB' database System.Data.SqlClient.SqlException: Could not locate entry in sysdatabases for database 'ReportServerTempDBTempDB'. No entry found with that name. Make sure that the name is entered correctly. at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection) at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection) at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj) at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj) at System.Data.SqlClient.SqlCommand.RunExecuteNonQueryTds(String methodName, Boolean async) at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(DbAsyncResult result, String methodName, Boolean sendToPipe) at System.Data.SqlClient.SqlCommand.ExecuteNonQuery() at Microsoft.ReportingServices.Common.DBUtils.ApplyScript(String connectionString, String script) at ReportServicesConfigUI.SqlClientTools.SqlTools.ApplyScript(String connectionString, String script)
It's driving me crazy, why is it looking for 'ReportServerTempDBTempDB' in the catalog instead of 'ReportServerTempDB'? Is it possible to migrate from 32bit to 64bit?
How can I use reporting services to get reports out of analysis Services Database ?
I found following on one site, but I tried to look for Analysis services database on the server I am deploying the model , but I am not able to find it.
'To use Reporting Services with data mining, you simply create a Reporting Services project with the BI Dev Studio and specify an Analysis Services database as your data source. Depending on the contents of the database, you will be presented with either an OLAP or Data Mining query builder as in Figure 3.23. You can always switch between the two modes by clicking the Switch '
Has anyone here been able to incorporate a way of data entry using reporting services. What I would like to see is a single page that not only provides the report but also allows for the entry of data. It would be much cleaner to have both together.
Sample Data (highly simplified): pID recordDetails recordDate userID 1 Baby said "dada" 1/1/2008 1 2 Baby said "daddy" 1/3/2008 1 3 Baby said "muma" 1/20/2008 1 4 Baby said "mommy" 1/21/2008 1 5 Baby said "wahwah" 2/5/2008 1 6 Baby said "teevee" 4/1/2008 1 7 Baby said "mush" 4/6/2008 1 8 Baby said "peepee" 4/9/2008 1 9 Baby said "ouch" 4/9/2008 1 10 Baby said "owwie" 4/9/2008 1 11 Baby said "peek-a-boo" 6/2/2008 1 12 Baby said "I see you" 6/3/2008 1 13 Baby said "Do not want" 6/20/2008 1 14 Baby said "Oh, no they didn't" 6/21/2008 1 15 Baby said "Emmancipation por favor?" 6/22/2008 1 16 Daddy said "Don't bite Daddy" 4/9/2008 2 17 Daddy said "peek-a-boo" 5/5/2008 2 18 Daddy said "Want to stay over at MJs?" 6/10/2008 2 19 Mommy said "Breakfast?" 1/20/2008 3 20 Mommy said "Someone tinkle tinkled!" 3/21/2008 3 21 Mommy said "Honey, where did Baby go?" 6/22/2008 3
However sometimes-- I want to just pull what Mommy and Daddy said
so like SELECT * from tblSample where userID in (@sample) except I want it to explicitly 0 record counts for when records were not created in relevant months.
How do I get it to count 0 records for months of no records?
REAL SQL Reporting Services Expression: SELECT i.FacilityID,NameAbbr as FacilityName, d.FirstDay, SUM(i.IncidentCount) AS TotalIncidents FROM vCIR_CategoryAWOLSummary AS i RIGHT OUTER JOIN FirstDay AS d ON d.FirstDay = i.FirstDay AND i.FacilityID IN (@facility) WHERE (d.FirstDay BETWEEN CAST(LTRIM(STR(MONTH(DATEADD(MONTH, - 12, GETDATE()))) + '/01/' + STR(YEAR(DATEADD(MONTH, - 12, GETDATE())))) AS datetime) AND CAST(LTRIM(STR(MONTH(DATEADD(MONTH, - 1, GETDATE()))) + '/01/' + STR(YEAR(DATEADD(MONTH, - 1, GETDATE())))) AS datetime)) GROUP BY i.FacilityID, NameAbbr, d.FirstDay ORDER BY d.FirstDay
FirstDay is just a table with every first day of many months (until 2020).
vCIR_CategoryAWOLSummary is just a view.
I need to create a line graph thus why I need the record count of 0. Help? Thanks!
This webcast from our partner, ETI, may be of interest to readers on the forum - we see questions quite often about high-performance interaction with other databases.
First, I am very new to programming, so please use the most basic descriptions in any replies. I have created a simple program in VB express and would like to be able to generate a report for a database used in my program through SQL Server Express. My goal was to have someone using my program on a laptop (not connected to a server or network) to be able to generate this report through my program in VB express without installing additional software, based on the local database on their computer. So, I thought using Reporting Services would best accomplish this. I went through the AdventureWorks tutorial and had to play with the connection settings so it would finally let me connect to the DB in the Visual Studio "Report Wizard". Now, when I try to connect to a datasource for my program, I get log on problems. I don't want to have to log on to anything to run this report. Since I actually got the AdventureWorks database report to work, I know there is a way to do this, but can't figure the specfiic settings. See the error below that I get.
Login failed for user ''. The user is not associated with a trusted SQL Server connection.
For anyone reading this, does this sound like a practical way to accomplish my objective or would there be an easier way to generate a report through VB express? Since I haven't gotten to the build process, I am still unclear as to how reporting services reports (if not using a web server) plays into the whole picture. Would a user have to install another program other than my VB express program to view reports that I create through Reporting Services (again, if I don't want to use a network server and instead have reports generated from their local DB on their computer).
Obviously, I have a lot to learn yet, but any help getting started would be greatly appreciated!
I need to implement SQL Reporting Services but do not wish to use the production database for running the reports against. I intend to set a seperate server for reporting services. The data only needs to be refreshed on a daily basis, during slient hours. What would be the best way to copy data from the production to the reporting services server?
I have a group of reports that are the same for 3 different companies, the difference is the data connection. Is there a way I can change the data connection based on a variable passed in at the time of report execution?
I am creating a report that shows orders created by our employees but I am having trouble with one employee. She changed her last name this year and so in the report, it shows a line her with her maiden name and one with her new name.
Is there a way to combine all the data from both names into one name so it appears on one line and looks like a single person instead of two.
