Speeding Up Site By Storing Query Results In File
Mar 4, 2008
Hello,
We have some queries that are long and intensive. We have thought about running the queries and storing the data in a text file for lookup from our website.
Example: Our online store only displays items that are in stock so when a user selects a category a query runs and grabs only items that are in stock and then displays them. There could be thousands of items the query needs to sort through before displaying the items that are in stock. What if we ran this query once every hours an stored the results in a txt file? The asp page would then go to the text file to grab the results instead of having to run the query every time a user selects a category. Will this speed up the site by not having to query every time? Would this be a correct way to eliminate queries that run thousands of times a day?
thanks
Andy
View 2 Replies
ADVERTISEMENT
Mar 28, 2001
Can someone tell me how to store the results of a multi-row query in a text or varchar variable so that I might do something like this:
declare @var1 text
select @var1 = {results of select date, count(*) from testtable2 group by date)
Thanks!
joe
View 2 Replies
View Related
Mar 7, 2005
I'm working on a reporting tool that could bring back hundreds of thousands of results back at once. I need some way to run the actual query only once a day, and then the reporting tool would just pull back this cached results. To be short, I need to figure out how to do this using a minimum amount of resources. Would a DataView work with something like this? How would I have it update only once a day? I appreciate any advice!
View 2 Replies
View Related
Jul 20, 2005
Consider this SQL Query:-----------------------------------------------------------------SELECT c.CASE_NBR, DATEDIFF(d, c.CREATE_DT, GETDATE()) AS Age,c.AFFD_RCVD, c.PRV_CRD_ISS, x.RegE, x.Type, x.Fraud,c.CUST_FN + ' ' + c.CUST_LN AS CustFullName,c.ATM_CKCD_NBR, x.TotalLoss, x.Queue, x.Status,c.QUEUE AS Expr1, x.CHECK_ACT_NBR, c.CUST_LN, c.SSN,c.CREATE_DTFROM (SELECT TOP 9999999 cl.CASE_NBR, cl.SSN, cl.CREATE_DT,SUM(cast(TRANS_AMNT AS float)) AS TotalLoss,glQueue.REFN_NM AS Queue,glStatus.REFN_NM AS Status,grRegE.REFN_NM AS RegE, grType.REFN_NM AS Type,grFraud.REFN_NM AS Fraud, cl.CHECK_ACT_NBRFROM (((((T_CASE_LST AS cl LEFT JOINT_INCIDENT_LST AS il ON cl.CASE_NBR = il.CASE_NBR)INNER JOIN T_GNRL_REFN AS glQueueON cl.QUEUE = glQueue.REFN_NBR)INNER JOIN T_GNRL_REFN AS glStatusON cl.STATUS_CD = glStatus.REFN_NBR)INNER JOIN T_GNRL_REFN AS grRegEON cl.REGE_CD = grRegE.REFN_NBR)INNER JOIN T_GNRL_REFN AS grTypeON cl.CASE_TYPE_CD = grType.REFN_NBR)INNER JOIN T_GNRL_REFN AS grFraud ON cl.FRAUD_CD =grFraud.REFN_NBRWHERE (((glQueue.REFN_DESC) = 'Queue')AND ((glStatus.REFN_DESC) = 'STATUS_CD')AND ((grRegE.REFN_DESC) = 'YesNo')AND ((grType.REFN_DESC) = 'Fraud_Code')AND ((cl.STATUS_CD) = 0))GROUP BY cl.CASE_NBR, glQueue.REFN_NM, glStatus.REFN_NM,grRegE.REFN_NM, grType.REFN_NM, grFraud.REFN_NM,grFraud.REFN_DESC, cl.CHECK_ACT_NBR,cl.SSN, cl.CREATE_DTHAVING (((grFraud.REFN_DESC) = 'YesNo'))) xLEFT OUTER JOIN T_CASE_LST c ON x.CASE_NBR = c.CASE_NBR-----------------------------------------------------------------1. Is there anything that can be done to speed up the query?2. This part of the query: ... AND ((cl.STATUS_CD) = 0 ... where the 0is actually a variable passed in via a VB application. 0 would be newcases, and normally return around 4000 - 5000 records.3. The SQL server, Web Server, and users, are all in different states.4. The time to return this query where cl.STATUS_CD = 0 is about 7 -12 seconds.5. Is this a reasonable time for this query? What can be done toincrease the time?6. The SQL server is indexed on T_CASE_LST.STATUS_CD andT_INCIDENT_LST.CASE_NBR, but not on any field from T_GNRL_REFN sinceT_GNRL_REFN is only a general lookup table, and contains less than 50records.7. I've built the query as a stored procedure, and it works, though nomeasurable speed increase was obtained.8. I have not attempted building a view to aid this, as I don't seethat helping... or will it?9. Well: any ideas?10. I would gladly rewrite the SQL Query if it could return the samedata faster using another method.11. Is there a way to accomplish the joins involved with theT_GNRL_REFN in another manner to make it quicker?12. Is there a better way to add the values in T_INCIDENT_LST than:.... SUM(cast(TRANS_AMNT AS float)) AS TotalLoss ... ?13. I don't care if its pretty, I just need it faster.14. How can I get the summing of T_INCIDENT_LST.TRANS_AMNT without aderived table...? I know that using the derived table is slowing itdown some.**** Any Ideas ****David
View 3 Replies
View Related
Dec 27, 2000
hi all...
i want to store the output of an sql query as a text file.this is for my project i am doing in java.can u help me how to do this..it is very urgent
thanking u
View 2 Replies
View Related
Dec 29, 2006
Hello there,I just want to ask if storing data in dbase is much better than storing it in the file system? Because for one, i am currenlty developing my thesis which uploads a blob.doc file to a web server (currently i'm using the localhost of ASP.NET) then retrieves it from the local hostAlso i want to know if im right at this, the localhost of ASP.NET is the same as the one of a natural web server on the net? Because i'm just thinking of uploading and downloading the files from a web server. Although our thesis defense didn't require us to really upload it on the net, we were advised to use a localhost on our PC's. I'll be just using my local server Is it ok to just use a web server for storing files than a database?
View 6 Replies
View Related
Mar 2, 1999
HI
Im running a query using isql from the command line and outputting the results
to a file. Ideally I'd like to have the file in csv format so that it can be used by a client to import the results into their system.
I can run a query and output the results to a file but get i dont get the csv format I require. I can separate the columns using /s parameter but I also additional space within the column fields.
I know it is possible to create the csv format using the SQL query tool and ISQL_w but I was wondering if there was some parameter that I can call that will format the output like the format options in ISQL_w allows me to do.
Thanks in advance
View 2 Replies
View Related
Jul 30, 2001
Is there a way in SQL Server 2000 to output your query results to a file (example comma-delimited) in the SQL CODE ITSELF? I know you can check the option under Tools->Options->Results but I want to have it do it in the SQL code. Thanks Eric
View 1 Replies
View Related
Apr 3, 2001
I have come across manual queries being run daily and the results saved to a txt file on the network.
I basically want to set run these up a stored procedure and set up as a scheduled task to run daily.
Does anyone know if you can automatically save the results of a query/stored procedure to a text file on a network??
Many Thanks..
View 1 Replies
View Related
Oct 16, 2000
Hi,
I'm using SQL Server 7.0.
I have a query (select * from table1) and I'd like to have the results of this query sent to a text file instead of the results windows when I run it from Query Analyzer.
Any suggestions?
Thanks in advance,
Darrin Wilkinson
View 3 Replies
View Related
Sep 23, 1999
Hi,
In 6.5 you could run a query and then make a flat file with whatever format
you wanted.
I cannot find that option in 7.0. Can someone please tell me where it is????
Thanks,
Dianne Watson
View 5 Replies
View Related
Aug 20, 1999
I want to write a small program (maybe just a stored procedure) that will save the results and column names of a simple query to CSV file. This will then be copied onto disk and distributed to user who'll use Excel or a similar package to open the CSV file.
Any suggestions as to the best way to achieve this? I was considering using a view and then BCP but I have problems then when trying to open the CSV file in excel (or other spreadsheet package). Doesn't like the datatypes and so on.
Is there a simple way to do this? And is it possible to save the results in such a way that Excel will choose the right datatypes for the columns. (not convert varchar's like '000122' to numbers.)
View 1 Replies
View Related
Feb 28, 2013
How can I export my query results to PDF file?
View 3 Replies
View Related
May 9, 2007
i jave the following query
select *,substring(stafflog,15,11) as test into #t1 from dbo.Customers
where stafflog like '%armagh%'
go
select left(stafflog,4) as Staff,count(left(stafflog,4)) as Total from #t1
where cast(left(test,charindex(' ', test))as smalldatetime) = cast(convert(varchar(8),getdate()-1,1) as datetime)
group by left(stafflog,4)
go
select Title,Address1,Address2,Town,County,Postcode,TelephoneDay,TelephoneWork,TelephoneEvening,
MobileTelephoneNo,Contact,Mail,Telephone,Terms,StaffLog
from #t1
where cast(left(test,charindex(' ', test))as smalldatetime) = cast(convert(varchar(8),getdate()-1,1) as datetime)
go
drop table #t1
go
i need this query to be scheduled to run at a certain time every day and the results to be put in a text file.
is there an easy way to do this or what should i be looking at doing to get it to work
View 6 Replies
View Related
Sep 16, 2015
After running a query (from the Query Builder) in SQL Server 2008 sometimes I can right-click on the results pane and "Save results as CSV file", other times it's not an option. After running a query for 24 hours (several million record results) I can't seem to do anything with the results. I have my settings:
Options | Query results | SQL Server | Default Destination for Results
set to "Results to File" and a path entered, but it doesn't work. Is there wording I can add to the end of my SQL statement such as "TO FILE xxx.csv" or something?
View 4 Replies
View Related
Apr 1, 2008
is there a way to print query results from mgt studio into a file with headings and formatting? When I "save as a" file, export file, spreadsheet, the results dont have headings and semi decent formatting.
View 7 Replies
View Related
Apr 18, 2001
I need to manage the number of rows affected by a query.
If I run this script on my database:
SELECT COUNT(*)
FROM TABLE_NAME
WHERE TABLE_NAME.FLAG = 'I'
AND NOT EXISTS
(SELECT*
FROMERR_TABLE_NAME
WHERE ERR_TABLE_NAME.FIELD_1 = TABLE_NAME.FIELD_1
AND ERR_TABLE_NAME.FIELD_2 = TABLE_NAME.FIELD_2)
the query analyzer gives me back the number of lines affected:
-----------
4,00
(1 row(s) affected)
I need to manage this amount (4,00), so I applied this modification:
DECLARE @NUM_ROWS INT
SET @NUM_ROWS = (SELECT COUNT(*)
FROM TABLE_NAME
WHERE TABLE_NAME.FLAG = 'I'
AND NOT EXISTS
(SELECT*
FROMERR_TABLE_NAME
WHERE ERR_TABLE_NAME.FIELD_1 = TABLE_NAME.FIELD_1
AND ERR_TABLE_NAME.FIELD_2 = TABLE_NAME.FIELD_2)
)
but it doesn't work!! the SRV returns me this:
Server: Msg 107, Level 16, State 3, Line 3
The column prefix 'TABLE_NAME' does not match with a table name or alias name used in the query.
Do you know another way to store the number of rows affected?
View 2 Replies
View Related
Jul 26, 2001
SQL Server is new to me.
I am using SQL Server 7.0 Query analyzer. I need to send my query results to a flat file. My select statement is 'SELECT * from table_a'.
Can someone help?
View 1 Replies
View Related
Dec 12, 2000
Is there a way to save the results of a query (run in the Query Analyzer) to a text file?
View 4 Replies
View Related
Apr 7, 2006
Hello folks!
I have written a query that I need to save results of. Is there some statement that I can place at the top of my query (like we can do in Oracle through spool command) to specify the directoryfilename to save query results to? Eventually I need to schedule this to run daily via a DTS package.
Please let me know if you've come across this before?
Thanks so much!
-Parul
View 3 Replies
View Related
Apr 19, 2013
I found this topic from this link: Save MySQL query results into a text or CSV file | a Tech-Recipes Tutorial
I am try to create the text file from query results but it didn't work and got this error: "Incorrect syntax near the keyword 'INTO'.
SELECT sale, del
FROM order
INTO OUTFILE 'C:/tmp/orders.csv'
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '
'
View 10 Replies
View Related
Sep 14, 2007
In sql mgt studio, I can save the query results to a file. How would I do this in an SSIS package?
View 2 Replies
View Related
Feb 9, 2008
I have posted this on previous forums, but the below code is a VB.net CLR stored proc that will accepts two parameters, a SELECT.... FOR XML query and a file path, and saves the resulting xml result into a file specified by the parameter. After compiling the VB code into a .dll than the T-SQL code to import the assembly, create the CLR proc and test it is included underneath. Hope some will find this useful
Code Snippet
'Requires the Trial or Release version of Visual Studio .NET 2005 Professional (or greater).
Imports System
Imports System.Data
Imports System.Data.SqlClient
Imports System.Data.SqlTypes
Imports Microsoft.SqlServer.Server
Imports System.Xml
Partial Public Class outputxml
<Microsoft.SqlServer.Server.SqlProcedure()> _
Public Shared Sub outputxml(ByVal XmlData As SqlXml, ByVal Filename As SqlString)
'Create the variables to hold the values that are supplied by the parameters
'input by the stored procedure
Dim xmlDoc As New XmlDocument()
Dim output As SqlPipe = SqlContext.Pipe()
Try
'Load the result set into the XmlDoc Variable and then save the results in the
'path provided by the stored procedure. The values are provided by the
'input parameters of the stored procedure
xmlDoc.LoadXml(XmlData.Value)
xmlDoc.Save(Filename.Value)
Catch ex As Exception
'If an error occurs catch the message and pipe it back to SQL
output.Send(ex.Message.ToString)
End Try
End Sub
End Class
T-SQL CODE BEGINS HERE
/*Alter the database to set trustworthy on in order
to allow the ability to set the assembly to external
access*/
ALTER DATABASE AdventureWorks SET trustworthy ON
--Import the assembly into SQL
CREATE ASSEMBLY outputxml
from 'C:outputxml.dll'
WITH PERMISSION_SET = EXTERNAL_ACCESS
-- Create the proc from the imported dll
CREATE PROCEDURE output
@xmldata XML,
@filename nvarchar(1024)
AS
EXTERNAL NAME outputxml.[outputxml.outputxml]
.outputxml
-- Test managed stored procedure
DECLARE @output xml
SET @output = (SELECT ProductID, Name, ListPrice
FROM Production.Product Product
FOR XML AUTO, ROOT('Catalog'), TYPE)
EXEC dbo.outputxml @output, 'c:Output.xml'
GO
View 2 Replies
View Related
Jul 9, 2014
SQL 2012R2 Std. I have a sql statement to retrieve data in XML format. I have to use EXPLICIT because I have to use tag <CDATA>
I would like to store the XML returned into 1 column containing the XML tags.
For example:
#tmpXML
row 1: <DOC><NET_INFO net_ID=34><NOTES><![CDATA[description]]></NOTES></NET_INFO></DOC>
row 2 <DOC><NET_INFO net_ID=35><NOTES><![CDATA[description2]]></NOTES></NET_INFO></DOC>
select *
into #tmpXML
FROM (SELECT 1 AS Tag
[Code] ....
But I keep getting an error message:
The FOR XML clause is invalid in views, inline functions, derived tables, and subqueries when they contain a set operator. To work around, wrap the SELECT containing a set operator using derived table syntax and apply FOR XML on top of it.
I have tried a couple of other ways but continue to get an error message.
View 3 Replies
View Related
Feb 11, 2008
I'm new to sql stored procedures but I would like to store the results
of an sql statement in a variable such as:
SET @value = select max(price) from product
but this does not work, can someone tell me how I would go
about in storing the results in a variable.
@value is declared as int
Thanks in advance,
Sharp_At_C
View 1 Replies
View Related
Jul 31, 2015
I have the following code below where I need to have all of the query results output into a .csv file to use in a VBA macro. The issue I am running into is that the data is not deliminating correctly and my rows are being shifted incorrectly. Any better way of out putting the results into a .csv file with a common delimiter.
-- Declare the variables
DECLARE @CMD VARCHAR(4000),
@DelCMD VARCHAR(4000),
@HEADERCMD VARCHAR(4000),
@Combine VARCHAR(4000),
@Path VARCHAR(4000),
@COLUMNS VARCHAR(4000)
[Code] ...
Output from query (please post in a text editor. The line starting with (only ) should be on line 1 after 20 pks and is shifted to a new line.):
557898^1^9885E25^80082^9.0 CM GLASS FIBER PADS 20PKS
(only 12 pks in stock that will ship today)
^12.00000000^.00000000^18.32000000^219.84000000000^28.30000000
^339.60000000000^9.98000000^35.2650176678445^DR9146322^0^
[Code] ....
View 1 Replies
View Related
Jan 2, 2014
Is there a way to export query results to an excel fie and add that file as an attachment in the email? All this has to be done using SQL query and it needs to be automated. My coworker tried using Openrowset and BCP, but it is not working.
View 3 Replies
View Related
Jan 21, 2008
I am using vs 2005 and sqlserver 2005 to manage a large database. I need to create a series of text files based on the value of one of the columns. Is there an sql command that will pipe the output directly to a file?
Thanks,
Mike
View 1 Replies
View Related
Oct 23, 2007
I'd like the results of my query to stream right to a file rather than be processed row by row in an ADO reader loop. Is this possible?
View 5 Replies
View Related
Oct 12, 2007
Hi All,
I have two SQL queries that we would like to automate. Ideally we want them to both be scheduled to run and dump their results to a single Excel spreadsheet with two workbooks, one for each query
Is it possible to do this? If not, sending each query to a seperate XLS or CSV file would be OK
Here are the queries:
SELECT p21_view_unvouched_po_currency_report.unvouched_document_type, p21_view_unvouched_po_currency_report.date_created, p21_view_unvouched_po_currency_report.unvouched_document_no, p21_view_unvouched_po_currency_report.line_number, p21_view_unvouched_po_currency_report.po_no, p21_view_unvouched_po_currency_report.po_line_number, po_line.created_by, p21_view_unvouched_po_currency_report.item_id, p21_view_unvouched_po_currency_report.item_desc, p21_view_unvouched_po_currency_report.qty_received, p21_view_unvouched_po_currency_report.qty_vouched, p21_view_unvouched_po_currency_report.order_date, p21_view_unvouched_po_currency_report.location_id, p21_view_unvouched_po_currency_report.supplier_id, p21_view_unvouched_po_currency_report.supplier_name, p21_view_unvouched_po_currency_report.extended_cost_home
FROM P21.dbo.p21_view_unvouched_po_currency_report p21_view_unvouched_po_currency_report, P21.dbo.po_line po_line
WHERE po_line.po_no = p21_view_unvouched_po_currency_report.po_no AND po_line.line_no = p21_view_unvouched_po_currency_report.po_line_number
and
SELECT p21_view_unvouched_po_currency_report.unvouched_document_type, p21_view_unvouched_po_currency_report.date_created, p21_view_unvouched_po_currency_report.unvouched_document_no, p21_view_unvouched_po_currency_report.line_number, p21_view_unvouched_po_currency_report.po_no, contacts.last_name, p21_view_inventory_return_hdr.buyer_id, p21_view_unvouched_po_currency_report.po_line_number, p21_view_unvouched_po_currency_report.item_id, p21_view_unvouched_po_currency_report.item_desc, p21_view_unvouched_po_currency_report.qty_received, p21_view_unvouched_po_currency_report.qty_vouched, p21_view_unvouched_po_currency_report.order_date, p21_view_unvouched_po_currency_report.location_id, p21_view_unvouched_po_currency_report.supplier_id, p21_view_unvouched_po_currency_report.supplier_name, p21_view_unvouched_po_currency_report.extended_cost_home
FROM P21.dbo.contacts contacts, P21.dbo.p21_view_inventory_return_hdr p21_view_inventory_return_hdr, P21.dbo.p21_view_unvouched_po_currency_report p21_view_unvouched_po_currency_report
WHERE p21_view_inventory_return_hdr.return_number = p21_view_unvouched_po_currency_report.unvouched_document_no AND contacts.id = p21_view_inventory_return_hdr.buyer_id
View 4 Replies
View Related
Mar 28, 2008
All,
I'm trying to store the results of my store procedure in a temp table. when I try it, I got the error saying...
"Insert exec cannot be nested"
I suspsect this is because I have a Insert Exec statement inside my stored procedure...
Is there any way to come over this issue ?
syntax of my code...
create table #Temp1 (ID int)
insert into #Temp1
EXEC SP1
when I try to run the above code I get the error "Insert exec cannot be nested"
SP syntax :
Create Procedure SP1
as
Begin
create table #Temp2
statements.....
Insert into #temp2
exec SP<Name>
staments.. cont...
END
View 5 Replies
View Related
Sep 5, 2014
I've got a query that returns the data I need. I want to put the query in a stored procedure such that, when the SP runs I get a pipe delimited text file on disk. I don't really want to mess with SSIS, etc. Is there a Q&D way to do this?
View 1 Replies
View Related
Jul 20, 2005
Can someone demonstrate a SIMPLE way to do this that does not requireadditional functions or stored procedures to be created? Lets say Iwant to execute the following simple query - "select * from clients" -and save the results to a text file, we will assume c:
esults.txtHow can I do this in one step?
View 3 Replies
View Related
