I am fetching large amount of data from teradata to sql server using linked server. I am facing below query:
A transport-level error has occurred when receiving results from the server. (provider: TCP Provider, error: 0 - The semaphore timeout period has expired.)
I am fetching large amount of data from teradata to sql server using linked server. I am facing below query:
A transport-level error has occurred when receiving results from the server. (provider: TCP Provider, error: 0 - The semaphore timeout period has expired.)
In a Library Management database we have these tables
1) Document ( DocNo , Doc_type , permalink,inDate)
2)Title(id, DocNo,Main_Title, Other_Title)
3)Author(id , Author_Name , Author_Family,Type--Like:main author , translator ,....)
4)Publisher(id,DocNo , Name,Publisedate,address)
5)Subject(id,DocNo,Subject)
6)Description(id,DocNo,ISBN,description)--one document may have some ISBN,etc
In document table I have 500,000 records.
I want to search a word in these tables ,for example i want to search 'Computer' ,this word may be in subject or title or description and etc. How can I do this with best performance?
I have a large excel spreadsheet created by finance user that contains several decades worth of sales data.
Here is a small sample:
Guest Count
Unit ID1/2/2011 1/9/2011
3 0
7 0
8 0
90 0
151696 1202
222769 1914
232704 2110
250 0
282838 1882
331089 691
363581 3064
371469 1062
I need to get this data into an SQL table in the following form so I can use it to further manipulate the data and update several other tables. I am thinking that UNPIVOT or CROSS APPLY might be the way to go, but am not sure how to code it.
The desired output:
Unit IDDate Guest Count
31/2/2011 NULL
71/2/2011 NULL
81/2/2011 NULL
91/2/2011 0
151/2/2011 1696
and so on ......
The spreadsheet has 2900 columns and 3500 rows so performance is definitely a consideration as well.
Hi,
I have a small Problem.
Lets say i have 6 rows in a table (two column month and MonthCost )naming jan,feb,mar,apr,may.jun and all of them have some data. but while displaying in Ui i need to show all the months from Jan to december and july to december with values as something, say 0
so the values displayed in gridview sholud be like this
Month MontCost
Jan 1
Feb 1
Mar 1
.
.
.
Jul 0
Dec 0
. I tried it can be done using a temp table in db , but think of a scenaio where the i want to display for Number of times( 1 to 12 * 50). I don't want to hit performance and i am uisng sql server 2000.
Any hints or suggestions are welcomed.
Hi,
I am trying to fetch data from IBM DB2 to SQL Server 2005.
The problem I am facing is when I create the OLE DB Connection (I am using the "IBM DB2 UDB for iSeries IBMDA400 OLE DB Provider") and see the "Preview", I get "System.Byte[]" in a couple of columns for all the rows, instead of the actual data.
The datatype of the original field is "Byte Stream".
I have tried all options, but, failed. I believe there is something in the "Force Translate" property of the OLE DB Connection. Right now it is set to "65535". I am not sure if that needs to be changed.
I was earlier using a DTS package, where I used ODBC for connecting to the same database. In ODBC, there is a "Translation" tab where there is a check box labelled: "Convert binary text (CCSID 65535) to text".
When I check this box, I am able to see the data correctly.
But, now I have moved to SSIS and I am facing the same problem as I am not using the ODBC connection.
Please help.
Thanks and Regards,
B@ns
i have table below
CREATE TABLE [dbo].[DR_Test](
[source_item_id] [int] NOT NULL,
[source_line_no] [int] NULL,
[buyer_id] [int] NOT NULL,
[seller_member_id] [int] NULL,
[code]...
the table contains more than 80 million records so when i fetch the data using buyer_id & timezone its taking lot of more than 1 hours or so....& where buyer_id is not unique.how to fetch the data fast or need to change the structure of the table
Hi,
I created a linked server as follows:
EXEC sp_addlinkedserver 'OracleLinkedServer', 'Oracle', 'MSDAORA', 'fcstage'
EXEC sp_addlinkedsrvlogin 'OracleLinkedServer', false, 'SA', 'fc_stage', 'password'
Now I try firing a simple select statement
SELECT FINANCIAL_TRX_INFO_ID FROM
[OracleLinkedServer]..[FC_STAGE].[WFS_FINANCIAL_TRX_INFO]
WHERE SFS_BUSINESS_SEGMENT IS NOT NULL
But I get the following error:
OLE DB provider "MSDAORA" for linked server "OracleLinkedServer" returned message "ORA-01426: numeric overflow
".
Msg 7330, Level 16, State 2, Line 1
Cannot fetch a row from OLE DB provider "MSDAORA" for linked server "OracleLinkedServer".
This seems to be a generic error statement. Can anyone tell me where am I going wrong.
Thanks.
Hello,I have one application which is having written in asp.net & plain asp.I am having one button on asp page,when i will click on that button, then itwill execute one other asp page.And after the execution of that second asp page, I redirect it to someASPX page with some values.On the ASPX page, it will connect to the Database, and insert the values.Thus, sometime, the following error is occuring :"Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding."Main thing is that i am testing it on my local machine, then also it is giving me this error ...Insert into statement is also not much complicated :just I inserted three values in one table.Is this occuring due to the mixer of asp & aspx??Plz give me some proper solution.I want it in efficient manner, because this application will be accessed by number of users at the same time.Plz help me.Thanks,Sandy
View 3 Replies View RelatedI'm trying to convert old DTS packages over to SSIS and also create new SSIS packages that use teradata as a warehouse source for many different departments.
So far i've tried what I can find here on the forums:
using special syntax and wrapping the sql command in an alias/variable name
Methods i've tried:
using an ODBC connection string that works just fine in VB for pulling data. - test connection succeeds but when I select data reader source I get:
Error at Data Flow Task [DataReader Source[185]]: Cannot acquire a managed connection from the run-time connection manager
Using an OLE DB connection where I get the standard error many have gotten and I tried the listed use special syntax and wrapping the sql command in an alias/variable name from :
MSDN
Error at Data Flow Task [Source - Testing]: An OLE DB error has occurred. Error code: 0x80040E21. An OLE DB record is available. Source: "OLE DB Provider for Teradata" Hresult: 0x80040E21 Description: "Multiple-step OLE DB operation generated errors. Check each OLE DB status value, if available. No work was done.".
Currently i'm testing/trying this on a very simple view that produces 2 columns and 173 rows of string data.
I have not tried the .net provider and am reluctant to accept that as the only solution if it works as it would involve permissions issues and red tape here at the bank and is a less user-friendly solution for my dev team.
Any ideas?
Driver version on my machine and likely the other 5 that would need this to work:
3.03.00.01 12/17/2003
I am extracting data from an SQLServer database to load into Teradata using DTS. The performance is abysmal. The same data in a text file loads quickly via multiload. I can move the data to other DBMSs via DTS quickly as well. Is there some way for me to improve the elapsed time/performance while using DTS? If not, what is the best way to move data from SQLServer into Teradata?
View 1 Replies View RelatedI recently installed the Teradata .NET data provider and I am trying to use named parameters and multi-value parameters in my SQL. I am finding information on this topic hard to come by.
In order to get parameters to work do I need to write my own Teradata data extension from scratch, expand an existing data extension to work with Teradata, or can I simple edit my .config files to point to a generic extension wrapper already existing?
Thanks!
Does anyone have any experience setting up paramters when using a Teradata data source? I am specifically looking for how to set up dates and multi-select drop down boxes as parameters. Using a fixed query against Teradata using SSRS is straight forward and is working well. Parameters, not so much.
R
hi,
I would like to know that, I have three instances of the same database at three different servers and I am trying to fetch the data from the select query. "select * from table_name"
I would like to know, whether the order of rows fetched by this query will be different on different servers of sql server or the same order of rows will be fetched.
For me the output is coming different on each server database with he same query . Pls let me know, is there any default order by or it takes it randomly.
Thanks
Gaurav Gupta
Hi,
I'm currently developing an application using C# and SQL Server Express, in which one of the functionalities is to allow the user to browse through the existing products, wether by browsing one by one (next, previous, first, last) or performing a search (by product name, id, etc.). I'm currently using the default DataSource and TableAdapter aproach, but I found out that when I execute the TableAdapter.Fill() method, all the data in the table is fetched... what with 10,000 records it takes a bit to perform, specially on slower connections.
So my doubt is, what would be the best aproach to solve this problem? I'm considering loading first position, and then when the user presses the next button I'd fetch second position, etc, this using ROW_NUMBER(). I could even cache every other 10 records or something like that. Since it is possible to know any records row number (as long as the records are ordered) it would even work when the user performs a search and goes directly into row number 5000 for instance.
Would this aproach be a good practice, because that's all I want, to do things the right way? Thank you for your time
EDIT: I've noticed that when creating a SqlDataReader there is the option to provide a CommandBehavior, but I'm not quite undestanding how this could be used, would some tweaking in this area do the job?
We have a large OLAP database, about 2.5 TB spread out over 3 data files on three different drives, and recently someone ran a query that created a table that continued to grow until the data files filled the available disk space (about 3 TB total - 1 TB per drive).
Tonight I plan on running a full backup (it's in Simple mode) and running a ShrinkFile on all three files sequentially with TRUNCATEONLY just so it will remove the space after the last extent. Any way to tell ahead of time how much space this will recover?
Granted running a DB Shrink is one of those things you just don't do, but this is a one-time shot and unavoidable to get the file size back under control.
I am attempting to do a rather simple purge task on a very large table. This task will need to take place daily and delete records older than 6 months out of the database. On first pass this will delete well over 130 million rows. I thought the best way to handle this is create a proc and call the proc from a SQL Agent Job that runs nightly. Here is an example of the script:
CREATE PROCEDURE usp_Purge_WCFLogger
AS
SET NOCOUNT ON
EXEC sp_rename 'dbo.logs', 'logs_work'
GO
SELECT * INTO dbo.Logs_Backup FROM dbo.Logs_Work WHERE TIMESTAMP < DATEADD(month, -6, GETDATE())
[Code] .....
I have a table with about 466 Million rows. In this table there is a int column called WeeksToRetain as well as a EventDate column containing the date the row was inserted. I am trying to delete all the rows that that should be deleted according to the WeeksToRetain. For example, if the EventDate is 5/07/15 with a 1 in the WeeksToRetain column the row should be removed by 5/14/15. I am not sure what days SQL considers the beginning and end of the week. However the core issue I am having is the sheer mass of deletions I must do and log growth.
So I am trying to do the delete in batches. More specifically I want to load a temporary table with a million rows, then use the temporary table to load a sub temporary table with 100,000 rows and join this temporary table to the table I want to delete from looping through 10 times to get 1 million. The Logging.EvenLog table which is the table I'm trying to purge has a clustered index on EventDate (ASC). I would like to run this in a schedule job with enough time between executions for log backups to run.
DECLARE @i int
DECLARE @RowCount int
DECLARE @NextBatchDate datetime
CREATE TABLE #BatchProcess
(
EventDate datetime,
ApplicationID int,
[Code] .....
I have the following scenario:
SQL database on SQL 2012
Large Production table 15 Million record
The table has 3 years of data
New monthly data is being added every month.
A New Monthly data is being loaded, checked and finally approved after 6 or 7 iteration before approval.Because of this iteration the monthly data set is being added then deleted then added then deleted few times.Because the table is big this process takes time, any thoughts on how to make the delete insert process faster.Keep in mind I cannot do much because it is a production table and is being access by other users to do other analysis.
Delete is done based on trx_date which is a year/month combo, like 201508.
The table has monthly sales by customer aggregated.
The table structure is:
CREATE TABLE [dbo].[Sales](
[batch_key] [int] NOT NULL,
[Company_key] [int] NOT NULL,
[customer_key] [char](22) NOT NULL,
[Trx_Date] [int] NOT NULL,
[account] [nvarchar](35) NOT NULL,
[code].....
Recently I installed the .NET data provider for Teradata.
In a regular C# application I can add the namespace, "using
Teradata.Client.Provider;" to connect to this data provider
directly without going though ODBC or OLD DB so I know it works.
However, when I open Reporting Services (new project -> Business Intelligence
Projects) I do not see how to add the Teradata .NET provider. I only
see the same standard data types as before. I have searched and I do
not see how to add the Teradata .NET provider to the available "Type"
list when you click €œAdd New Data Source€? to your report.
Thanks!
SQL 2012
I have a source table in the staging database stg.fact and it needs to be merged into the warehouse table whs.Fact.
stg.fact is not a delta feed it is basically an intra-day refresh.
Both tables have a last updated date so its easy to see which have changed.
It will be new (insert) or changed (update) data that I am interested in, there are no deletions.
As this could be in the millions of rows that are inserts or updates then this needs to be efficient.
I expect whs.Fact to go to >150 million rows.
When I have done this before I started with T-SQL Merge statement and that was not performant once I got to this size.
My original option was to do this is SSIS with a lookup task that marks the inserts and updates and deal with them seperately. However I set up the lookup tranformation the reference data set will have a package variable in the SQL commnd. This does not seem possible with the lookup in 2012! Currently looking at Merge Join transformation and any clever basic T-SQL that could work as this will need to be fast, and thats where I think that T-SQL may be the better route.
Both tables will have >100,000,000 rows
Both tables have the last updated date
The Tables are in different databases but on the same SQL Instance
Each table holds 5 integer columns, one Varchar, one datatime
Last time I used Merge it was a wider table with lots of columns so don't know if this would be an option.
We have a table to 100M rows and up until now we were fine with an non clustered index a varchar(4000) because we never went above 900 bytes (yes it is a bad design).We have the need to support international character sets now so the column was updated to nvarchar(4000) and now we have data past the 900 byte limit.
The data is long, seems useless but is needed by the business and they need to be able to search "where bigcolumn like 'test%'". With an index, even with a huge amount of data, it was 'fast'. Now of course without an index it is unusable. The wildcard is always at the end of the search. I made a full text index on the column and basic queries such as: select * from ourtable where contains(bigcolumn, 'AReallyLongStringofTextHere') works fine unless there is a space in the data. We loose thousands of returned rows because of spaces in the data.
I have tried select * from ourtable where contains(bigcolumn, '"AReallyLongStringofTextHere that includes spaces"') but not all of the data is returned. I get 112 rows with the contains statement. The table scanning statement of "select * from ourtable where bigcolumn like 'AReallyLongStringofTextHere that includes spaces%' returns 1939 rows.I understand that a full text index is breaking the long string up since it contains spaces. Is there a way to retain the entire string as 1 index entry or is there a way to fix my query to return all of the rows?
hello i'm using visual studio 2005 and asp.net 2 i have an error in the code that fetching data from databse in grid here is the code :Dim con As SqlConnection con = New SqlConnection("Data Source=local;AttachDbFilename='D:New Folderhorushorus.mdf';Integrated Security=True") Dim cmd As New String("SELECT TOP (10) serial, name, gender, dateofbirth FROM(dbo.students)ORDER BY RAND(CONVERT(varbinary(4), NEWID()))") Dim cd As SqlDataAdapter cd = New SqlDataAdapter(cmd, con) Dim ds As New DataSet() cd.Fill(ds, "students") grid.DataSource = ds.Tables("students").DefaultView grid.DataBind()that code is selecting random records from the database and showing it in grid the error is in the cd.fill(ds,"students") line so can any one help me in that problem.thanks
View 14 Replies View Relatedhello, i'm using asp.net 2 with VB, i have a table in my db named 'exams' and i have 6 columns in this table one for question number and one for the question and 3 columns for 3 answers and a last column for the right answer, now in my page i want to show 7 questions and each question has 3 answers that i can choose between them i want the question appear in datatable and the answers in radiolist i have a little code for that but it shows an error it says "no row at position 1" so hope u guys can help : Dim con As New SqlConnection("Data Source=.SQLEXPRESS;AttachDbFilename=|DataDirectory|aspnetdb.mdf;Integrated Security=True;User Instance=True") Dim a As New ArrayList(8) Dim i As New Integer i = 0 Dim d As New DataSet() Dim cmd As New SqlDataAdapter("select top(7) question from exams", con) cmd.Fill(d, "exams") a.Add(d.Tables.Item(0, 1)) i = i + 1 Dim ta As New DataTable ta.Rows(i).Item(0).text = d.Tables(0).Rows(0)(i)and by the way this code is for the questions i still dont have the code for the answers that will appear in a radiolist hope u can help .thanks
View 3 Replies View RelatedHi All,
I am getting problem in fetching data from SQL using C++ program.
How to Fetch SQL Unicode characters?
I am using ODBC driver calls in C++ to fetch SQL data.
It is working fine with character data.
But for Unicode data it displays “????� instead of data.
Can anybody please help?
Regards
Ketaki
Hi i have 6 tables
1.location
id primary key
authorization string
count int
2.account
acc_id primary key
loc_id foreign key of location table
hsr string
3.Line
id primary key
intaralata string
interalat string
4.trunk
id primary key
intaralata string
interalat string
5.RCF
id primary key
intaralata string
interalat string
6.TSG
id primary key
intaralata string
interalat string
now i need to fetch the field authorization from location table by the following conditins
LOCATION.AUTHORIZATION using ACCOUNT.LOC_ID. If any (LINE/TRUNK/TSG/RCF.INTERLATA/INTRALATA= ’F’), and this value is blank, send LOCATION.AUTHORIZATION
can anyone help me out in this scenario. i need the sql query for fetching that fiels.
thanks in advanvce
Hi i am firing a query in SQL SERVER CE as given below
SELECT Ticker_Master.Ticker, Ticker_Master.Company_Name, Capital.DataSet_Date, Capital.Quick_Ratio, Dividend.Payout
FROM Ticker_Master INNER JOIN Capital ON Ticker_Master.Ticker = Capital.Ticker INNER JOIN Dividend ON Capital.Ticker = Dividend.Ticker AND Capital.DataSet_Date = Dividend.DataSet_Date AND Capital.Region = Dividend.Region
WHERE (Capital.Quick_Ratio = @PARAM1) AND (Capital.DataSet_Date BETWEEN @PARAM3 AND @PARAM4) AND (Capital.Region = @PARAM5) OR
(Capital.DataSet_Date BETWEEN @PARAM3 AND @PARAM4) AND (Capital.Region = @PARAM5) AND (Dividend.Payout = @PARAM2)
On execution is asks 4 the parameter values as follows :
@PARAM1
@PARAM3
@PARAM4
@PARAM5
@PARAM2
After values are given on execution it throws error as
Duplicated parameter names are not allowed. [Parameter Name = @PARAM3]
Can anybody tell me what's wrong ?????????????????
thanx in advance
Hi,
I am getting an error while fetching data from IBM DB2:
I am using an OLE DB Connection provider - "IBM DB2 UDB for iSeries IBMDA400 OLE DB Provider".
I am also setting the "Force Translate" value in "Data Link Properties" of the Connection to "37" because I am getting some BYTE_STREAM data which I need to translate.
I am using the OLE DB Source to extract data using a SQL Query. I can see the data correctly when I do a "Preview".
However, when I execute the task, I get the following error.
It will be great if someone can help me with this.
Error: 0xC0202009 at Customer Telephone, CUSTOMER_TELEPHONE Source [238]: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80040E00.
Error: 0xC0047038 at Customer Telephone, DTS.Pipeline: SSIS Error Code DTS_E_PRIMEOUTPUTFAILED. The PrimeOutput method on component "CUSTOMER_TELEPHONE Source" (238) returned error code 0xC0202009. The component returned a failure code when the pipeline engine called PrimeOutput(). The meaning of the failure code is defined by the component, but the error is fatal and the pipeline stopped executing. There may be error messages posted before this with more information about the failure.
Error: 0xC0047021 at Customer Telephone, DTS.Pipeline: SSIS Error Code DTS_E_THREADFAILED. Thread "SourceThread0" has exited with error code 0xC0047038. There may be error messages posted before this with more information on why the thread has exited.
Error: 0xC0047039 at Customer Telephone, DTS.Pipeline: SSIS Error Code DTS_E_THREADCANCELLED. Thread "WorkThread0" received a shutdown signal and is terminating. The user requested a shutdown, or an error in another thread is causing the pipeline to shutdown. There may be error messages posted before this with more information on why the thread was cancelled.
Error: 0xC0047021 at Customer Telephone, DTS.Pipeline: SSIS Error Code DTS_E_THREADFAILED. Thread "WorkThread0" has exited with error code 0xC0047039. There may be error messages posted before this with more information on why the thread has exited.
Thanks and Regards,
B@ns
Hello All,
I have to fetch data in an ssis package from a set of web services. what is the best way of doing this?
The web services are session based, this means that multiple calls are needed to complete one operation. like one to log on, second onwards to execute calls, and the last one to log out.
(there is some cookie management also required to logon successfully into the web services).
Should we write a custom task which will fetch the data for us? Or just write a C# component which is invoked from SSIS?
Regards,
Abhishek.
Our system runs a SQL Server 2012 DB, it has a table (table_a) which has over 10M records. Our system have to receive data file from previous system daily which contains approximate 3M updated or new records for table_a. My job is to update table_a with the new data.
The initial solution is:
1 Create a table (table_b) which structur is as the same as table_a
2 Use BCP to import updated records into table_b
3 Remove outdated data in table_a:
delete from table_a inner join table_b on table_a.key_fileds = table_b.key_fields
4 Append updated or new data into table_a:
insert into table_a select * from table_b
As the test result, this solution is very inefficient. Step 3 costs several hours, e.g. How can I improve it?
Hi there,
I'm using a query to fetch data from a table where one of the criteria is IN(...) clause for the key column of the table.Now the data being retrieved is ordered by the key column of the table even though I haven't specified any order by clause.
I want to know if there a way in which the data being fetched is in the order of my IN(...) clause.
Thanx
Aby
so async cursor population is supposed to create the cursor and return the cursor id quickly, while the server works on async populating the results. For a keyset-driven cursor, SQL Server stores the key sets in tempdb, which it then uses to fetch data for cursor results. Anyway, this works fine for smaller tables, but I'm finding for large result sets, the async cursor population is very slow and indeed seems to approximate synchronous time. The wait stat I get while it is running (supposedly asynchronously) is TRANSACTION_MUTEX.
Example:
--enable async cursor
exec dbo.sp_configure 'cursor threshold', 0; reconfigure;
declare @cursor int, @stmt nvarchar(max), @scrollopt int, @ccopt int, @rowcount int;
--example of giant result set
set @stmt = 'select * from sys.all_objects o1, sys.all_objects o1';
[code]...
Note that using the SQL "select * from sys.all_objects o1" is much faster than "select * from sys.all_objects o1, sys.all_objects o2". However, if cursor population is async, I'd expect the time to return a cursor id to be similar between the two.
S/W Technologies : C#.Net 2005, ASP.Net 2005, SQL Server 2005
Greetings everyone,
Heres my code for login verification, which is written in the login button click.
SqlCommand cmd = new SqlCommand("Select uid,pass from UserRegistration where uid='" + txtuname.Text + "' and pass='" + txtpass.Text + "'", con);
con.Open();
dr = cmd.ExecuteReader();
if (dr.HasRows) { <code......> }
Now, my problem is, the SQL Server 2005 is fetching rows without checking case. For e.g. if I enter a password as "MYSTERY" or "mystery" which is stored as "Mystery" in the database. The datareader shows positive in HasRows property. So, Can someone suggest me how to fetch case sensitive data from SQL SERVER 2005. Thanks.