SQL Equivalent For Excel Match + Lookup Formula
Nov 29, 2007
Hi there,
Not sure if this is the right thread to post this on but here goes:
I currently have an excel file, we will call it price for now. In this file I have details similar to those below:
100 200 300 400
100 $4 $5 $6 $7
200 $20 $30 $40 $50
300 $100 $200 $300 $400
400 $15 $15 $15 $15
At present in Excel, one uses the match and lookup function to determine the price of the length and height dimension. Example: 300 x 200 = $40
How can one perform this same task in SQL 2005? Is this even possible?
Your responses would be greatly appreciated.
Thanks,
chris_kzn
View 3 Replies
ADVERTISEMENT
Jul 24, 2007
I am doing a lookup that requires mapping 2 columns in the column mapping section. When I do this, I get the error "Row yielded no match during lookup" . The SQL that I captured in SQL profiler does find the record when I run it in Management Studio. I have already tried trimming everything to no avail.
Why is this happening?
I tried enabling memory restrictions but then I my package hangs and I get a SQLDUMPER_ERRORLOG.log file with the following logged:
07/24/07 13:35:48, ERROR , SQLDUMPER_UNKNOWN_APP.EXE, AdjustTokenPrivileges () failed (00000514)
07/24/07 13:35:48, ACTION, SQLDUMPER_UNKNOWN_APP.EXE, Input parameters: 4 supplied
07/24/07 13:35:48, ACTION, SQLDUMPER_UNKNOWN_APP.EXE, ProcessID = 5952
07/24/07 13:35:48, ACTION, SQLDUMPER_UNKNOWN_APP.EXE, ThreadId = 0
07/24/07 13:35:48, ACTION, SQLDUMPER_UNKNOWN_APP.EXE, Flags = 0x0
07/24/07 13:35:48, ACTION, SQLDUMPER_UNKNOWN_APP.EXE, MiniDumpFlags = 0x0
07/24/07 13:35:48, ACTION, SQLDUMPER_UNKNOWN_APP.EXE, SqlInfoPtr = 0x0100C5D0
07/24/07 13:35:48, ACTION, SQLDUMPER_UNKNOWN_APP.EXE, DumpDir = <NULL>
07/24/07 13:35:48, ACTION, SQLDUMPER_UNKNOWN_APP.EXE, ExceptionRecordPtr = 0x00000000
07/24/07 13:35:48, ACTION, SQLDUMPER_UNKNOWN_APP.EXE, ContextPtr = 0x00000000
07/24/07 13:35:48, ACTION, SQLDUMPER_UNKNOWN_APP.EXE, ExtraFile = <NULL>
07/24/07 13:35:48, ACTION, SQLDUMPER_UNKNOWN_APP.EXE, InstanceName = <NULL>
07/24/07 13:35:48, ACTION, SQLDUMPER_UNKNOWN_APP.EXE, ServiceName = <NULL>
07/24/07 13:35:48, ACTION, SQLDUMPER_UNKNOWN_APP.EXE, Callback type 11 not used
07/24/07 13:35:48, ACTION, SQLDUMPER_UNKNOWN_APP.EXE, Callback type 15 not used
07/24/07 13:35:49, ACTION, SQLDUMPER_UNKNOWN_APP.EXE, Callback type 7 not used
07/24/07 13:35:49, ACTION, SQLDUMPER_UNKNOWN_APP.EXE, MiniDump completed: C:Program FilesMicrosoft SQL Server90SharedErrorDumpsSQLDmpr0033.mdmp
07/24/07 13:35:49, ACTION, DtsDebugHost.exe, Watson Invoke: No
Why am I getting this error with "Enable Memory Restriction"?
View 12 Replies
View Related
Sep 1, 2015
In SQL reporting, How do I add the formula in the Formula bar?All the data is coming from a sproc.
View 3 Replies
View Related
Jul 23, 2005
I've got a data set on which I'm trying to perform the same as excel'svlookup formula using SQL, but do not know how. Can anyone help?If you picture me having two tables Table_A and Table_B and they looklike this;Table_AName SportJohn FootballJohn RugbySteve CyclingSteve RunningTable_BName AgeJohn 23Steve 24Peter 25How do I extract All from Table_B and one Sport from Table_AI.E I want to end up with something like this...Name Age SportJohn 23 FootballSteve 24 CyclingPeter 25 NULLThanks in advance,Ciarán
View 2 Replies
View Related
Sep 29, 2006
Hi all,
I don't understand what's happening here.
I have a Conditional Split with 3 outputs. On the first output I have a lookup, when I execute the package I have 56 rows going through the Conditional Split, all rows are then going to the 2nd and 3rd output but the lookup on the first output generates an error "Row yielded no match during lookup".
I don't understand why the lookup is generating an error while there is no row going through it.
Any idea ?
Sébastien.
View 6 Replies
View Related
Nov 15, 2007
Could someone from the dev team please share as much details as you're comfortable with regarding the formula or logic used to calculate the confidence for a fuzzy lookup match?
Particularly I'm trying to understand how the confidence drops when there are several fuzzy matches for one input row.
With 1 row in the lookup table, that fuzzy match shows 98% confidence. If there's a duplicate row in the lookup table, then fuzzy lookup returns two matches, both with 46% confidence. If there's a triplicate row in the lookup table, then fuzzy lookup returns three matches, all with a 30% confidence. Makes sense because confidence tells you how confident it is that this particular match is the best match.
Now take a different match which is lower quality. Single row = 77% confidence, duplicate row = 43% confidence, triplicate row = 30% confidence. So you see that confidence doesn't shrink consistently. That's why I would like to know the formula.
View 1 Replies
View Related
Sep 8, 2006
In SSIS. I am having trouble exporting records
that don't match from a lookup transformation. I get the following
error:
Row yielded no match during lookup.
I would really like to have a list of all records that did not match so
that I could send an email of those missing rows
Please give me solution with example
Thanks
View 9 Replies
View Related
Sep 14, 2006
I have configured a lookup transformation to 'redirect error' all no-matched rows to a text file using the flat file destination.
Now I want to send the same text file as an email.I Know email can be send using the send email task but i need to know where to place send email task and how to check whether flat file contains the error data.
Can we use the send email task on eventhandler and invoke the same in case of such error "row yielded no match during lookup" so that we can send the such non matching rows as an email.
Or else any other way to send an email after generating the text file ocntaining the non matching rows.
Please suggest using steps or example
View 4 Replies
View Related
May 29, 2007
Hello,
I have a peculiar problem in my project. My project design is like this
The number in (...) are count of records.
File feed (1000)
|
|
Fuzzy Lookup
against Table2
|
|
Split Fz Lookup results
(_Similarity >= 0.60 && _Confidence >= 0.85)
| |
| |
| Write matches to Table1 (250)
|
Fuzzy Group
Remaining rows (750)
|
|
Split Fz Group results
| |
| |
Write Canonicals Write Dupes
to Table2 to Table1
(300) (450)
This is basically a customer de-dupification project.
The Table2 has the canonicals and Table1 has the dupes (of the canonicals).
I already have some data in these tables and the new data is matched against the existing data
in these tables and classified as new customers and duplicate customers.
In the above process one could notice that the rows identified as dupes of already exsting canonicals
by the Fuzzy Lookup task are written into the dupes table (Table1) and will not be processed further down
the line in the project.
But in my case I see that those matches identified by Fuzzy lookup are further being included in the
Fuzzy Grouping also.
When I run this in debug mode in BIDS, it shows the correct numbers as I have depicted in the
illustration above. But, after execution, when I query the tables it shows that all 1000 rows
went through Fuzzy Grouping.
Any thoughts?
Btw, is there anyway to upload attachments to the postings here?
View 1 Replies
View Related
Apr 10, 2008
I have a fuzzy lookup task that compares a source list of contacts to a reference list of contacts with the default settings. I did some testing by adding seed data that I knew would produce somewhat high similarity hits. All of the seeded contacts but one came back with the expected high sim values. When I looked for the one that didn't, I noticed another match had come up but it had a very low similarity of .17. I then did some research and discovered the reason was the MaxOutputMatchesPerInput setting which was set to 1. I then set it to 3 and reran the package and sure enough my seeded contact that was missing before now showed up. I thought the best match would show up if the MaxOutputMatches was set to 1? That is not the case in my testing.
For example, Donna Mizeman was in the reference list. I added Don Miseman to the source list to seed it. The only match that came back was something like Dieman Abdul .... So the initial match had a similarity of .17 but when MaxOutputMatchesPerInput is set to 3 the best match (seeded) has a similarity of .72.
Anyone have an explanation for this?
-Mike
View 1 Replies
View Related
May 29, 2007
Hello,
For one of my SSIS projects that does a fuzzy lookup on a table, I opted to create an index and
to maintain the stored index. The index got created and subsequent project execution was able to
use that index.
Now I want to update certain rows in that table. When I run the update statement I get the following error.
How can I retain the index and still be able to update the table?
update location_stage set batchid = 'APR07N'
where batchid is null and eventid = '20070528020041';
Msg 6549, Level 16, State 1, Procedure sp_FuzzyLookupTableMaintenanceInvoke, Line 0
A .NET Framework error occurred during execution of user defined routine or aggregate 'sp_FuzzyLookupTableMaintenanceInvoke':
System.Data.SqlClient.SqlException: Transaction is not allowed to roll back inside a user defined routine, trigger or aggregate because the transaction is not started in that CLR level. Change application logic to enforce strict transaction nesting.
System.Data.SqlClient.SqlException:
at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)
at System.Data.SqlClient.SqlInternalConnectionSmi.ExecuteTransaction(TransactionRequest transactionRequest, String transactionName, IsolationLevel iso, SqlInternalTransaction internalTransaction)
at System.Data.SqlClient.SqlInternalTransaction.Rollback()
at System.Data.SqlClient.SqlTransaction.Rollback()
at Microsoft.SqlServer.Dts.TxBestMatch.TableMaintenance.TranWrap(DataCleaningOperation c)
. User transaction, if any, will be rolled back.
The statement has been terminated.
View 3 Replies
View Related
Mar 14, 2006
I am having problems with a lookup transformation. I have a row in my lookup table for blank ('') source data. If I test the join using SQL the match is made, but the Lookup transform doesn't consider it a match and sends it to error output. Is there a property that I don't have set correctly or something else I am forgetting?
View 5 Replies
View Related
Oct 8, 2007
Hello,
The illustration below is for a customer dedupification project.
The Source file, containing customer name and address records, is conditionally split based on 7 ranges of substring(city,1,2) to distribute the load across 7 different threads for parallelization. Each customer record in the source file
is looked up against a reference table named Location_Stage for its existence using the Fuzzy Lookup
transformation.
The reference table Location_Stage has around 10 miilion+ records. The source file would normally have around 1 million
records.
I am wondering :
- if it would be possible to partition the Match Index of the reference table (NOT the reference table) into
7 partitions based on 7 ranges of substring(city,1,2) and maintain these partitions on different drives?
- if it is possible to specify a particular partition to be used by a FzLkup transformation?
- if the partitioning approach will improve the performance of the Fuzzy Lookups?
Source File Feed
|
Split data into 7 groups based on substring(city,1,2)
|
------------------------------------------------------------------------------------------------------------------------------------------
| | | | | | |
UnionAll UnionAll UnionAll UnionAll UnionAll UnionAll UnionAll
| | | | | | |
FzLkup FzLkup FzLkup FzLkup FzLkup FzLkup FzLkup
| | | | | | |
Split Split Split Split Split Split Split
| | | | | | |
------------- -------------- -------------- -------------- -------------- -------------- --------------
| | | | | | | | | | | | | |
<- - - - - - - Write the Canonicals and Dupes from each of these splits into database - - - - - - - - ->
Please advice.
Thanks.
View 3 Replies
View Related
Nov 7, 2007
I have a very basic Lookup in my SSIS package that looks up against two columns and outputs a row to a table. Now currently if there is no exact match, it writes a null in my destination table. How do I simply drop all those rows that dont produce an exact match? I tried using the 'Ignore' error output, but with that it writes NULLS into my destination table. With the 'Redirect' it is looking for a place to redirect the error (NULL) rows, and I dont want to deal with the hassle or writing these NULL values to a file or table just to delete them afterwards. I just simply want to forget about all those rows that dont produce an exact hit and only fill in the destination table with those that do produce a hit. How can I drop these lookup rows that dont produce an exact match?
View 5 Replies
View Related
Jul 6, 2006
Hi all,
In BOL it says: "The Lookup transformation performs an equi-join between values in the transformation input and values in the reference dataset. Using an equi-join means that each row in the transformation input must match at least one row from the reference dataset. If there is no matching entry in the reference dataset, no join occurs and no values are returned from the reference dataset. This is an error, and the transformation fails, unless it is configured to ignore errors or redirect error rows to the error output. "
I have a lookup transformation which is supposed to find a match on two fields in the reference dataset (a table in my case) but strangely, when I execute my package and the reference table is empty the lookup still finds match for each row of my input dataset.
Does anyone have an idea why? I could'nt find anything about that in BOL.
Sébastien.
View 4 Replies
View Related
Oct 9, 2006
The Lookup Transformation fails to match this datatype when full caching is enabled. When partial caching is activated (Edit > Advanced, Enable Memory Restrictions > Enable Caching) the lookup works.
This appears to be a bug to me.
View 4 Replies
View Related
Jan 7, 2008
I have two colums with two different date ranges
1/2/08 11:41 AM
1/4/08
12/27/07 9:38 AM
12/27/07
12/27/07 12:55 PM
1/2/08
1/4/08 11:20 AM
1/4/08
In the third column I'm trying to stated if the dates are different but when i change the first column to show just the date and not the time it still says they are different. How can i round or change the first column to just reflect the date and not the time so i can compare it to the second column.
View 4 Replies
View Related
Jul 10, 2007
I have a data list that will grow over time. The values are listed vertically in a column; most recent value at the bottom. I am trying to figure out how to setup a formula to figure out the standard deviation on the most recent 30 values automatically. For instance if the column contains 30 values and I add the 31st value, I'd like to have to have the standard deviation displayed in a cell and automatically shift from calculating on values 1-30 to values 2-31. Is this possible?
Jeff.
View 1 Replies
View Related
Feb 12, 2008
Is there a property or a format type that I can use on a field to enter an excel formula and have it actually be a formula once it is saved to excel out of reporting services.
This is a 'what if' report. That the users want to fiddle with after it is generated.
I have item number, cost, sell price, and discount from the database. I calculate the gross profit and the percent from those fields. They want to take this report, save it to excel and then have the gross profit change when they enter a new discount amount.
I have tried to just enter the fields like they would look in excel but the report just shows A3 - A2. If I use the = before it will not run and says A3 is not defined. I have tried quotes, parenthesis, square brackets any thing that looked remotely possible in properties menu to have the values save as a formula.
I don't think it is possible but would like a second opinion.
View 1 Replies
View Related
Jun 22, 2009
In SQL server Reporting service we need to export excel formula for summing column values. scenario : After generating report we are exporting report to excel file using report viewer.when user will modify a column value we need to calculate(update) automatically sum of the column values.Basically we are setting excel formula.
View 3 Replies
View Related
Jul 1, 2015
is it possible to replicate this in SSRS I wonder??I have included the code of the fields used and a snapshot of some data, and also how the Pivot looks in Excel.
SELECT
TARNSubmissionID,
ISSBand,
BPTLevelAchieved,
FinancialYearOfDischargeOrDeath,
FinancialQuarterOfDischargeOrDeath,
FinancialMonthOfDischargeOrDeath,
CalendarMonthNameOfDischargeOrDeath,
[code]...
View 4 Replies
View Related
Dec 14, 2007
Hi,
I am trying to fetch rows from Excel file and upload them into a SQL Server 2005 table.
I need to perform a couple of lookup transformations on the data before inserting only those rows that do not exist in the database.
I have redirected the error output of the lookup transformation to an OLE DB Destination.
However, when I execute the package, even if the rows exist in the DB, they are getting inserted again.
The lookup is failing when it should not!
Any help available?
Thanks in advance.
Regards,
B@ns
View 14 Replies
View Related
Aug 27, 2015
i want to use lookup transformation using Excel as a source.i am having two excel files .
file1 one of the column contains 'Andhrapradesh'
file2 one of the column contains 'ap'
here want to match these using lookup.
View 5 Replies
View Related
Oct 31, 2007
We did some "at scale" fuzzy lookup tests today and were rather disappointed with the performance. I'm wanting to know your experience so I can set my performance expectations appropriately.
We were doing a fuzzy lookup against a lookup table with 25 million rows. Each row has 11 columns used in the fuzzy lookup, each between 10-100 chars. We set CopyReferenceTable=0 and MatchIndexOptions=GenerateAndPersistNewIndex and WarmCaches=true. It took about 60 minutes to build that index table, during which, dtexec got up to 4.5GB memory usage. (Is there a way to tell what % of the index table got cached in memory? Memory kept rising as each "Finished building X% of fuzzy index" progress event scrolled by all the way up to 100% progress when it peaked at 4.5GB.) The MaxMemoryUsage setting we left blank so it would use as much as possible on this 64-bit box with 16GB of memory (but only about 4GB was available for SSIS).
After it got done building the index table, it started flowing data through the pipeline. We saw the first buffer of ~9,000 rows get passed from the source to the fuzzy lookup transform. Six hours later it had not finished doing the fuzzy lookup on that first buffer!!! Running profiler showed us it was firing off lots of singelton SQL queries doing lookups as expected. So it was making progress, just very, very slowly.
We had set MinSimilarity=0.45 and Exhaustive=False. Those seemed to be reasonable settings for smaller datasets.
Does that performance seem inline with expectations? Any thoughts to improve performance?
View 4 Replies
View Related
Sep 26, 2007
I'm working with an existing package that uses the fuzzy lookup transform. The package is currently working; however, I need to add some columns to the lookup columns from the reference table that is being used.
It seems that I am hitting a memory threshold of some sort, as when I add 3 or 4 columns, the package works, but when I add 5 columns, the fuzzy lookup transform fails pre-execute:
Pre-Execute
Taking a snapshot of the reference table
Taking a snapshot of the reference table
Building Fuzzy Match Index
component "Fuzzy Lookup Existing Member" (8351) failed the pre-execute phase and returned error code 0x8007007A.
These errors occur regardless of what columns I am attempting to add to the lookup list.
I have tried setting the MaxMemoryUsage custom property of the transform to 0, and to explicit values that should be much more than enough to hold the fuzzy match index (the reference table is only about 3000 rows, and the entire table is stored in less than 2MB of disk space.
Any ideas on what else could be causing this?
View 4 Replies
View Related
Sep 23, 2015
Say I want to lookup a value in another dataset, but there is a grouping that requires you to know what the values for each level is in order to get to the correct detail record.  Can you still use the lookup function with more than one field to compare against? So for example
Department
\___SalesPerson
    \___Measure
I want to be able to add a new row at the Measure level, but lookup each field from another dataset. In order to do that I will need the Department AND SalesPerson values to do the lookup, but I dont think the Lookup function will let us do that will.
View 2 Replies
View Related
Jun 27, 2007
Hi All,
Actually this is in regard to SCD Type 2 Dimension, Scenario is like that I am moving Fact table from some old source and I have dimensionA description value in fact which I want to replace with appropriate id from Dimension Table and that Dimension table is SCD Type 2 based on StartDate and EndDate and Fact Table doesn't contains direct date value rather there is timeId in Fact so to update the value in Fact table I have to Join Time Dimension table and other Dimension Table to replace fact Description with proper Id.
Lets assume DimensionA Structure
id
Description
StartDate
EndDate
Fact Table
id
measure1
measure2
TimeId
Description
Time Dimension
TimeId
Date
Day
Hour ...
View 1 Replies
View Related
Oct 4, 2007
I am designing a ssis package,This is intends to mine text data(Data extracted from websites).
Term lookup/Term extraction has been used as tools for mining.
I have lookup terms defined with me for reference table,but the main problem lie in extracting the nearby text/number/charcters to these lookup terms during mining.
For example :
I found noun "Email" 200 (frequency score) times in my text,Now I want to extract nearby email address(this is also true for PhoneNumber,Address attributes also).so how can I achieve this with SSIS.
If u have some idea/suggestion to carry out this challenge with or without Term Extraction/Term Lookup,plz do write here.
View 1 Replies
View Related
Jan 18, 2008
Having a hard time writing a formula in RS...
Trying to say if the funding date is equal to today or before the beginning of the month then sum the loan amount.
Looking for something like this SUM(IF(Funding Date,>=Date(),Loan Amount))
Know it not right but someone please help!
View 1 Replies
View Related
Jan 26, 2008
I am create a database and want to store some value automatically in some field i.e say i have 3 column 1) salary(int),2)tax(int) and 3rd field total salary(float) . i want to automatically fill total salary field as Column1-Column2 while i daont have any Idea How to do that?
please help me?
any idea or tutorial..example anything...
View 3 Replies
View Related
Nov 17, 2007
hi,my users can make posts in my web application, i mean they fill a form and the information they filled will be saved in sql server 2000 and can be shown in web application,now i want to give each post an Id and save it in the database, how can i do that? does sql server have the abilities or i should do sth in my c# application
thanx
View 1 Replies
View Related
Oct 2, 2005
i wonder what is the best approach to use !!!i have creditLimit column in Customer Table the default value will be 500 and this limit to allow users to send sms from my website ..... every month they will be allowed to send 500 sms referring to the credit limit column.... now !!if the user sent today some sms and after few days sent another 20 and after one week he sent 150 SMS as a total so that means he has only 350 SMS as credit to use this month!!!so what is the best approach to implement this solution ? shall i have another field in the table called 'CreditUsed' and that will be updated each time the user will send SMS and this value will be compared with the credit limit ORi use the formula for the column to calculate the credit left and do the maths !!!what is the best approach as you think ???thnaks for reading this question !!
View 7 Replies
View Related
Mar 30, 2006
hi there.
I have asked this question before in a different section of the forum without much reply, and its probably because i was asking the wrong people.
I need to calculate a qouta for an election.
SqlCommand SqlCmd1 = new SqlCommand("SELECT count(vote)FROM PRTest", SqlCon1);
int quota = (count(Vote) + 1) / ((11) + 1);
My problem is this: how to i get the count(vote) value from the Sql Statement to the formula.
View 2 Replies
View Related
