Full Text With Fuzzy Matching
Jan 11, 2007
Andrew Worral writes "I am currenly working on a website that uses Full Text search to search the name of companies.
We are having trouble figuring out what tools are best suited for this with SQL 2005 Standard/Enterprise and how to implement them.
The first issue to address would be Misspelling of words. Such as Looking For "Davids Shoe Repare" and returning "David's Shoe Repair"
Besides the spelling in "Davids Shoe Repare" there is also the issue of the " ' " in David's which we have not come up with a good solution for yet. So a search for David will not returns "David's"
I have done a little looking into Fuzzy matching with Integration Services but I am not sure this is the right tool, nor am I sure of the overhead involved and any speed issues with this. Nor am I in any way overly familiar with Integration Services.
What would you suggest?
Thank you in advance!
-Andy"
View 7 Replies
ADVERTISEMENT
Aug 30, 2007
I'm not quite sure that this is possible but, I figured that I would check with you experts out there before trying a new approach. I've done quite a bit of research and have not seen anyone quite figure this out yet.
We have a SQL Server 2005 application that stores and indexes documents to the database as an image data type. I'm able to do full-text queries against the documents without any trouble. I begin to run into problems when trying to pattern match social security numbers and drivers licenses stored in a full-text index. I have a user defined function that I call which runs my regular expression that checks for hits of a ssn or license number in the index. I have no problem getting hits when the data sits in a column.
I do need to mention that I have no trouble when searching for a ssn with a fixed value and where I know the ssn (ex: 123-45-6789). I am actually trying to find the existence of the pattern of ###-##-#### (ex: ^d{3}-d{2}-d{4}$) anywhere in the index.
Any help would be very much appreciated.
View 1 Replies
View Related
Oct 3, 2007
A column in my database contains phrases such as "Extreme Golf: The Showdown" or "Welcome to Happy Land". I need to write a search engine so that users could type in phrases such as "Golf Extreme Showdown" or "Happy Land" and the correct, or closest matched results will be returned. I don't need variations of words, just phrase keyword match based search. I know I could do this by using multiple LIKE %% statements OR'd together, but this would be too performance intensive. So, I have heard I should use charindex somehow to achieve this in a stored procedeure. Does anyone have any clue how to solve this problem? Thanks!
View 7 Replies
View Related
Oct 11, 2006
Hi *,
does anyone know if MS supports some kind of breaking strategy within Fuzzy Lookup/Grouping?
Besides that, I'd like to perform a address cleansing operation on a CRM database. I don't have a reference table (Street, Zip, LastLine, etc.) for that. Where can I get an appropriate database? Anyone has some experience with this issue?
Thanks a lot.
S.
View 5 Replies
View Related
May 30, 2007
Will the fuzzy grouping task match a null value to an empty string (or spaces)? I've got 5 columns I'm matching on, and one of them may be null for certain rows but an empty string for others. Given the 4 other columns may match, will this difference stop similar columns being grouped together?
(Someone's modified my grouped data since it was deduped, which takes a while, and I'm hoping for a quick answer on this).
Thanks in advance.
Ben
View 3 Replies
View Related
Jul 31, 2015
I need a script which will tell me Table name and column name where a given text is matching.
Example: search string = "I want to search something"
Output: Table T1 , Column C3.
DB: Microsoft SQL Server 2005 / 2008
View 1 Replies
View Related
Sep 17, 2015
I have two tables in SQL Server--an expression table and a translate table.
DECLARE @EXPRTABLE TABLE
(
expression varchar(100)
)
INSERT INTO @EXPRTABLE
SELECT 'CALL("-","","TEST")'
[Code] ....
I am trying to (for lack of a better way to explain it), overlay the pattern onto the expression and extract the text where the "#" symbols are.
So,
CALL("-","","TEST") using pattern CALL("#","#","#") would return -,,TEST
SUB(2,67,TEST,4,) using pattern SUB(2,#,TEST,4,) would return 67
View 8 Replies
View Related
Aug 14, 2007
Dear Friends,
i think fuzzy lookup
COMPARES WHAT WE ARE MAPING THE COLUMNS WITH SPELLING (IT WILL REJECT ATLEAST 1 LETTER IS DIFFRENT IN ANY RECORD MAPPED COLUMN) EX: RAVI != REVI
what is fuzzy grouping ???? please explain
regards
koti
View 3 Replies
View Related
Sep 7, 2007
Hi, i'm trying to do a full text search on my site to add a weighting score to my results. I have the following database structure:
Documents: - DocumentID (int, PK) - Title (varchar) - Content (text) - CategoryID (int, FK)
Categories: - CategoryID (int, PK) - CategoryName (varchar)
I need to create a full text index which searches the Title, Content and CategoryName fields. I figured since i needed to search the CategoryName field i would create an indexed view. I tried to execute the following query:
CREATE VIEW vw_DocumentsWITH SCHEMABINDING ASSELECT dbo.Documents.DocumentID, dbo.Documents.Title, dbo.Documents.[Content], dbo.Documents.CategoryID, dbo.Categories.CategoryNameFROM dbo.Categories INNER JOIN dbo.Documents ON dbo.Categories.CategoryID = dbo.Documents.CategoryID
GOCREATE UNIQUE CLUSTERED INDEX vw_DocumentsIndexON vw_Documents(DocumentID)
But this gave me the error:
Cannot create index on view 'dbname.dbo.vw_Documents'. It contains text, ntext, image or xml columns.
I tried converting the Content to a varchar(max) within my view but it still didn't like.
Appreciate if someone can tell me how this can be done as surely what i'm trying to do is not ground breaking.
View 2 Replies
View Related
Sep 12, 2007
Hello everyone !
I want to perform Full Text Search with SQL Server 2000. My documents (.doc, .xls, .txt, .pdf) are stored in a SQL Server field which is binary (the type of the column is image).
I would like to know, how you can extract pieces of text from the documents.
Example:
I have a ASPX page with codebehind in C# making the search in a table in SQL server that is full text indexed.
I make a search looking for the word "peace", than SQL server will take care about the search and return it to me the rows that match with that. But also I'd like to extract the 50 characters before and after where sql server found the word "peace" to show in the result page.
Does anyone has any idea how to work around it ?
Best regards.
Yannick
View 5 Replies
View Related
Apr 21, 2015
I have a column in a table that has a type TEXT,when I pull the length of a row it returns 88222 but when I select from that column it dows not show all the text in the result set.
View 3 Replies
View Related
Mar 31, 2004
I am using the following plumbing code to search a database column for a keyword. I can't use full-test indexing so I came up w/ this work around. But It has many flaws so I'm looking for a better way. Thx in advance.
'Open sql connection
SqlConnection1.Open()
Dim datareader2 As SqlClient.SqlDataReader
datareader2 = cmdFindRowsWithKeyword.ExecuteReader
Dim strMsg As String
Dim intRowToFlag As Integer
Dim strRowsToGet As String
Dim strKeywordAsTyped As String
Dim strKeywordAllCaps As String
Dim strKeywordAllLower As String
Dim strKeywordFirstLetterCap As String
Dim FirstLetter As String
While datareader2.Read
intRowToFlag = datareader2(0).ToString
strMsg = datareader2(1).ToString
'Assign keyword as typed to variable
strKeywordAsTyped = txtSearchFor.Text
'Assign keyword as typed to variable then convert it to all uppercase
strKeywordAllCaps = txtSearchFor.Text
strKeywordAllCaps = strKeywordAllCaps.ToUpper
'Assign keyword as typed to variable then convert it to all lowercase
strKeywordAllLower = txtSearchFor.Text
strKeywordAllLower = strKeywordAllLower.ToLower
'Assign keyword as typed to variable then convert it so just the first letter is in uppercase
strKeywordFirstLetterCap = txtSearchFor.Text
FirstLetter = strKeywordFirstLetterCap.Chars(0)
FirstLetter = FirstLetter.ToUpper
strKeywordFirstLetterCap = strKeywordFirstLetterCap.Remove(0, 1)
strKeywordFirstLetterCap = strKeywordFirstLetterCap.Insert(0, FirstLetter)
'If the string contains the keyword as typed in all caps all lowercase or w/ the 1st letter in caps then flag that row.
If strMsg.IndexOf(strKeywordAsTyped) <> -1 Or strMsg.IndexOf(strKeywordAllCaps) <> -1 Or strMsg.IndexOf(strKeywordAllLower) <> -1 Or strMsg.IndexOf(strKeywordFirstLetterCap) <> -1 Then
cmdFlagRowsWithKeyword.Parameters("@recid").Value = intRowToFlag
SqlConnection2.Open()
Dim datareader3 As SqlClient.SqlDataReader
datareader3 = cmdFlagRowsWithKeyword.ExecuteReader
datareader3.Close()
SqlConnection2.Close()
End If
End While
datareader2.Close()
View 2 Replies
View Related
Oct 21, 2007
Can you not add a text column to a full text index?? If I change it to a nvarchar it works fine but if I change it to a text column it wont index. Anyone know how to fix this?
View 1 Replies
View Related
Mar 29, 2007
I have a text column which my users use extensively for like queries. The table has 1.3 Millon rows and has seen som eperformance issue ie it does a table scan and blocks other processes.
I was wondering if anyone how to handle this coulmn can I create indexes or better can text column support full text indexing.
Anyone who can shed any ideas as what might be the best possible solution will be great
AdAnup
View 1 Replies
View Related
Apr 15, 2008
I've been looking into ways to accomplish a fuzzy search and SSIS makes that possible if I want to do a bulk import or something like it. But what it I just want to look stuff up at any given time not haveing to run the package?
Is it possible to expose the fuzzy lookup outside of SSIS to for example t-sql?
Here's an example:
I want to lookup the music artist "Notorious BIG" but in the database it is "Notorious B.I.G." if I use the SSIS fuzzy lookup I basically get what I'm looking for. But how would I call this from a web application? So then I tried Full text search but this doesn't really work out as well.
Will I have to re-write the logic that the fuzzy lookup uses to enable it to work? i.e. using Full Text Indexes and FreeTextTable, ContainsTable, SoundEx and the like to somewhat even come close to what the Fuzzy Lookup has?
View 6 Replies
View Related
Feb 4, 2007
Hi All
I have a strange request that might not be possible based on the laws of relational databases but I thought I'd give it a try.
I have three tables which for simplicity I will call A, B and C. Table A contains my master records, Table B contains user details and the final table contains some extra data
In my initial search when joining A and B, I return 100 records. I then need to search in table C for these 100 records based on a criteria. the expected result should return all 100 rows for the ones that match and also the ones that do not match. The problem is that in Table C, not all the 100 IDs exist, so there will not be a corresponding record. Unfortunately, our users still want to see all 100 records in the output. Is this possible
As always any help or direction would be appreciated.
View 5 Replies
View Related
Oct 15, 2007
i am tring to create the full text catalog I am following the direction at http://technet.microsoft.com/en-us/library/ms189520.aspx but on the last command it gives me an error
'PK_problem' is not a valid index to enforce a full-text search key. A full-text search key must be a unique, non-nullable, single-column index which is not offline, is not defined on a non-deterministic or imprecise nonpersisted computed column, and has maximum size of 900 bytes. Choose another index for the full-text key.
also I tried the wizard in the managment express but it does not have a option when I right click on the table for Full-Text Index
what am I doing wrong???
View 11 Replies
View Related
Oct 18, 2007
I created a new catalog and a new index but when I start a search the grid come back empty I didnt insert any data into the table until after I created the catalog and index I dont think it has indexed the data yet. How do I force this or make it???? I have another example where I inserted data and then created the catalog and index and did a search and the proper rows came back but then I inserted new data and searched on a keyword that should have brought it up but nothing showed?
View 1 Replies
View Related
Jun 24, 2006
Hi,
I have a SQL statement that works great when I don't use a SQL Parameter, but when I do it just takes the @Searchfor as literal text "@SearchFor" instead of the string @SearchFor represents. Any ideas? Below is the two versions of the sql statements
sqlComm.Parameters.Add(new SqlParameter("@SearchFor", strSearchFor));
sqlComm.CommandText = "SELECT RANK, intID, chTitle, chDescription "
"FROM FREETEXTTABLE( tblItems, *, 'ISABOUT("+ strSearchFor +" WEIGHT(1.0))') a " +
"JOIN tblItems b on a.[KEY] = b.intID ORDER BY RANK DESC; ";
sqlComm.Parameters.Add(new SqlParameter("@SearchFor", strSearchFor));
sqlComm.CommandText = "SELECT RANK, intID, chTitle, chDescription " +
"FROM FREETEXTTABLE( tblItems, *, 'ISABOUT(@SearchFor WEIGHT(1.0))') a " +
"JOIN tblItems b on a.[KEY] = b.intID ORDER BY RANK DESC; ";
View 1 Replies
View Related
Jul 16, 2006
I have a Full-text search that is being performed on a variable (@Description) see part of querie below:
WHERE (CONTAINS([Description], @Description)
This search only seems to work when a text fo 3 or greater characters is used Ball, but not for "an" or "a". it also does not search on part of a word i.e. "Gard" of "Garden"
Two things:
1) How do I perform the CONTAINS search for part of a word or "a".
2) How do I perform a search that returns all values, when I leave the input feild blank it returns no records.
Many thanks in advance
View 5 Replies
View Related
Feb 15, 2007
hello
in Full Text Search
Are there method when add record in Field for properties "Full Text Index " , update catalogs ?
thanks
View 2 Replies
View Related
Apr 23, 2007
Hi I have a full text index on my product table. When I do a search for Record, it returns all values for Record and Records.Now If I do a search with a spelling mistake say Recod . it doen't return anything.How can I get the full text to return my query even if there is a spelling mistake ? Thanks My query:SELECT * From Product WHERE FREETEXT (description, @SearchString)
View 5 Replies
View Related
Sep 21, 2007
I have implemented Full text search in my web application.
I am using sql server 2005 database.
I used “contains “Keyword for full text search.
These are syntax as given below: CONTAINS ( { column | * } , '< contains_search_condition >' ) < contains_search_condition > ::= { < simple_term > | < prefix_term > | < generation_term > | < proximity_term > | < weighted_term > } | { ( < contains_search_condition > ) { AND | AND NOT | OR } < contains_search_condition > [ ...n ] }
My search gives correct results according to AND NOT and OR.
But, it is not working if I used AND.
Please give me solution.
Its very urgent for me……
View 6 Replies
View Related
Sep 30, 2007
full text indexing
Hi,
In SQL Server 2005, if I set full text indexing enables in column MyDesc and
1. use “Select * from MyTable where MyDesc LIKE ‘%abc%’� would this be using full text indexing? Or have to use Contains to get it be in use?
2. Once I create the full-text index, should I be setting it to populate periodically? Isn’t it populating itself?
View 2 Replies
View Related
Dec 10, 2007
hi all.i want to search, for example :"test string" in database : table have column(name) , i want to search all rows with column(name) is "test " or "string" or "test string"i don't want to use(full text search of sqlserver 2005 ) can i help me.thanks in advance
View 1 Replies
View Related
Dec 28, 2007
How can (is full-text indexed) property TRUE in column properties in moment create table with right-click on tables and select NEW TABLE?
View 1 Replies
View Related
Jan 4, 2008
how can install full text searching in SQL SERVER 2005
By mohsen
View 1 Replies
View Related
Jan 5, 2008
how can TRUE full text indexed property in column property in SQL server 2005?
by mohsen
View 1 Replies
View Related
Jan 6, 2008
how can enabled full-text searching in sql server 2005 on windows XP?
Thanks, mohsen
View 2 Replies
View Related
Jan 6, 2008
right-click do on database and selected properties,selected files page check box "use full-text indexing" is disable.
how can enable check box above for sql server 2005 on windows xp?
thanks,mohsen
View 2 Replies
View Related
Jan 10, 2008
do support full-text searching for sql 2005 express edition? thanks,mohsen
View 1 Replies
View Related
Jan 16, 2008
how can enabled full-text searching for table columns for sql server 2005 enterprise edition?
View 1 Replies
View Related
Mar 5, 2008
How might it be possible to have on the fly full-text indexing utilizing the FREETEXT and CONTAINS statements so that when I enter a piece of data into the database it is immediatly searchable using the full-text indexing features?
View 4 Replies
View Related
