Andrew Worral writes "I am currenly working on a website that uses Full Text search to search the name of companies.
We are having trouble figuring out what tools are best suited for this with SQL 2005 Standard/Enterprise and how to implement them.
The first issue to address would be Misspelling of words. Such as Looking For "Davids Shoe Repare" and returning "David's Shoe Repair"
Besides the spelling in "Davids Shoe Repare" there is also the issue of the " ' " in David's which we have not come up with a good solution for yet. So a search for David will not returns "David's"
I have done a little looking into Fuzzy matching with Integration Services but I am not sure this is the right tool, nor am I sure of the overhead involved and any speed issues with this. Nor am I in any way overly familiar with Integration Services.
I'm not quite sure that this is possible but, I figured that I would check with you experts out there before trying a new approach. I've done quite a bit of research and have not seen anyone quite figure this out yet.
We have a SQL Server 2005 application that stores and indexes documents to the database as an image data type. I'm able to do full-text queries against the documents without any trouble. I begin to run into problems when trying to pattern match social security numbers and drivers licenses stored in a full-text index. I have a user defined function that I call which runs my regular expression that checks for hits of a ssn or license number in the index. I have no problem getting hits when the data sits in a column.
I do need to mention that I have no trouble when searching for a ssn with a fixed value and where I know the ssn (ex: 123-45-6789). I am actually trying to find the existence of the pattern of ###-##-#### (ex: ^d{3}-d{2}-d{4}$) anywhere in the index. Any help would be very much appreciated.
A column in my database contains phrases such as "Extreme Golf: The Showdown" or "Welcome to Happy Land". I need to write a search engine so that users could type in phrases such as "Golf Extreme Showdown" or "Happy Land" and the correct, or closest matched results will be returned. I don't need variations of words, just phrase keyword match based search. I know I could do this by using multiple LIKE %% statements OR'd together, but this would be too performance intensive. So, I have heard I should use charindex somehow to achieve this in a stored procedeure. Does anyone have any clue how to solve this problem? Thanks!
does anyone know if MS supports some kind of breaking strategy within Fuzzy Lookup/Grouping?
Besides that, I'd like to perform a address cleansing operation on a CRM database. I don't have a reference table (Street, Zip, LastLine, etc.) for that. Where can I get an appropriate database? Anyone has some experience with this issue?
Will the fuzzy grouping task match a null value to an empty string (or spaces)? I've got 5 columns I'm matching on, and one of them may be null for certain rows but an empty string for others. Given the 4 other columns may match, will this difference stop similar columns being grouped together?
(Someone's modified my grouped data since it was deduped, which takes a while, and I'm hoping for a quick answer on this).
Hi, i'm trying to do a full text search on my site to add a weighting score to my results. I have the following database structure: Documents: - DocumentID (int, PK) - Title (varchar) - Content (text) - CategoryID (int, FK) Categories: - CategoryID (int, PK) - CategoryName (varchar) I need to create a full text index which searches the Title, Content and CategoryName fields. I figured since i needed to search the CategoryName field i would create an indexed view. I tried to execute the following query: CREATE VIEW vw_DocumentsWITH SCHEMABINDING ASSELECT dbo.Documents.DocumentID, dbo.Documents.Title, dbo.Documents.[Content], dbo.Documents.CategoryID, dbo.Categories.CategoryNameFROM dbo.Categories INNER JOIN dbo.Documents ON dbo.Categories.CategoryID = dbo.Documents.CategoryID GOCREATE UNIQUE CLUSTERED INDEX vw_DocumentsIndexON vw_Documents(DocumentID) But this gave me the error: Cannot create index on view 'dbname.dbo.vw_Documents'. It contains text, ntext, image or xml columns. I tried converting the Content to a varchar(max) within my view but it still didn't like. Appreciate if someone can tell me how this can be done as surely what i'm trying to do is not ground breaking.
Hello everyone ! I want to perform Full Text Search with SQL Server 2000. My documents (.doc, .xls, .txt, .pdf) are stored in a SQL Server field which is binary (the type of the column is image). I would like to know, how you can extract pieces of text from the documents. Example: I have a ASPX page with codebehind in C# making the search in a table in SQL server that is full text indexed. I make a search looking for the word "peace", than SQL server will take care about the search and return it to me the rows that match with that. But also I'd like to extract the 50 characters before and after where sql server found the word "peace" to show in the result page. Does anyone has any idea how to work around it ? Best regards. Yannick
I have a column in a table that has a type TEXT,when I pull the length of a row it returns 88222 but when I select from that column it dows not show all the text in the result set.
I am using the following plumbing code to search a database column for a keyword. I can't use full-test indexing so I came up w/ this work around. But It has many flaws so I'm looking for a better way. Thx in advance.
'Open sql connection SqlConnection1.Open()
Dim datareader2 As SqlClient.SqlDataReader datareader2 = cmdFindRowsWithKeyword.ExecuteReader Dim strMsg As String Dim intRowToFlag As Integer Dim strRowsToGet As String Dim strKeywordAsTyped As String Dim strKeywordAllCaps As String Dim strKeywordAllLower As String Dim strKeywordFirstLetterCap As String Dim FirstLetter As String
'Assign keyword as typed to variable strKeywordAsTyped = txtSearchFor.Text 'Assign keyword as typed to variable then convert it to all uppercase strKeywordAllCaps = txtSearchFor.Text strKeywordAllCaps = strKeywordAllCaps.ToUpper 'Assign keyword as typed to variable then convert it to all lowercase strKeywordAllLower = txtSearchFor.Text strKeywordAllLower = strKeywordAllLower.ToLower 'Assign keyword as typed to variable then convert it so just the first letter is in uppercase strKeywordFirstLetterCap = txtSearchFor.Text FirstLetter = strKeywordFirstLetterCap.Chars(0) FirstLetter = FirstLetter.ToUpper strKeywordFirstLetterCap = strKeywordFirstLetterCap.Remove(0, 1) strKeywordFirstLetterCap = strKeywordFirstLetterCap.Insert(0, FirstLetter)
'If the string contains the keyword as typed in all caps all lowercase or w/ the 1st letter in caps then flag that row. If strMsg.IndexOf(strKeywordAsTyped) <> -1 Or strMsg.IndexOf(strKeywordAllCaps) <> -1 Or strMsg.IndexOf(strKeywordAllLower) <> -1 Or strMsg.IndexOf(strKeywordFirstLetterCap) <> -1 Then
cmdFlagRowsWithKeyword.Parameters("@recid").Value = intRowToFlag SqlConnection2.Open() Dim datareader3 As SqlClient.SqlDataReader datareader3 = cmdFlagRowsWithKeyword.ExecuteReader datareader3.Close() SqlConnection2.Close()
Can you not add a text column to a full text index?? If I change it to a nvarchar it works fine but if I change it to a text column it wont index. Anyone know how to fix this?
I have a text column which my users use extensively for like queries. The table has 1.3 Millon rows and has seen som eperformance issue ie it does a table scan and blocks other processes.
I was wondering if anyone how to handle this coulmn can I create indexes or better can text column support full text indexing.
Anyone who can shed any ideas as what might be the best possible solution will be great
I've been looking into ways to accomplish a fuzzy search and SSIS makes that possible if I want to do a bulk import or something like it. But what it I just want to look stuff up at any given time not haveing to run the package?
Is it possible to expose the fuzzy lookup outside of SSIS to for example t-sql?
Here's an example: I want to lookup the music artist "Notorious BIG" but in the database it is "Notorious B.I.G." if I use the SSIS fuzzy lookup I basically get what I'm looking for. But how would I call this from a web application? So then I tried Full text search but this doesn't really work out as well.
Will I have to re-write the logic that the fuzzy lookup uses to enable it to work? i.e. using Full Text Indexes and FreeTextTable, ContainsTable, SoundEx and the like to somewhat even come close to what the Fuzzy Lookup has?
I have a strange request that might not be possible based on the laws of relational databases but I thought I'd give it a try.
I have three tables which for simplicity I will call A, B and C. Table A contains my master records, Table B contains user details and the final table contains some extra data
In my initial search when joining A and B, I return 100 records. I then need to search in table C for these 100 records based on a criteria. the expected result should return all 100 rows for the ones that match and also the ones that do not match. The problem is that in Table C, not all the 100 IDs exist, so there will not be a corresponding record. Unfortunately, our users still want to see all 100 records in the output. Is this possible
As always any help or direction would be appreciated.
i am tring to create the full text catalog I am following the direction at http://technet.microsoft.com/en-us/library/ms189520.aspx but on the last command it gives me an error
'PK_problem' is not a valid index to enforce a full-text search key. A full-text search key must be a unique, non-nullable, single-column index which is not offline, is not defined on a non-deterministic or imprecise nonpersisted computed column, and has maximum size of 900 bytes. Choose another index for the full-text key.
also I tried the wizard in the managment express but it does not have a option when I right click on the table for Full-Text Index
I created a new catalog and a new index but when I start a search the grid come back empty I didnt insert any data into the table until after I created the catalog and index I dont think it has indexed the data yet. How do I force this or make it???? I have another example where I inserted data and then created the catalog and index and did a search and the proper rows came back but then I inserted new data and searched on a keyword that should have brought it up but nothing showed?
Hi, I have a SQL statement that works great when I don't use a SQL Parameter, but when I do it just takes the @Searchfor as literal text "@SearchFor" instead of the string @SearchFor represents. Any ideas? Below is the two versions of the sql statements sqlComm.Parameters.Add(new SqlParameter("@SearchFor", strSearchFor)); sqlComm.CommandText = "SELECT RANK, intID, chTitle, chDescription " "FROM FREETEXTTABLE( tblItems, *, 'ISABOUT("+ strSearchFor +" WEIGHT(1.0))') a " + "JOIN tblItems b on a.[KEY] = b.intID ORDER BY RANK DESC; ";
sqlComm.Parameters.Add(new SqlParameter("@SearchFor", strSearchFor)); sqlComm.CommandText = "SELECT RANK, intID, chTitle, chDescription " + "FROM FREETEXTTABLE( tblItems, *, 'ISABOUT(@SearchFor WEIGHT(1.0))') a " + "JOIN tblItems b on a.[KEY] = b.intID ORDER BY RANK DESC; ";
I have a Full-text search that is being performed on a variable (@Description) see part of querie below: WHERE (CONTAINS([Description], @Description) This search only seems to work when a text fo 3 or greater characters is used Ball, but not for "an" or "a". it also does not search on part of a word i.e. "Gard" of "Garden" Two things: 1) How do I perform the CONTAINS search for part of a word or "a". 2) How do I perform a search that returns all values, when I leave the input feild blank it returns no records. Many thanks in advance
Hi I have a full text index on my product table. When I do a search for Record, it returns all values for Record and Records.Now If I do a search with a spelling mistake say Recod . it doen't return anything.How can I get the full text to return my query even if there is a spelling mistake ? Thanks My query:SELECT * From Product WHERE FREETEXT (description, @SearchString)
I have implemented Full text search in my web application. I am using sql server 2005 database. I used “contains “Keyword for full text search. These are syntax as given below: CONTAINS ( { column | * } , '< contains_search_condition >' ) < contains_search_condition > ::= { < simple_term > | < prefix_term > | < generation_term > | < proximity_term > | < weighted_term > } | { ( < contains_search_condition > ) { AND | AND NOT | OR } < contains_search_condition > [ ...n ] } My search gives correct results according to AND NOT and OR. But, it is not working if I used AND. Please give me solution. Its very urgent for me……
full text indexing Hi, In SQL Server 2005, if I set full text indexing enables in column MyDesc and 1. use “Select * from MyTable where MyDesc LIKE ‘%abc%’� would this be using full text indexing? Or have to use Contains to get it be in use? 2. Once I create the full-text index, should I be setting it to populate periodically? Isn’t it populating itself?
hi all.i want to search, for example :"test string" in database : table have column(name) , i want to search all rows with column(name) is "test " or "string" or "test string"i don't want to use(full text search of sqlserver 2005 ) can i help me.thanks in advance
right-click do on database and selected properties,selected files page check box "use full-text indexing" is disable. how can enable check box above for sql server 2005 on windows xp? thanks,mohsen
How might it be possible to have on the fly full-text indexing utilizing the FREETEXT and CONTAINS statements so that when I enter a piece of data into the database it is immediatly searchable using the full-text indexing features?
