I need some suggestions on validating a string of text based on some business rules using T-SQL. I have a string similar to the following:
This is some text (1.00); this is some more (there's more text here) text (2.00); this is yet more text (1.10)
The above example illustrates a valid string. You'll notice the multiple sets of parens The parens that contain numbers must be followed by a semi-colon except for the last set of parens. Furthermore, any parens followed by a semi-colon must contain only numbers and not text. I can easily identify the positions in the string by using PATINDEX.
Get the closing paren not followed by a semi-colon, ignoring the last closing paren PATINDEX('%)[^;]', @string)
Get the closing paren followed by a semi-colon, ignoring the last closing paren PATINDEX('%);%', @string)
My question is, is there a way to quickly validate the data between the parens without visiting each character. This is SQL 2000 so using CLR regular expressions is out.
I need to make sure that a string contains at least one number, at least one letter and has no repeating chars. Anyone done something similar? What's the best way to do it? Thanks.
Is there a way in TSQL to validate a string to ensure it has only numbers or letters (no special characters like @,%, etc)?
I want to write a function that validates a number that should only contain letters and numbers. The function should return 1 if valid 0 if not. If it contains anything other than 0-9 or A-Z or a-z then it should return 0.
Any ideas on how to do this in TSQL? I could do this in C# easily with regular expressions, but I don't want to use SQLCLR or any other external resource like that...just straight TSQL.
I am taking a five-digit zipcode from a database where it is stored as an int, and I would like to store it as a char(5).
I set my OLE DB source to pull from a proc as:
Code Snippet
SELECT
cust.CustomerID as CustomerID, case
when cust.Zip > 99999 then Null when cust.Zip < 0 then Null else REPLICATE('0',5 - LEN(cast(cust.Zip as varchar(5))))
+ cast(cust.Zip as varchar(5)) end as CustomerZip
However, when I do that, SSIS sees the external column as a string of length 8000, and insists on giving me a truncation warning on validation.
I also tried bringing the Zip field in as an int and doing a data conversion in the SSIS data stream. This worked great until I added a Derived Column transform to do the REPLICATE function as above to add leading zeros to the zip code. Once I did that, SSIS decided that there was a chance of truncation again and started with the truncation warnings.
BTW - I don't think the TRIM() function is really necessary in the above, but I tried it out of desperation and it made no difference.
This is really starting to drive me nuts. Does anyone have any thoughts on how to convert an integer zipcode to a char(5) string with leading zeros without incurring these validation warnings?
I am currently in the process of migrating data from Sybase to Sql server and would like to know how to test the data migrated.
As of now, we took one table data from both source and destination and compared it in Excel to check if the data migrated looks good (note, we used SSIS to migrate data). However, I would like to check if there are any other best & easy ways to apprach data validation post migration.
While run time these values are lets suppose @SSN = '999-000-000' & @State='ABC'
Now the Result is displayed with the state data Like 'AB' only.
Output: 1 999-000-000 AB
instead it should give system generated error.
Here I have 2 Questions: 1. Why it is taking 1st 2 Charecters? 2. Why it does not have any system generated for length?
I can do validation with Length function for these 2 variables however if have 100 variables then it should not feasible case. So, what is the reason behind?Â
Hi guys I am fairly new to t-sql. I am sure there are SPs or scripts that I can use to create a procedure that will do the data validation in the staging table...
Client send us data and often some of the records have bad values... what i have to create is a process that will check for those values and update a flag in the staging table for each column if the data is not valid....
Please help me out if you have something that can be used for this.
Being relatively new to SSIS, I'm looking for advice, or a best practice, regarding data validation before extracting the data for a transformation.
One of my project's require that certain data be validated in staging tables before it is loaded. The validations include checking for null values, verifying that a field is populated with apropriate values etc... The entire batch of data (good records and bad records) may be rejected depending on the validations.
I have a couple of different thoughts on how this could be handled...
Run a series of validation queries on the data before executing an SSIS package Run some kind of validation transformation (does one exist or should I write a custom transformation?) Place contraints on the target tables so that bad records error out on the load Something else... I could be missing the completely obvious
#3 doesn't seem to viable as the entire load may be rejected if some of the data is bad...
Hello all...I am trying to validate that the new work date that is being imported from the text file does not exist in the table. So in other words I do not want duplicate data. This is what I haveSqlDataReader dr = new SqlDataReader(); SqlParameter sp = new SqlParameter("@WorkDate", Data.SqlDbType.datetime, 8, Data.ParameterDirection.Input); if (dr.HasRows) sp = "@WorkDate"; else *How does that look? Am I in the ball part…
I have one question regarding SQL Server replication.
I have setup SQLServer merge replication at my work, one of our subscriptions has unreliable connection to the publisher, after setting up the replication to this subscriber, it happened once that the connection was really bad so the merge agent for this subscription was failing before synchronizing data changes, and after that , the connection went good & the merge agent completed the synchronization successfully.
After a while I decided to validate all subscriptions, I discovered that this subscription has different row count for many articles although the merge agent was finishing all synchronizations successfully!!
so my question is:
how could this happen? I mean finishing the sync with success and still have different records!!
I thought that SQL Server will keep make the sync fail just because it can't do it!!
Is it right or there is something I misunderstood here?
How can I prevent this from happening again? at least I want SQL Server to inform me by failing the merge agent synchronizations.
I new here so hopefully I'm asking this question in the correct forum. I'm have a flat file that contains numbers that I need to verify that they begin with certain prefixes so they load to the correct client. For example in the flat file if I'm loading data for client A and their account numbers begin with 045XXXXXXX then it loads the data. But if there is a record that begins with 037XXXXXXX it should be loaded to client B instead and that records gets written to a error file.
So to summarize what I need , I'm looking for a check to kick out records if I'm loading client A's data versus if I'm loading client B's data.
Hello everyone, I have spent hours trying to figure out the validation issue with SQL Server Reporting Services with no luck.I am hoping someone can help me with the issue. Basically I am looking for a way to validate the data entered by the user in the parameterized Reports. say, the field is defined as int datatype. If user enter String, it should display the message saying string cannot be entered. Currently, My report does not check the user entered datatype, runs the underlying query and displays the error thrown by the DB ( oracle in my case). I get : An error has occured during report processing Cannot read the next data row.... ORA-01722: invalid number
Instead of this, validation even before query is executed would have been perfect. Is there a way to do that?
I want to make a conditional split based on the data type provided by the input. For example : If the comming (Column x) is of data type (numeric) then pass , else do not pass.
I am working on a coupon redemption project that appears to require dynamic data validation and need some help. For example, a coupon could require a person to buy XXX number of XXX items from XXX manufacture before the transaction is approved. The logical operator between each validation could also be “AND� and “OR�. My first attempt at this has been something along the lines of creating a coupon table with a child table of validation logic. However, being able to apply this logic (and at what layer) has turned out to be a challenge. I am very concerned about speed as all requirements have to be met before a transaction is approved. Any help with trying to figure out the best solution for this would be great@!
Coupons C_ID C_DESC MFG_ID MFG_OFFERNUM
Coupon_Validation CV_ID C_ID MFG_ID (Manufacture) MFG_PARTNUM (Manufacture Part Number) CV_QTYREQUIRED (Number to meet requirements) CV_ITEMLOGIC (AND, OR)
We have the following scenario: We receive CSV files every month for which SSIS packages were built to process the data. The following problems occur from time to time:
1. The structure of the CSV file changed (e.g. column added or removed) 2. There were no footers in the data, but now footers started to appear 3. Date format changed (e.g. used to be mm/dd/yyyy, but became mm.dd.yyyy) 4. Number format changed (e.g. from 2000 to 2,000)
Currently we have person who manually opens each file, and using our "validation document" validates to ensure none of these or similar problems occur. We would like to move away from this manual process if possible. I understand that items 3. and 4. could be caught by loading data into a staging table with VARCHAR data types, and performing validation before moving it any further.
Item 2 is a bit questionable (meaning depending on the footer size SSIS load could fail or not).
Item 1, however, is a sure fail of the SSIS package that directly loads the data into a table.
Thus I feel the two possible options are:
1. Create a custom script that will run through the file, row by row, apply all the necessary validations and report an error or continue if all checks out
2. Use some 3rd party tool to validate the files (semi-manually) before kicking off the SSIS processing.
Im newto SSIS. I want to develop package for data validation.
FirstName
1. Mandatory  field checking: if Null, reject the record  2. If field length > 50, then reject the record
SSN
1.  If field length > 12, then reject the record 2. If SSN is not in valid format, issue warning and process rhe record  without SSN value. 3. Valid format: 9 digit numeric values should present after striping off  all non-numeric characters. 4. Only send 9 digits to MDM
Like these i have 30 rules. And I have to shop the error msg if the validation fails like "Mandatory feild is missing".
I wanted to know more about validation of SSRS parameters. I have a simple report which has a parameter called startdate of DateTime datatype. The datetime parameter in SSRS takes manual input as well. So, the user can enter any junk value. I want to ensure that the input parameter is in correct format and I want to display an error msg when the format is incorrect. My report has the following VB code for validation:
Public Function Validate( ByVal startdate As String) As Boolean If IsDate(startdate) = True Then Return True Else Return False End If End Function
And my report has a textbox which has the expression property set to;
=Code.Validate(Parameters!startdate.Value) the textbox on the report has to display if the entered date is valid or not.
But, when i enter an erroneous date, SSRS doesn't render the report and throws a generic error. This happens even before the code written for validating the parameter executes.
Also couldn't find a way to disable the manual input for the datetime parameter. Even that would solve the problem.
Another alternative was to make the startdate parameter as string, but i want the calendar control button to be provided for the user.
In my database some of the store procedures getting the data from xml nodes.so I need to implement the validation to xml data for prevent sql injection.
I am trying to executed a packege so that it loads data from from the excel file to the SQL Server Server database. When I execute it, it prompts the following error message and 1 warning The excel file has three colums, Week, Item and Value
Error 4 Validation error. Data Flow Task: OLE DB Source [94]: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80040E14. An OLE DB record is available. Source: "Microsoft OLE DB Provider for Oracle" Hresult: 0x80040E37 Description: "ORA-00942: table or view does not exist ". Test - GET NW PERF 1.dtsx 0 0
Warning
Warning 1 Validation warning. Data Flow Task: OLE DB Destination [36]: The external metadata column collection is out of synchronization with the data source columns. The column "DAY" needs to be added to the external metadata column collection. The column "TCH_AVAIL" needs to be added to the external metadata column collection. The column "PDROP" needs to be added to the external metadata column collection. The column "P_HR" needs to be added to the external metadata column collection. The column "SFAIL" needs to be added to the external metadata column collection. The "external metadata column "VALUE" (90)" needs to be removed from the external metadata column collection. The "external metadata column "ITEM" (89)" needs to be removed from the external metadata column collection. Not in use - GET NW STATS.dtsx 0 0
I've created a merge replication a few days ago. it works correctly, but today when click synchronize publications, some of then encounter this error :
Data validation failed for one or more articles. When troubleshooting, check the output log files for any errors that may be preventing data from being synchronized properly. Note that when error compensation or delete tracking functionalities are disabled for an article, non-convergence can occur. (Source: MSSQL_REPL, Error number:
I want to do something with error checking in my company. For this we have a selection of different tables and the data needs to meet various validation rules else it is classed as an error.
To deal with this I'm currently thinking of this approach:
1. Create a view pulling all of the various data together from the multiple tables. 2. Create an empty 'errors' data table. 3. Create an Excel file with a button to call a Check for Errors Script
Then in the the script:
1. Clear the 'errors' data table 2. Call multiple scripts, each of which uses the new view, applies the checks for that specific error and writes any erroring data into the 'errors' data table (along with a text string with the unique error code for filtering / sorting purposes). 3. After calling all the scripts, the table can be refreshed in excel when when used with a pivot table can show the various errors, and let us drill down into all the data so we can fix them.
Also.. Ideally, I'd like some way to write comments in an excel column for each entry and error code and be able to write that back into a comment table.
The issue is in the data flow for loading and setting the Fact table dimension keys (the dimensions are all loaded fine). After 16 rather pedestrian Lookup Transformations, I have an escalating problem adding additional Lookup transforms to the Data Flow. The problem is not in execution; the problem is adding more transforms in design mode.
Lookup # Fields in Data Flow Time to validate that lookup <17 47 Sub-second 17 48 2 sec 18 49 4 sec 19 50 8 sec 20 51 16 sec 21 52 32 sec 22 53 64 sec
While I€™m intrigued by the mathematical progression that is forming here, the issue is that I have at least 6 more Lookups to perform. I hope you can see my dilemma.
I have gone to where it takes a little over 4 minutes each to validate the lookup transform and its associated Derived Column transform and Union transform (Total 12 Minutes). Not only does this add up to many idle minutes to each design step, BUT it breaks the debugger as it pre-validates the ENTIRE data flow before it ever switches into debugging mode.
Some notes: 1. It doesn€™t matter what order the Lookup transforms occur in, the timings are exactly the same. 2. I tried many Data Flow execution optimizations, but they don€™t improve the validation times (or even get a chance to improve the execution times!)
I realize this may be somewhat of a unique problem.
I have a package set up basically with two consecutive data flows. The first flow takes data from an OLE DB Source and stores it into a Flat File Destination. The second flow uses this same flat file as a source, alters the data, and stores the data in the same flat file, overwriting the old file. I set DelayValidation to True on the flat file. Still, here are the error messages I am receiving:
Error: 0xC020200E at DO, Flat File Destination [7676]: Cannot open the datafile "C:Temp.txt".
Error: 0xC004701A at DO, DTS.Pipeline: component "Flat File Destination" (7676) failed the pre-execute phase and returned error code 0xC020200E.
I am new to SSIS, so I'm sure I have a setting wrong or something. Is the problem that SSIS is trying to write to a file from which it is simultaneously reading data?
In C# .NET I have the possible to create some validations of my data, with regulary expressions. Do SQL have the same feature? I will like to do an data validation of all my insert statement inside the sql-server. Is that possible?
I'm developing a database-driven program using SQL server 2000 and Visual Basic 2005.
Most of the guys say professional programming is doing the validation stuff (such as the constraints and data integrity stuff like" [0-9][1] " and the use of LIKE IN keywords etc.) in the databse itself.
say i did the data validation contraints in SQL server itself. and now i connect the database with the interface made in 2005. and say a person enters some invalid data through the interface. but the error messages are generated by SQL server. how am i to display the SQL server generated error messages in the VB made interface??
PLS HELP ME .. if the question is not clear pls tell so that i can explain it further.
I'm developing a database-driven program using SQL server 2000 and Visual Basic 2005.
Most of the guys say professional programming is doing the validation stuff (such as the constraints and data integrity stuff like" [0-9][1] " and the use of LIKE IN keywords etc.) in the databse itself.
say i did the data validation contraints in SQL server itself. and now i connect the database with the interface made in 2005. and say a person enters some invalid data through the interface. but the error messages are generated by SQL server. how am i to display the SQL server generated error messages in the VB made interface??
PLS HELP ME .. if the question is not clear pls tell so that i can explain it further.
I am trying to insert a row into a table of Microsoft SQL Server 2000.
There are various columns.
[SNO] [numeric](3, 0) NOT NULL , [DATT] [char] (32) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL , [DATTA] [char] (3000) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL , [CODECS] [char] (32) COLLATE SQL_Latin1_General_CP1_CI_AS NOT NULL ,
The [DATTA] column is causing a problem. Even if I am trying to put only 1700 character string into [DATTA], the java code throws the following exception:-
StaleConnecti A CONM7007I: Mapping the following SQLException, with ErrorCode 0 and SQLState 08S01, to a StaleConnectionException: java.sql.SQLException: [Microsoft][SQLServer 2000 Driver for JDBC]Connection reset
at com.microsoft.jdbc.base.BaseExceptions.createException(Unknown Source)
Why is it throwing an exception even though the sum-total of this row doesn't exceed 8000 characters?
Hello, I am tring to add a string my database. Info is added, but it is the name of the string, not the data contained within. What am I doing wrong? The text "Company" and "currentUserID" is showing up in my database, but I need the info contained within the string. All help is appreciated!
Imports System.Data Imports System.Data.Common Imports System.Data.SqlClientPartial Class _DefaultInherits System.Web.UI.Page
Protected Sub CreateUserWizard1_CreatedUser(ByVal sender As Object, ByVal e As System.EventArgs) Handles CreateUserWizard1.CreatedUser 'Database ConnectionDim con As New SqlConnection("Data Source = .SQLExpress;integrated security=true;attachdbfilename=|DataDirectory|ASPNETDB.mdf;user instance=true") 'First Command DataDim Company As String = ((CType(CreateUserWizard1.CreateUserStep.ContentTemplateContainer.FindControl("Company"), TextBox)).Text) Dim insertSQL1 As StringDim currentUserID As String = ((CType(CreateUserWizard1.CreateUserStep.ContentTemplateContainer.FindControl("UserName"), TextBox)).Text) insertSQL1 = "INSERT INTO Company (CompanyName, UserID) VALUES ('Company', 'currentUserID')"Dim cmd1 As New SqlCommand(insertSQL1, con) '2nd Command Data Dim selectSQL As String selectSQL = "SELECT companyKey FROM Company WHERE UserID = 'currentUserID'"Dim cmd2 As New SqlCommand(selectSQL, con) Dim reader As SqlDataReader '3rd Command Data Dim insertSQL2 As String insertSQL2 = "INSERT INTO Company_Membership (CompanyKey, UserID) VALUES ('CompanyKey', 'currentUserID')"Dim cmd3 As New SqlCommand(insertSQL2, con) 'First CommandDim added As Integer = 0 Try con.Open() added = cmd1.ExecuteNonQuery() lblResults.Text = added.ToString() & " records inserted."Catch err As Exception lblResults.Text = "Error inserting record." lblResults.Text &= err.Message Finally con.Close() End Try '2nd Command Try con.Open() reader = cmd2.ExecuteReader()Do While reader.Read() Dim CompanyKey = reader("CompanyKey").ToString() Loop reader.Close()Catch err As Exception lbl1Results.Text = "Error selecting record." lbl1Results.Text &= err.Message Finally con.Close() End Try '3rd Command Try con.Open() added = cmd3.ExecuteNonQuery() lbl2Results.Text = added.ToString() & " records inserted."Catch err As Exception lbl2Results.Text = "Error inserting record." lbl2Results.Text &= err.Message Finally con.Close()End Try
I have a customer table with a postcode and a suburb fields and cutomer info which is manually entered by data entry people...
I am trying to compare the entries against a postcode table with the correct postcodes which have fields postcode and suburb and based on the postcode entered in the customer table it should be the same as the suburb in the postcode table, if they are not the same output them to a table for manual checking..How would I go about this
i'm using sql server 2005 express edition as the database server for my web application. Right now, i want to deploy it and the problem is, my hosting is using sql server 2000. So, is it possible to deploy my database that has been created in sql server 2005 in the sql server 2000? I'm afraid that the connection string for sql server 2000 is different from sql server 2005. In addition, i'm using forms authentication in my web application. So, some data regarding to the forms authentication like dbo.asp_net.membership table, dbo.asp_net.roles table is in my database. Is it the table that has been created like dbo.asp_net.membership can be used in sql server 2000 since it is created by asp.net forms authentication function?
