I have a Problem with my SSIS and it is that when you open the package it says "Validating <Data Flow>" and it takes a loooooong time to finaly open ( by long I mean like 3 or 4 hours to just for open!!!). In 3 of the DF, I have conections to a Unisys server and in exactly in this DF is where it takes its time. This Unisys system its kind a slow but not that slow. I am running on a server that have 4 G and 8 processors, so it cant be the machine.
my question is, How can I make it validate faster ? I have tried to work Offline but when I run the package I need to be online.
I'm stuck here when I try to open an existing package which contains several data flows, SSIS tries to validate each data flow and after a while a Visual Studio error message pops up and I can't do anything.
The error message says : "Unable to cast COM object of type 'System._ComObject' to interface type 'Microsoft.SqlServer.Dts.Pipeline.Wrapper.IDTSObject90'. This operation failed because the QueryInterface call on Com component for the interface with IID '...GUID...' failed due to the following error : The application called an interface that was marshalled for a different thread. (Exception from HResult: RPC_E_WRONG_THREAD)"
I have a MsAccess db containing a table called Employees which i am transforming to a staging table in Sql server 2005. Everything is working fine. I am using Foreach File enumerator and uploading the files one by one.However I now plan to validate the schema of MsAccess before uploading it. For eg: My employee table in msaccess is as follows :
Code Snippet Employees empId int, empName varchar(60), empAge int
Since the files come from different vendor, while looping, i want to perform a check if the empid or empAge are not of type long or string etc. If they are of type smallint,i have no problem.
However if they are larger datatypes than the the ones kept in Sql server, then the file needs to be logged in the db with the reason and moved to the error folder. In short, if the datatypes in access tables are smaller than those in Sqlserver, allow it, otherwise reject it. THe schema of Sqlserver table is same as of that of Employees in msaccess.
I want to compare the schema of the incoming access tbl fields with my desired schema and all mdb's having data types that are higher or incompatible with the desired schema should be moved to the error.
I have an SSIS package that wont run whne i try to run it in one go, but if i run it step by step it will succeed as the reason its failing is in the 2nd step it references a table that doesn't exist (its created in the 1st step.
So at the start package is trying to validate, but it won't as the table is created in step 1.
So this is why it works if i execute this step by step, is there a way of turning off the pre run validation on SSIS scripts ?
Hello All again, I have another question, I would like to validate the data in a sql table before my code runs and inserts duplicate information. Can anyone help out with this via example?
Has anyone figured out how to test if a string is a valid date without risking a sql conversion error?
I'd really like to avoid parsing the strings myself in an import procedure I am writing.
I can run an insert/select with the text being converted to datetime data, but if it hits text that won't convert correctly (such as '13-1-1999') it fails with a sql error. This could mess me up quite a bit.
In my SSIS package I have flat files as a source, I have to load numbers of flat files into SQL target table. I am using For each loop container for that. I am doing it correctly. My aim is to validate the source data from all angle before writing it into target sql table. I am using below points to validate the source data , if I found any bad data I am redirecting those data to error output.
To Checking
1. To check whether data type of column. 2. To check whether buisinesskey column null.
Is there any thing which I am missing to validate source data.
I have one table called product 1 and the second table is called product 2 i have DirNo, ProdNo, UpdCode and BranchOffice as fields i whant to loop trought every record in table product 1 and check in table product 2 if the record is the same and exist i f it does go to next record, if not delete the record in product 1 and move to the next record and so on. could someone please help me. iam very new at this. thanks
This question is rather open ended. Basically, I'm wondering the different approaches people use when validating input into a stored procedure. The rest of my post just describes a rather simple approach I'm using, and some ideas I have, but I'm eager to know what others are doing. So, if you'd like to comment on this, feel free to do so without reading the following.
I often find myself calling a stored procedure, and needing to validate some of the inputs before proceeding with the rest of the tasks. Such as, making sure a matching record isn't already in the database before adding a new record. Ideally, this sort of validation will report back to the user interface immediately, so that the user can get real-time response to their form input, instead of the all-too-common approach of redirecting to an error page if something goes wrong in the database transaction.
It's also convenient, at times (though perhaps not efficient in all cases) to let your database perform business rules validation of inputs (let the user interface make sure data types are valid, and such), so that you duplicate less code in the event that the stored procedure is called in more than one context.
To address these issues, I've taken to making two stored procedures instead of one. Let the stored procedure be called AddRecord. I'd create that, as well as the procedure named AddRecord_Validate, which would take identical inputs as AddRecord, whenever possible. Nearly the first thing done in AddRecord would be to execute AddRecord_Validate. The user interface would also call AddRecord_Validate, and return any validation errors to the user interface.
This seems rather convenient to me, in many cases, but often it doesn't work as well as I prefer. It's not uncommon for me to end up performing the same queries in the _Validate SP as I do in the parent SP. For example, let's say I'm passing a parameter that I use to look up a record I plan to edit. In my validation, I query the database to verify that the record is found. But in the parent SP, I run the same query again in order to get the stored ID of the record I need to edit. This ends up duplicating queries, making the pair of procedures less efficient on the whole, as well as introducing the likelihood of bugs when code changes in one of the SPs, but not the other.
So I've been brainstorming other approaches. The first idea is to execute a single stored procedure, but when I want to run it in "validation" mode, I simply rollback the transaction. This would let me know if the stored procedure would have run with the specified input, but won't ultimately change anything. Unfortunately, I see some badness in this, such as the entire SP perhaps taking a lot longer to execute that simple validation would have, and side-effects such as incrementing Sequences, or locking the database, and basically just doing a whole lot more work during validation than needs to be done.
The next idea was to require that the user specify the "run mode" via a parameter, either 'validate' or 'run'. Then, all of the validation would be performed at the top of the stored procedure, and a simple IF statement would exit the SP before actually making any changes if it's run in 'validate' mode. Otherwise, it will continue, and do the real work. The only real downside I see to this is forcing the developer to deal with an extra parameter. And, perhaps, it's not really a "relational" approach.
i am using visual studio 2008 to create an inventory management system and i am having trouble checking for a constraint violation. i have a partnumber colum in the database that needs to be unique and i am using datasets / tableadapters to interact with the database. when a user adds a new part number to the database i need to make sure its not already there i have tried to use catch ce as constraintexception debug.print ce.tostring end try
but the program still crashes with
System.Data.ConstraintException was unhandled ( see full error below)
no matter where i put the try statement it says its unhandeled
this error triggers when when i leave the combo box (when its validated) i have even tried placing it here
Private Sub PartNumberComboBox_Validating(ByVal sender As Object, ByVal e As System.ComponentModel.CancelEventArgs) Handles PartNumberComboBox.Validating
Try
Catch CE As ConstraintException
Debug.Print(CE.Message)
Catch EX As Exception
Debug.Print(EX.Message)
End Try
End Sub
if someone could point me in the right direction id greatley appriciate it
System.Data.ConstraintException was unhandled Message="Column 'PartNumber' is constrained to be unique. Value '36LDVRRP' is already present." Source="System.Data" StackTrace: at System.Data.UniqueConstraint.CheckConstraint(DataRow row, DataRowAction action) at System.Data.DataTable.RaiseRowChanging(DataRowChangeEventArgs args, DataRow eRow, DataRowAction eAction, Boolean fireEvent) at System.Data.DataTable.SetNewRecordWorker(DataRow row, Int32 proposedRecord, DataRowAction action, Boolean isInMerge, Int32 position, Boolean fireEvent, Exception& deferredException) at System.Data.DataTable.SetNewRecord(DataRow row, Int32 proposedRecord, DataRowAction action, Boolean isInMerge, Boolean fireEvent) at System.Data.DataRow.EndEdit() at System.Data.DataRowView.EndEdit() at System.Windows.Forms.CurrencyManager.EndCurrentEdit() at System.Windows.Forms.CurrencyManager.ChangeRecordState(Int32 newPosition, Boolean validating, Boolean endCurrentEdit, Boolean firePositionChange, Boolean pullData) at System.Windows.Forms.CurrencyManager.set_Position(Int32 value) at System.Windows.Forms.ComboBox.OnSelectedIndexChanged(EventArgs e) at System.Windows.Forms.ComboBox.WmReflectCommand(Message& m) at System.Windows.Forms.ComboBox.WndProc(Message& m) at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m) at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m) at System.Windows.Forms.NativeWindow.DebuggableCallback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam) at System.Windows.Forms.UnsafeNativeMethods.SendMessage(HandleRef hWnd, Int32 msg, IntPtr wParam, IntPtr lParam) at System.Windows.Forms.Control.SendMessage(Int32 msg, IntPtr wparam, IntPtr lparam) at System.Windows.Forms.Control.ReflectMessageInternal(IntPtr hWnd, Message& m) at System.Windows.Forms.Control.WmCommand(Message& m) at System.Windows.Forms.Control.WndProc(Message& m) at System.Windows.Forms.GroupBox.WndProc(Message& m) at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m) at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m) at System.Windows.Forms.NativeWindow.DebuggableCallback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam) at System.Windows.Forms.UnsafeNativeMethods.CallWindowProc(IntPtr wndProc, IntPtr hWnd, Int32 msg, IntPtr wParam, IntPtr lParam) at System.Windows.Forms.NativeWindow.DefWndProc(Message& m) at System.Windows.Forms.Control.DefWndProc(Message& m) at System.Windows.Forms.Control.WmCommand(Message& m) at System.Windows.Forms.Control.WndProc(Message& m) at System.Windows.Forms.ComboBox.WndProc(Message& m) at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m) at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m) at System.Windows.Forms.NativeWindow.DebuggableCallback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam) at System.Windows.Forms.UnsafeNativeMethods.DispatchMessageW(MSG& msg) at System.Windows.Forms.Application.ComponentManager.System.Windows.Forms.UnsafeNativeMethods.IMsoComponentManager.FPushMessageLoop(Int32 dwComponentID, Int32 reason, Int32 pvLoopData) at System.Windows.Forms.Application.ThreadContext.RunMessageLoopInner(Int32 reason, ApplicationContext context) at System.Windows.Forms.Application.ThreadContext.RunMessageLoop(Int32 reason, ApplicationContext context) at System.Windows.Forms.Application.Run(ApplicationContext context) at Microsoft.VisualBasic.ApplicationServices.WindowsFormsApplicationBase.OnRun() at Microsoft.VisualBasic.ApplicationServices.WindowsFormsApplicationBase.DoApplicationModel() at Microsoft.VisualBasic.ApplicationServices.WindowsFormsApplicationBase.Run(String[] commandLine) at Inventory_System.My.MyApplication.Main(String[] Args) in 17d14f5c-a337-4978-8281-53493378c1071.vb:line 81 at System.AppDomain._nExecuteAssembly(Assembly assembly, String[] args) at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args) at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly() at System.Threading.ThreadHelper.ThreadStart_Context(Object state) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Threading.ThreadHelper.ThreadStart() InnerException:
I am using the following SQl statement in a DTS vb file. The data is coming from an Excel spreadsheet and being put into a SQl server table.
oCustomTask1.SourceSQLStatement = "select `Order No`,`Country`,`Desc`,`Amount` from `Sheet1$` WHERE `Order No` = 1 "
I want to validate the data being put into the table to ensure no duplicates records are entered UNLESS the record has changed in any way. E.g. If record 1 had a column called "NumberOne" which changed to "Number1" change this information and move the original value to another table.
Hi all,I am a newbie to sql and I need your help.I want to update column (email) from one table to another validating theCustomerid column in both table. Update the email address in productiontable with the email address in temp table if the customerid is same in bothtables.What would be the query?Thanks,
I'm totally new to SSIS and need some direction.I'ved worked with the Import/Export wizard to create a package that imports a text file into a SQL Server table. However, I'm told the format of the text file changes over time and that's not good. I need to program in a format validation check on the source file before it gets imported. If it changes, I'm suppose to throw an alert or something.Let's say the file has the columns: field1 (string[10]), field2 (date), field3 (integer), field4 (decimal).I did some testing and tried to change the data to a longer string in field1, and SSIS recognizes that and errors out. How do I get it to send the bad record to an bad record file? Do I just set a destination file connection for bad records and connect the red arrow from the source file to the destination file?I forget if the source file connection recognizes a bad date and errors out. I'll have to check again.But when I changed the data in the datafile for field3 from integer to decimal. It didn't recognize that as an error. It read it in "successfully". That's not good.Similar thing happened when I changed field4 from decimal to integer in the data file. But I'm not too worried about that.Any hints on how to do this or a better approach on checking for file format changes would be appreciated.Ken
I have one date parameter in my report. The user will be able to manually enter or use the calendar control. In the case of entering manually, is there any way to validate the input string? How can we prevent users from entering alphabets?
Also, is there any way to reduce the length of the parameter textbox. Can you please help?
I want to do some error checking on the parameters found in a SQLDataSource before I run the insert. The problem is these are ControlParameters and I want to do this dynamically so I can't just call the Control.Text property and grab its value. So how can I get access to what the ControlParameter evaluates to? Secondly, is there a way to access what the update parameters evaluate to in order to check them before they're inserted - if so how do I get access to these? Here's an example of one of the data sources i'm using:<asp:SqlDataSource ID="sqlContact" runat="server" ConnectionString="<%$ ConnectionStrings:strConn %>" SelectCommand="SELECT [ContactID], [FirstName], [LastName], , [Address], [Phone], [Grade], [Contacted], [ListServe] FROM [Contact]" UpdateCommand="UPDATE Contact SET FirstName = @FirstName, LastName = @LastName, Email = @Email, Address = @Address, Phone = @Phone, Grade = @Grade WHERE ContactID = @ContactID" DeleteCommand="DELETE FROM [Contact] WHERE ContactID = @ContactID" InsertCommand="INSERT INTO [Contact] ([FirstName],[LastName],,[Address],[Phone],[Grade],[Contacted],[ListServe]) VALUES (@FirstName,@LastName,@Email,@Address,@Phone,@Grade,0,0)"> <UpdateParameters> <asp:Parameter Name="FirstName" Type="String" /> <asp:Parameter Name="LastName" Type="String" /> <asp:Parameter Name="Email" Type="String" /> <asp:Parameter Name="Address" Type="String" /> <asp:Parameter Name="Phone" Type="String" /> <asp:Parameter Name="Grade" Type="String" /> </UpdateParameters> <InsertParameters> <asp:ControlParameter Name="FirstName" Type="String" ControlID="txtContactFirst" PropertyName="Text" /> <asp:ControlParameter Name="LastName" Type="String" ControlID="txtContactLast" PropertyName="Text" /> <asp:ControlParameter Name="Email" Type="String" ControlID="txtContactEmail" PropertyName="Text" /> <asp:ControlParameter Name="Address" Type="String" ControlID="txtContactAddress" PropertyName="Text" /> <asp:ControlParameter Name="Phone" Type="String" ControlID="txtContactPhone" PropertyName="Text" /> <asp:ControlParameter Name="Grade" Type="String" ControlID="ddlContactGrade" PropertyName="SelectedValue" /> </InsertParameters> </asp:SqlDataSource> An Event is fired when I click add on a button which looks similar to this: //Add btnAddContact_Clickprotected void btnAddContact_Click(object sender, EventArgs e) { InsertRow(sqlContact); }The InsertRow() function is then what i'm using to evaluate the values... So how can get the values those controlparameters actually are in order to evaluate them before I actually insert. Or is there a better way to do it?
I periodically transfer large databases across servers and have had problems with tables on the target being empty (to rule out errant deletes by other developers). I'd like a way to do a table by table row count comparison between the two servers, similar to the way sp_compare_db (found here at Swynk) does on a single server. Even a method of referencing tables by including the server name would help.
I am looking for a store procedure which validates certain keywords like delete,truncate,update,insert etc. and restricts them to be used by users in all of my store procs which takes strings as inputs.
The problem: I need to validate all stored procedures on my server that they have not been altered after the application has been created.
Our server installation creates the user stored procedures for the application. Once created, I need a way to validate that the proc code has not been altered.
Previously, we were validating against the value of stats_schema_ver and crdate in the sysobjects table for the given stored procedure. During installation we would create a hash value of the crdate and stats_schema_Ver values for all our user defined procs. Then during validation, we would recreate the hash value and compare to that created during installation. If they differed, we knew that the proc code had been altered in some way.
Any alteration of the stored proc would update the stats_schema_ver- if the proc was dropped and recreated, it would update the crdate.
In 2005, the stats_shema_ver value is no longer updated when the proc is altered.
Perhaps this method was completely off-base to begin with...
So, how *does* one verify after installation that no proc code has been altered since installation?
The other option was to create a hash of the proc text definition- but that seems absolutely tedious to rehash the current proc definition every time the proc is accessed. There's also the problem that if you use the "with encryption" directive, it's no longer easy to get the text definition of the proc back out.
Appreciate any general thoughts or specific feedback.
I created a column in a table with datatype uniqueidentifier which is set as RowGUID and the default value to newsequentialid(). I get the following error whenI try to save the table
Error validating default value for column 'ColumnName'.
I am attempting to open up my ssis integration project but when I try to open up packages it says @ the bottom "Validating Sequence Container" and it just stays there. I cannot click on anything and nothing is responsive. Does anyone know why this is happening?
I just this day started using SQL sever 2005. I created a database and then created a table. Then I started adding some fields. I wanted to add a check constraint to one of the fields called state but I keep getting the same error. I right click on the field while editing the table and select check constraint. I then click add on the check constraint dialogue and in the expression caption I input the following:
<code>
@State In('CA', 'AZ', 'UT', 'CO')
</code>
I am using a book and have straight copied the above example from the book. However when I input the check constraint I get the following error;
"Error validating constraint 'ck_myfirstdatabase'
I have tried this with other fields and other types of check constraints and I still get the same error. I have tried to delete the database and recreate it. I have tried everything I can think of and I cannot seem to get check constraints to work. I have no idea why I keep getting this message. I have checked the examples a thousand times, the syntax is definately correct. This is getting extremely annoying as I cannot continue unless I do this. I'm all out of ideas. Can anyone please tell me why it could not be working? Any ideas would be greatly appreciated.
I'm trying to import a text file with a column that is actually a date value, like this "19550218" (yyyymmdd).
What is the best way to test if the string-date values in this column represent valid sql dates? Should I use SSIS to copy that value into an actual sql date field and see if an error is generated? I have tried using the VB.NET
"System.Datetime.ParseExact"
function in a Script task but found ParseExact less than reliable.
TIA,
Barkingdog
P.S. In fact, thought not speedy, I think that SSIS should come with a number of datatype validation routines "out of the box". Importing clean data is relatvely easy. It's handling the exceptional cases, like above, that really burn up my time.
P.P.S An before some says "regular expressions" I don't know enough about them to use them wisely.
I am curious why my stored procedures are parsing properly when you do not reference a table with its schema. The stored procs then fail when you run them.
It seems that the parser does not validate that a tables schema is missing.
This is an example stored procedure against the Person.Address table in the Adventureworks database. It will execute fine if I change the FROM clause to Person.Address.
CREATE PROCEDURE [dbo].[Address_Load] @AddressID [int] AS BEGIN SET NOCOUNT ON;
DECLARE @intError int
BEGIN TRY SELECT A.[AddressID] , A.[AddressLine1] , A.[AddressLine2] , A.[City] , A.[StateProvinceID] , A.[PostalCode] FROM [Address] A WHERE A.[AddressID] = @AddressID
IF @@ROWCOUNT = 0 BEGIN RAISERROR('Record not found', 16, 1) -- Record not found. END
-- Return success RETURN 0 END TRY BEGIN CATCH SET @intError = ERROR_NUMBER();
-- Log error here
RETURN @intError; END CATCH END
The stored proc parses fine and gets saved to the database but when executing it I get the following
Msg 208, Level 16, State 1, Procedure Address_Load, Line 10
Invalid object name 'Address'.
Is there any way to change this so the parsing will generate an error and not allow this into the database?
Hello, I am trying to validate a user via a stored procedure. This works OK until I try to validate the password, at which point the validation fails (even though the password is OK). Below is my sproc. If I comment out AND m.Password = @Password the sproc is OK. I am passing in the password as a string when testing the sproc. Any ideas? J. ALTER Proc [dbo].[pru_VerifyApproveUser] @UserName nvarchar(256),@Password nvarchar(128),@ApplicationName nvarchar(256)AS BEGIN IF EXISTS(Select * from dbo.aspnet_Membership WHERE Email = @UserName)BEGIN DECLARE @UserId uniqueidentifierSELECT @UserId = NULLSELECT @UserId = u.UserId FROM dbo.aspnet_Membership m, dbo.aspnet_Users u, dbo.aspnet_Applications aWHERE LoweredUserName = LOWER(@UserName) ANDu.ApplicationId = a.ApplicationId AND LOWER(@ApplicationName) = a.LoweredApplicationName ANDu.UserId = m.UserId AND m.Password = @Password -- If @Userid is Null - then the above credentials are not satisfied-- e.g. wrong password/username combo.. IF (@UserId IS NULL) BEGIN RETURN 2 END Update dbo.aspnet_Membership SET IsApproved = 1 WHERE Email = @UserName RETURN 1 END END
---Checks that input only contains numbers if PatIndex('%[^0-9]%','11') > 0 Begin Print 'Not all numbers' End Else Begin Print 'All numbers' End
---Checks that input only contains letters if PatIndex('%[^a-z]%','aaaaa') > 0 Begin Print 'Not all letters' End Else Begin Print 'All letters' End
--Checking for mixed input If PatIndex('%[^0-9][0-9]%','abc') > 0 Begin Print 'Alpha numeric data' End Else Begin Print 'Either all numbers or all letters' End
--Checks that value must start with a letter and a number If PatIndex('[^0-9][0-9]%','A1anamwar11') > 0 Begin Print 'Starts with a letter and a number' End Else Begin Print 'Does not start with a letter and a number' End
--Checks that value must End with a letter and a number If PatIndex('%[^0-9][0-9]','A1anamwar11a1') > 0 Begin Print 'Ends with a letter and a number' End Else Begin Print 'Does not End with a letter and a number' End
--Checks that value must Start with a letter and Ends with a number If PatIndex('[^0-9]%[0-9]','namwar1') > 0 Begin Print 'Starts with a letter and ends with a number' End Else Begin Print 'Does not start with a letter and ends with a number' End
How can I validate the date that is coming across in a 8 byte character field via an expression? It will either be a valid date or 0. If it is zero I want to make it a null.
It works in a straight select query, but when I put it in the formula of the table design window, I get an error "Error validating the formula for column 'test_fci'"
I don't know if it's relevant but repl_value is itself a computed column with the formula:
(repl_value_e_g + repl_value_aux)
Is it possible to use the system functions in a computed column? If not, how would I pass those values into a udf and use it for the formula?
Hello. You can compile a proc with a non-existant table. proc compiles do fail though on exsiting tables with non-existing columns.
Anyways, question is, how can you scan thru all procedure code to find tables that do not exist in the database? Is there any new DM feature that provides this info? Meaning, if I have a proc and inside it, refer to a table name that does not exist, I want to know the proc will not execute clean. Can the deferred name resolution be turned off somehow?
