Best Practice For Add, Edit Records Into Database With Lots Of Fields ?
Feb 7, 2006
What's the best practice for adding / editing a record into a database with lots of fields ?
I am not talking about the mechanics of it, as there are a lot of trivial examples using ADO.NET, stored procs, etc.
Deleting is easy, you just pass in (a few) primary key/keys to uniquely identify the record.
But in the real world when you have, say, a table with 100 fields! Do you code the INSERT sproc by hand, with 100 parameters... then call it with your ADO.NET code ? sounds like a lot of work to me...
What about updating! That's even worst, sometimes you may need to update only 3 or 4 fields, but using sprocs you would have to pass the whole 100 parameters in again, and "update" the whole record (when in fact you are only changing 3 or 4 fields).
With the update i could write different sprocs targeting only the fields i wish to update, but that sounds like duplicating work, vs having one generic update proc.
Sometimes i just feel like bypassing sprocs and having inline sql as it would be less work... but i know it is untidy.. and more potential to be buggy.
So come on guys (and gals)... let's hear your thoughts on how you would handle the insert / update scenarios when you have lots of fields ? Northwind examples are too trivial :-)
I've got about 6 formats of flat files coming through data flows & heading into a relational db, then later to a data warehouse. In each file type, I've got about 60-70 columns to perform basically two levels of validations on - first is straight data type conversions, then 2nd is finer level stuff. Some of the data from each file type overlaps in other files, so for instance some lookup codes are maintained for both.
The data is pretty dirty, so I'm keeping everything as varchar coming into the staging area, just so I can get the data in the system, b/c the users demand on some form of of the data making it into the system, no matter the dirtiness.
So then I'm running my two steps - first converting data types from varchar to bit/datetime/int, etc. as applicable. And then , I'll be running finer levels of validation, doing range checks, etc. My question is - with so many columns, what's the most efficient & best way of doing all these checks in the data flow, and recording errors out of each check? Do I put one column check after another with the success constraint, do the lookup/range check/other, and then record the error if an error is encountered, then move on to the next column? Or would it be better to multicast out the stream to 60 flows and do all the needed checks, then union all the good stuff back together at the end? Anything to help save some headache - b/c this data is dirty and there's a lot of it.
Using ASP.net 2.0 or Visual Web Developer Edition 2005 I could not find a single example with Datagrid Control or Form View Control, to achieve the basic functionality ofAdd/Edit/Delete Records with Remote SQL Server 2000 Database. When I searched for these examples I found many videos and examples using local database and in learning path of Visual web express editions, very goodexamples and videos using local SQL Server 2005 database, BUT not with the remote database. My question Is it possible to get the basic functionality of Add/Edit/Delete Records with Remote SQL Server 2000 Database using ASP.Net 2.0 Datagrid and FormView controls? This question looks like, a lazy developer question!! but, in my learning path I found GREAT videos in Visual Studio Web Developer Express edition and learned simple way of drag and drop the controls assign create local database and their connections, and few clicks to add/edit/delete records using datagrid and formview controls. In real life those are not much useful because many of web interfaces are with sql server 2000 and we need to convert them into ASP.Net. Similarly I wanted to drag and drop couple of controls and less coding etc. and accomplish basic functionality of add/edit/delete records using remote database sql server 2000. If possible please help me out. Wondering is it possible to get the functionality of add/edit/delete records using datagrid/formview with remote sql server 2000, the way they explained in bigginers learning videos ... I am referring to (http://msdn2.microsoft.com/en-us/express/aa700802.aspx) If possible and you think it is possible please guide me to that URL where I can learn the great functionalty n implement..!!
I have an application that reads a monitoring devices that produces 200 digital outputs every second and I would like to store them in a table. This table would get quite big fairly quickly as ultimately I would like to monitor over a hundred of these devices.
I would like to construct queries against each of the individual digital channels or combinations of them.
M first thought is to set up a table with 200 separate columns (plus others for date stamp, device ID etc) however, I am concerned that a table with 200 boolean (1-bit) fields would be an enormous waste of space if each field takes maybe one to four bytes on the hard disk to store a single bit. However, this would have the advantage of make the SQL queries more natural.
The other alternative is to create a single 200 bit field and use lots of ANDing and ORing to isolate bits to do my queries. This would make my SQL code less readable and may also cause nore hassle in the future if the inputs changed, but it would make the file size smaller.
In essence I am asking (hoping) the following : If I create a table with 200 boolean fields, does SQL server express automatically optimise the storage to make it more compact? This means that the server can mess around at the bit level and leave my higher level SQL code looking cleaner and more logical.
Apparently, deleting 7,000,000 records from a table of about 20,000,000 is not advisable. We were able to take orders at 8:00AM, but not at 7:59.
So, what's the best way of going about deleting a large number of records? Pretty basic lookup table, no relationships with other tables, 12 or so address-type fields, 4 or 5 simple indexes. I can take it down for a weekend or night, if needed.
DTS the ones to keep to another table, drop the old and rename the new table? Bulk copy out, truncate and bring back in? DTS to text, truncate and import back? Other ways?
Never worked with such a large table and need a little experienced guidance.
I have a database where several thousand records have NULL in a binary field. I want to change all the NULLs to false. I have Visual Studio 5, and the database is a SQL Server 5 database on a remote server. What is the easiest way to do this? Is there a query I can run that will set all ReNew to false where ReNew is Null? This is a live database so I want to get it right. I can't afford to mess it up.Diane
I managed to find the 'Deleting Duplicate Records' from SQLTeam.com (thanks, by the way!!).. I managed to modify it for one of my tables (one of 14).
-- Add a new column
Alter table dbo.tblMyDocsSize add NewPK int NULL go
-- populate the new Primary Key declare @intCounter int set @intCounter = 0 update dbo.tblMyDocsSize SET @intCounter = NewPK = @intCounter + 1
-- ID the records to delete and get one primary key value also -- We'll delete all but this primary key select strComputer, strATUUser, RecCount=count(*), PktoKeep = max(NewPK) into #dupes from dbo.tblMyDocsSize group by strComputer, strATUUser having count(*) > 1 order by count(*) desc, strComputer, strATUUser
-- delete dupes except one Primary key for each dup record deletedbo.tblMyDocsSize fromdbo.tblMyDocsSize a join #dupes d ond.strComputer = a.strComputer andd.strATUUser = a.strATUUser wherea.NewPK not in (select PKtoKeep from #dupes)
-- remove the NewPK column ALTER TABLE dbo.tblMyDocsSize DROP COLUMN NewPK go
drop table #dupes
Now that I've got that figured out, I need to write the same thing to fix the other 13 tables (with different column info)- and I'll need to run this daily.
Basically I've put together some vbscript that gathers inventory data and drops it into an MSDE db (sorry - goin for 'free' stuff right now). Problem is it has to run daily so that I'm sure to capture computers that turned on at different times etc which ever-increases my database 'till I bounce off the 2GB limit of MSDE.
So the question is, what would be the best way to do this? Can I put the code into a stored procedure that I can execute each day?
writing the query for the following, I need to collapse the continuity. If the termdate for an ID is one day less than the effdate of the next id (for the same ID) i need to collapse the records. See below example .....how should i write the query which will give me the desired output. i.e., get min(effdate) and max(termdate) if termdate is one day less than the effdate of next record.
Hi guys... I need to do some statistical reporting on the answers to registration questions. The answers are stored in pipe-delimited varchar fields to allow for one-or-many answers to a single question... here's an example:
Q: What are your fav colors? A: [checkbox list of colors]
Now the data is stored like this:
Reg001 ----------- red|blue|green|yellow
Now I need to determine how many people selected each individual answer, so like, how many people picked red, how many people picked yellow.... to report like this:
Red - 12 Blue - 23 Green - 15 Yellow - 9
What is the best, or suggested way, to go about splitting this data up and reporting on it? Can it be done in T-SQL, or will it need to be a combination of T-SQL and .Net language?
As we all know, there is a 8060 bytes size limit on SQL Server rows. Ihave a table which requires a number of text fields (5 or 6). Each ofthese text fields should support a max of 4000 characters. We currentlystore the data in varchar columns, which worked fine untill ourappetite for text fields increased to the current requirement of 5, 6fields of 4000 characters size. I am given to review a design, whichesentially suggests moving the text columns to a separate TextFieldstable. The TextFields table will have two columns - a unique referenceand a VARCHAR (4000) column, thus allowing us to crossreference withthe original record. My first impresion is that I'd rather use the SQLServer 'text' DB type instead, which would allow me the samefunctionality with much less effort and possibly better performance.Can anyone advise on advantages and disadvantages of the two optionsand what the best practice in this case would be.Any advise will be well appreciated.Tzanko
Hi every body! I created a report project by SQL Server Business Intelligence Development.I use toolbox chart to display report.But I want to edit format of catagory fields when it appear.That mean I have a report have two catagory(category1 and category2),one is parent of another.I want to edit style of catagory( such as catagory1 will have lines to separate with another category1......) Please help me! Thank you very much.
There's lots of smart people here who can, I am sure, offer insight on this one.
I need to add handling of erroneous data to my package.
This is what my SSIS package does. The source table has thousands of records in it, most of which have been processed already (have a status of "Finished".) A typical run would have 400-600 records to process. The runs happen overnight.
Steps: 1) Execute SQL task that Updates all records in input table with status "New" or "Error" to "Process" 2) Data Flow task that takes as input all records with status "Process" and outputs to destinate OLEDB table 3) Execute SQL task that Updates all records with status "Process" to "Finished"
Erroneous records are identified at step 2 (Data Flow task) and need to be marked as "Error" so that they are not marked as "Finished" by step 3. They will be picked up and reprocessed by step 1 next time around.
I am having trouble seeing what would be the best way to achieve this? I am concerned that any steps I take might be prone to deadlocking as the updated data is also the source data?
My initial thought is to output some data to an "error" table in step 2, and then insert another Execute SQL task before step 3 that will update the source table to mark all these records as "Error" records. Does this make sense? Is there a best-practice way to achieve this?
Hi!I am able to add records to my SQL 2005 database using bound forms in myAccess application, but if I try to edit any records (not only the newrecords I create) using the same bound forms, I receive a write conflict.For the record, I am currently the only user of this application and my SQLServer database has only one userID. I used this userID to create all ofthe objects in the database and to connect to the database from Access. Thedata sources for the bound forms are linked tables, the record source typeis set to "Dynaset" and the record locks are set to "No Locks". "Opendatabases using record-level locking" is checked.The write error says that another user has changed the current record sinceit was opened (since I'm currently the only user this isn't possible) andasks me to copy my changes to the clipboard or drop the changes.Can anyone suggest what might be causing this? Thanks!
I'm new to reporting services and this is a very general question. I'm working on a large sales stats report with many results. I want to be able to compare many results for two dates. These results include, average sales value per day, average sales per weekday, sales with payment received, etc.
So basically there is lots of analysis needed mainly based on one database table (a fairly standard orders table).
What seemed the most logical thing to do is get all the relevant order rows for these two date ranges, A and B, and append a period column to the results, and then do all the maths/aggregate functions in Reporting Services. Thus only having to connect to database once.
And use a matrix with date period columns. So my query gives me results like:
Period order_total, is_weekday, no_weekdays_in_period.....
A 123 0 22....
A 54 1 22....
B 134 0 20...
Does this make the most sense? Or should I do the maths (grouping and aggregate functions) in lots of different queries (in which case, is Reporting Services worthwhile using?)?
when I hit F12 to preview my page and forms etc it goes to this error
This error (HTTP 500 Internal Server Error) means that the website you are visiting had a server problem which prevented the webpage from displaying. For more information about HTTP errors, see Help.
AND when I try to hold up to an SQL database it wont let me and says the server doesnt exist still. Whats wrong with my SQL server and what can I do to resolve this problem?
Good Morning, I work for a company that has sees alot of people come and go. The one thing I have noticed is that people use their admin accounts to log into SQL and create sp, views and databases.When the user leaves I am stuck with all these objects that are owned by somone no longer working for the company. So my question to you guys is: What is the best practice to use in creating new objects? Thanks for your guru-ness!
Can you tell me what is the best practice for SQL database migration fromone DB server, to another one, new DB server. The old DB server will beremoved.1. Backup from old and restore all databases on the new server2. Export data and copy/import on a new server3. Something else...Thanks in advance for any good advice...Regards
hi experts, i have a postcode database that i need to update. the database cnotains of 6tables, the file i ahev has all the information at once, so i have to organize it and insert records into the appropriate tables. this is the first time i'm doign this so i would like to know what the best way to do? do i need to create a stored procedure or a script, or may be something special and efficient that i do not know yet. any advise will be very appreciated thanks in advance
I'm currently building a database that is going to have at least 6 different types of users accessing it via a web application.
A user will have different information collected about them. A few of of the users will have the same or similiar information collected about them.
I will be using role based authentication.
Now my query is this;
Scenario 1: Would it be a good idea to keep all the users common information in one table. Ie. thier username,email,password,name. Then create a tblDetails for each type of user that would contain the different data. That way i can just check one table to verify thier login credentials.
OR Scenario 2: Would it be best to create a seperate table for each type of user and then log them in based on the credentials stored in each type of users respective table.
Hope this is clear. I'm leaning towards scenario 1, although I've used scenario 2 before. Just wondering which would be preferred.
I am new to working with Database using .net C#. i will really appreciate if someone can point me to a link where i can find best practices of working with sql express database with my c# web service applications.
preferably i would like to have answers for:
- to have one db or to have one for testing and one real db?
- db security
- use windows authentication or db user authentication
i have a table with some data imported from another database. In one table shows some records 'manchester customer [][] phoneline [][] january' when I select these data it display in multilines. that mean 2 squares represent enter key. I need to remove these from the records and replace space. so that i can select field value as a single line how can i do this ?????? pat
I have a table that contains some duplicate records. The entire record is not duplicated, but there are two fields which if both are duplicated, result in what I call a duplicate record. I can see that I have these duplicates by doing the following:
SELECT DISTINCT ReferenceNumber, ShipToCompanyID FROM tabMedia WHERE (MediaTypeID = '1') AND (FinishDate > '2011-01-01') GROUP BY ReferenceNumber, ShipToCompanyID HAVING (COUNT(ReferenceNumber) > 1) AND (COUNT(ShipToCompanyID) > 1) ORDER BY ReferenceNumber
but of course, this only returns two fields. I need the entire record returned where the duplicate exists...
Hi, I would create a vb2005 application that allow me to edit a sdf database stored into a pda connected at pc with activesyn directly on my desktop as well as "SQLCE Database View". How can i do it??
I'm not sure this is the exact forum in which to ask this question, so if there is a more appropriate forum, please point me in that direction.
Basically I'm looking for advice and best practices for dealing with an upgrade plan for a hosted database. Here is the situation: - Hosted software application (.Net) - Uses SQL Server 2005 - When we upgrade the software application to a new version, there will be schema and data changes that need to be applied to the database - For the software application, we have considered upgrading an offline machine and then switching old one instantaneously. - For the database, I'm trying to figure out if we can keep everything live for upgrades, or if we should have a small amount of scheduled downtime to upgrade.
So we have identified 2 basic directions: 1) Keep the database live and apply the new script changes just after the application software is upgraded. This means zero downtime, but we'd have to design and test our application to be backward compatible, so that the new version of the application code could handle both the new schema design, and the old schema design. It seems that this option would add a huge amount of complexity because we would essentially be testing a 3rd app/schema combination. 2) Take a small amount of scheduled downtime and upgrade the application code and database all at once.
Obviously option #2 is the most desirable from a development process point of view, and #1 is the most desirable option from a business point of view (zero downtime).
So, I'm looking for some advice on this. Do most people take the whole system down? Are there 100% live systems that deal with this? Are there other approaches I'm not considering?
Any information would be greatly appreciated. Thanks.
Hi everyone, I have some trouble writing a SQL stored procedure thatcan do the following:We have data in one table in numeric form, but we want to sum the datain this table based on the values of two different alpha fields. Toillustrate, let me write the following example:Table with these records:A B 1.1 2.2 Blah1 Blah1A B 2.3 5.6 Blah2 Blah2B C 7.8 9.1 Blah3 Blah3B C 4.5 1.0 Blah4 Blah4R F 1.1 4.3 Blah5 Blah5B A 3.1 2.7 Blah6 Blah6I need to write a query that will return the following result set fromthe above table:A B 3.4 7.8 Blah1 Blah1B C 12.3 10.1 Blah3 Blah3R F 1.1 4.3 Blah5 Blah5B A 3.1 2.7 Blah6 Blah6If the alphanumberic keys are the same, the sum the numeric columns upwhilst displaying one one of the records, the blah fields don't matterif one only one is displayed. Can anyone recommend the best way to dothis? I'm running MS SQL 2005.Thanks,Herman
I am trying to write a query that will return a full record with a particular distinct field (the rest of the record being the first such record that includes the distinct field).
For example, for the following:
Fruit Like? Colour Apple Y Green Orange N Orange Banana Y Yellow Grape Y Green Grapefruit N Yellow
I would want to return (assuming Colour was the distinct field):
Fruit Like? Colour Apple Y Green Orange N Orange Banana Y Yellow
How do I do this? I've tried using a join (of all different kinds) with a subquery that uses SELECT DISTINCT but this doesn't seem to work. I've tried GROUP BY but none of the aggregate functions seem to just take the first found field.
hi i have a database with a table with several columns: ID - primery key customer company now lets say i want to edit one row in the database and i have the primery key ID for that row. so i click on that row (which is displayed in a gridview) when i click I want the text that is in the customer cell to appear in a textbox1 and the same with compeny text I want it to apear in another textbox2. so what code can i write in the GridView1_SelectedIndexChanged eventhandler to populate the textboxes? thank you
I'm relatively new to administering SQL Server, hence this basic question.
What are the standard practice tool(s)/methods for allowing users access to a database to edit data? In this case, these are engineers (not programmers) that need to edit some values. The practice in the past has been to install Enterprise Manager on their PC. However, this does not seem proper to me and I was thinking more along the lines of having them use Access instead.
I compare Enterpise Manager in SQL Server as an equilvalent to Enterprise Manager in Oracle - these are tools only for use by database administrators. Is this a correct correlation, or is Enterpise Manager in SQL Server a tool that users (and programmers?) are commonly allowed to use? In the case of common users, I would think not.
