Need Some Advise On Loading Data From Excel Sheet

I insert/update thousands of line items daily to my MS SQL 2k db each day from multiple excel sheets that are uploaded. In Microsoft's infinite wisdom Excel and MS SQL is not "Fully" compatible and some characters throw off the uploads, cause errors in loading, etc. Each Excel sheet has from a few rows up to 50,000 rows or more. We load around 100 of these Excel sheets each day depending on what our users upload.

Our main problem appears to be with "Special Characters", anything that is not a number or letter seems to be an issue in loads. We have written our scripts to ignore a certain set of characters such as #,!, -, ', ", [, ], {, }, +, =, *, %, ~, `, <,>, etc. But we still get errors. This has become a frustrating nightmare. Any help in the right direction would be greatly appreciated.


I have tried ASP scripts, VB created exe's, converting the Excel sheet to a text file, then uploading, and other various means to get this process error free. Some files never have issues loading, some excel files will error out and not at the same point each time. We can run the same file 5 times in a row and it will stop/error at a different point each time without any rhyme or reason.

Now we are not just doing an "Insert", there are several variables that are at work when loading the data, like combining exact items into one row, associating data with ID's in another table, etc. It is not just a simple, take this data and place it here scenerio which makes this a serious headache to figure out how to make this error free and troubleshoot.

Is there some information or a direction I should look to consider a solid solution to importing data from Excel sheets to a MS SQL 2k db? These files are loaded into a specific folder and on upload they are also recorded in a table marked ready for update in the db. Our scheduler runs the exe associated with that users ID and loads their data, overwriting their previous data load, then marks the file as done.

Is there a proven method, some external program that can be used to make this a solid process, or any direction you can provide for me to research?