Stored Proc Takes 45+ MINUTES!

Hi guys -

I have been googling all day and not found a reasonable solution to my problem.

Here is what I am trying to do. The T-SQL code is included at the bottom of this message.

- The user can specify columns that are stored in a CVColumn table (the data for which comes from an external source). For example, we may provide LastName and FirstName columns and the user could add ShoeSize and FavoriteColor.

- When we get the data from the other source, we create it as a dataset in the *code* with each user-defined field as a column of the table and each person as a row in the table. This is used *everywhere* including binding to grids, etc... So the dataset may look like this:
PersonID
LastName
FirstName
ShoeSize
FavoriteColor

- The dba has required that the data be stored in a normalized fashion. Since we don't know what the columns are - the data is stored in the *table* as:
PersonID
ColumnName
ColumnValue

- When the user gets the data from the external source the first time, the code loops through every person row and every column of the row to insert fields into the table using an Insert stored proc. This performs reasonable well.

- When the code retrieves the data from the database, it needs to reformat the data into one row per person, one column per field so that all of the other code can do the binding, etc.

- So I created a temp table with the appropriate rows and columns and then updated that table with the appropriate data. *This* is the stored proc that has been running now for almost an hour.

- There *must* be a way to do this that performs better?

Here is my T-SQL:




Code Snippet
DECLARE @columnNamesWithSizes varchar(8000)
DECLARE @delim varchar(1)
DECLARE @values varchar(8000)
-- Get the set of column names
SET @columnNamesWithSizes='swiCMKey VARCHAR(4092), swiCMContactManager VarChar(100), swiCMContactManagerID int,
swiCMFolder VarChar(4092), swiCMCVFolder VarChar(4092)'
SELECT @columnNamesWithSizes=COALESCE(@columnNamesWithSizes + ',', '') + ColumnName + ' VARCHAR(50)' FROM CVColumn
-- Create the table with the desired set of columns
EXEC ('CREATE TABLE ##tempTable ( ' + @columnNamesWithSizes + ')')

-- Insert the primary key into each row
EXEC ('INSERT INTO ##tempTable (swiCMKey) SELECT DISTINCT ContactID as swiCMKey FROM CVContact')
-- Use a cursor to loop through all of the records
SET @delim=''''
DECLARE @swiCMKey VARCHAR(4096)
DECLARE contactList CURSOR FOR
SELECT swiCMKey FROM ##tempTable
OPEN contactList
FETCH NEXT FROM contactList INTO @swiCMKey

-- Loop until all rows in temp table have been processed.
WHILE @@FETCH_STATUS = 0
BEGIN
SET @values=null
-- Retrieve all of the columns for this contact
SELECT @values = COALESCE(@values + ',', '') + ColumnName + '=' + @delim + COALESCE(Value,'') + @delim
FROM CVContact
WHERE CVContact.ContactID=@swiCMKey

-- Perform the update to the temp table with the column values
EXEC ('UPDATE ##tempTable SET ' + @values + ' WHERE swiCMKey=' + @delim + @swiCMKey + @delim)

-- Get the next one
FETCH NEXT FROM contactList INTO @swiCMKey
END
CLOSE contactList
DEALLOCATE contactList
-- Return the results
SELECT * from ##tempTable
-- Kill the temp table
DROP TABLE ##tempTable






Any ideas or othre suggestions would be much appreciated!

(BTW - up until today the data from the code-based dataset was stored as XML and then read back as XML. However, we found with 80,000+ people that it took 5 minutes to read the XML. But that is faster than this stored proc!<G>)