SSIS Development Suggestions
Mar 20, 2006
I could use some help from those of you who have SSIS deployed in a production environment. The developer creates a SSIS package on a workstation with BIDS. Package accesses a network drive and a SQL database on a development server. The developer can test the SSIS package successfully in BIDS.
The only way to use the ServerStorage protect level is that the developer needs to save the SSIS package directly to the development server (we are storing the package in SQL Server). This is not a problem, but it possesses a problem when we need to migrate the SSIS package to the production server. This is because in development our databases are suffixed with the word test. Also, the SQL Server user id that has access to the tables has a different password in production. We run our SSIS packages with the SQL agent. The SQL agent id has permissions to the network drives; the connection information within the package has permissions to the SQL tables.
How can the DBA, who would ideally does not want to and may not be able to due to audit restrictions, update the server name, user password, and database name without using BIDS? I tried using the configuration file option, but that does not seem to work. It keeps deploying the XML configuration file to my local machine even after successfully migrating the package to SQL Server.
To me, the DBA should simply need to run the deployment wizard and target the production server. At the time of the deployment, the wizard should prompt for items that need to be updated, and the settings should be stored in SQL Server. I know this scenario sounds exactly like those presented in the docs, but I must be doing something wrong.
View 5 Replies
ADVERTISEMENT
Jul 20, 2005
I'm getting ready to start a fairly large application to be hosted onSQL-Server. In the past I have used Access 2002 (as a front end, wasn'thappy with the number of bugs I ran into), PowerBuilder 6.5, Oracle Forms 7years ago (also no happy with the number of bugs). I was curious what thestate of the art was these days and what current recommendations were for4GL type of development tools.Thanks,Doug Broadwell
View 5 Replies
View Related
Oct 26, 2006
We have been facing some delevelopment issues with SSIS. These are as follows:
We want to execute some tasks in a particular package depending on a condition. Example: If record count is more than 0 then go to next task else stop the package.
We tried to write a script, in which we evaluate the value of the variable. This variable is populated with the record count value.
We cannot find any particular function which can exit the package successfully. We can fail the Task, but that means the package is failed. wo dont want to fail the package.
In short we want to conditionally execute some tasks in a package. Something like an "If container" similar to "For container".
We have one main package from which we execute 5 other packages. We want to assign values for connection information and variables in this main package only. The other 5 packages should pick values from this main package.
We are scheduling our package from Windows Scheduled Tasks. When the package is run in this manner, it doesnt read values of variables and connections from configuration file. It uses the values that were provided during development which are embedded in the code.
Thanks
Anup
View 3 Replies
View Related
May 15, 2006
In the past I wrote DTS transforms entirely by myself. With SSIS, our team of several developers now wants each member to develop a piece of the same package.
Do SSIS packages support this type of simultaneous multi-developer creation or is it a "one developer at a time" type product?
TIA,
Barkingdog
View 1 Replies
View Related
Jun 29, 2007
I have seen a number of posts regarding parallel development of SSIS packages and need some further information.
So far we have been developing SSIS packages along a single development stream and therefore have managed to avoid parallel development of our packages.
However, due to business pressures we will soon have multiple project streams running in parallel, and therefore multiple code branches, as part of that we will definitely need to redevelop the same SSIS packages in parallel. Judging from your post above and some testing we have done this is going to be a nightmare as we cannot merge the code. We can put in place processes to try and mitigate this but there are bound to be issues along the way.
Do you know whether this problem is going to be fixed? We are now using Team Foundation Server but presumably the merge algorythm used is same/similar to that of VSS and therefore very flaky?
However, not only are we having problems with the merging of the XML files, but we also use script tasks within the packages which are precompiled, as the DTSX files contain the binary objects associated with the script source code, if two developers change the same script task in isolated branches the binary is not recompiled as the merge software does not recognise this object.
Do you know whether these issues have been identified and are going to be fixed to be in line with the rest of Microsoft Configuration Managment principles of parallel development?
Many thanks.
View 7 Replies
View Related
Nov 1, 2007
I have a SSIS package that runs fine when I execute it in visual studio but when I run it on the server as a job it fails. This is nothing to do with protection levels etc as the job is running. It fails part way through with this as the error log
Started: 2:42:48 PM
Progress: 2007-11-01 14:43:04.16 Source: Import Hon File Validating: 0% complete End Progress
Progress: 2007-11-01 14:43:04.16 Source: Import Hon File Validating: 50% complete End Progress
Progress: 2007-11-01 14:43:04.22 Source: Import Hon File Validating: 100% complete End Progress
Progress: 2007-11-01 14:43:04.22 Source: Import Med File Validating: 0% complete End Progress
Progress: 2007-11-01 14:43:04.22 Source: Import Med File Validating: 50% complete End Progress
Progress: 2007-11-01 14:43:04.22 Source: Import Med File Validating: 100% complete End Progress
Progress: 2007-11-01 14:43:04.22 Source: Import MedFac File Validating: 0% complete End Progress
Progress: 2007-11-01 14:43:04.22 Sour... Process Exit Code 1. The step failed.
It is failing on the last file it is trying to import but it works OK on my PC.
Can anyone give me any pointers on what might be causing this.
Thanks
View 5 Replies
View Related
Apr 25, 2008
Having a difficult time setting up a development environment and a set of standards for SSIS package development.
First of all, you can't run the dataflow object "SQL Server Destination" in BIDS because BULKCOPY can only be run from the actual server. So how do you test/debug a package with this object in it?
Second of all, if you create an SSIS package on a developer computer in BIDS, and then import it into the SSIS package store on your development SQL server, you can't run the package from Management Studio on the developer PC. You get the error "DTS_E_PRODUCTLEVELTOLOW" when it tries to run any of the SSIS. Do I have to have SSIS installed on the developer client machine? How do I do that without installing a full server instance on each client machine (not to mention the licsense issues)?
Lastly, what protection level would you suggest using for production? We are having issues with ODBC connection passwords being decrypted and thus package steps failing in using "EncryptSensitiveWithUserKey". What exactly does this protection level do? Our network is physically very locked down, so we arent worried about SSIS package security too much, just looking for a way for them to work reliably without having to setup complicated security scenarios.
thanks for your time.
View 4 Replies
View Related
Dec 13, 2007
Does anyone here have a favorite site or set of sites with resources on SSIS custom component (tasks, transforms, log providers, etc.) development? I've been searching around to try to find this on my own, but so far I've had little luck.
Darren - I've noticed that you always seem to have the most insightful responses to questions related to SSIS .NET development (such as your recent response to evaluating expressions in a custom component) so I am particularly interested in anything that you have to share.
Thanks in advance, everyone!
View 6 Replies
View Related
Jul 10, 2006
When I run a package I created in the development Studio it runs fine but if I create a job and run it I get an error "The AcquireConnection method call to the connection manager "ODS" failed with error code 0xC0202009"
I have the package setup to use a XML config file and it works fine on all the other packages but this one will not work.
View 6 Replies
View Related
Jan 2, 2008
Posts 7
Dear all
I have developed some packages (around 40) on my local system.
now i m trying to move the integration services project on the production server.
when i double click on the intergration services from the local drive of the server all the packages are up(works fine till here)
now i open any of the package-->this is what happens
Prompt1)-->TITLE: Microsoft Visual Studio
------------------------------
There were errors while the package was being loaded.
The package might be corrupted.
See the Error List for details.
------------------------------
BUTTONS:
OK
------------------------------
i press ok and then the prompt comes
Prompt2-->There were build errors.Do you want to continue with the last build and continue..yes..no..
i click yes ..and then error comes
prompt3-->
Error loading RTS-IMRB-DISTRIBUTION.dtsx: The connection "Excel Connection Manager" is not found. This error is thrown by Connections collection when the specific connection element is not found.
and finally the prompt for the OLE DB error
prompt 4-->[OLE DB Destination [14]] Error: SSIS Error Code DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER. The AcquireConnection method call to the connection manager "NDELNTX46.IMRB RTS.siddharth" failed with error code 0xC0202009. There may be error messages posted before this with more information on why the AcquireConnection method call failed.
Possible work around suggested to me was that in the base system(read local) before exporting go to package properties and the security and set the security as "Encrypt all with password" and also put some Password which i did and it worked for one.
other thing i thought off was creating the configuration file but first i thought of fully running one complete package on the production and then think of the configuration file.
so i went inside the invidual excel connection and pointed it to the correct new file location and also edited the OLE DB database connection and put my user name and password and checked save password and trying to run the entire thing.
but still the same problem..
one more thing i observed was that my OLE DB was going off everytime i am running the package.and i have to enter my password again...so othe save password i guess is not working
I know its a little long post..but i wanted to explain the entire process and problem to all with description.
all suggestions are welcome
Thanks in advance
cheers & Happy new year
sid
View 4 Replies
View Related
Nov 26, 2015
When deploying a project from within a SSIS project in Visual Studio 2012 to SQL Server 2012 Integration services server I get the follwoing error message:
Failed to deploy project. For more information, query the operation_messages view for the operation identifier '10'. (Microsoft SQL Server, Error: 27203)
For the given operation id there is no entry in view catalog.operation_messages.
View 3 Replies
View Related
Dec 20, 2007
Please can anybody help me in transferring existing SSIS Packages saved in a shared folder location from development server 2ED to Live server TWD1.
Both has SQL server 2005 running and has visual studio 2005
Currently about 25 SSIS packages are executed from the development server transferring data on Live server TWD1...these ETL process is called from development server but executed on live server.
Now the problem is when i call these packages from the shared folder from live server it crashes.....i need to changes something to shift the whole package to the live server..and execute on live server itself instead of recreating the whole 25 process from scratch.....also i use optimize for many tables ..and run in a single trancastion....so how can i see the mappings of source and destination tables.
Please let me know the process how i can achieve this.
Thanks
George
View 5 Replies
View Related
Jul 2, 2007
here is my schema...
Board of Zoning Appeals
Parcel#
BZACase#
ApplicantID
OwnerID
DateFiled
Size
Zoning
VU (Variance of Use)
BZACase#
ProposedUse
Comments
VDS (Variance of Developmental Standard)
BZACase#
OrdinanceReq
RequestedDim
ProposedUse
Comments
SE (Special Exception)
BZACase#
CurrentUse
ProposedUse
OrdinanceReq
RequestedDim
Comments
Applicant
ApplicantID
FirstName
LastName
CompanyName
Line1
Line2
City
State
Zip
PhoneNum
Owner
OwnerID
FirstName
LastName
CompanyName
Line1
Line2
City
State
Zip
PhoneNum
Now i know what im doing with the applicantID and ownerID...but the BZAcase# is a number/unique identifier that looks like this....2007-VU-000, 2007-VU-001, 2007-VU-003....so my question is
1. how do i get the last three numbers to increment each time a new application is created?
2. how do i retrieve the last record in the table???
3. Do you have any other suggestions?? i have to have the number and what type of form they applied for in the "case#"???
View 11 Replies
View Related
Mar 26, 2003
I am requesting suggestions to solve my problem.
Background: We are changing the way we pay commissions to our rep groups. We used to pay when the order was placed, now we want to pay when the invoice is paid.
Problem: The commision information is currently stored in the customer order, not in the invoice. These orders get deleted a couple weeks after the order was completed (shipped).
I want to create another, rather dynamic, table/structure that will store the order number and the commission percentage.
This info in this table should:
Be deleted: if the order has been deleted and the invoice either does not exist or was payed some period of time ago (maybe 6 months)
Be updated: if the customer order has been updated (i.e. the commission was changed)
Be inserted: if the order exists but the order number is not in the new table.
That is it in a nutshell.
Thanks,
Brian
View 1 Replies
View Related
Sep 25, 2006
hi
i have written a procedure for stock report.
its working fine. please go through the sp and give me some Suggestions. please tell me where i need to improve my code. thanks
Note: User is required to execute this procedure daily.
i am taking the sum of issues,purchases,returns,physical adjustments for each and every product from last updated date to today's date and storing it in a table i,e stock_Dump. from this table i generate the date wise stock report
SET QUOTED_IDENTIFIER OFF
GO
SET ANSI_NULLS ON
GO
CREATE PROCEDURE dbo.spUpdateStock
@strReturn varchar(70) output
AS
BEGIN
declare @maxDt smalld atetime
if exists(Select * from Stock_Dump where Txn_Date=
Convert(varchar,Getdate(),101))
BEGIN
set @strReturn='Stock Table already generated
for the day. cannot generate it again'
END
ELSE
BEGIN
TRUNCATE TABLE Stock_Dump_Temp
select @maxDt=max(Txn_Date) from Stock_Dump
/* insert (Opening stock) Closing stock for all
all the products from last max Date*/
INSERT INTO Stock_Dump_Temp Select Product_code,
convert(varchar,GetDate(),101) as Txn_Date,
Closing_Stock as Opening_Stock ,
0,0,0,0,0,0,0 from Stock_Dump Where
Txn_Date=Convert(varchar,@maxDt,101)
/* Issues*/
INSERT INTO Stock_Dump_Temp Select Product_code,
convert(varchar,GetDate(),101) as Txn_Date,0,
Sum(Qty) as Issue_Qty,0,0,0,0,0,0 from Issue_Details
Where Issue_No IN(Select Issue_No from Issue_Hdr
Where Issue_Date > Convert(varchar,@maxDt,101) and
Issue_Date <= Convert(varchar,getdate(),101))
Group by Product_Code
/* Goods receipt*/
INSERT INTO Stock_Dump_Temp Select Product_code,
convert(varchar,GetDate(),101) as Txn_Date,0,0,
Sum(Qty) as Purchase,0,0,0,0,0 from Dlv_note_Details
Where Dlv_Note_No IN(Select Dlv_Note_No from
Dlv_Hdr Where Dlv_Note_Date > Convert(varchar,@maxDt,101) and
Dlv_Note_Date <= Convert(varchar,getdate(),101))
Group by Product_Code
/* Rejection after receipt*/
INSERT INTO Stock_Dump_Temp Select Product_code,
convert(varchar,GetDate(),101) as Txn_Date,0,0,
0,Sum(Qty) as Rejected,0,0,0,0 from
Rejection_Details Where Rejection_No IN
(Select Rejection_No from Rejection_Hdr Where
Rejection_Date > Convert(varchar,@maxDt,101) and
Rejection_Date <= Convert(varchar,getdate(),101))
Group by Product_Code
/* Issues returns*/
INSERT INTO Stock_Dump_Temp Select Product_code,
convert(varchar,GetDate(),101) as Txn_Date,0,0,
0,0,Sum(Qty) As Issue_Returns,0,0,0 from
Issue_Return_Details Where Issue_R_No
IN(Select Issue_R_No from Issue_Return_Hdr
Where Return_Date > Convert(varchar,@maxDt,101) and
Return_Date <= Convert(varchar,getdate(),101))
Group by Product_Code
/* Physical Stock + */
INSERT INTO Stock_Dump_Temp Select Product_code,
convert(varchar,GetDate(),101) as Txn_Date,0,0,
0,0,0,Sum(Var_Qty) as Phy_Qty_P,0,0 from
Physical_Details Where Var_Qty>0 and Txn_No
IN(Select txn_No from Physical_Hdr Where
Txn_Date > Convert(varchar,@maxDt,101) and
Txn_Date <= Convert(varchar,getdate(),101))
Group by Product_Code
/* Physical -*/
INSERT INTO Stock_Dump_Temp Select Product_code,
convert(varchar,GetDate(),101) as Txn_Date,0,0,
0,0,0,0,Sum(Var_Qty) as Phy_Qty_M,0 from
Physical_Details Where Var_Qty<0 and Txn_No
IN(Select txn_No from Physical_Hdr Where
Txn_Date > Convert(varchar,@maxDt,101) and
Txn_Date <= Convert(varchar,getdate(),101))
Group by Product_Code
/* insert all the records into actual table i,e Stock_dump from Stock_dump_temp (temporory table)*/
INSERT INTO Stock_Dump Select Product_code,Txn_Date,
Sum(Opening_Stock) as Opening_Stock,Sum(Issue_Qty) as
Issue_Qty,Sum(purchase) as Purchase,Sum(Rejected) as
Rejected,Sum(Issue_Returns) as Issue_returns,
Sum(Phy_Qty_P) as Phy_Qty_P,Sum(Phy_Qty_M) as
Phy_Qty_M,0 as Closing_Stock from Stock_Dump_Temp
Group By ProducT_Code,Txn_Date
/* update closing stock*/
UPDATE Stock_Dump Set
Closing_Stock=abs((Opening_Stock+Purchase+Issue_Returns+Phy_Qty_P)-(Issue_Qty+Rejected+Phy_Qty_M))
Where Txn_Date=Convert(varchar,Getdate(),101)
/* delete unwanted records */
DELETE From Stock_Dump Where Opening_Stock=0 and
Issue_Qty=0 and Purchase=0 and Rejected=0
and Issue_Returns=0 and Phy_Qty_M=0 and Phy_Qty_P=0
set @strReturn='Stock Table Update Successfully'
return
END
END
GO
SET QUOTED_IDENTIFIER OFF
GO
SET ANSI_NULLS ON
GO
suji
View 1 Replies
View Related
Nov 21, 2006
I have a database which contains more than 20000 stored procedureswhich were created withansi nulls off. This i found out using the querySELECT name,AnsiNullsOn FROM(SELECT name, OBJECTPROPERTY(id, 'ExecIsAnsiNullsOn') AS AnsiNullsOnFROM sysobjects WHERE type = 'P' ) A WHERE AnsiNullsOn=0Is there any way that i can set this property to 1 for all the storedprocedures i have??I know the alternate method is to drop the procedure and execute thescripts again with AnsiNullsOn = 1.Is there any other simple ways?? It will be very helpful for me..
View 2 Replies
View Related
Dec 8, 2006
I want to transform textfiles to sql server set based and not row based.what would be the best way to transfer.
let me know.
View 10 Replies
View Related
May 4, 2008
Hello all!
I have this simple sp.
SELECT VisName
FROM tblVis
WHERE (VisID = 1)
Now I have lets say VISID 1 to 50. I'm using this SP to change the text on a button. Now I have 50 buttons. So I run this SP, then I run this in my vb.net code
Code Snippet
Dim constr As New SqlConnection(PVDBConn)
Try
'Variable to hold the results
Dim results As String = String.Empty
cmdUpd = New SqlCommand("SelVis1Name", constr)
cmdUpd.CommandType = CommandType.StoredProcedure
constr.Open()
'Set results to the value returned from ExecuteScalar()
results = CType(cmdUpd.ExecuteScalar(), String)
constr.Close()
'Set our buttons text to that value
Button1.Text = results
Catch ex As Exception
MsgBox(ex.Message.ToString)
End Try
At any time, when I start my program, I may need to label 10 buttons, or up to 50. Now I will have this number in a text file. Can I grab that number from a text file, and pass it into a SP?
And can I write this SP only once, to work for more than one label per time. Or do I have to write this sp 50 times?
TIA!
Rudy
View 5 Replies
View Related
Feb 9, 2008
I have a database that will be used by two or more organizations. I would like to use pass phrase encryption to encrypt a couple of columns.
I'm looking for suggestions on how I might set up the db to let the organization change the pass phrase that is used for their encryption?
I don't really want to hard code it into stored procedures or select statements with parameters. I will be using SSL if that should make a difference with what you suggest.
Any thoughts are appreciated.
Thank you
View 4 Replies
View Related
Sep 5, 2006
I have two stored procedures (l'll call them P1 & P2). P1, after a lot of processing, creates a temporary table that is used by P2 after an "exec P1" is done. I've separated the logic into two stored procedures because, ultimately, other sprocs will need the output of P1.
I get an error if I use #tempTable as the output table in P1 because it no longer exists after P1 finishes. ##tempTable works, but I'm concerned about concurrency issues. Any suggestions on what construct(s) I should be using?
Thanks in advance!
View 2 Replies
View Related
Oct 4, 2005
Hi All,
I am new to SQL Server but have been doing database programming since last 3 years. I recently attended MOC (Microsfot Official Curriculum) training on SQL Server and have started to use at my company. I am comfortable with SQL but want to dig deeper into T-SQL side. I searched on the Internet but not many good books available in that either they are ranked very low or are very old i.e. written around 1999/2000 or covers SQL Server 2000 as a whole. Can anybody suggest me any T-SQL book which was written recently and focuses purely or majorly on T-SQL?
Thanks to all for your time and advice in advance.
Regards:
Prathmesh
View 3 Replies
View Related
Jun 29, 2006
I'm looking for some help on how i should index this table.
current table has about 500k records in it.
the fields in the table are:
member_num (varchar(12), not null)
first_name (varchar(20), null)
last_name (varchar(20), null)
ssn (varchar(50), null)
address1 (nvarchar(200), null)
address2 (nvarchar(200), null)
city (nvarchar(200), null)
state (nvarchar(200), null)
zip (nvarchar(100), null)
phone1 (nvarchar(50), null)
all of the fields are searchable through an asp.net webform.
my first stab at this consisted of creating a clustered index on member_num and then creating a separate index for each of the remaining fields.
View 4 Replies
View Related
Sep 7, 2007
What I have.
I have a spreadsheet that is used in 4 or more locations on a daily basis by 1-3 ppl per locations. The spreadsheet is used to gather Quality Control information. So everyday there are a couple of spreadsheets from each system that is used to generate weekly and monthly reports. This is becoming to much work and I would like to automate the process.
What I have access to.
I currently run a Sharepoint 2007 Server for all our collaboration and document needs.
I also have the ability to setup any sql server.
What I want.
I want the QC techs in each system to be able to upload the data at the end of each day and be done with it. This way they do not have email or do a weekly report.
I would prefer to use Sharepoint and create reports weekly and monthly that can be pulled just by going to a site.
I'm knowledgeable in Sharepoint and Excel. I have some skills in VBA. I haven't dealt with SQL any, but willing to learn.
Also I'm knowledgeable in Microsoft Access as well.
Any suggestions on how I could accomplish this would be appreciated.
TIA
View 1 Replies
View Related
Apr 26, 2004
Hi Guys,
Well, as a VB/VBA applications developer I'm not well prepared for this, but it looks like I will be riding herd on a production SQL Server.
TSQL I know well enough to get along, but where can I get a fast fix on all the logins, security, and process management info? Today we had a DTS package crash overnight and it took me forever to figure out that it had left half a dozen tables locked. (Note that the scripts for the DTS package are being re-written as we speak with use of transactions and NOLOCK.) Meanwhile tech support was handling a whole mess of grumpy users.
Are there any books you would recommend as resources/references? Is there a particular author who is good at writing the stuff you really need to know in English that can be read by a mere mortal like I? I am fond of the Microsoft resources/help files but I'd like to have somthing that holds highlighter and post-it flags a bit better. Not to mention something that focuses more on the beast as a whole rather than the minutia at length.
Thanks for any suggestions!
View 14 Replies
View Related
Jul 20, 2007
I have a set of tables with about the same structure
dataID, recordID, 15 other columns
dataID is unique but is never referenced in queries
recordID is one of the most referenced columns but only has a cardinality of about 30%
The current structure has a clustered PK on (dataID,recordID)
Someone suggested reversing the clustered PK to (recordID,dataID) because of the number of references to recordID but that didn't seem to boost performance any
After staring at this for a while I came up with something but I'd like some advice whether it makes sense or not.
create a non-clustered PK on dataID
create a non-unique clustered index on recordID
Let me know if any other information is needed. Thanks
View 5 Replies
View Related
Aug 15, 2006
I would be teaching an applied database course to buisness majorundergrads. I'm looking for a book that introduces database conceptsusing SQLServer as the database. I would really appreciate if you couldrecommend me a few such books.ThanksNemo
View 4 Replies
View Related
Jul 20, 2005
I'm trying to count the number of records in 'game_dates' where thecolumns home_team_id or away_team_id have the same value. E.g., iwant to know the number of records for each team_id where team_id ishome_team_id or away_team_id.I'm doing this in two separate select statements now. Example:SELECT count(home_team_id),home_team_id FROM gamesWHERE league_id = 218 and ((home_score IS NOT NULL OR away_score ISNOT NULL) OR (home_score <> 0 OR away_score <> 0))GROUP BY home_team_idandSELECT count(away_team_id),away_team_id FROM gamesWHERE league_id = 218 and ((home_score IS NOT NULL OR away_score ISNOT NULL) OR (home_score <> 0 OR away_score <> 0))GROUP BY away_team_idand then combining the results. Is there anyway to combine these toqueries into one query? ...and have a single result set returned withtwo columns (count,team_id)?Thanks,Glenn
View 3 Replies
View Related
Jul 10, 2007
The Background:
I have a SQL Serever with more than 10 million records.
I have to update/delete/insert records on a daily basis.
I am using the standard edition of SQL Server.
The tables are constantly having data inserted into them and the server has different jobs running all day long.
My Problem:
I cannot create index on my database and the database is getting really slow as each month/year passes.
Any/All suggestions are welcomed.
View 10 Replies
View Related
Aug 16, 2007
Hi All,
Need some suggestions for senior management for DR Purposes:
Background:
WSS/MOSS2007 is being used as a Document Management solution.
17 Servers geographically dispersed around the UK. Each server runs WSS 3, SQL Server 2005 and IIS. Each server is linked into a PiP cloud via 2MB MPLS.
At each location; We are looking at 20 core databases; each pre-sized to 10GB. If I take one site as an example, the previous nights backup totalled 135GB.
The company has taken a centralised view on backup's, so SQL Server Data and Log files are replicated using Double-Take to a central location where by the files are taken onto tape daily (Full backup of all files).
As a precaution, I take a Full SQL Server backup daily and also Tran Logs every 4 hours locally and keep it there for 2 days; however if the site goes boom I loose those, so for this purpose; please forget they exist.
As I expect; when I restore the mdf and ldf files from tape, I will get errors when I attach those files into SQL Server for transactional inconsistencies which I'm well aware of.
Other options I've considered are:
1) DB Mirroring. Not a bad option, but still have to get the DB to the Mirror Server in the first place. Also DB Mirroring is not recommended for more than 10 mirrored databases.
2) Log Shipping. Same issue as above; Have to get the data here in the first place. Then once Log Shipping is setup; if I have a failure; I need to start the whole lot off again.
3) Transactional Replication. Issue is with the initial replication getting the data from A to B, then if I need to use it in a DR situation; I will get issues saying this table is being used for replication. This can be worked around, but it's a not a quick process...
4) 3rd Party Backup Compression. E.G. Litespeed; Redgate SQL Backup, etc. Good; Tests have shown a 42% compression for us, however if I refer to the earlier example of 135GB, this compresses to 81GB. Throw in the theoretical max for a 2MB link of 19GB / 24 Hours, this would take 4 Days to copy.
Other thoughts I've come up with are:
A) Split the tables into different file groups; not sure how easy this would be as the DB's and Tables already exist.
B) Full/Diff/Tran. Still have the issue of scheduling the full backup over the weekend and taking 4 days to get here.
C) Local Tape Backups. Issue is relying on someone to change the tape on a daily basis. It's not centrally managed and how do we restore in a DR situation ?
Could someone give me some pointers please?
Thanks
Steve
SQL DBA.
View 6 Replies
View Related
Jan 28, 2008
Hi folks, I have a very typical database for an ASP.net application. There is a table which will contain a hierarchical data..much like files-folders structure of a file system.
But we know that the table will be a giant one in production. There will be a huge collection of data need to persist in it. we are already facing some performance problem with some queries during the QA/test machine.
Currently there is a table which is keeping all file and folder information and another table maintaing their hierarchy relation using two column namely, parentID and childID.
My first question is, would it be better to keep this hierarchy relation into the same table rather using a different one? (much like managerID and empID in AdventureWorks sample?)
My Second question, what is the best way to design this kind of structure to get the highest performance benifit?
All kind of thoughts will be appreciated much! thanks
View 26 Replies
View Related
Nov 20, 2007
I am working with an application currently that stores the majority of its data in the following schema.
factid1 int,
factid2 int,
factid3 int,
factid4 int,
Value decimal 14,4
To extract data from this table we are using a 4 table join to each of the factid's
Our where clause in this query is based on (where factid1 = something)
So we have a composite clustered index led by factid1.
Our plan is to reduce the size of this table by introducing the kind of schema, we would like to introduce this to keep the table size to a minimum and hopefully increase the performance of our extracts from this table.
factid4 int
intersectid int
value decimal 14,4
And then the intersect table with fact2,fact2,fact3 combinations
factid1 int
factid2 int
factid3 int
intersectid
This kind of schema reduces the size of this table substantially but performance of our extract is very poor.
Does anyone have any suggestions on schemas that will give us high performance?
Or does anyone think that the original schema will outperform any alternative schema.
View 4 Replies
View Related
Mar 26, 2007
Greetings all,
I'm a developer tasked with securing up a SQL Server 2005 SP2 database. I'm not exactly a DBA but I'm giving it my best shot. I was hoping someone could offer some suggestions/tips on how I could approach this task. The amount of documentation on this type of thing is somewhat overwhelming. I'm a little pressed for time and was hoping someone could offer some help. Maybe even provide some feedback as if I'm in the "weeds" or not.
Ok, here's the deal...
At the moment I am using Windows authentication. From what I have read this is the preferred method over SQL authentication. I'd like to continue using this approach if possible.
The database can be has 3 principals
1. ASP.NET (Network Service on Windows Server 2003)
2. Windows Service running on the host server
3. A Data Access Layer assembly running on some other server
All the principals access the db using stored procedures only. Each uses a subset of all the stored procedures, some of them overlap.
My initial though was this:
For the ASP.NET I would perform the following:
1. sp_grantlogin [NT AUTHORITYNETWORK SERVICE]
2. sp_grantdbaccess [NT AUTHORITYNETWORK SERVICE]
3. Grant Execute on [For each sproc used] to [NT AUTHORITYNETWORK SERVICE]
For The Windows Service and the Data Access Layer principal, I was thinking something like this:
1. Create a separate windows login for each principal
2. Create a db login for each principal login From Windows
3. Grant execute on each of the sprocs used for each role
Question: How do I Deny Select, Insert, Update and Delete privs for all tables regardless of the principal (public user)?
Again, any help and or suggestions would greatly be appreciated.
Thank!
View 5 Replies
View Related
Jun 27, 2007
Hello,
I have a matrix with a dynamic number of columns (1-10). The trouble is that hiding one ore more columns still leaves space reserved for all 10 columns, which is ugly. This is because the size of the TextBox that oversees the columns is not dynamic, and it is set to the size of all 10 text boxes.
In other words, a matrix with 5 columns looks like this:
Item Total
Col1 Col2 Col3 Col4 Col5 Total
5 5 5 5 5 25
While a matrix with 2 columns (the last 3 have visibility set to false) looks like this:
Item Total
Col1 Col2 Total
5 5 25
I am going crazy trying to solve this one. Does anyone have any ideas at all that can help me? Merging all the columns into a single column would not work well for me, as each column is a drill-down for the others. And making each column small (.1in), doesn't work either because there is no "no-wrap" property.
ANY suggestions would be appreciated. How have others dealt with this issue?
Michael
View 2 Replies
View Related
