This has probably been covered in other posts. I have been working with SSIS for the past month and I am trying to follow best practices on various items. Having worked with a different ETL tool prior to this, I am wondering what is the best approach to use for Connections and File Paths.
What I would normally do with DataStage for this would be to assign a Job Variable (and eventually Sequence Variable) of the type: Path. So, if I was developing a job I would create SourceFilePath, ErrorFilePath, etc. I would use these variables in a FlatFile or Dataset Stage. For instance I would assign a filename for a source as: #SourceFilePath#SourceFile1.txt. During execution the job would load the variable and then the filename would be: C:MyDocumentsDatafilesSourceFile1.txt.
When its time to move to another environment, I don't have to worry about changing values for file connections because it is managed dynamically by a config file or whatever method.
What is the best practice that emulates this behaviour for SSIS? I've been thick and can't get my head around this. Any direction to blogs or user sites would be great. Examples, even better!
Hi I have sql 2000/2005 installation path errors in some of prodution servers. Like we have standards that backup files should go to E: drive and data files to f: and log files to h:. Can any one help me in this issue what can be done with out reinstallation,
We have a growing issue where we have a relative dtsconfig file (which stores the absolute base path of the ETL packages). This way we can keep the ETL projects failry portable - only having to modify one value in the dtsconfig file. The master package that defines the dtsconfig location (which is config/Default.dtsconfig) usually interpretates this location to be relative the project. The problem is that every now and again when you open this package in .NETStudio, the path is interpreted differently and causes: config/Default.dtsconfig to state invalid path. But when we delete the variable (which defines the dtsconfig path), save/close and open/recreate it works again. This may or may not be supported MS method, but I was curious to know why this gets messed up. Is there somehwere in the .NET framework that defines what "/" is relatively under?
For example: Our absolute config path is "D:Program FilesMicrosoft SQL Server90DTSPackagesETLProjectETLBaseconfigDefault.dtsconfig" but using: "config/Default.dtsconfig" for xml file value works. However, sometimes we will get an error stating that this file cannot be found, and when we just try to delete (without saving and closing) and immediatelly try to put "config/Default.dtsconfig" again and hit next, we get an error and the path is now:
Right now, we have the production and development databases and ASP pages on one server. This is causing a strain on the performance, of course. Well, the finally, the customer agreed to get 2 less powerful servers. This is what we are planning to do with the 2 extra servers.
Server 1 : Original Server (fast) We will only have the Production Database (NO IIS)
Server 2: New Machine23 (faster box than Server 2) Offload IIS away from the production Database so we can allow more memory for Database processing.
Server 3: New Machine 3 (slower box we just got) Move the entire development environment to one server (SQL Server Database + IIS with all the ASP pages)
Does this sound OK? or should we have IIS sitting on one box (holding pages for Development and production) and the other machine just holds the Development database.
I would like to hear other opinions/stories on different IIS & SQL Server configurations to maximize performance.
I am trying to set up a TEST ENIVORNMENT for a reservation software package. I need this setup so that I can run various scheduling scenarios in order to optimize operations. Below I have been given instructions from the software vendor on how to set up my SQL database. However, I am a little confused on what to do for a couple of the steps. They are as follows:
***In SQL server 2000 enterprise manager on the test laptop, change the 2 path settings in the table SGCONFIG. The should be changed to C:StratagenAdept5Server. ** YOU WILL NEED TO DO THIS STEP EVERYTIME YOU RESTORE A BACKUP ADEPT5_CLASTRAN.BAK FROM THE PRODUCTION TO THE TEST ENVIRONMENT
QUESTION: How do you change the path settings on a table?
***Open the .udl files in the apps folder and the server apps folder and check to see that they are pointing to the test laptop server, not the production server.
QUESTION: What are the .udl files and how do you check to see if they are pointing at the test laptop server only?
Thanks sincerely for your help. I am trying to meet a deadline for a meeting tomorrow. Therefore, I am desperate. Please send me email. rtanner@clastran.com.
I am trying to set up a TEST ENIVORNMENT for a reservation software package. I need this setup so that I can run various scheduling scenarios in order to optimize operations. Below I have been given instructions from the software vendor on how to set up my SQL database. However, I am a little confused on what to do for a couple of the steps. They are as follows:
***In SQL server 2000 enterprise manager on the test laptop, change the 2 path settings in the table SGCONFIG. The should be changed to C:StratagenAdept5Server. ** YOU WILL NEED TO DO THIS STEP EVERYTIME YOU RESTORE A BACKUP ADEPT5_CLASTRAN.BAK FROM THE PRODUCTION TO THE TEST ENVIRONMENT
QUESTION: How do you change the path settings on a table?
***Open the .udl files in the apps folder and the server apps folder and check to see that they are pointing to the test laptop server, not the production server.
QUESTION: What are the .udl files and how do you check to see if they are pointing at the test laptop server only?
Thanks sincerely for your help. I am trying to meet a deadline for a meeting tomorrow. Therefore, I am desperate. Please send me email. rtanner@clastran.com.
What are the SQL 2005 Upgrade Paths? For example, is there a direct in-place upgrade for SQL 6.5? SQL 7.0? Can someone provide a link please? Thanks in advance.
I have created over a hundred reports and each of them are scheduled to output a pdf. The problem is this was temporary and each week I would like to output the reports to a directory that is based upon time. For example, I have 13 groups of reports and each group has 9 reports. Each group has it's own directory and folder. IE group A would be saved in something like \client filesgroup aweekly reportsMar 17 2008. The next week the reports for the same group need to be saved in \client filesgroup aweekly reportsMar 24 2008. This would be the same for each of the groups; group A, group B etc.
The question is there a way to set these paths dynamically or at least iterate through the subscriptions and change the paths?
I have an ActiveX Script Task in SQL Server 2000. It chooses one of two possible success-paths depending on if a file exist or not. (Part of the old ActiveX Script for choosing next step is below)
I need to rewrite this for a Script Task in SQL Server 2005 but it seems like it doesn't have this functionality and objects.
Does anyone know how to write the code for choosing the next step in a Script Task or knows another way to solve my problem?
The package is not suppose to fail if the file is missing, it's must succeed and that's why I need two success-paths.
Regards, Sara
'********************************************************************** ' Visual Basic ActiveX Script '************************************************************************ Function Main ()
Dim pkg Dim stpContinuePkg Dim stpExit
SET pkg = DTSGlobalVariables.Parent
SET stpContinuePkg = pkg.Steps("DTSStep_DTSActiveScriptTask_4") SET stpExit = pkg.Steps("DTSStep_DTSActiveScriptTask_21")
If ...... Then Main = DTSStepScriptResult_ExecuteTask stpContinuePkg.DisableStep = False stpExit.DisableStep = True Else ....... End If
I want to store each SQL Server's (prod, test, etc) connection string in XML to make deployment & configuration easy. However, the connection string must be encrypted. Does anyone have any ideas or suggestions on how to accomplish both. I know how to encrypt it but I dont think SSIS can work with it.
[[Disclaimer: I am by no means a highly-skilled DBA. I have just enough db skills to get my job done, which involves light sql server and oracle tasks. I appreciate the help, guys.]]
I am trying to replace the path section of filenames listed in a collumn of my database. The filenames are UNC paths, and all files have 8 digit filenames with the same extension (they are all tif images). The length of the paths in the filenames will change, however.
Here is what my old paths look like:
\imageimagesNYY-MMABCDEFGH.TIF
..and here is the new path (all of the images are being consolidated into one directory):
\als-imagealscom31imagesdbABCDEFGH.TIF
------------------------------------------------------------- I have tried using the following query to do this:
update image set filename=replace(filename,'\imageimages5 0-08','\als-imagealscom31imagesdb')
This returns insanely crazy results like 460239 row affected, when it should be about a thousand. -------------------------------------------- I have also tried the following:
update image set filename=right(filename,12) + '\als-imagealscom31imagesdb' where left(filename,21) = '\imageimages5 0-08'
This affects zero rows.
Please guide me down the one true path. My coworkers are clueless and I'm quickly running out of disk on the old box.
Ive just installed VS2005 Pro and SQL Server Developer edition.
Now I want to set the default paths for project, templates, settings. But I want for both VS and SQL files.
For example, right now the default folders for SQL Studio are:
.My DocumentsSQL Server Management StudioProjects
.My DocumentsSQL Server Management StudioSettings
.My DocumentsSQL Server Management StudioTemplates
.My DocumentsSQL Server Management StudioBackup Files
... and so on.
For VS Studio:
.My DocumentsVisual Studio 2005Projects
.My DocumentsVisual Studio 2005Templates
.My DocumentsVisual Studio 2005Code Snippets
... and so on.
I would like for all sub-folders(from SQL Server and VS) to be in one same folder(Development), for example:
.My DocumentsDevelopmentProjects
.My DocumentsDevelopmentSettings
.My DocumentsDevelopmentTemplates
.My DocumentsDevelopmentBackup Files
.My DocumentsDevelopmentCode Snippets
... and so on.
I have tried changing in each application´s options the default path for each one, pointing to each folder I listed above.
My problem is that both VS and SQL Server still keeps creating folders in the old locations, for example: ".My DocumentsSQL Server Management StudioProjects". This example happens when I create a new query, and try to save it, it automatically creates that folder.
It only works for a few of them, like the settings folder. Ive managed to create a single folder for SQL and VS setting files.
Is there a way I can join both Application folders? I want to keep both projects files in one folder, both setting files in one folder, and so on...
Currently I have an sql string which looks like this
INSERT INTO tblPDFFiles (fileType,PDFcontent) SELECT 'First test file', BulkColumn FROM OPENROWSET(Bulk 'C://Test.pdf, SINGLE_BLOB) AS BLOB
But the files i am trying to access is on a different shared server called 'test2008' now im told I can access this doing
INSERT INTO tblPDFFiles (fileType,PDFcontent) SELECT 'First test file', BulkColumn FROM OPENROWSET(Bulk ' \test2008 estpdf.pdf', SINGLE_BLOB) AS BLOB
Which I made sure had the pdf there I get the following error
SSIS package "Package.dtsx" starting.
Error: 0xC002F210 at Execute SQL Task, Execute SQL Task: Executing the query "INSERT INTO tblPDFFiles (fileType,PDFcontent) SELECT 'First test file', BulkColumn FROM OPENROWSET(Bulk '\test2008 est.pdf', SINGLE_BLOB) AS BLOB" failed with the following error: "Cannot bulk load because the file "\test2008 est.pdf" could not be opened. Operating system error code 5(Access is denied.).". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly.
Task failed: Execute SQL Task
Warning: 0x80019002 at Foreach Loop Container: The Execution method succeeded, but the number of errors raised (1) reached the maximum allowed (1); resulting in failure. This occurs when the number of errors reaches the number specified in MaximumErrorCount. Change the MaximumErrorCount or fix the errors.
Warning: 0x80019002 at Package: The Execution method succeeded, but the number of errors raised (1) reached the maximum allowed (1); resulting in failure. This occurs when the number of errors reaches the number specified in MaximumErrorCount. Change the MaximumErrorCount or fix the errors.
SSIS package "Package.dtsx" finished: Failure.
How do I give access to this other server to my SSIS package
I was wondering if anyone has a neat (preferably automated) method of creating small testing databases from large production instances. My requirement would be to copy the schema and a subset of configuration data from a production database into a test database. The subset of data would be a full copy of a subset of tables, rather than a subset of data within one or more tables. There is a mixture of SQL2000 and SQL2005 servers involved in this requirement. I'm familar with the scripting mechanisms of Enterprise Manager and Management studio and DTS packages, sufficent to perform a process like this manually, but want to productionise and schedule this process to be performed automatically. I'm sure this must be a commonly performed task, so I'm interested to know if anyone has a "best practice" for this requirement.
Another DBA on my team is trying to tell me that it is a widely accepted best practice to only have a single version of SQL server in the enterprise environment. This seems counter intuitive to me and I cannot find a confirming source for this assertion.
I would think that it largely depends on your environment but that once a new version of SQL server has been decided on for a given application, for whatever reasons, that new version should be used moving forward on all new development.
It also seems like that as other databases grow and hardware comes to its end of life that older databases be either migrated or ported to the new version as seems appropriate.
Is there any reason an enterprise should always be on X version and only X version?
Hi,It appears that binary_checksum can give the same checksum fordifferent strings, which is a bit worrying. (I guess the algorithm isthe problem in the context of a repeating pattern.)e.g.select binary_checksum('A'),binary_checksum('AAAAAAAAAAAAAAAAA'),binary_checksum('AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA A'),binary_checksum('AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA AAAAAAAAAAAAAAAAA')My question...Is this approach to generating checksums adequate for managing theobject scripts in the SQL Server to ensure that they haven't changed. Iguess that the probability of somebody making a change to a script andending up with the same checksum is almost negligible. Has anybody usedthis approach in an FDA validated production environment, i.e. 'no ifs,no buts'? Would it stand up to scrutiny?Any experiences, thoughts?RegardsLiam
We have multiple development environments on one machine for our VisualStudio .net projects.These are web applications so we have organized them by IP Address andHost Headers.When I installed Reporting Services, it installed into my default website (localhost) and that is fine.However, we want to be able to create our reports in our different .netprojects. The only projects that will allow me to create reports arethe projects using the default website.When I right click to create a new project, the Business Intelligenceprojects (reports) type is not there.Can I do this? If so, how do I configure it?
Hi there.I've been searching for this error specifically but I haven't found anything yet.I have these two tables (USERS and REQUESTS):USERS ( [LOGIN] [varchar] (10) NOT NULL , [NAME] [varchar] (20) NOT NULL)where LOGIN is the primary key.The problem comes when I try to create the "REQUESTS" table.In these requests there's one user who types the request. After one or two days, there's other user who aproves the request. The problem is that I need two foreign keys referencing the table "USERS".CREATE TABLE REQUESTS ([ID] [numeric](5, 0) NOT NULL ,[DATE] [datetime] NOT NULL ,[NOTES] [varchar] (100) NOT NULL ,[TYPED_BY] [varchar] (10) NOT NULL ,[APROVED BY] [varchar] (10) NULL) ON [PRIMARY]GOALTER TABLE REQUESTS ADD CONSTRAINT [PK__REQUESTS__07DE5BCC] PRIMARY KEY ( [ID]) ON [PRIMARY] GOALTER TABLE REQUESTS ADD CONSTRAINT [FK__REQUESTS__TYP__15702E88] FOREIGN KEY ([TYPED_BY]) REFERENCES [USERS] ([LOGIN]) ON UPDATE CASCADE ,CONSTRAINT [FK__REQUESTS__APR__12742E08] FOREIGN KEY ([APROVED_BY]) REFERENCES [USERS] ([LOGIN]) ON UPDATE CASCADEAnd SQL returns:Introducing FOREIGN KEY constraint 'FK__REQUESTS__APR__12742E08' on table 'REQUESTS' may cause cycles or multiple cascade paths. Specify ON DELETE NO ACTION or ON UPDATE NO ACTION, or modify other FOREIGN KEY constraints.Could not create constraint. See previous errors.Ok, after that, I tried creating a new table to store aprovals (Table with two fields: "REQUEST_ID" and "APROVED_BY").So, I removed "APROVED_BY" field from "REQUESTS" and its FK constraint.The same error comes up.I don't think this structure goes into "cycles" or "multiple cascades".How can I do this?Thanks in advanceRegardsRoland
I'm curious if there's a "best practice" for setting up the data directories MS SQL will use for each operation? I've allocated independent disks for things like C: (OS), E: (DATA), etc etc etc but I'm not familiar w/ MS SQL to understand how DBA's commonly configure the folders under each unique disk for things like DATA, LOGS, BACKUP, INDEXES, and TEMPDB. Should I have an identically name folder as show below in my example?
You can see I've just mirrored the drive name to a new folder under the partition so data is being written to: F:DATA and E:LOGS. Is this considered correct / good practice? I assume naming the folder in each mount point to whatever I logically called the drive is correct but if I should change how I configure my drive paths above. I'm trying to learn common good SQL Server practices and while I work on properly installing SQL Server 2012/2014, I want to make sure I configure my partition names SQL will utilize correctly.
Here a code for finding all minimal loops (cyclic paths) in a graph with vertexes of degree >= 3. Almost obviously that before seeking for loops we should eliminate from the graph all its vertexes of degree < 3 (degree of a vertex is the number of edges outcoming from the vertex). Note: there are no any 'parent' - 'child' nodes here. All vertexes are absolutely equitable. if object_id('g3')>0 drop table g3 if object_id('g3x')>0 drop table g3x if object_id('g3y')>0 drop table g3y if object_id('g3l')>0 drop table g3l GO create table g3y(v1 int, v2 int) -- ancillary table GO create table g3x(n int, v1 int, v2 int) -- ancillary table GO create table g3l(nl int, v1 int, v2 int) -- table for storing of 'detected' loops GO create table g3(v1 int, v2 int) -- table of test data with pairs of adjoining vertexes -- each vertex is named by an arbitrary number GO insert into g3 select 2, 3 union all select 2, 4 union all select 1, 4 union all select 3, 5 union all select 5, 6 union all select 1, 6 union all select 4, 7 union all select 6, 8 union all select 3, 9 union all select 1, 7 union all select 2, 7 union all select 1, 8 union all select 5, 8 union all select 2, 9 union all select 5, 9 ----union all /* select 2, 13 union all select 3, 13 union all select 13, 14 union all select 12, 14 union all select 12, 15 union all select 11, 15 union all select 11, 13 union all select 10, 11 union all select 10, 12 union all select 10, 14 union all select 10, 15 */ GO insert into g3 select v2, v1 from g3
declare @i int, @n int, @v1 int, @v2 int set @i=1
while 0=0 begin set @n=1 truncate table g3x truncate table g3y select top 1 @v1=g3.v1, @v2=g3.v2 from g3 left join g3l on (g3.v1=g3l.v1 and g3.v2=g3l.v2)or(g3.v1=g3l.v2 and g3.v2=g3l.v1) where g3l.nl is null if @@rowcount=0 break insert into g3x select @n, @v1, @v2
while @v1<>(select top 1 v2 from g3x order by n desc) begin set @n=@n+1 insert into g3x select top 1 @n, v1, v2 from g3 where v2=@v1 and v1<>@v2 and v1=(select top 1 v2 from g3x order by n desc)
if @@rowcount=0 begin insert into g3x select top 1 @n, v1, v2 from g3 where v2 not in (select v1 from g3x union all select v2 from g3x) and v1=(select top 1 v2 from g3x order by n desc) and not exists (select 0 from g3y where g3y.v1=g3.v1 and g3y.v2=g3.v2) if @@rowcount=0 if @n>2 begin insert into g3y select v1, v2 from g3x where n=@n-1 delete from g3x where n=@n-1 set @n=@n-2 end else begin insert into g3l select 0, v1, v2 from g3x break end end else begin insert into g3l select @i, v1, v2 from g3x set @i=@i+1 end end end select * from g3l order by nl Below is what we get:
7 5 9 7 9 3 7 3 5 Of course, in general case not all found by the code loops are minimal. But this is exactly my approach: firstly find any possible loops (avoiding excessiveness!!), then, in WHILE loop, try to mark out minimal loop(s) from intersection of two non-minimal loops... seems it will be an interesting t-sql job.
Hello,There are three tables:OS-GroupOFCompanies (Table1)GoC_GroupOFCompaniesID (PK)OS-Organization (Table 2)Org_OrganizationID (PK)OS-UnitAddress (Table 3)Unit_UnitAddress (PK)Scenario:(1)GoC_GroupOfCompanies has -one to many- relationship with Org_OrganizationID.(2)GoC_GroupOfCompanies has -one to many- relationship with Unit_UnitAddress.(3)Org_OrganizationID has -one to many- relationship with Unit_UnitAddress.Following Error message appeared after trying to save the relationship (3) described above.'OS-Unit-UnitAddress' table saved successfully'OS-Organization' table- Unable to create relationship 'FK_OS-Organization_OS-Unit-UnitAddress'.ODBC error: [Microsoft][ODBC SQL Server Driver][SQL Server]Introducing FOREIGN KEY constraint 'FK_OS-Organization_OS-Unit-UnitAddress' on table 'OS-Organization' may cause cycles or multiple cascade paths. Specify ON DELETE NO ACTION or ON UPDATE NO ACTION, or modify other FOREIGN KEY constraints.[Microsoft][ODBC SQL Server Driver][SQL Server]Could not create constraint. See previous errors.NB: ON UPDATE is much needed.How to manage the situation?Please guide.ThanksSuryaPrakash****************************************** This message was posted via http://www.sqlmonster.com** Report spam or abuse by clicking the following URL:* http://www.sqlmonster.com/Uwe/Abuse...f935efcabb55ee9*****************************************
I am on SQL Server 2008 R2. I have a table that contains a field called [Location]. In that field is a UNC path to the physical file on the repository. Is there a way in SQL Server that I can say give me the select sum([Location] UNC file) where criteria? I saw some posts about xp_filesize or xp_GetFileDetails, but I do not see them in master. I am unable to add anything and wondering if there is any native functionality that would allow me to accomplish this!?
We are upgrading a production database server to new hardware. The server is currently running SQL Server 2000 Standard Edition. We are thinking about installing SQL Server 2000 Enterprise Edition, however that would mean the test server (2000 Standard) and production server (2000 Enterprise) have different edtions of SQL Server. How much of a risk does this present? Later in the year we would upgrade test to SQL Server 2000 EE, but for a couple of months the environments would be different.
If I have stored procedures that reference linked servers, is there any way to avoid changing the procedures when moving between environments (dev, qa, prod)?
Do i resort to dynamic sql? Use case statements to switch between environments?
I modified an existing package created by someone (who left the organization). Its a ftp bulk insert from flat files to the tables. The need was to updates 2 tables out of 9 with additional columns. I made the changes to the package and it runs successfully on my laptop.
After that I build the package and tested it on the development server, although the package runs successfully but the data is written to my DB tables in my laptop instead of development server DB tables!.I see "." in the place of the connection manager server name. The DB name is the same in all the three environment (local, development & production). There is a configuration table used in the package.
Is there a way to simulate hosting three environments on one Report Server. Due to resource limitations, I would like to set up three environments DEV, TEST, STAGE on one report server such that the data source for each environment would be different. Can I use a different folder structure for each and then deploy the reports by providing server url accordingly?
e.g. http://servername/DEV_Reports or should it be (http://servername/ReportServer/DEV_Reports)
SSIS is behaving differently in different environments but the code is same.
One thing is nor working correctly that is I am converting a string data type column to float data type in data conversion. In our local environments the package is working fine but in production environments it is not working correclty. It is unable to convert the data it is throwing an error.
"The data value cannot be converted for reasons other than sign mismatch or data overflow"
Is it possible to config environments ( dev,tst,prd ) in  VS Tabular solution , like it is done in SSIS ?  i find Variable tab , but i dont find the way to set a connection (string) to be change dynamically between the environments.
Is there documentation somewhere about multiple execution paths in SSIS control flow? I didn't find documentation anywhere. I have a situation where I have two tasks that take considerable time, but could be executed in parallel (to speed up things) and I was wondering whether SSIS supports parallelism.
To illustrate the issues in simultaneous execution, I created a test SSIS package. In the package, I have five tasks, let's call them T1, T2, T3, T4 and T5. The taks are connected with "green arrows" like this:
T1->T2
T1->T3
T2->T4
T3->T4
T5 is not connected. The tasks can be e.g. Send Mail tasks, that's not relevant to this issue. I put a breakpoint in each task and execute the package.
When I execute package, T1 and T5 become active, i.e. the arrow that displays where the package execution currently is, is in two tasks simultaneously. Now F10 (step over) doesn't seem to work "Unable to step. Not implemented". If I press F5 nothing happens. After I press F5 for a second time tasks T1 and T5 and executed. Why don't they execute with the first pressing of F5? I would additionally like to know whether these two tasks are executed in parallel or sequentially, i.e. in the same thread or in two threads? Is there documentation of this?
The execution stops at T2&T3. Again, pressing F5 doesn't do anything, but the second time I press F5 T2 and T3 are executed.
