Integration Services :: Schema Separation - Accessing Multiple Databases
Nov 8, 2015
We have a system that uses 3 databases, one for Membership db standard MS membership only the application has access to that data, one with User Data which we would like to make multi-tenant using Schema-Separation, and a third read-only reference db which is Common Market data for all users.we anticipate Tenant numbers in the thousands.Current we have multiple queries which create joins between the Main db and the Reference database using something like
Selec S.*, M.ScheduleDate, M.substation from Sites S left outer join Market.dbo.MarketUnit M where S.MarketUnitID = M.MarketUnitID
i'm planning to have a new schema for each Tenant on the Main Database, so I would create a Schema T1 for the first customer, a user T1User with access to T1 schema. and grant T1User access to Market.dbo. My First question is are there any concerns about the above T1User setup? My second question is, are there any tools which would automate the setup of the multi-tenant with schema separation, or should I just script the whole Main Database schema creation and replace schema name globally and then execute the script?
My Third question, how about upgrade and updates... currently using VS to compare dev/qa/prod database to identify changed which need to be promoted, and pushing updates... this could be a big pain to promote code to thousands of Schemas. grantedwe will likely keep the overall number of schemas spread over different SQL servers.
View 2 Replies
ADVERTISEMENT
Apr 15, 2015
How to read multiple excel sheets in same excel file with different table schema.
Basically need to load data into tables from these excel sheet.
So I know how to dynamically read multiple excel sheets in same excel file with same table schema and load into one table.
But how to do this dynamically for multiple excel sheet with different table schema and load into different tables?
View 7 Replies
View Related
Jul 18, 2007
I just want to verify my understanding of schemas...
Using separate schemas to segregate data is only a logical mechanism. Separate schemas do not physically segregate data onto different groups of pages or files. Partitioning is the mechanism for physical segregation of data.
Is that everyones understanding as well?
View 1 Replies
View Related
May 10, 2005
I am using SQL Server 2000.
I am working on a project where there will be multiple databases on a single instance of SQL Server. Each database will have the exact same schema but will be accessed by different groups of users.
What is the easiest way to sychronize changes between the databases, so that if I add a column to one database, it will be reflected in the other databases. If I add, remove, or alter a stored procedure, I want the change to be made in the other databases. I want the data in each database to remain isolated. In other words, I do not want replication of the data, only the schema of the databases. I would like to have a single "master" database that I use to make any schema changes and all the other databases be schema mirrors of this database each with their own data.
I have looked into SQL Server replication, but this didn't seem to work the way I wanted and I wasn't able to publish column changes etc.
View 3 Replies
View Related
Jul 23, 2005
I'm looking for a tool that can extract only the schema from a databasein a form that can be used to generate that schema in another emptydatabase. This is to facilitate our disaster recovery processes wherewe need the objects only, not the data, and need to replicate this toour disaster recovery site over the WAN. There are plenty of toolsthat can handle a single database, but does anyone know of any toolsthat could handle multiple databases where many of the objects (storedprocs and views) are dependent on objects in other of the databases(tables). This is a home-grown ETL suite so making changes to the codeto remove these dependencies would take way too much effort. I amlooking for something that can either extract the schema for all 3databases and handle the object creation ordering to account for thedependencies (a simple method would be to extract by object type acrossall databases, e.g. tables for all dbs before views before procs), or abackup/restore tool that allows you to restore the objects only withoutdata. Worst case we could write something to generate the DDL or useSQL DMO, but ideally we would prefer to purchase a (relativelyinexpensive) tool to do it.Thanks,Simon
View 3 Replies
View Related
Nov 22, 2007
Hello,
My next project(in VB.NET 2005) would involve accessing different DBs and tables in remote SQL servers
What are the steps that need to consider first? Are there any settings that needs to be done in the servers?
k
View 1 Replies
View Related
Aug 8, 2006
I have several database files, all in the same directory. I want to write SQL statements that access two databases at once. For example, if only one database would be involved, I would write:
SELECT Test_Source_ID_column, Data_column
INTO Test_Destination_table
FROM Test_Sources_table
The above query copies two columns (Test_Source_ID_column and Data_column) from Test_Sources_table into a brand new table called Test_Destination_table.
Now the above query is fine for both the old and new tables being in the same database. But what if Test_Sources_table is in file Old.dbf and I want the new table Test_Destination_table to be in New.dbf?
Both Old.dbf and New.dbf are in the same directory. Therefore, there is no security issue about using ......etcpasswords. I should therefore be able to use relative paths. So, I also want to know how I can do that select on two databases in the same directory without typing long, long paths.
Is there a way to do queries that access multiple database files (all database files being in the exact same directory) AND use relative paths or no paths (and certainly not type a full path)?
NOTE 1: I'm using SQLEXPRESS, therefore do not have the SQL Server 2005 tools. I have Visual Studio 2005 Developer Edition with its tools.
NOTE 2: One table is accessed through ODBC. The other table is accessed through SQLEXPRESS.
View 5 Replies
View Related
May 16, 2006
Hi ,
I have around 5 databases with same structure used to store data for different locations and services.
I have created an intermediate database which will have all the stored procedures to access data from above 5 databases.
My queries are common to all the databases. so,I would like to pass database name as an argument to my stored proc and execure query on specified database.
I tried this doing using "USE Databasename " Command. but it says ...We can not use "USE command " in stored proc or triggers. so ..what i did is ..
--------------------------------------------------------------------------------------------------------------
CREATE PROCEDURE TestDB_access(@dbname varchar(100))
AS
BEGIN
DECLARE @localDB varchar(100)
Set @LocalDB=@dbname
if @LocalDB='XYZ'
EXEC ('USE XYZ ')
elseif @LocalDB='ABC'
EXEC ('USE ABC')
Select * from Mytable
END
---------------------------------------------------------------------------------------------------------------
When I run this from my database , it gives me an error "unable to find table "mytable".
What is the best way to make my queries work with all the databases.
Thanks in advance
Venu Yankarla
View 4 Replies
View Related
Nov 7, 2007
I recently installed Integration Services on one of our SQL 2005 servers running SP2. For starters I keep getting a warning message during the system test that indicates that there is a difference between the version that is installed and the version I am trying to install. I am using the same media I used for the initial server installation. I am assuming that this message is showing up due to the fact that the media is pre SP 2 and the server is now running SP2 allong with build/hotfix #3200. None the less the installation seems to have completed without errors. The problem I am having is when I connect to Integration services and try to expand the different directories I get the following error
TITLE: Microsoft SQL Server Management Studio
------------------------------
Failed to retrieve data for this request. (Microsoft.SqlServer.SmoEnum)
For help, click: http://go.microsoft.com/fwlink?ProdName=Microsoft+SQL+Server&LinkId=20476
------------------------------
ADDITIONAL INFORMATION:
Library not registered. (Exception from HRESULT: 0x8002801D (TYPE_E_LIBNOTREGISTERED)) (Microsoft.SqlServer.DTSRuntimeWrap)
------------------------------
Library not registered. (Exception from HRESULT: 0x8002801D (TYPE_E_LIBNOTREGISTERED)) (Microsoft.SqlServer.DTSRuntimeWrap)
------------------------------
BUTTONS:
OK
------------------------------
I have tried a number of different fixes that i have found on the web but none have resolved the issue. If there is anyone out there that has experienced this issue and has been able to resolve it your feed back would be greatly appreciated.
Thanks,
Brian
View 2 Replies
View Related
Oct 6, 2015
How can I get table name and schema names of the source and destination in ssis package to insert into audit table....??
View 3 Replies
View Related
Aug 10, 2015
Here is my requirement, How to handle using SSIS.
My flatfile will have multiple columns like :
ID key1 key2 key3 key 4
I have SP which accept 3 parameters ID, Key, Date
NOTE: Key is the coulm name from the Excel. So my sp call look like
sp_insert ID, Key1, date
sp_insert ID, Key2,date
sp_insert ID, Key3,date
View 7 Replies
View Related
Jun 16, 2015
I have a requirement where in i have around 15 different flat files , filenames are fixed but folder path can be changed(i think i should use a variable for folder path). These 15 files data should go to their respective tables in the database.
Whether I need to create separate data flow task for each file or separate package? In addition to these, example : while importing product data into product table, if product ID already exists, we need to ignore it and upload only the new records.
View 4 Replies
View Related
Aug 25, 2015
I have an excel file that has multiple sheets and I need to import data from each separate sheet to a separate table using SSIS.
E.g. Sheet A data should go to Table A and Sheet B data should go to Table B and so on. Is it possible to do this with out using script task.
View 6 Replies
View Related
May 8, 2007
Hi!
We have multiple databases (1 per client) and we want to use reporting services for ad-hoc reporting. Our authentication is done by using client's database so security is custom.
What is the best way to set this up? Is it possible to somehow allow users/admins to use only their database and never see any others? I know we can install a separate instance of RS and do that but I am wondering if there is a better way to run multiple clients doing different reports using exclusively their databases without having to create a new instance.
Thanks
View 13 Replies
View Related
Jun 19, 2008
I have an XML document that I am trying to use within a data flow (XML Source)--the problem is that I keep getting an error message stating that the XML document has multiple namespaces and therefore, SSIS will not create the XSD for me. If I take out the second namespace, I am able to successfully get the task to execute, but this XML is created with 2 namespaces and there is no way to get around this (I cannot get the report server to change these parameters)--my question is: know a way to get SSIS to handle multiple namespaces so that I can process this XML document and extract the necessary data elements from it.
View 11 Replies
View Related
Oct 26, 2015
1. SQL 2008 R2
2. Maintenance Plan -
a. Daily, Weekly, Monthly schedule
b. Full Back Up - All Database, To Disk, for every database and Disk:DirectorySubDirectory*.bak Compress backup
3. SQL Agent Job
a. Runs the Sub_Plan's at scheduled time I've allocated for each Maintenance Plan.
4. These creates Multiple .bak for all database's that list in the directory.
How can I zip them into a compressed Directory.
View 13 Replies
View Related
Mar 31, 2009
I have a situation where i need to unpivot multiple columns using ssis. The data looks like
Name Age products1 products2 orders1 orders2
abc 23 cycle radio 12 24
as
Name Age Products orders
abc 23 cycle 12
abc 23 radio 24
Is it possible to do this using the unpivot task in ssisMy actual data is has 18 columns which needed to be unpivoted into one and another 18 into another one.when using unpivot task it gives an error saying only one pivotvalue key is allowed.
View 6 Replies
View Related
Oct 14, 2008
I'm pretty new to SSIS but I've managed to cobble together a number of individual packages to refresh SQL tables from a 3rd-party database.
Now, what I'd like to do is have a single package that I can use to invoke each of the individual ones. Since it will run on a quad, I'd like to invoke them such they'll run in parallel.
View 2 Replies
View Related
Sep 21, 2015
I have a sequence container in my Package and this sequence has more than one control flow tasks.
Can I create the checkpoints such that only the failed component inside the sequence container runs again and not the other successful components/tasks in the sequence container?
View 3 Replies
View Related
Dec 3, 2015
I have 5-6 table and wanna to make sure if I'm using any of them in my Packages and I do have 100 Packages.shortest way to search these tables in Packages.
View 2 Replies
View Related
Oct 26, 2013
I have an EXECUTE SQL Task, which gets a result-set from SQLServer using OLEDB Connection.
The result set is mapped to an object variable , say @[User::FilePath]
There are 33 row is the above resultset.
Then, I have a For-each loop, inside which, I have a Script task .
I am trying to put the above @[User::FilePath] recordset into a DataTable using DataAdapter.Fill() function. I perform some read operations to its rows.
The problem is , in the First Iteration of For-Each-Loop, the number of rows in data-table shows 33.
But from the Next Iteration, it comes out to be 0. (ZERO!!)
This causes my package to fail.
View 5 Replies
View Related
May 11, 2015
I need to do something like this in SSIS:From one SQL table I need to get some id values, I am using a simple sql query:Select ID from Identifier where value is not null.I've got this result:As a final result I need to generate and set a variable in SSIS with the final value:
@var
= '198','120','ACP','120','PQU'
Which I need to use later in a odbc expression.How can I do this in SSIS?
View 4 Replies
View Related
Apr 23, 2015
Using sql server management studio I want to do 2 things.
- Import all txt files from a directory as tables into my DB.
- Add an additional field to each new table which contains the table name.
View 10 Replies
View Related
Aug 20, 2015
I have an SSIS package in VS 2010 that uses flat files to load database tables. I would like to check for the flat files existing before continuing to run the package. The flat files each have their own connection manager. I was wondering if I could use the connection managers to determine the file names instead of creating a Script Task and hard-coding each of the file names to check.
View 4 Replies
View Related
Aug 31, 2015
I have two records in the source with information ID, RevisionID, Description, Region
There are two lookup files one with ID,Description amd other with ID, Region
I wish to update my two source records with performing lookup with these two files.To get the correct description and region data. How to do this in ssis DFT.
View 4 Replies
View Related
May 21, 2015
I was to split each record into multiple columns. The problem is some records need to be split into only 1 column, others may need to be split into more. Also need to remove the "/"'s. This is all dependent on where a "/" is found. Been beating my head for a while and getting nowhere.
So:
create table #foo (myPK int, c1 nvarchar(425))
insert into #foo values (1,'/folder1')
insert into #foo values (2,'/lvl1/folder2')
insert into #foo values (3,'/folder1/lvl2/folder3')
insert into #foo values (4,'/f1/folder2/lvl3/fldr4')
Should return as:
folder1
lvl1
folder2
folder1
lvl2
folder3
f1
folder2
lvl3
fldr4
View 10 Replies
View Related
Sep 9, 2014
I have multiple sequence containers in my package. I only want to have one sendmail task for the failure/completion of the package. If I put the sendmail task in the last sequence container and the first seqence fails, the sendmail task will not be reached and therefore, no email will be sent out.Is there a way to have one sendmail task for all the sequence containers and allow it to send mail regardless of what sequence fails/completes?
View 4 Replies
View Related
Apr 10, 2007
Hello!
I'ave got a problem of setting more than one Variable in ReadOnlyVariables Property of ScriptComponent...I provide comma separated list of names ( As described in the help ) byt VS Studio Editor can not be opoened claiming that there is no a variablle with such a name...Looks like it doesn't treat the list as a collection of names...
Please help.
Vladimir
View 4 Replies
View Related
Jun 11, 2015
I am able to collect data from Progress DB, using ODBC Connectivity. The problem I am facing is, i have to iterate thru multiple servers. How do i configure ODBC source dynamically. It creates problem. Using expression, i tried to set the connectionstring dynamically, but it fails.
View 2 Replies
View Related
Oct 28, 2015
I have a requirement to load multiple flat files in target table .
I have created the package which used to load files into target table using For each loop container.
But now requirement has been changed now I have to take only those files from table where status="Success" and max JobId. By the query I am to get those records which need to load into table.
Below query I am using to get the files which need to load.
select [JobLogKey],[SrcNm],[DestNm]
FROM [ConfigRep].[dbo].[JobLog]
Where [JobId]=
(Select Max(cast([JobId] as Int)) Jobid
FROM [ConfigRep].[dbo].[JobLog]
Where [JobStat]='Success')
Output:-
JobLogKey SrcNm DestNm
268 H:Data PlatformSource FileClient2LocHGSSpecLocation.txt Location.txt
269 H:Data PlatformSource FileClient1LocHGSSpecLocation.txt Location.txt
I have to load using above 2 files which are under SrcNm. I have created one variable called FileToLoad as Object and mapping to result set of above query. I have create JobId,SrcNm and DestNm variable to catch the record at every loop. I have created 2 For each Loop container
Below screen shot of outer Foreach loop. Till here Its working fine. Inner for each loop container not executing any task under that. How to get it done.
View 3 Replies
View Related
May 29, 2015
how do you load the multiple flat files to into destination dynamically?
View 9 Replies
View Related
Oct 12, 2014
I have one scenario
Table
Col1. Col2
1. A,b,c,df,ghf
2. C,b
3. B
Output should be
Col1. Col2
1. A
1. B
1. C
1. Df
1. Ghf
2. C
2. B
3. B
View 9 Replies
View Related
Oct 25, 2015
I have been tasked to do the following using SSIS.
We received two csv files each week and we would like to load these files to two different sql server tables using SSIS.
These files should be archived into a folder after each load.
How can I achieve this?
View 6 Replies
View Related