We are working on a project, (C#.Net 2003, MS SQL 2000) where database is growing as 2 to 3 GB per day (Scanned Documents are storing in Database). There is only one table in the database. Currently database size is grown upto 200 GB. For safety reason we are planning to split the database accoring to one key field in the table say Book_no.
Please guide/suggest me how to split the database now on the SQL query like "SELECT * FROM master_records WHERE book_no=1"
And later how to merge/combine all these splitted database into one.
I thought of using "Data Export" and later "Import with Append" but don't know whethere it will effective or not? Any method or tool available to Split the database table and later combine them to make again a single one?
I have one table 'tbUser' having coloumnsUserID,UserName,Category UserID UserName Category 1 Shailaja H 2 Anagha H 3 Raju L 4 Kiran L I want output as UserID UserName HigherCategory LowerCategory 1 Shailaja H 2 Anagha H 3 Raju L 4 Kiran L
Hi all,Here is my situation.I have an application that uses SQL 2000. The client has a programthat uses MSDE. The client MUST have the MSDE version of the Database.What I have done is to create a SQL 2000 Database that is exactly likethe MSDE version. What I wish to do, is at regularly scheduledintervals, update the SQL 2000 Data with the newest MSDE Data.On my clients server, they have both MSDE and SQL 2000 installedthe msde has a user name and password to access it, whereas the SQL2000 uses authenticated mode to access it.What I'm trying to do is to write a bunch of SQL Queries, that I canrun that will copy over the new Data, but I'm having problems accessingthe msde database in query analyzer from the SQL DatabaseCan anyone give me hints on the best way to try thisThanksScott
My company use SQL server 2005 standard. considering deal with huge data, how if we want to split data (date range yearly or monthly) in order to ease transaction. that's simply for us to use query, but how if we want to split data that can be easily execute by operator (non-admin privilege). Is there any another way?
i am facing a problems, hope that anyone who know the answer can help me.
i am using sql server 2000 and vs.net. i wan to retrieve a data from the table which only select 3 top row based on their different contingentID. the table have different contingent and each contingent have 5 row data. i wan to retrieve 3 top row based on the total group by contingentID and sum up the total to order by the total and generate the ranking based on total. what i can do is only retrieve every row of data in that table. so anyone know the answer please help me, or if u dun understand my question , can message me..thank your
hai all, This is my first question to this forum. here is my situtation: I am into report testing I need to test a report for which i have write a query,iam using qery analyser for runing query
Database : sql server tabel name :job_allocations column naME :technicain code
Based on techincain code in joballocation tablei need to get technician cost from other table for the particular technician.
Based on the technician code user chooses column will be updated if single data will be TC01 if more than one then data will be TC01:TC02:TC03
user can choose any number of techincian for a job
MY problem is :How to split tha when there is multiple technician and calculate cost for the job Ineed it in single excecution query
I'm trying to figure out how in a Data Flow Transform I can split some data.
I have data coming through that has a PK (col1) and datetime col (col2). This data may contain multiple rows for the PK, col1. I want to be able to take the min datetime for each row for col1 and send down one path and all other rows down another path eg. There are 20 rows in the Data Flow coming through. Col1 has value 1 for first 10 rows, value 2 for remaining 10 rows and there are different values for Col2 for all rows. Take the Min value in Col2, where col1=1 and also the min value in Col2, where col1=2 and send down one path. Send all other rows down another path.
I thought of using the Conditional Split Transform but my Expression knowledge isin't experienced enough (I'm looking into this) and I'm not sure if it can be done.
I also though of using the Multi-Cast Transform and then using the Aggregate function, on each data set. While I can do this easily for the first data flow.
SELECT col1, MIN(col2)
FROM tbl1
GROUP BY col1
which is easily done in the Aggregate transform, the other side of the data flow I can't see how can be done using the Aggregate Transform
SELECT col1
FROM tbl1
WHERE col2 NOT IN (SELECT MIN(col2) FROM tbl1 GROUP BY col1).
Are either methods feasilble or is there another way. I want to avoid putting this data into temp tables in a SQL database and manipulating the data from there. The data has been extracted from a flat file source. Any help and ideas welcome.
I'm trying to get a database from SQL2005 to SQL2000. I already recreated the database in SQL2005 and I'm just trying to get some data from just a few tables.
It doesn't appear the DTS or any export functionality is available in my version of SQL Express 2005...so how do I do this. I've tried installing everything I can find.
I've also tried BCP but I keep getting these errors:
SQLState = 08001, NativeError = 53 Error = [Microsoft][SQL Native Client]Named Pipes Provider: Could not open a connection to SQL Server [53]. SQLState = HYT00, NativeError = 0 Error = [Microsoft][SQL Native Client]Login timeout expired SQLState = 08001, NativeError = 53 Error = [Microsoft][SQL Native Client]An error has occurred while establishing a connection to the server. When connecting to SQL Server 2005, this failure may be caused by the fact that under the default settings SQL Server does not allow remote connections.
This is a LOCAL database and I have tried -T for trusted, I have also tried a username and password combination. The database is set to Allow Remote connections. Nothing is working.
I would like to know if it is possible to split a database across different servers, in the same manner you can split it over multiple drives on the same server We are trying to balance the load cause we are finding that the current server can't handle the load
How can i write a query to split a database column and shows 2 new columns. In my database column I have 2 mixing items and need to split out to 2 columns. Normally I have to write a query and change parameter and run another query. For example a database column with average number and range number. Thanks Daniel
What's the best way to convert a large set of records from a simple schema where all fields are in one table to a schema where fields are split across two tables? The two table setup is necessary for reasons not worth getting into here.
Doing this via cursor is pretty straightforward, but is there a comparable set-based solution?
Here are sample create table commands. Obviously, the example below is simplified for discussion purposes.
-- One record from here will produce a record in TargetParentRecords and a record in TargetChildRecords for a total of two records. CREATE TABLE OriginalSingleTableRecords ( ID INT IDENTITY (1, 1) NOT NULL,
ColumnA VARCHAR(100) NOT NULL, ColumnB VARCHAR(100) NOT NULL,
I have Data split into 3 text files with 3 fields repeated in each to link then (key). I want to import this data into one table. I used DTS to create 3 tables with the data. Now i want to combine the 3 tables into only one (that i already created). How can i do this? Note: the field names in the source tables are different from the destination table.
i using SSIS to update data on tables based on another tables as follow
start connect to DS --> using Lookup to located the key fields on the 2 tables --> on error start insert into the second table else if the recorde is already exist start update
my problem is Performance
--source table rows count almost 60 million
--distination table almost 3 million
my package execute from 2 days and still working till now updated 122,000 and insert 4 million
I found string from net how to split column data to row
SELECT A.JbIDFull, A.ProdID, A.OrderQty, A.OtherDetails, A.OrderDate, c.Item FROM (SELECT JbIDFull, DeptID, ProdID, OrderQty, OtherDetails, OrderDate, LamORSteachType, JBID, RMIDs, RMQty FROM tblJobCardforProduction WHERE JBID = '2' AND DeptID = '3') A CROSS APPLY dbo.SplitStringNEW(a.RMIDs,';') b CROSS APPLY dbo.SplitStringNEW(a.RMQty,'/') c
When I apply one cross apply it's working fine but when i apply for one more column is replacing three time one row, this data i want to split RMIDs and RMQty with same JbIDFull
we have some reference tables in in a specific database. that other applications need to have access to them. Is it possible to create a view in the application's database to retrive data from ref database while users just have access to the application Database not the view's underlying tables?
If you are having trouble accessing Meta Data Services on W2k3 and SQL 2000 and get the error that the "msdb database could not be opened" I found that hot fix 912812 on the operating system is the culprit. Remove it and Meta Data goes back to working, although I am being asked to approve the ActiveX control every time the page refreshes.
Hope this helps somebody, to took me long enough to track it down.
If anyone has a resolution to the ActiveX question I'd love to know the answer.
I am trying to break apart a list of filenames that was inserted into a database. It only breaks out the first one then moves onto the next record. If I do them individually then seem to work but not the whole table when queried. I need to break out each file into a temp table then insert them into a documents field in a database.
my filenames look like so and can have from 1 file name to 10 file names in the string.
This is my current method, I needed to create a cursor around it to go through all the records, split out the filenames and insert into a temp table. But if there is a better way ill do it. The problem with this is only the first file is getting inserted into the temp table and nothing else even if the filename has 4 files in it.
Create table #tempFiles (OldStrId int, OldPercent int, strfilename varchar(max), RequestId int, OblId int) declare @OldStr int, @OldPer int, @FileName varchar(max), @intcount int; Declare filenames CURSOR FOR Select intSTRBonusID, intPercentID, strFileName from tblSTR where strFileName > '' UNION ALL Select intSTRBonusID, intPercentID, strFileName from tblSTRHist where intPercentID in (61,62) and strFileName > ''
Hi This is probably a very basic question for most people in this group. How do i split the data in a column in to 2 columns? This can be done in access with an update query but in MS SQL server I am not sure. Here is an example of what i want to acheive
Hi, I have a scenario, where I have a string column from database with value as "FTW*Christopher,Lawson|FTW*Bradley,James". In my report, I need to split this column at each " | " symbol and place each substring one below the other in one row of a report as shown below .
I'm using sql 2008 and triying to build a dynamic sql script to split the records 50/50.I know using newid() with order by clause selects randomly but how should I build the select statement to split the data 50/50 so i don't need to run the script manually everytime ?
1 ,AU-Australia 1,MM-Myanmar 1,NZ-New Zealand 1,PG-Paua New Guinea 1,PH-Phlippines
Note: we are getting source data from sqlserver tables.
I googled and found below way but did't get the output as required
SELECT A.id, a.country, Split.a.value('.', 'VARCHAR(500)') AS String FROM (SELECT id, country , CAST ('<M>' + REPLACE(country, ' ', '</M><M>') + '</M>' AS XML) AS String FROM #t3) AS A CROSS APPLY String.nodes ('/M') AS Split(a);
This is probably an easy question, and I just can't find the solution. I've searched extensively, but I am probably just not searching for exactly what I need.
Basically, I have a Conditional Split. What I need to do is for each row coming out of my split, I need to SELECT some data from another database based on one of the fields and then place the data from the DB into a file for later processing.
Seems pretty simple, considering the power of SSIS. Using tools such as OLE DB Command didn't help - the data that comes out of the OLE DB Command is the input data, not the data returned by the command.
- 500 GB DW - 5 GB in smaller DBs - 220 GB TempDB - 350 GB in Log files.
My machine is Fujitsu Primergy 64 cores (with HT) and 192 GB RAM.
I have several IO locations:
- 540 GB in-server HDD 15k RAID10 - 1 TB HDD 15k RAID10 on SAN (separete controller) - 2 TB HDD 15k RAID10 on SAN (same controlller as below) - 800GB SSD RAID10 on SAN (same controller as above)
Data warehouse has 2 fact tables that are absolutely crucial and quite large.
Now i want to organize DB into several Filegroups and put them on different drives. Filegroups I'm thinking of:
- FILEGROUP1: for 1st crucial Fact Table - FILEGROUP2: for 2nd crucial Fact Table - FILEGROUP3: for tempDB - FILEGROUP4: for dimensions data - FILEGROUP5: for the rest of facts data - FILEGROUP6: for dimensions indexes - FILEGROUP7: for the rest of facts indexes - FILEGROUP8: for 1 log file of one smaller DB (its in full-recovery and its quite large) - FILEGROUP9: for the rest of log files - FILEGROUP10: others
How should I organize them across available drives? I was thinking about sth like:
I know that having multiple filegroups on the same drive is pointless regarding performance, but in future i could actually add some more drives, so i want to separate them now.
Also - how much files per filegroups should i create? Considering 1 or 2. Except TempDB where I am going for 4.
I have a field in a table that contains addresses e.g
15 Green Street 5F Brown Steet 127 Blue Street 1512 Red Road
I want to output the numbers into one column and the address to another column as i need to produce a report that only shows streets and roads but no numbers.
So basically no matter how many characters before the first space which can be numbers or letters i want these output into two columns.
I have a source table which I'm splitting 3 ways based on a column value, but the target is the same OLE DB destination table. One conditional path is to a Multi-Cast two way split to same OLE DB gestination table. The default split is to a flat file for logging unknown record types. For a test I have data for only the 3 column values I want, but I'm having trouble with the process completing. If I pre-filter the data going into the source table by one or two values I can get the process to complete even if one split is to the multicast. If I include all three data types in the source table, I get different results depending on the order in which the conditions are specified - sometimes only two split paths are executed; other times all three are executed, but in some cases only one path of the multicast split is executed. In any case, when the three source data types are used in the test, the process never competes - the pathes are in a yellow condition and never complete.
Am I creating some kind of deadlock situation by having the source data directed to the same target table via 4 splits? Any help you can provide is appreciated. Thanks.
I am using conditional split Checking to see if a record exists and if so update else insert. But this cause database dead lock any one has suggestion?
I'm encountering a very peculiar situation when I'm trying to compare source and target data using conditional split. Following is the Data Flow and how I'm trying to achieve this.
Source Data : Col_A (PK) Col_2 1 100 8 500 Target Data : Col_A (PK) Col_2 1 100 3 700 8 500 Look-up Target on Col_A to check for existing records. Now we have four columns in Look-up match output: Col_A, Col_B, Lkp_Col_A (Target Col), Lkp_Col_B (Target Col).
Conditional Split: Compare Col_B with Lkp_Col_B
Update target if there is any change in the existing value of Col_B.When I'm running the package for every record in source, the conditional split fails and even when there is no change in Col_B, some of the records (Not all and quite randomly) get updated with the same value. If I run the package for few records, it works absolutely fine.
