Design Data Storage For Feature Similar To Facebook Groups
Mar 13, 2008
Ok so facebook groups have 100,000's of members. Members can be part of an unlimited number of groups, and a group can have an unlimited number of members.
Comma Deliniated String seems absurd. Many-2-Many Database relationship seems like it won't scale well t the 10's of thousands and 100's of thousands of members (especially if you have 1000-5000 groups). A table for each group would work but thats a bit over the top in my opinion. XML file doesn't seem to be any better than the above options.
I am no database guru, but I can't figure out a scalable method of doing this, be it with or without a database. I need something that can support 10 groups that have 20 members each OR 1000 groups with 100,000 members each.
Any help, suggestions, or kicked in the right direction would be most appreciated.
I am building partitiong tables, partitioning on different file groups:
the question is:
Partitioned table referred to old data that are not frequent accessed for reporting can be stored on separate location(External storage, tape and so on) or to make partitioning functioning must all file groups must be presents?
If not, how can I separate old data from current ones (still using partitioning) to reduce the size of DB?
What it is the best for storage data and easy to access it when needs arise (eg reporting): Tape, external storage, others?
select distinct  case      when LastStatusMessageIDName = 'Program completed with success' then 'Office 2013 SP1 Installed Successfully'      when LastExecutionResult = '2013' then 'Machine Does not have Office 2013'      when LastExecutionResult = '17023' then 'User cancelled installation'      when LastExecutionResult = '17302' then 'Application failed due to low disk space.'
[Code] .....
The below is the output for the given query,here i want to see only one comment value in my list and the count is also sum of all where comment should be Application will be installed once machine is online(Bold columns o/p)
Comment Machine Name Application will be Installed once machine is Online 4 Application will be Installed once machine is Online 12 Application will be Installed once machine is Online 42 Application will be Installed once machine is Online 120 Machine Does not have Office 2013 25 User cancelled installation 32 Application failed due to low disk space 41 Office 2013 SP1 already Exist 60
I need o/p like below:in single line
Application will be Installed once machine is Online 178 Machine Does not have Office 2013 25 User cancelled installation 32 Application failed due to low disk space 41 Office 2013 SP1 already Exist 60
I have a question about the storage design wizard in the analysis manager.
We are working with different seasons in our reports and every week we update the data of the seasons in our cubes. But as seasons end, and at some point the data for old seasons doesn't change anymore, I don't think it is necessary to update every season every week (which we now still do now for seasons in, for example, 2004). It's a waste of time. So my question now is. How can I storage the data of previous seasons (and work with them in the report) and still be able to update the current season? Can I use the storage design for this??
I have a new SQL 2005 (SP2) Reporting Services server to which I've just upgraded and deployed some SSRS 2000 reports.
I have a subreport that contains a matrix with two groups. The report data seems to be inexplicably repeating the data for the first row in the group for all rows in the group. Example:
ID1 ID2 DisplayData
1 1 A
1 2 B
1 3 C
2 1 A
2 2 B
2 3 C
Parent group is on ID1, child group is on ID2, report would show:
1 1 A
2 A
3 A
2 1 A
2 A
3 A
Is this a matrix bug in 2005 SP2, or do I need to do something differently? I can no longer pull a comparison version from an SSRS 2000 server to verify, but I believe it was working as expected before...
Fact table gets update every day with thousands of records, I been increase the aggregations manually. Is there a way I can increase aggregations (design storage) programmatically . My cub gets process ones in day with VB program. I want to increase aggregations before processing cub through VB.
I'm trying to work out how facebook displays so much information easily. Each time you visit your profile a load of new info is shown. Is each piece of info, which is stored in different table, gathered right there and then? If so, isn't that a helluva lot of connections? I'm trying to implement something into my website where it records history, a it like on facebook when it says "Steve has written on Paul's wall" the names are clickable. Has the information been stored like:
Name1ID ActionID Name2ID23432 56 32432
If so, wouldn't 3 connections be needed? 2 to retrieve the names and 1 for the action? It would be easier to store the info as "Steve has written on Paul's Wall" in 1 field, but it wouldn't be dynamic. And if the page shows a lot of information like that, it's hundreds of conections on just 1 viewing. I'm still learning and am also wondering how many connections per page would take up a lot of resources?
How to implement distinct storage tiers on SQL Remote BLOB Storage (RBS)?
I want to use this SQL Feature to move files(images, videos, pdf files) from a database to a distinct database dedicated to RBS. Then I want to have several storage tiers, where objects will be saved and moved according access frequency. Old data will be arquived in cheap storage, but it must be always accessible if needed.
Description: - 1st and main tier: new and frequently accessed objects stored in high performance storage; - 2nd tier: automatically move older or less accessed objects to an inexpensive and different storage tier; - in all cases, all objects must be accessible to all users, but accessing to archived objects(2nd tier) will be much slower;
I have a table that I am basically reduplicating a couple of times for each part of this database that I want to create.Each table basically has the same data: The tables will be called motherTable, fatherTable, sonTable, daughterTable and so on.I am pretty much using the following in each column: UserID, MotherID(or FatherID or SonID, etc., etc. and so on for each unique table), FirstName, LastName, MiddleName, BirthPlace, Photo, Age.I don't see an option to copy a table and just modify the second ID part and rename that table accordingly.How can I make this an easier way of creating these similar tables without retyping all these columns over and over again?Thanks in advance.
finding is queries to find duplicate data.Basically I have a picklist table in a database and I have discovered that there are what looks like duplicate data (because the name is the same) but there is a different number on the end, as you can see from an example below.
NO, ENTRY 24, John Doe|26|05768 24, John Doe|26|5768
Do you know if there is a sql query that can be ran against this table that will look through the ENTRY column and select fields that are similar and not duplicates (duplicates can't exist due to PK constraints)
Table One is an older database and has the column employee id, which would always consist of first letter of the last name an underscore and a numeric value. So for example, data looks like
employeeID R_12345678 S_5555555 T_777777 U_7777
Our new data structure simply removed the letter & underscore.
employeeID 12345678 5555555 777777 7777
Now my question is, how could I join on oldDB.employeeID to newDB.employeeID since the data is very similar, but not exactly the same?
I'm not quite sure if this is the correct forum to post this, if not please advise where should I post.
I have 7 databases with same structure, but different data in it, I need to have a query to consolidade some info from all of them in one report, is it possible just in onw script? how should I do it?
thanks,
Marcus
PS: I'm a beguinner in this so I apologize if the question seems stupid, or wrong.
i want to know how many data can storage into sql server compact edition. I've got a db into a pocket pc that has a table with about 2000 records inside; are they too records?
If you see below there are 2 customer names on 1 loan, most of them share the same lastname and address, I want to separate it with fields,LoanID, customer 1 Firstname, Customer 1 Lastname, Customer 2 FirstName, Customer 2 Lastname, Adddress,zip
LEFT JOIN Status As S on S.LoanID = L.LoanID LEFT JOIN Borrower B on B.LoanID = L.LoanID LEFT JOIN MailingAddress MA on MA.LoanID = L.LoanID where S.PrimStat = '1' and B.Deceased = '0'
I have two tables in my database called CartItems and OrderItems. Istore all of a session's shopping cart items in the CartItems tableusing the sessionID as the identifier (called cartID in my DB). Afteran order is placed and is approved, I would like to copy all of theitems in the CartItems table for that given cartID to the OrderItemstable given a new orderID.I will know the cartID and orderID ahead of time and would like tosend them both into a stored procedure and have the transfer takeplace.Example:take this data...CartItems (table)--------------------------------------cartID | itemID | quantity | price--------------------------------------12345 2 1 12.9512345 7 2 17.95and make it this data...OrderItems (table)--------------------------------------orderID | itemID | quantity | price--------------------------------------00001 2 1 12.9500001 7 2 17.95via some stored procedure that I send (@cartID,@orderID)Any help would be greatly appreciated!!
It seems that there should be a solution for my situation, but for the life of me I can't seem to figure it out.
I need to compare two "like" tables, containing similar data. Tbl 1 is "BOOKED" (which is a snapshot of inventory) and tbl 2 is "CURRENT" (the live - working inventory table). If I write my query as follows the the subsequent result is "duplicate" data.
Code Block SELECT booked.item, booked.bin, booked.quantity, current.bin, current.quantity FROM BOOKED LEFT JOIN CURRENT ON booked.item = current.item
No matter what type of join I use, there is duplicate data displayed for each table. For example, if there are more bins in the BOOKED table that contain a certain product then the CURRENT table will repeat data and vica versa.
As follows:
Item Bin Quantity Bin Quantity
12345 A01 500 A01 7680
12345 B01 6 A01 7680
12345 C01 20 A01 7680
54321 G10 1032 E15 1163
54321 G10 1032 F20 523
54321 G10 1032 H30 750
98765 Z20 7000 Z20 8500
98765 Y15 2500 Y15 3000
98765 X10 1200 Y15 3000
What I would like to do is display Bin and Quantity only once and the repeating values as NULL or [BLANK]. Or, to display all of the bins from both tables and only the quantities from each table in relation to the bin found in that table, returning a "0" if no quantity exists.
This is what I'm after:
Item Bin Quantity Bin Quantity
12345 A01 500 A01 7680
12345 B01 6 B01 0
12345 C01 20 C01 0
54321 G10 1032 E15 1163
54321 F20 0 F20 523
54321 H30 0 H30 750
98765 Z20 7000 Z20 8500
98765 Y15 2500 Y15 3000
98765 X10 1200 X10 0
Is this possible? If so, how?
I also might add that it is ok for each table to contain multiple entries for any given item. This is basically being requested as an inventory variance report - inventory before physical count and immediatly after physical count - and will only be run once a year.
----------------------------------------------- Just thinking out loud here: What if I created three subqueries, the first containing only BOOKED information, the second containing only CURRENT information and the third being a UNION of both tables? Something like this:
Code Block SELECT q3.bin, q1.item, ISNULL(q1.quantity, 0) as QTY_BEFORE, ISNULL(q2.quantity, 0) as QTY_AFTER
FROM
(select item, bin, quantity from BOOKED)q1 Left Join
(select item, bin, quantity from CURRENT)q2 on q1.item = q2.item Left Join
(select bin, item from BOOKED UNION CURRENT)q3 on q1.item = q3.item
Order By q1.item
I don't know if I wrote the UNION statement correctly, but I will have to try this when I get back to work...
I'm consistently getting >10 minute render times in a matrix based report whose underlying query only takes 1-2 seconds to run in both RS's data tab and mgt studio. The bad render times occur in preview tab, Report Viewer and when requested from Report Manager. I believe most of the time being taken is RS's attempt to pivot and present the data.
I can run a homegrown query that pivots (with unknown number of cols) the data itself in under 4 seconds but would be concerned about mapping an unknown number of columns to a table region in RS.
The matrix I'm testing with is 67 columns across (just over 2 months of daily columns) and is supposed to be 207 rows long. There is no aggregation required by RS when it is pivoting the data because each intersection occurs at most one time in the query's result set. The number of data points actually being "bucketed" is around 13,800 values. Row headers consist of 3 columns.
RS is also cutting off the number of rows it is supposed to be showing. The number of rows cutoff seems to be 100% repeatable based on the date range chosen. RS doesnt tell me when it cuts off rows that should have been displayed. From what I can tell so far, it never cuts off columns.
The number of rows cut off seems consistent across designer's preview, report viewer and when requested thru report manager. The "run" command (exclamation icon in data tab) seems to always return the correct number of unpivoted rows, just as in mgt studio.
Are there known bugs and or limitations in RS's matrix data region feature? Are there some settings or workarounds that might make it work more to my liking?
When trying to process my minding model (using Association Rules) i receive the following errors
Errors and Warnings from Response
Error (Data mining): The 'Items Recommendations' mining model has 64675 attributes. This number of attributes exceeds the attribute limit of 5000 allowed by the current version of the algorithm associated with the mining model.
Errors related to feature availability and configuration: The 'Unlimited data mining attributes.' feature is not included in the 'Standard Edition' SKU.
I searched through the internet and found that a hotfix can be installed to make it work (http://support.microsoft.com/kb/932609). I emailed microsoft and they provided me back with the hotfix and a password.
I tried to install the fix but i couldnt. It needed a prerequisit fix. So i installed SQL Server 2005 Service Pack 2 from (http://support.microsoft.com/?kbid=921896)
Then tried to install the fix for my unlimited attributes problem.
I tried to process my module after installing the fix and restarting my computer, but am still gettin the same error.I created another new model but the problem still exists.
Am i missing something here? Any idea to make it work? Is there any alternative? I need to use all the elements in my datasource view.
hi all, I have a field which name is Information and it type is Varchar (8000),but some time data access than 8000 character, my client told me,make this field to store Unlimited data. So how can i achive this task, i m using VS 2003 (ASP.NET with VB.NET) with SQL 2000. Thanks Shally
Hi All,As per BOL, XML data type can store up 2 GB of data.My question is when a row is inserted in a table, for its xml column,2GB of space will be resered.In other words, how xml is internally stored. Is storage allocation issimilar to varchar(max) data type?Thanks in advance for everything.
I am planning on doing database mirroring using two (2) servers for each instance and a SAN to store the data and log files for both the primary server and mirrored server. How do I arrange the SAN 4 Physical Drives? My options are: · 2 Raid 1 Mirrors giving 250 GB to each SQL engine €“ This though has both the transaction logs and data on the same physical drive even if we split it up further into logical drives · A Raid 10 - The transaction logs and data can be on separate drives · A Raid 5 using the 4 Drives. (How SQL will see these drives I€™m not sure when it€™s 2 SQL engines) · Or I could get a 5th drive and have a mirror set for transaction logs and a RAID 5 configured for the data.
Hello, So, here's my dumb question; if I wanted to store some *.gif images in some database (SQL2K possibly 2K5) field and wanted to pull the information from that to display on the web form, am I actually storing the image in the database or am I storing the location of the image in the database?I ask this because I was under the impression that the location to the image file is what was being stored but another person was saying that it was the actual image. I guess I'm confused... Thanks in advance....
User-B Custom fields: Blood type <char 3> Date of birth <date> Referred by <varchar 50>
Different users can have different custom fields in their address book. As you can see, while the standard fields for each user can be
stored in a single table. However, I have several methods by which I can store the CUSTOM fields.
------------------------------------------------ Method 1: Create 2 separate tables called CustomField and CustomValue:
CustomField has fields: FieldID <int> FieldName <varchar 25> UserID <int>
CustomValue has fields: ValueID <int> Value <varchar 50> FieldID <int>
------------------------------------------------ Method 2: Create a separate Field and multiple Value tables for each data type: CustomField, CustomCharValue, CustomIntValue, CustomMoneyValue, etc...
CustomField has fields: FieldID <int> FieldName <varchar 25> FieldType <smallint> (determines which TABLE, below, contains the data) UserID <int>
I'm thinking that while Method 1 will be easier to implement, Method 2 may offer me better performance if coded correctly. I'm going
to assume that I'll have at least 1-5 million records to work with over the course of my first year and I will need the ability to sort
records based on values in the custom fields as well.
My first question is: Which method should I be considering and is there an alternative or hybrid that I should be considering?
My second question is: What statements should I use in my stored procedure that will enable me to retrieve a list of USERID, CustomFieldIDs and their values as one resulting table that I can query at will and with solid performance?
I trying to fully understand when to use different data types in sql server.I want to know what Microdoft means when they say"Varchar is the actual length of the data entered plus 2 bytes".example e.g. what would the storage of varchar (50) be?
Don't know if this is the right forum to be asking this, but I'll give it a try...
I'm relativelly a beginner in SQL Server and T-SQL in general. The problem I'm trying to solve is the following:
The big picture is that I have data coming from different data sources which I need to store on a database for later reference. Each data source might have a different set of measurements. For example, data source 1 might log Pressure and Humidity while data source 2 logs Pressure and Temperature. Once the data is present on the DB, the users can go ahead and retrieve data for a given [datasource/measurement/time interval] to generate reports or charts.
My implementation so far consists of two tables: series_info and series_data. series_info holds general information for a given series of measurements for a given data source (Pressure for data source 1, Pressure for data source 2, Humidity for data source 1 and Temperature for data source 2, in our example). Each series has a bigint index as primary key.
The table series_data contains all data relative to the series from series_info. Each piece of data has a bigint as a primary key, an associate time (which is always crescent) and a foreign key to the series it represents (in series_info).
Alright, everything is cool so far. However, whenever a user wants to retrieve data for given [data source/measurement/time interval], this takes very long, since all data is interposed in series_data and for every search it's necessary to find where the desired data actually lies.
One obvious solution for this would be to dynamically create a new table to hold the data for each series, but that would just make my database disorganized, since there would be thousands and thousands of tables.
Another thing that comes to my mind is to create a table with information of where lies the data for a given [data source / measurement] for given dates. So when the user requested data for a given [data source/measurement] between, say, january and february, we would first look at this intermediate table and find out that the data lies between indexes 1000 and 2000 on the series_data table, so the next SELECT command to series_data would already contain a restriction like WHERE index>=1000 and index<=2000. This should probably improve the speed of retrieval.
What do you guys (or girls) think? Maybe there's simply a classical solution for such a case.
Hello, I am designing a program for work with SQL Server express 2005. But I don't know what is the data storage limit in this version of SQL Server. What i want is storing about 30000 records in a table of the database. Hasn't SQL Server express 2005 any problem or restrictions for storing the data? Please advice in this regards, Thank you, Mona
I am using SQL server 7 with ASP. I have two working environment means one is korean and second it english. - one Korean OS server have SQL server 7.0 and it is my database server - second Korean OS server is only webserver - English OS is win2k and it is only Web server.
1) When i used both Korean server as my webserver + database server then there is no problem to add Korean Data to SQL server On korean OS.
2) But when I try to user English OS server as my webserver and Korean Os server as my database server then I am not able to store Korean Data in Database server insted of it stored some mis/junk/acssi characters in database.
-- I allready try with Korean version of MDAC of English os -- I also try with OEM feature in SQL server client network utility -- When I am use CODEPAGE in my .ASP page then data storage work fine .. but at the time of getting it back there is problem.
If u need any more information about problem then let me know.
So please help me in this regards.
Thanx in advance Anis Vora Partner Global SoftWeb Solutions www.globalsoftweb.com
User-B Custom fields: Blood type <char 3> Date of birth <date> Referred by <varchar 50>
Different users can have different custom fields in their address book. As you can see, while the standard fields for each user can be
stored in a single table. However, I have several methods by which I can store the CUSTOM fields.
------------------------------------------------ Method 1: Create 2 separate tables called CustomField and CustomValue:
CustomField has fields: FieldID <int> FieldName <varchar 25> UserID <int>
CustomValue has fields: ValueID <int> Value <varchar 50> FieldID <int>
------------------------------------------------ Method 2: Create a separate Field and multiple Value tables for each data type: CustomField, CustomCharValue, CustomIntValue, CustomMoneyValue, etc...
CustomField has fields: FieldID <int> FieldName <varchar 25> FieldType <smallint> (determines which TABLE, below, contains the data) UserID <int>
I'm thinking that while Method 1 will be easier to implement, Method 2 may offer me better performance if coded correctly. I'm going
to assume that I'll have at least 1-5 million records to work with over the course of my first year and I will need the ability to sort
records based on values in the custom fields as well.
My first question is: Which method should I be considering and is there an alternative or hybrid that I should be considering?
My second question is: What statements should I use in my stored procedure that will enable me to retrieve a list of USERID, CustomFieldIDs and their values as one resulting table that I can query at will and with solid performance?
I am a Windows developer for the IBM Tivoli Storage Manager Server (TSMS) product. Our product installation is built with InstallShield and uses the Windows Installer.
On a new installation of Windows 2003 x64 Storage Server R2, at a customer's site, the TSMS product fails to install. The install of the OS has version 3.01.400.3959 of the Windows Installer and I see no newer version that installs.
Part of our product is 32 bit (console) and another part is x64 (server). When installing I can see that the install's default is being redirected/reset to C:Program Files (x86)TivoliTSM after it is explicitly set by a custom action to ..Program Files.. . I further observe that our custom actions to write 64 bit registry entries are being refused.
REGSAM samMask = KEY_ALL_ACCESS; if ( regIsWow64Process () ) samMask = samMask | KEY_WOW64_64KEY; lStatus = RegCreateKeyEx( hLocalConnectKeyRoot, szSubkey, 0L, NULL, REG_OPTION_NON_VOLATILE, samMask, NULL, hKey, &dw ) ; The above fails to create the key.
We have tried four versions of our TSMS spanning many changes but the install acts the same. This does not happen on any other Windows OS we test on but we do not test on Windows 2003 Storage Server R2 being that it is an OEM product. We did test on Windows server 2003 R2 x64 and do not see this problem.
Do you have any suggestions on how to tackle this problem? I have full installation traces but can only see that the registry work is being refused. I can't see why.
