Advanced Schema Design Question For Experts. Please Help. Thanks
Jul 14, 2005
QUESTIONs:
What schema is the best for high speed search for a classified web application?
Is our schema design looks OK? It is a STAR schema and will be used for OLTP type app. Is this OK? or Are we missing something?
Please let me know if you need more diagramatic description.
BACKGROUND:
Our group is making a classified website (like classified.yahoo.com) where people can place online ad to sell items.
like cars, computers, electronics etc. Users will fill out webform for each category(car or computers) with all
attributes of the item forsell to post an add.
Main 3 operations the web users will perform:
Quick Searching(most frequent): category=car, subcategory=sedan, country=USA city=LosAngeles Zip=empty
Advanced Search(less frequent):
User can include all fieldsattributes or then can user a subset to query to do advanced search like :
category=car, subcategory=sedan, country=USA, city=Los Angeles, Year= in(00, 03), transmission=Auto, engine=V6, Maker=Honda, Model=Accord, color=Red and Price < 10000
category=car, subcategory=SUV, country=USA, city=NY, Year= in(99, 00, 03), transmission=Auto, Maker=Toyota, and Price between 11000 and 14500
Insert(least frequent): By filling out web form. For Car, the form will have different dimensionsattributes (year, make, model, transmission, mileage, color, price etc)
Current Schema design:
Set of core dimensionlookup tables: stores corecommon attributes for lookups eg. status={open, new, expired}, country={USA, Canada, India,...}
One custom Lookup table: for all custom attribute lookup: transmission{auto, manual}, engine type{V4, V6} for car, processor{PII, PIII, PIV), RAM(512MB, 1GB, 2GB} for Computer & so
3 Fact Tables:
Main factPivot table that stores all sell common attributes eg. price, title, year_made, post_date, expire_date, user_name, description etc.
One fact table to store custom string, int, float, date field values of ads.
One fact table to store custom dropdown field selection values
Concerns and issues:
Looking at the schema, it seems to be a STAR schema with multiple fact tables where all core lookup tables connected to the main Pivot table and
custom lookup table connected to the 2nd and 3rd fact table.
Quick search only queries the Pivot fact table. While Advanced search query requires to join 3 fact tables. Both query requires to join 3 fact tables with all
dimension tables(15 to 20 each having avg of 20 values) to get the look up names so that users sees text instead of ids.
Search speed is the Main concerns. Insertionupdate speed doesn't matter that much as that is less frequently done.
View 5 Replies
Apr 16, 2008
I have the following query. I have two problems with this. For each Region I should only have a CustomerNumber listed once with their SOExtChargeAmount summed. I have some showing up more than once.
The second problem is that after my UNION ALL I need to sum 'AllOthers' as one row of data. I can't figure this out. Any help would be absolutely awesome. Thanks.
SELECT
Region, Location, WarehouseCode, CustomerNumber, CustomerName, MonthLessEleven, MonthLessTen, MonthLessNine, MonthLessEight, MonthLessSeven,
MonthLessSix, MonthLessFive, MonthLessFour, MonthLessThree, MonthLessTwo, MonthLessOne, CurrentMonth, CurrentYearTotal, LastYearYTD,
LastYearTotal, CustRank
FROM
(
SELECT
casewhen substring(gla.Account,5,3) = '936' then 'North Region'
when substring(gla.Account,5,3) = '908' then 'East Region' ELSE 'Unknown'END AS Region,
case
when substring(gla.Account,5,3)='900' then 'ALE'
when substring(gla.Account,5,3)='902' then 'ATO'
when substring(gla.Account,5,3)='904' then 'BOW'
when substring(gla.Account,5,3)='906' then 'BRY'
when substring(gla.Account,5,3)='908' then 'BPT'
when substring(gla.Account,5,3)='910' then 'BYD'
when substring(gla.Account,5,3)='912' then 'BUF'
when substring(gla.Account,5,3)='914' then 'CLE'
when substring(gla.Account,5,3)='916' then 'GRN'
when substring(gla.Account,5,3)='920' then 'DXN'
when substring(gla.Account,5,3)='924' then 'CTH'
when substring(gla.Account,5,3)='926' then 'ELC'
when substring(gla.Account,5,3)='928' then 'FTL'
when substring(gla.Account,5,3)='930' then 'FTW'
when substring(gla.Account,5,3)='932' then 'I35'
when substring(gla.Account,5,3) IN ('936','000') then 'GAI'
when substring(gla.Account,5,3)='939' then 'STW'
when substring(gla.Account,5,3)='940' then 'GRE'
when substring(gla.Account,5,3)='942' then 'HEN'
when substring(gla.Account,5,3)='944' then 'FTS'
when substring(gla.Account,5,3)='948' then 'JAC'
when substring(gla.Account,5,3)='952' then 'JEN'
when substring(gla.Account,5,3)='956' then 'KIL'
when substring(gla.Account,5,3)='957' then 'MCA'
when substring(gla.Account,5,3)='958' then 'MIN'
when substring(gla.Account,5,3)='960' then 'NOC'
when substring(gla.Account,5,3)='962' then 'ODE'
when substring(gla.Account,5,3)='964' then 'BTP'
when substring(gla.Account,5,3)='966' then 'RA'
when substring(gla.Account,5,3)='968' then 'RIF'
when substring(gla.Account,5,3)='970' then 'SWD'
when substring(gla.Account,5,3)='971' then '3PS'
when substring(gla.Account,5,3)='972' then 'ROC'
when substring(gla.Account,5,3)='976' then 'SJO'
when substring(gla.Account,5,3)='978' then 'SMB'
when substring(gla.Account,5,3)='980' then 'STO'
when substring(gla.Account,5,3)='982' then 'TOL'
when substring(gla.Account,5,3)='984' then 'VEL'
when substring(gla.Account,5,3)='985' then 'CFP'
when substring(gla.Account,5,3)='986' then 'CLM'
when substring(gla.Account,5,3)='988' then 'WHI'
when substring(gla.Account,5,3)='992' then 'WRA'
when substring(gla.Account,5,3)='995' then 'ADM' ELSE 'Unknown'END AS Location,
ihh.WarehouseCode,ihh.CustomerNumber, cm.CustomerName,
SUM(CASE WHEN DATEDIFF([MONTH], ihh.SOTransDate, getdate()) = 11 THEN ihd.SOExtChargeAmount ELSE 0 END) AS MonthLessEleven,
SUM(CASE WHEN DATEDIFF([MONTH], ihh.SOTransDate, getdate()) = 10 THEN ihd.SOExtChargeAmount ELSE 0 END) AS MonthLessTen,
SUM(CASE WHEN DATEDIFF([MONTH], ihh.SOTransDate, getdate()) = 9 THEN ihd.SOExtChargeAmount ELSE 0 END) AS MonthLessNine,
SUM(CASE WHEN DATEDIFF([MONTH], ihh.SOTransDate, getdate()) = 8 THEN ihd.SOExtChargeAmount ELSE 0 END) AS MonthLessEight,
SUM(CASE WHEN DATEDIFF([MONTH], ihh.SOTransDate, getdate()) = 7 THEN ihd.SOExtChargeAmount ELSE 0 END) AS MonthLessSeven,
SUM(CASE WHEN DATEDIFF([MONTH], ihh.SOTransDate, getdate()) = 6 THEN ihd.SOExtChargeAmount ELSE 0 END) AS MonthLessSix,
SUM(CASE WHEN DATEDIFF([MONTH], ihh.SOTransDate, getdate()) = 5 THEN ihd.SOExtChargeAmount ELSE 0 END) AS MonthLessFive,
SUM(CASE WHEN DATEDIFF([MONTH], ihh.SOTransDate, getdate()) = 4 THEN ihd.SOExtChargeAmount ELSE 0 END) AS MonthLessFour,
SUM(CASE WHEN DATEDIFF([MONTH], ihh.SOTransDate, getdate()) = 3 THEN ihd.SOExtChargeAmount ELSE 0 END) AS MonthLessThree,
SUM(CASE WHEN DATEDIFF([MONTH], ihh.SOTransDate, getdate()) = 2 THEN ihd.SOExtChargeAmount ELSE 0 END) AS MonthLessTwo,
SUM(CASE WHEN DATEDIFF([MONTH], ihh.SOTransDate, getdate()) = 1 THEN ihd.SOExtChargeAmount ELSE 0 END) AS MonthLessOne,
SUM(CASE WHEN DATEDIFF([MONTH], ihh.SOTransDate, getdate()) = 0 THEN ihd.SOExtChargeAmount ELSE 0 END) AS CurrentMonth,
SUM(CASE WHEN DATEDIFF([YEAR], ihh.SOTransDate, getdate()) = 0 THEN ihd.SOExtChargeAmount ELSE 0 END) AS CurrentYearTotal,
SUM(CASE WHEN DATEADD([year], - 1, getdate()) > ihh.SOTransDate AND DATEDIFF([YEAR], ihh.SOTransDate, getdate()) = 1 THEN ihd.SOExtChargeAmount ELSE 0 END) AS LastYearYTD,
SUM(CASE WHEN DATEDIFF([YEAR], ihh.SOTransDate, getdate()) = 1 THEN ihd.SOExtChargeAmount ELSE 0 END) LastYearTotal,
ROW_NUMBER() OVER(PARTITION BY
casewhen substring(gla.Account,5,3) = '936' then 'North Region'
when substring(gla.Account,5,3) = '908' then 'East Region' ELSE ' 'END
ORDER BY
casewhen substring(gla.Account,5,3) = '936' then 'North Region'
when substring(gla.Account,5,3) = '908' then 'East Region' ELSE ' 'END,
SUM(CASE WHEN DATEDIFF([YEAR], ihh.SOTransDate, getdate()) = 0 THEN ihd.SOExtChargeAmount ELSE 0 END) desc) AS CustRank
FROMMAS_BIF_AR1_CustomerMaster AS cm INNER JOIN
MAS_BIF_ARN_InvHistoryHeader AS ihh ON cm.CustomerNumber = ihh.CustomerNumber INNER JOIN
MAS_BIF_ARO_InvHistoryDetail AS ihd ON ihh.InvoiceNumber = ihd.InvoiceNumber INNER JOIN
MAS_BIF_GL_Account AS gla ON ihd.SOGLSalesAcct = gla.AccountKey
GROUP BYihh.CustomerNumber, cm.CustomerName, gla.Account, ihh.WarehouseCode
) X
WHERECustRank < 20
union all
SELECT
Region, Location, WarehouseCode, 'AllOthers', CustomerName, MonthLessEleven, MonthLessTen, MonthLessNine, MonthLessEight, MonthLessSeven,
MonthLessSix, MonthLessFive, MonthLessFour, MonthLessThree, MonthLessTwo, MonthLessOne, CurrentMonth, CurrentYearTotal, LastYearYTD,
LastYearTotal, 20
FROM
(
SELECT
casewhen substring(gla.Account,5,3) = '936' then 'North Region'
when substring(gla.Account,5,3) = '908' then 'East Region' ELSE 'Unknown'END AS Region,
case
when substring(gla.Account,5,3)='900' then 'ALE'
when substring(gla.Account,5,3)='902' then 'ATO'
when substring(gla.Account,5,3)='904' then 'BOW'
when substring(gla.Account,5,3)='906' then 'BRY'
when substring(gla.Account,5,3)='908' then 'BPT'
when substring(gla.Account,5,3)='910' then 'BYD'
when substring(gla.Account,5,3)='912' then 'BUF'
when substring(gla.Account,5,3)='914' then 'CLE'
when substring(gla.Account,5,3)='916' then 'GRN'
when substring(gla.Account,5,3)='920' then 'DXN'
when substring(gla.Account,5,3)='924' then 'CTH'
when substring(gla.Account,5,3)='926' then 'ELC'
when substring(gla.Account,5,3)='928' then 'FTL'
when substring(gla.Account,5,3)='930' then 'FTW'
when substring(gla.Account,5,3)='932' then 'I35'
when substring(gla.Account,5,3) IN ('936','000') then 'GAI'
when substring(gla.Account,5,3)='939' then 'STW'
when substring(gla.Account,5,3)='940' then 'GRE'
when substring(gla.Account,5,3)='942' then 'HEN'
when substring(gla.Account,5,3)='944' then 'FTS'
when substring(gla.Account,5,3)='948' then 'JAC'
when substring(gla.Account,5,3)='952' then 'JEN'
when substring(gla.Account,5,3)='956' then 'KIL'
when substring(gla.Account,5,3)='957' then 'MCA'
when substring(gla.Account,5,3)='958' then 'MIN'
when substring(gla.Account,5,3)='960' then 'NOC'
when substring(gla.Account,5,3)='962' then 'ODE'
when substring(gla.Account,5,3)='964' then 'BTP'
when substring(gla.Account,5,3)='966' then 'RA'
when substring(gla.Account,5,3)='968' then 'RIF'
when substring(gla.Account,5,3)='970' then 'SWD'
when substring(gla.Account,5,3)='971' then '3PS'
when substring(gla.Account,5,3)='972' then 'ROC'
when substring(gla.Account,5,3)='976' then 'SJO'
when substring(gla.Account,5,3)='978' then 'SMB'
when substring(gla.Account,5,3)='980' then 'STO'
when substring(gla.Account,5,3)='982' then 'TOL'
when substring(gla.Account,5,3)='984' then 'VEL'
when substring(gla.Account,5,3)='985' then 'CFP'
when substring(gla.Account,5,3)='986' then 'CLM'
when substring(gla.Account,5,3)='988' then 'WHI'
when substring(gla.Account,5,3)='992' then 'WRA'
when substring(gla.Account,5,3)='995' then 'ADM' ELSE 'Unknown'END AS Location,
ihh.WarehouseCode,ihh.CustomerNumber, cm.CustomerName,
SUM(CASE WHEN DATEDIFF([MONTH], ihh.SOTransDate, getdate()) = 11 THEN ihd.SOExtChargeAmount ELSE 0 END) AS MonthLessEleven,
SUM(CASE WHEN DATEDIFF([MONTH], ihh.SOTransDate, getdate()) = 10 THEN ihd.SOExtChargeAmount ELSE 0 END) AS MonthLessTen,
SUM(CASE WHEN DATEDIFF([MONTH], ihh.SOTransDate, getdate()) = 9 THEN ihd.SOExtChargeAmount ELSE 0 END) AS MonthLessNine,
SUM(CASE WHEN DATEDIFF([MONTH], ihh.SOTransDate, getdate()) = 8 THEN ihd.SOExtChargeAmount ELSE 0 END) AS MonthLessEight,
SUM(CASE WHEN DATEDIFF([MONTH], ihh.SOTransDate, getdate()) = 7 THEN ihd.SOExtChargeAmount ELSE 0 END) AS MonthLessSeven,
SUM(CASE WHEN DATEDIFF([MONTH], ihh.SOTransDate, getdate()) = 6 THEN ihd.SOExtChargeAmount ELSE 0 END) AS MonthLessSix,
SUM(CASE WHEN DATEDIFF([MONTH], ihh.SOTransDate, getdate()) = 5 THEN ihd.SOExtChargeAmount ELSE 0 END) AS MonthLessFive,
SUM(CASE WHEN DATEDIFF([MONTH], ihh.SOTransDate, getdate()) = 4 THEN ihd.SOExtChargeAmount ELSE 0 END) AS MonthLessFour,
SUM(CASE WHEN DATEDIFF([MONTH], ihh.SOTransDate, getdate()) = 3 THEN ihd.SOExtChargeAmount ELSE 0 END) AS MonthLessThree,
SUM(CASE WHEN DATEDIFF([MONTH], ihh.SOTransDate, getdate()) = 2 THEN ihd.SOExtChargeAmount ELSE 0 END) AS MonthLessTwo,
SUM(CASE WHEN DATEDIFF([MONTH], ihh.SOTransDate, getdate()) = 1 THEN ihd.SOExtChargeAmount ELSE 0 END) AS MonthLessOne,
SUM(CASE WHEN DATEDIFF([MONTH], ihh.SOTransDate, getdate()) = 0 THEN ihd.SOExtChargeAmount ELSE 0 END) AS CurrentMonth,
SUM(CASE WHEN DATEDIFF([YEAR], ihh.SOTransDate, getdate()) = 0 THEN ihd.SOExtChargeAmount ELSE 0 END) AS CurrentYearTotal,
SUM(CASE WHEN DATEADD([year], - 1, getdate()) > ihh.SOTransDate AND DATEDIFF([YEAR], ihh.SOTransDate, getdate()) = 1 THEN ihd.SOExtChargeAmount ELSE 0 END) AS LastYearYTD,
SUM(CASE WHEN DATEDIFF([YEAR], ihh.SOTransDate, getdate()) = 1 THEN ihd.SOExtChargeAmount ELSE 0 END) LastYearTotal,
ROW_NUMBER() OVER(PARTITION BY
casewhen substring(gla.Account,5,3) = '936' then 'North Region'
when substring(gla.Account,5,3) = '908' then 'East Region' ELSE ' 'END
ORDER BY
casewhen substring(gla.Account,5,3) = '936' then 'North Region'
when substring(gla.Account,5,3) = '908' then 'East Region' ELSE ' 'END,
SUM(CASE WHEN DATEDIFF([YEAR], ihh.SOTransDate, getdate()) = 0 THEN ihd.SOExtChargeAmount ELSE 0 END) desc) AS CustRank
FROMMAS_BIF_AR1_CustomerMaster AS cm INNER JOIN
MAS_BIF_ARN_InvHistoryHeader AS ihh ON cm.CustomerNumber = ihh.CustomerNumber INNER JOIN
MAS_BIF_ARO_InvHistoryDetail AS ihd ON ihh.InvoiceNumber = ihd.InvoiceNumber INNER JOIN
MAS_BIF_GL_Account AS gla ON ihd.SOGLSalesAcct = gla.AccountKey
GROUP BYihh.CustomerNumber, cm.CustomerName, gla.Account, ihh.WarehouseCode
) X
WHERECustRank > 19
View 8 Replies
View Related