Pivot Transformation Help
Oct 15, 2007
Hi Guys, wondered if anyone could point me in the right direction with this one.
I'm fairly new to SSIS and one data flow transformation i'm trying to get my head around is the pivot transformation. Perhaps if i explain what i want it to do it will make my problem seems clearer.
I have a table which contains a list of people and their demographic and personal details, each with a unique identifier number. When each persons details are entered one of the pieces of data captured is the month they were registered and their ethnic group classification. Within my data flow i want to take this table of registrations and pivot it like you would in Excel e.g. take the whole list, add months as the columns and ethnic categories as the rows and then a count of the unique identifiers as the value, giving you a simple count of all of the customers and what different ethnic groups per month were registered. Simple enough??
Now, i've tried to do this using the pivot and i seem to be getting lost so after reading the chapter from my wrox book i've tried to outline how i think it should be configured to see if it makes anymore sense, i'd appreciate it if anyone could point out where i might be going wrong.......
In my example i'm assuming my input columns will be the unique identifier, the ethnic group and the month?
I'll then select the ethnic group as the row and the month as the pivot and the unique code as the value using the pivotusage properties of 1, 2, 3. still on the right track??
I then create an output column of each month and set the pivotkey value to the month name and the lineageID to the ID of the unique identifer column.
Can anyone spot if i've missed something out here?
many thanks in advance for any help
View 3 Replies
ADVERTISEMENT
Oct 4, 2007
Does anyone have any ideas how to resolve this problem?
I have set my PivotKey to be a Column containing the ColumnNames which I want to Pivot. I have taken the LineAgeId value from the Input column and used it as my SourceColumnID in the Ouput Columns tab.
Error 1 Validation error. Data Flow Task: Pivot [231]: Output column "ATTRIBUTENAME" (489) cannot be mapped to PivotKey input column. Package.dtsx 0 0
View 6 Replies
View Related
Jan 24, 2008
All,
Is there a way to do the opposite of what€™s Pivot transformation does in SSIS?
I have data like :
Monday, Tuesday, Wednesday, Thursday, Friday
2 3 4 4 3
And I want transform it into
Day_name , QTY
Monday 2
Tuesday 3
Wednesday 4
Thursday 4
Friday 3
thanks
View 1 Replies
View Related
Dec 4, 2007
I have a table with following structure:
ProductID OrderType Value
1001 BaseSales 2000
1001 Incremental 1000
1001 TotalSales 3000
1002 BaseSales 2002
1002 Incremental 1003
1002 TotalSales 3005
I would like to get the data in following format:
ProductID BaseSales Incremental TotalSales
1001 2000 1000 3000
1002 2002 1003 3005
Thanks for any help
Regards
Josh
View 1 Replies
View Related
Apr 21, 2008
Hey,
I am using the pivot component in SSIS to pivot a set of data, and up until now it has worked fine. However lately it has started to produce duplicates, meaning 2 rows for a combination of the keys (type 1) used. It is completely random, meaning a combination of keys for one run can work correctly, and in the next run it inserts an extra record for the combination for just one of the pivoted values. This also means that for a batch of 10 mill rows input the output is completely random, because the number of duplicates vary.
Are there any memory restrictions or similar that causes this, or do you guys have any idea what might be wrong?
Thanks in advance
View 5 Replies
View Related
Mar 6, 2008
Hi
I have an Issue. Please find the sample of my data source
StudentID
Fee 1
Fee 1 Currency Type
Fee2
Fee 2 Currency Type
1
10
USD
20
INR
2
45
EUR
20
USD
If I need to transform this data, it should be like below Table.
ID
StudentId
FeeType
FeeAmount
CurrencyType
1
1
Fee 1
10
USD
2
1
Fee 2
20
INR
3
2
Fee 1
45
EUR
4
2
Fee 2
20
USD
I have selected columns Fee 1 and Fee 2 in Unpivot Transformation. I set the "FeeAmount" as Destination Column Name and "FeeType" as Pivot Key Value Column Name. I left the Pivot Key Value as is for all the selected columns. So far no issue that I am able to get the Fee Type and Fee Amount columns for my destination table.
But the issue is how to get the Currency Type for each corresponding coulmns? I was not able to get the currency type for each student. I hope you got my point. Please let me know If have further queries and hel me out to find this.
View 5 Replies
View Related
Jun 5, 2006
Hi,
If you have two synchronous transformation components and the input of the second is connected to the output of the first, does the first transformation process (loop through) all rows in the buffer before outputting these rows to the second transformation? Or does the first transformation output each individual row to the second transormation as soon as it has finished processing it?
Thanks in advance,
Lawrie.
View 5 Replies
View Related
Apr 22, 2015
tell me the difference between Audit transformation and rowcount transformation.
Because audit and rowcount transformation will provide the environment variables.
Only difference i am finding is rowcount returns the count of rows its updating .
Apart from these is there any other difference?
Tell me the scenario where i need to use the audit transformation.
View 3 Replies
View Related
May 19, 2006
Hi all,
In MyDatabase, I have a TABLE dbo.LabData created by the following SQLQuery.sql:
USE MyDatabase
GO
CREATE TABLE dbo.LabResults
(SampleID int PRIMARY KEY NOT NULL,
SampleName varchar(25) NOT NULL,
AnalyteName varchar(25) NOT NULL,
Concentration decimal(6.2) NULL)
GO
--Inserting data into a table
INSERT dbo.LabResults (SampleID, SampleName, AnalyteName, Concentration)
VALUES (1, 'MW2', 'Acetone', 1.00)
INSERT €¦ ) VALUES (2, 'MW2', 'Dichloroethene', 1.00)
INSERT €¦ ) VALUES (3, 'MW2', 'Trichloroethene', 20.00)
INSERT €¦ ) VALUES (4, 'MW2', 'Chloroform', 1.00)
INSERT €¦ ) VALUES (5, 'MW2', 'Methylene Chloride', 1.00)
INSERT €¦ ) VALUES (6, 'MW6S', 'Acetone', 1.00)
INSERT €¦ ) VALUES (7, 'MW6S', 'Dichloroethene', 1.00)
INSERT €¦ ) VALUES (8, 'MW6S', 'Trichloroethene', 1.00)
INSERT €¦ ) VALUES (9, 'MW6S', 'Chloroform', 1.00)
INSERT €¦ ) VALUES (10, 'MW6S', 'Methylene Chloride', 1.00)
INSERT €¦ ) VALUES (11, 'MW7', 'Acetone', 1.00)
INSERT €¦ ) VALUES (12, 'MW7', 'Dichloroethene', 1.00)
INSERT €¦ ) VALUES (13, 'MW7', 'Trichloroethene', 1.00)
INSERT €¦ ) VALUES (14, 'MW7', 'Chloroform', 1.00)
INSERT €¦ ) VALUES (15, 'MW7', 'Methylene Chloride', 1.00)
INSERT €¦ ) VALUES (16, 'TripBlank', 'Acetone', 1.00)
INSERT €¦ ) VALUES (17, 'TripBlank', 'Dichloroethene', 1.00)
INSERT €¦ ) VALUES (18, 'TripBlank', 'Trichloroethene', 1.00)
INSERT €¦ ) VALUES (19, 'TripBlank', 'Chloroform', 0.76)
INSERT €¦ ) VALUES (20, 'TripBlank', 'Methylene Chloride', 0.51)
GO
A desired Pivot Table is like:
MW2 MW6S MW7 TripBlank
Acetone 1.00 1.00 1.00 1.00
Dichloroethene 1.00 1.00 1.00 1.00
Trichloroethene 20.00 1.00 1.00 1.00
Chloroform 1.00 1.00 1.00 0.76
Methylene Chloride 1.00 1.00 1.00 0.51
//////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
I write the following SQLQuery.sql code for creating a Pivot Table from the Table dbo.LabData by using the PIVOT operator:
USE MyDatabase
GO
USE TABLE dbo.LabData
GO
SELECT AnalyteName, [1] AS MW2, AS MW6S, [11] AS MW7, [16] AS TripBlank
FROM
(SELECT SampleName, AnalyteName, Concentration
FROM dbo.LabData) p
PIVOT
(
SUM (Concentration)
FOR AnalyteName IN ([1], , [11], [16])
) AS pvt
ORDER BY SampleName
GO
////////////////////////////////////////////////////////////////////////////////////////////////////////////////
I executed the above-mentioned code and I got the following error messages:
Msg 156, Level 15, State 1, Line 1
Incorrect syntax near the keyword 'TABLE'.
Msg 207, Level 16, State 1, Line 1
Invalid column name 'AnalyteName'.
I do not know what is wrong in the code statements of my SQLQuery.sql. Please help and advise me how to make it right and work for me.
Thanks in advance,
Scott Chang
View 6 Replies
View Related
Jul 8, 2015
I have two data tables:
1) Production data with column headers: Key, Facility, Line, Time, Output
2) Costs data with column headers: Key, Site, Cost Center, Time, Cost
The tables have a common key named obviously as Key. The data looks like this:
Key
Facility
Line
Time
Output
Alpha
I would like to have two pivot tables which I can filter with ONE slicer based on the column Key. The first pivot table shows row labels Facility, Line and column labels Time. Value field is Output. The second pivot table shows row labels Site, Cost Center, and column lables Time. Value field is Cost.How can I do this with Power Pivot? I tried by linking both tables above to a table with unique Keys in PowerPivot and then creating a PivotTable where I would have used the Key from the Keys table.
View 5 Replies
View Related
Oct 13, 2015
Can I force the following measure to be visible for all rows in a pivot table?
Sales Special Visibility:=IF(
  HASONEVALUE(dimSalesCompanies[SalesCompany])
  ;IF(
    VALUES(dimSalesCompanies[SalesCompany]) = "Sales"
    ;CALCULATE([Sales];ALL(dimSalesCompanies[SalesCompany]))
    ;[Sales]
  )
  ;BLANK()
)
FYI, I also have other measures as well in the pivot table that I don't want to affect.
View 3 Replies
View Related
Oct 14, 2015
I have a simple pivot table (screenshot below) that has two variables on it: one for entry year and another for 6 month time intervals. I have very simple DAX functions that count rows to determine the population N (denominator), the number of records in the time intervals (numerator) and the simple percent of those two numbers.
The problem that I am having is that the function for the population N is not overriding the time interval on the pivot table when I use an ALL function to do so. I use ALL in other very simple pivot tables to do the same thing and it works fine.
The formula for all three are below, but the one that is the issue is the population N formula. Why ALL would not work, any other way to override the time period variable on the pivot table.
Population N (denominator):
=CALCULATE(COUNTROWS(analyticJudConsist),ALL(analyticJudConsist[CurrentTimeInCare1]))
Records in time interval (numerator):
=COUNTROWS(analyticJudConsist)
Percent:
=[countrows]/[denominatorCare]
View 13 Replies
View Related
Aug 17, 2015
How can I apply "Min" formula under a "new measure" (calculated field) within a pivot table under Power pivot 2010?Can see that neither does it allow me to apply "min" formula directly "formula box" nor could find any other option.Intent formula: "=Min(1,sum(a:b))" this isn't allowed so all I can do is "=sum(a:b)".
View 3 Replies
View Related
Mar 11, 2015
I have simple pivot table (below screenshot with info redacted) that displays a population number ("N" below), this is the denominator, a cumulative numerator number (below "#") and a simple cumulative percent that just divides the numerator by the denominator. It cumulates from top to bottom. The numerator and percent are cumulative using the below functions. There are two problems with the numerator and percent:
1. When there is not a number for the numerator, there is no value displayed for both the numerator and the percent..There should be a zero displayed for both values.
2. When there has been a prior number for the numerator and percent (for a prior month interval) but there is no number for the numerator in the current month interval, the prior month number and percent are not displayed in the current month interval--see the 3rd yellow line, this should display "3" and "16.7%" from the second yellow line.Here is the formula for the numerator:
=CALCULATE(count(s1Perm1[entity_id]),FILTER(ALL(s1Perm1[ExitMonthCategory]),s1Perm1[ExitMonthCategory] <= MAX(s1Perm1[ExitMonthCategory])))
Here is the formula for the percent:
=(CALCULATE(countrows(s1Perm1),FILTER(ALL(s1Perm1[ExitMonthCategory]),s1Perm1[ExitMonthCategory] <= MAX(s1Perm1[ExitMonthCategory]))))/(CALCULATE(COUNTROWS(s1Perm1),ALL(s1Perm1[Exit],s1Perm1[ExitMonthCategory])))
View 24 Replies
View Related
Sep 18, 2015
I have data in my Powerpivot window which was generated by a sql query. This data includes a field named 'Cost' and every row shows a value for 'Cost' greater than zero. The problem is that when I display this data in the pivot table all entries for Cost display as $0. At first I thought that maybe Cost was set to a bogus data type (such as 'text) but it is set to ''Decimal Number' so that's not the problem.Â
What is happening and how do I fix it so that my pivot table reflects the values for 'Cost'?
View 3 Replies
View Related
Nov 23, 2015
I have a data table that contains budget and actual data by month. Â I use the data to create a pivot that shows actual results next to budgeted results. Â I need a column that shows that variance between those columns. Â I think my issue is that the "Type" field contains actual and Budget. Â I sum on "Type". Â I can't seem to create a sum since those items are in the same field or am I missing something?
Table design
Month|Division|Subdivision|Type|Dept|Rate|Units|Amount
October|DC|Day|Budget|125|10.00|100|1000
October|DC|Day|Actual|125|10.00|110|1100
Output Design
DC
DAY
Actual
Budget
125 AvgOfRate
AvgOfRate
SumOfUnits
SumOfUnits
SumOfAmt
SumOfAmt
View 4 Replies
View Related
Oct 9, 2015
How to get a list of values to actually display in correct order in either a slicer or when on an axis on a pivot table?
I currently have the below list and have tried to add a preceding numeric (ex. "1. <=0") or preceding blank space, neither of which is visually great. Is there another way?
<= 0
1 - 6
7 - 12
13 - 18
19 - 24
25 - 30
31 - 36
37 - 42
43 - 48
49 - 54
55 - 60
61 - 66
67 - 72
73 - 78
79 - 84
85 - 90
91 - 96
97 - 102
> 102
View 8 Replies
View Related
Apr 13, 2015
I am using excel 2010 and creating pivot table from Power Pivot. I created a pivot table with department slicers. All is good, the problem I am having is whilst in an unfiltered position (ALL) of the slicers (departments) I get 200 pages, now when I click on a given department with say 10 pages, I still get the same 200 pages with the first 10 pages showing the data from the clicked department and 190 blank pages.
All I want is to get a WYSIWYG (What you see is what you get) of what is on the screen as my print, but I am getting extra blank pages right after the data. Â How do I resolve this.
Below are the steps I go thru to printÂ
1. Select slicers in unfiltered position (ALL)
2. Select entire pivot table
3. Select Page layout and select print area.
4. Save
5. Click on Print Preview to preview the print
6. Click on a given department in the slicer and repeat item 5, but this gives me blank pages after the data.
Do I need any other step?Â
View 2 Replies
View Related
Jul 5, 2006
I am using the pivot task to to a pivot of YTD-Values and after that I use derived columns to calculate month values and do a unpivot then.
All worked fine, but now I get this error message:
[ytd_pivot [123]] Error: Duplicate pivot key value "6".
The settings in the advanced editor seem to be correct (no duplicate pivot key value) and I am extracting the data from the source sorted by month.
Could it be a problem that I use all pivot columns (month 1 to 12) in the derived colum transformation and they aren´t available at this moment while data extracting is still going on?
any hints?
Cheers
Markus
View 3 Replies
View Related
Sep 21, 2007
Say, I have the following temporary table (@tbl) where the QuestionID field will change values over time
Survey QuestionID Answer
1 1 1
1 2 0
2 1 1
2 2 2
I'd like to perform a pivot on it like this: select * from @tbl Pivot (min(Answer) for QuestionID in ([1], [2])) as PivotTable
...however, I can't just name the [1], [2] values because they're going to change.
Instead of naming the values like this:
for QuestionID in ([1], [2], [3], [4])
I tried something like this:
for QuestionID in (select distinct QuestionID from @tbl)
but am getting a syntax error. Is it possible to set up a pivot like this:
select * from @tbl Pivot (min(Answer) for Question_CID in (select distinct @QuestionID from @tbl)) as PivotTable
or does anyone know another way to do it?
View 3 Replies
View Related
Jul 8, 2015
Is it possible to generate automatic refresh of excel 2013 table which displays some table of a power pivot model on file open?? I dont want to use pivottable (which supports this ...)
View 2 Replies
View Related
Apr 29, 2015
I have a pivot table that connects to our data warehouse via a PowerPivot connection. Â The data contains a bunch of comment fields that are each between 250 and 500 characters. Â I've set the columns in this pivot table to have the 'Wrap Text' set to true so that the user experience is better, and they can view these comment fields more clearly.
However, whenever I refresh the data, the text wrapping un-sets itself. Â Interestingly, the 'Wrap Text' setting is still enabled, but I have to go and click it, then click it again to actually wrap the text. Â This is very burdensome on the user, and degrading the experience.
Any way to make this text wrapping stick so that I don't have to re-set it every time I refresh the data?
View 2 Replies
View Related
Dec 5, 2006
Hello I trying to do some transformation using javascript. I have a table with 40 columns that I need dynamicaly mapped to another table. For example I need to test each Column of tableA if Col1 of TableA looks like an email address map it to TableB email if it looks like a zip map it to Email...
I need to do this with Javascript
if someone could point me in the right direction that would be great
lets say I start small test each col in TableA for an @ sign if I find it map to Email if I can get the idea of how to do that I can do this
thanks
View 1 Replies
View Related
Feb 21, 2006
Hi there,
I want to check whether the values in a column within a defined list of values or not.
These values aren't in a table or a file - I just want to "code" them in the package so to speak.
I was thinking of using a script component but I'm not sure how to "fail" a row if it the column's value is not part of the pre-defined list.
Any suggestions.
Thanks.
View 1 Replies
View Related
Apr 28, 2008
plz tell me abt look up transformation , i need to look up with 3 source files ,1 flat file another 2 are compared with databases, how can we use that ....
plz tell me.
View 2 Replies
View Related
Dec 11, 2007
I am a bit familiar with SSIS, but I have not used it in about a year. However, this is my question. We have a couple of reports that are being done in SSRS and our customer also requires some of these reports to be exported to a flat file. Somebody suggested using SSIS to do this.
What transformation element should I use to transform the data to a flat file destination. For instance, I have a column with a value of 10250. The customer specifies that this column must be right-justified with leading zeros. So I need to take that and convert it to a field length that is 12 characters long. It would ultimately be 000000010250.
The data source is returning columns that are both numeric and strings.
Thanks for the help.
View 1 Replies
View Related
May 11, 2006
Hi,
we are trying to implement this logic in SSIS data flow task
We have 2 datasets
Dataset 1 has Emplid and Actiondate. (from Oracle Database instance I)
Dataset 2 has Emplid, ProjectID ,Project Start_Date ,Project_End_Date (from Oracle Database
instance II)
Logic is
select ProjectID from Dataset 2
where Dataset 1.Emplid = Dataset 2.Emplid
and Dataset 1.Actiondate between Dataset 2.Project Start_Date and Dataset 2.Project_End_Date.
for first condition we have used merge join between 2 datasets.
for second condition we are trying to use look up transformation... it did'nt help...
if i want to write an query where do i write that???
please help
Thanks in Advance.
View 5 Replies
View Related
Aug 16, 2002
I am trying to transform data from a text file to a SQL Server table and one particular column I want to split it to three other columns. It is erroring out when I try to split the column from the source in the Active X script while doing the transformation.
Is it possible to have more number of Columns in the destination when compared to the source.
If so can someone help me with this.
Thanks,
Hari
View 1 Replies
View Related
Sep 14, 2000
We are transferring data between AS/400 and SQL Server 7.0 using DTS. Some of these transfers may need to be very close to real time. It doesn't seem like a continuously running job is the best solution for that.
Do you know any tools or utilities that can help us to move the data?
Thank you,
Anastasia.
View 2 Replies
View Related
Dec 2, 2001
Please help,I am using SQL Server 7 and this is the first time I am using DTS. Can I import a text file with 3 fields to a table which has those 3 fields and also another 2 fields? Can somebody help with the ActiveX script to accomplish this? And one of the extra 2 fields in the table is a timestamp field. It needs to be timestamped automatically. Can somebody please help? Thank you in advance
View 2 Replies
View Related
Feb 19, 2003
i have something like this:
select * from accounts
name type amount
==== ==== ======
mary saving 123.00
mary chequing 246.00
mary investment 135.00
john saving 678.00
john chequing 987.00
john investment 0.00
what should i do to present the data in the following format?
name saving cheq investment
==== ====== ==== ==========
mary 123.00 246.00 135.00
john 678.00 987.00 0.00
Thanks.
View 3 Replies
View Related
Mar 5, 2007
I am importing user id's from CSV file into my Database. The source user id is of 4 character long and the destination user_id field is of 100 varchar. I just query the data so that i wan'a check that is said user id is already exist in the database or not . So i got this error. Read the script and error below
'************************************************* *********************
' Visual Basic Transformation Script
'************************************************* ***********************
' Copy each source column to the destination column
Function Main()
DIM v_user_id,rs, ConStr,sql
set rs = CreateObject("Adodb.Recordset")
set con = CreateObject("Adodb.Connection")
ConStr = "Provider=SQLOLEDB;Server=192.168.1.71;Database=tes tkaanza;uid=sa;pwd=sa"
v_user_id = DTSSource("Col001")
rs.open "Select user_id from tbl_gc_user_hdr where user_id = " + v_user_id,ConStr,0,1
if Not rs.eof then
MsgBox "Record Found"
else
MsgBox "Record Not Found"
end if
rs.Close
End Function
i got error from this line :
rs.open "Select user_id from tbl_gc_user_hdr where user_id = " + v_user_id,ConStr,0,1
Error message :
Error Source: Microsoft OLE DB Provider for SQL Server
Error Description:The conversion of the varchar value '1000015151910165' overflowed an int column. Maximum integer value exceeded.
Error Help File:
Error Help Context ID:0
Thank you in advance
View 1 Replies
View Related
Apr 26, 2004
What is the fonction to use in order to format a date to char?
for instance have the date under the following format :
ddmmyy or dd/mm/yyyy ...
Thanks
View 1 Replies
View Related
