Hi Guys, wondered if anyone could point me in the right direction with this one.
I'm fairly new to SSIS and one data flow transformation i'm trying to get my head around is the pivot transformation. Perhaps if i explain what i want it to do it will make my problem seems clearer.
I have a table which contains a list of people and their demographic and personal details, each with a unique identifier number. When each persons details are entered one of the pieces of data captured is the month they were registered and their ethnic group classification. Within my data flow i want to take this table of registrations and pivot it like you would in Excel e.g. take the whole list, add months as the columns and ethnic categories as the rows and then a count of the unique identifiers as the value, giving you a simple count of all of the customers and what different ethnic groups per month were registered. Simple enough??
Now, i've tried to do this using the pivot and i seem to be getting lost so after reading the chapter from my wrox book i've tried to outline how i think it should be configured to see if it makes anymore sense, i'd appreciate it if anyone could point out where i might be going wrong.......
In my example i'm assuming my input columns will be the unique identifier, the ethnic group and the month?
I'll then select the ethnic group as the row and the month as the pivot and the unique code as the value using the pivotusage properties of 1, 2, 3. still on the right track??
I then create an output column of each month and set the pivotkey value to the month name and the lineageID to the ID of the unique identifer column.
Can anyone spot if i've missed something out here?
Does anyone have any ideas how to resolve this problem?
I have set my PivotKey to be a Column containing the ColumnNames which I want to Pivot. I have taken the LineAgeId value from the Input column and used it as my SourceColumnID in the Ouput Columns tab.
Error 1 Validation error. Data Flow Task: Pivot [231]: Output column "ATTRIBUTENAME" (489) cannot be mapped to PivotKey input column. Package.dtsx 0 0
I have a table with following structure: ProductID OrderType Value 1001 BaseSales 2000 1001 Incremental 1000 1001 TotalSales 3000 1002 BaseSales 2002 1002 Incremental 1003 1002 TotalSales 3005
I would like to get the data in following format: ProductID BaseSales Incremental TotalSales 1001 2000 1000 3000 1002 2002 1003 3005
I am using the pivot component in SSIS to pivot a set of data, and up until now it has worked fine. However lately it has started to produce duplicates, meaning 2 rows for a combination of the keys (type 1) used. It is completely random, meaning a combination of keys for one run can work correctly, and in the next run it inserts an extra record for the combination for just one of the pivoted values. This also means that for a batch of 10 mill rows input the output is completely random, because the number of duplicates vary.
Are there any memory restrictions or similar that causes this, or do you guys have any idea what might be wrong?
I have an Issue. Please find the sample of my data source
StudentID Fee 1 Fee 1 Currency Type Fee2 Fee 2 Currency Type
1 10 USD 20 INR
2 45 EUR 20 USD
If I need to transform this data, it should be like below Table.
ID StudentId FeeType FeeAmount CurrencyType
1 1 Fee 1 10 USD
2 1 Fee 2 20 INR
3 2 Fee 1 45 EUR
4 2 Fee 2 20 USD
I have selected columns Fee 1 and Fee 2 in Unpivot Transformation. I set the "FeeAmount" as Destination Column Name and "FeeType" as Pivot Key Value Column Name. I left the Pivot Key Value as is for all the selected columns. So far no issue that I am able to get the Fee Type and Fee Amount columns for my destination table.
But the issue is how to get the Currency Type for each corresponding coulmns? I was not able to get the currency type for each student. I hope you got my point. Please let me know If have further queries and hel me out to find this.
If you have two synchronous transformation components and the input of the second is connected to the output of the first, does the first transformation process (loop through) all rows in the buffer before outputting these rows to the second transformation? Or does the first transformation output each individual row to the second transormation as soon as it has finished processing it?
1) Production data with column headers: Key, Facility, Line, Time, Output 2) Costs data with column headers: Key, Site, Cost Center, Time, Cost
The tables have a common key named obviously as Key. The data looks like this:
Key Facility Line Time Output Alpha
I would like to have two pivot tables which I can filter with ONE slicer based on the column Key. The first pivot table shows row labels Facility, Line and column labels Time. Value field is Output. The second pivot table shows row labels Site, Cost Center, and column lables Time. Value field is Cost.How can I do this with Power Pivot? I tried by linking both tables above to a table with unique Keys in PowerPivot and then creating a PivotTable where I would have used the Key from the Keys table.
Can I force the following measure to be visible for all rows in a pivot table?
Sales Special Visibility:=IF( Â Â HASONEVALUE(dimSalesCompanies[SalesCompany]) Â Â ;IF( Â Â Â Â VALUES(dimSalesCompanies[SalesCompany]) = "Sales" Â Â Â Â ;CALCULATE([Sales];ALL(dimSalesCompanies[SalesCompany])) Â Â Â Â ;[Sales] Â Â ) Â Â ;BLANK() )
FYI, I also have other measures as well in the pivot table that I don't want to affect.
I have a simple pivot table (screenshot below) that has two variables on it: one for entry year and another for 6 month time intervals. I have very simple DAX functions that count rows to determine the population N (denominator), the number of records in the time intervals (numerator) and the simple percent of those two numbers.
The problem that I am having is that the function for the population N is not overriding the time interval on the pivot table when I use an ALL function to do so. I use ALL in other very simple pivot tables to do the same thing and it works fine.
The formula for all three are below, but the one that is the issue is the population N formula. Why ALL would not work, any other way to override the time period variable on the pivot table.
Population N (denominator): =CALCULATE(COUNTROWS(analyticJudConsist),ALL(analyticJudConsist[CurrentTimeInCare1])) Records in time interval (numerator): =COUNTROWS(analyticJudConsist) Percent: =[countrows]/[denominatorCare]
How can I apply "Min" formula under a "new measure" (calculated field) within a pivot table under Power pivot 2010?Can see that neither does it allow me to apply "min" formula directly "formula box" nor could find any other option.Intent formula: "=Min(1,sum(a:b))" this isn't allowed so all I can do is "=sum(a:b)".
I have simple pivot table (below screenshot with info redacted) that displays a population number ("N" below), this is the denominator, a cumulative numerator number (below "#") and a simple cumulative percent that just divides the numerator by the denominator. It cumulates from top to bottom. The numerator and percent are cumulative using the below functions. There are two problems with the numerator and percent:
1. When there is not a number for the numerator, there is no value displayed for both the numerator and the percent..There should be a zero displayed for both values. 2. When there has been a prior number for the numerator and percent (for a prior month interval) but there is no number for the numerator in the current month interval, the prior month number and percent are not displayed in the current month interval--see the 3rd yellow line, this should display "3" and "16.7%" from the second yellow line.Here is the formula for the numerator:
=CALCULATE(count(s1Perm1[entity_id]),FILTER(ALL(s1Perm1[ExitMonthCategory]),s1Perm1[ExitMonthCategory] <= MAX(s1Perm1[ExitMonthCategory]))) Here is the formula for the percent: =(CALCULATE(countrows(s1Perm1),FILTER(ALL(s1Perm1[ExitMonthCategory]),s1Perm1[ExitMonthCategory] <= MAX(s1Perm1[ExitMonthCategory]))))/(CALCULATE(COUNTROWS(s1Perm1),ALL(s1Perm1[Exit],s1Perm1[ExitMonthCategory])))
I have data in my Powerpivot window which was generated by a sql query. This data includes a field named 'Cost' and every row shows a value for 'Cost' greater than zero. The problem is that when I display this data in the pivot table all entries for Cost display as $0. At first I thought that maybe Cost was set to a bogus data type (such as 'text) but it is set to ''Decimal Number' so that's not the problem.Â
What is happening and how do I fix it so that my pivot table reflects the values for 'Cost'?
I have a data table that contains budget and actual data by month. Â I use the data to create a pivot that shows actual results next to budgeted results. Â I need a column that shows that variance between those columns. Â I think my issue is that the "Type" field contains actual and Budget. Â I sum on "Type". Â I can't seem to create a sum since those items are in the same field or am I missing something?
How to get a list of values to actually display in correct order in either a slicer or when on an axis on a pivot table?
I currently have the below list and have tried to add a preceding numeric (ex. "1. <=0") or preceding blank space, neither of which is visually great. Is there another way?
I am using excel 2010 and creating pivot table from Power Pivot. I created a pivot table with department slicers. All is good, the problem I am having is whilst in an unfiltered position (ALL) of the slicers (departments) I get 200 pages, now when I click on a given department with say 10 pages, I still get the same 200 pages with the first 10 pages showing the data from the clicked department and 190 blank pages.
All I want is to get a WYSIWYG (What you see is what you get) of what is on the screen as my print, but I am getting extra blank pages right after the data. Â How do I resolve this.
Below are the steps I go thru to printÂ
1. Select slicers in unfiltered position (ALL) 2. Select entire pivot table 3. Select Page layout and select print area. 4. Save 5. Click on Print Preview to preview the print 6. Click on a given department in the slicer and repeat item 5, but this gives me blank pages after the data.
I am using the pivot task to to a pivot of YTD-Values and after that I use derived columns to calculate month values and do a unpivot then.
All worked fine, but now I get this error message:
[ytd_pivot [123]] Error: Duplicate pivot key value "6".
The settings in the advanced editor seem to be correct (no duplicate pivot key value) and I am extracting the data from the source sorted by month.
Could it be a problem that I use all pivot columns (month 1 to 12) in the derived colum transformation and they aren´t available at this moment while data extracting is still going on?
Say, I have the following temporary table (@tbl) where the QuestionID field will change values over time
Survey QuestionID Answer 1 1 1 1 2 0 2 1 1 2 2 2
I'd like to perform a pivot on it like this: select * from @tbl Pivot (min(Answer) for QuestionID in ([1], [2])) as PivotTable
...however, I can't just name the [1], [2] values because they're going to change.
Instead of naming the values like this: for QuestionID in ([1], [2], [3], [4])
I tried something like this: for QuestionID in (select distinct QuestionID from @tbl)
but am getting a syntax error. Is it possible to set up a pivot like this: select * from @tbl Pivot (min(Answer) for Question_CID in (select distinct @QuestionID from @tbl)) as PivotTable
Is it possible to generate automatic refresh of excel 2013 table which displays some table of a power pivot model on file open?? I dont want to use pivottable (which supports this ...)
I have a pivot table that connects to our data warehouse via a PowerPivot connection. Â The data contains a bunch of comment fields that are each between 250 and 500 characters. Â I've set the columns in this pivot table to have the 'Wrap Text' set to true so that the user experience is better, and they can view these comment fields more clearly.
However, whenever I refresh the data, the text wrapping un-sets itself. Â Interestingly, the 'Wrap Text' setting is still enabled, but I have to go and click it, then click it again to actually wrap the text. Â This is very burdensome on the user, and degrading the experience.
Any way to make this text wrapping stick so that I don't have to re-set it every time I refresh the data?
Hello I trying to do some transformation using javascript. I have a table with 40 columns that I need dynamicaly mapped to another table. For example I need to test each Column of tableA if Col1 of TableA looks like an email address map it to TableB email if it looks like a zip map it to Email...
I need to do this with Javascript
if someone could point me in the right direction that would be great
lets say I start small test each col in TableA for an @ sign if I find it map to Email if I can get the idea of how to do that I can do this
plz tell me abt look up transformation , i need to look up with 3 source files ,1 flat file another 2 are compared with databases, how can we use that ....
I am a bit familiar with SSIS, but I have not used it in about a year. However, this is my question. We have a couple of reports that are being done in SSRS and our customer also requires some of these reports to be exported to a flat file. Somebody suggested using SSIS to do this.
What transformation element should I use to transform the data to a flat file destination. For instance, I have a column with a value of 10250. The customer specifies that this column must be right-justified with leading zeros. So I need to take that and convert it to a field length that is 12 characters long. It would ultimately be 000000010250.
The data source is returning columns that are both numeric and strings.
Hi, we are trying to implement this logic in SSIS data flow task
We have 2 datasets
Dataset 1 has Emplid and Actiondate. (from Oracle Database instance I) Dataset 2 has Emplid, ProjectID ,Project Start_Date ,Project_End_Date (from Oracle Database instance II)
Logic is
select ProjectID from Dataset 2 where Dataset 1.Emplid = Dataset 2.Emplid and Dataset 1.Actiondate between Dataset 2.Project Start_Date and Dataset 2.Project_End_Date.
for first condition we have used merge join between 2 datasets.
for second condition we are trying to use look up transformation... it did'nt help... if i want to write an query where do i write that???
I am trying to transform data from a text file to a SQL Server table and one particular column I want to split it to three other columns. It is erroring out when I try to split the column from the source in the Active X script while doing the transformation.
Is it possible to have more number of Columns in the destination when compared to the source.
We are transferring data between AS/400 and SQL Server 7.0 using DTS. Some of these transfers may need to be very close to real time. It doesn't seem like a continuously running job is the best solution for that.
Do you know any tools or utilities that can help us to move the data?
Please help,I am using SQL Server 7 and this is the first time I am using DTS. Can I import a text file with 3 fields to a table which has those 3 fields and also another 2 fields? Can somebody help with the ActiveX script to accomplish this? And one of the extra 2 fields in the table is a timestamp field. It needs to be timestamped automatically. Can somebody please help? Thank you in advance
name type amount ==== ==== ====== mary saving 123.00 mary chequing 246.00 mary investment 135.00 john saving 678.00 john chequing 987.00 john investment 0.00
what should i do to present the data in the following format?
name saving cheq investment ==== ====== ==== ========== mary 123.00 246.00 135.00 john 678.00 987.00 0.00
I am importing user id's from CSV file into my Database. The source user id is of 4 character long and the destination user_id field is of 100 varchar. I just query the data so that i wan'a check that is said user id is already exist in the database or not . So i got this error. Read the script and error below '************************************************* ********************* ' Visual Basic Transformation Script '************************************************* ***********************
' Copy each source column to the destination column Function Main()
DIM v_user_id,rs, ConStr,sql set rs = CreateObject("Adodb.Recordset") set con = CreateObject("Adodb.Connection")
ConStr = "Provider=SQLOLEDB;Server=192.168.1.71;Database=tes tkaanza;uid=sa;pwd=sa" v_user_id = DTSSource("Col001") rs.open "Select user_id from tbl_gc_user_hdr where user_id = " + v_user_id,ConStr,0,1
if Not rs.eof then MsgBox "Record Found" else MsgBox "Record Not Found" end if rs.Close End Function
i got error from this line : rs.open "Select user_id from tbl_gc_user_hdr where user_id = " + v_user_id,ConStr,0,1
Error message :
Error Source: Microsoft OLE DB Provider for SQL Server Error Description:The conversion of the varchar value '1000015151910165' overflowed an int column. Maximum integer value exceeded. Error Help File: Error Help Context ID:0
