I need to get a list of customer ids and then use them as a parameter to select from a transaction table and then create a file for each customer. I have used an execute SQL task to get the list of customers and have put the result set into a variable.
How do go through the recordset to create a file for each customer?
I am trying to create a text file from an SQL query on a SQL table. I would like the SSIS package to prompt for the file name and path. The text file is tab delimited and the text qualifier is a double quote.
Thanks,
Fred
I have a select Script as follows:
SELECT c.ABC AS 'ABC'
, a.Qty AS 'Quantity_Recived'
, b.PC AS 'PC'
, b.PC AS 'PC'
, 'I' AS 'Flag'
FROM TNRInventory.dbo.tInventoryAlloc AS a
LEFT OUTER JOIN vwInventoryAllocMapping AS vwMap ON a.TNRAllocTypeID = vwMap.TNRInventoryAllocID
LEFT OUTER JOIN ABC.dbo.ZREFRESHTAB AS b ON a.DispenserID = b.Asset
LEFT OUTER JOIN ABC.dbo.TableJoinKey AS c ON a.TitleID = c.TITLE_ID
WHERE (vwMap.DataSourceID = 3) and vwMap.[DataSourceAllocName] = 'I'
group by c.SKU_NO , vwMap.[DataSourceAllocName],a.Qty , b.Profit_Center
order by c.SKU_NO,vwMap.[DataSourceAllocName]
GO
i have to send the result of aforesaid script in batch of 300 records per file (tab delimited text file)
now the file name must be dynamically created as each file will contain 300 records.
I have found some document related to same issue on this url
http://forums.microsoft.com/TechNet/ShowPost.aspx?PostID=1238184&SiteID=17
but still there is a catch.
Can any one guide/suggest me better way to do the aforesaid.
Thanks
hi , I am trying to copy files dynamically from a remot folder by using File System task ,
the files are going to be generated daily with f1yyyy-mm-dd.txt
I created 3 file connections :
- Existing remote folder (RemotFolder) and set that to var v_remotefolder
- Existing destination folder (DestFolder)
- existing file (filename) and set the expression to var (string ) =v_fname
in the File System task I set the the
Destination connection = DestFolder
Operation =Copy file
Source = filename
expression >>Property (source) = @[User::v_remotefolder] + @[User::v_fname]
I know I need the date too ,but I 'm getting error on this step.
Can you please tell me if you know how to resolve this without using For each loop since I don't have control on the remote folder and I don't want to copy unneeded files (the files are 10 GB daily) and get delete it once every month.
-I know this not uncommon task but I can't find any example.
Thanks
I have text data files from a third party and they use comma as field delimiters and enclose the text for each column in double-quotes. Not a problem for most of the data files until they start sending files where there is " within the column values. SSIS package fails with the error:
The column delimiter for column "Column 1" was not found.
Any ideas on how to resolve this issue will be greatly appreciated.Thankspcp
I've just started using SQL Server 2005 Integration Services and
I've come up against a situation where I need to name output files
dynamically (i.e. based on a timestmap). Looking through this
newsgroup, and other web resources it looks like my only option is to use a Script Task/ActiveX Script Task to rename the file after it has
been created with a generic file name.
I was wondering if there was a different approach or if there was a way to pass in a variable to the file connection manager that I could append to the file name at run-time.
Thanks
Bill
I am pretty new to SQL Server 2005 and SSIS. I am trying to develop a package that will dynamically load files into SS2005 based on the contents of a configuration table. The configuration table (see below for example) contains the path to the file, a flag indicating whether or not to process the flag, the type of file (specifies the nature of data -- financial, order, etc.) and some parameters specific to each file.
FileName
ProcessFlag
Type
ExcelTab
Param1
C:File1.xls
TRUE
1
Sheet1$
C:File2.xls
TRUE
1
Sheet1$
C:File3.xls
TRUE
1
Sheet1$
C:File4.xls
TRUE
2
Sheet1$
C:File5.xls
FALSE
2
Sheet1$
C:File6.csv
TRUE
3
C:File7.txt
TRUE
4
Right now I basically have a seperate sequence for each of the file types. The task in each sequence are virtually identical with the exception of the the data flow source in the data flow task (since the source file could be .xls, .txt, .csv). The first sequence ran fine in isolation, but when I linked a second sequence I started getting a Package Validation Error:
DTS_E_CANNOTACQUIRECONNECTIONFROMCONNECTIONMANAGER
I use the same variables (FileName, ExcelTab) for each of the sequences, so I am not sure if that causes te error. I also tried changing the ValidateExternalMetadata setting to false, since the connection variable wont be applicable until that sequence is being processed. I am not sure where to go here ... should I rearchitect how the package all together?
Is there a better/most efficient way to to architect this package to handle the different file types (develop a package for each type which is called by a master package, create one package with a different sequence container for each type, etc).
Any help would be greatly appreciated!!
Hello,
I am trying to create a sp dynamically. However each time, I run my sp, which creates the dynamic SQL for the sp, then it automatically changes:
CREATE PROCEDURE .....
to
Alter PROCEDURE ....
Then I am obvoiusly getting the error msg:
Invalid object name.
How can I get rid of that?
I am pretty new to SQL Server 2005 and SSIS. I am trying to dynamically load files into SQL Server 2005 using files/paths contained in tables. I have a key table in my SSIS package that defines which files should be processed, some default values that will be associated with each file (ie CompanyKey and PeriodKey) and the file path (see below for example):
CompanyKey
PeriodKey
ProcessFlag
FileName
2
61
FALSE
2005TB200501.xls
2
62
FALSE
2005TB200502.xls
2
63
FALSE
2005TB200503.xls
2
64
TRUE
2005TB200504.xls
2
65
TRUE
2005TB200505.xls
2
66
TRUE
2005TB200506.xls
2
67
TRUE
2005TB200507.xls
2
68
TRUE
2005TB200508.xls
2
69
TRUE
2005TB200509.xls
2
70
TRUE
2005TB200510.xls
2
71
TRUE
2005TB200511.xls
2
72
TRUE
2005TB200512.xls
The package I am developing is supposed to loop through this table checking for files with ProcessFlag = TRUE. For those files, it will load all of the records/columns (prefaced with the company and period keys) into a common table.
Do I have to manually create data sources for each file or is there a way to dynamically define the connection and process the connection?
Any assitance you could provide would be greatly appreciated!
Thanks.
Vik
Hi,
There is one manual process done by my friend. The process is " In query analyser, she runs a sql query and the resultset of Sql query, she copies and pastes into a excel workbook".
I wanted to write a SSIS package to automate it and providing a UI, in which she can give connection details and paste the SQL query and the package will in turn generate excel workbook in the target path". I thought of trying ExecuteSQL task, but the output resultset of ExecuteSQL task cannot be mapped dynamically to excel source(correct me if I am wrong). If I use Dataflow task, then also whether dynamic mapping of SQL output to Excel destination input is possible?
Can you please tell me the best approach for achieving the above functionality?
Thanks,
Venkataraman R
Hi! I am using VS 2003 (v 1.1)
I need to generate tables with the values from the Backend (SQLServer 2000) database in C#.Net.
How can i create the tables, tablerows, cells, etc. from the codebehind page of C#. I am having a very little knowledge about dynamic generation.
Give me the complete code with can example (if possible)
Thanks & Regrads
Jai Shankar
Hi,
I have N1 table where columns name(id,Field). Base on the fields of this table I want to create N2 table from SP where data from N1 will be columns in N2.
id Field
-- ------
1 ID
2 First
3 Last
Create table N2(ID,First,Last)
regards,
Mark
I'm currently developing an RDF application which need to handle lots of datatypes. But I want to use SQL-Servers capabilities for efficient querying and sorting. Therefore, I've wanted to create a Main Table which stores a Reference to the Table where the Data is stored. The Data itself should get stored in a Datatype-specific Table.
The Typed Table might get created by something like:
public void CreateTypedLiterals(Type type)
{
String sql = String.Format(
"CREATE TABLE [Literals_{1}] (" +
"ID int DISTINCT NOT NULL, Value {1})",
// BUG: does not work
// WARNING: introduces a potential sql-injection problem
type.ToString()
);
...
As you can see on the statements this solution makes many troubles. So I've wanted to implement it in a more fine way using a DataTable:
[SqlProcedure]
public void CreateTypedLiterals(Type type)
{
DataTable TypedLiterals =
new DataTable(
String.Format("Literals_{0}", type.ToString()));
TypedLiterals.Columns.Add(
"ID", typeof(int), "DISTINCT NOT NULL");
TypedLiterals.Columns.Add("Value", type);
...
But I have totally no Idea how to fetch this result into the existing Database. It might be cool to simply access the Database as it would be a .NET Dataset in the form:
using(Microsoft.SqlServer.Server)
{
CurrentDatabase.Tables.Add(TypedLiterals);
}
But this is afaik not possible. Has anybody an idea how to solve this issure?
I am trying to dynamically create subtotals base on values in a column. I also have to group by year. I have gotten the group by year. I used the (previous function) to check for the year and col1 not being equal. I am getting the first subtotal back, don't know how to proceed in my function to return the remaining subtotals
i.e.
col1 col2 col3
aaa 111 2005
bbb 222 2006
aaa 333 2006
ccc 444 2005
ccc 555 2006
bbb 666 2006
bbb 777 2006
ddd 888 2007
2005
aaa subtotal: 1
ccc subtotal: 1
total: 2
2006
aaa subtotal: 1
bbb subtotal: 3
ccc subtotal: 1
total: 5
2007
ddd subtotal: 1
total: 1
grandtotal:
aaa 2
bbb 3
ccc 2
ddd 1
totalnumber 8
Hi there!
I have a certain scenario and would like to know if it would be recommendable to try and realize it through RS.
I have a matrix of sales data (countries, regions, categories, total sales, Year-To-Date, changes, etc.) that need to be shown on charts. The charts themselves have a single format (bars). The out put needed is a relatively large number of charts (3000+) that would allow for a great number of comparisons (country to country, region to region, etc.). All these charts share a single layout. What would be varying greatly are the values and the labels attached to the elements on the chart (labels of the bars, title of the chart, legend, footnotes, etc.)
My plan is the follwoing:
- create a .net module that would prepare the data for the chart. This would be done with two tables
- parent table with general information
- child table with the the values for the bars and the labels
- Create a parametrized report in RS that accepts the Id of a report from the parent table
- Call this report for each and every report that can be found in the parent table
I would really appreciate any information of the feasabilitiy of this plan. I have worked a lot with Crystal before, but I don't have it anymore and will be using RS from now on. I haven't really started working with RS as of yet, but would need to know if this is "doable" with RS!
Thank you very much in advance for your replies!
Greetings
NB
Every day an application creates new tables and dumps static info into them.
I would like to create a package to dynamically export those database tables to raw files for long term archive, one file per table. Here is what I have so far and the issue I am having.
1) Get a list of un-archived tables.
2) Foreach table do the following.
a. Export the table into raw file.
b. Zip the raw file.
c. Update archive tracking table.
As long as the metadata for each table is the same this package seems to work fine. However, I have many tables with different metadata. How can I dynamically get the package update the metadata column collection when it hits a new table? When it hits a table with different metadata I am getting warnings like this:
The column "some_column" needs to be added to the external metadata column collection.
The "external metadata column "someother_column" (103)" needs to be removed from the external metadata column collection.
Then I get this error:
Error: 0xC004706B at dump the table into a raw file, DTS.Pipeline: "component "OLE DB Source" (1)" failed validation and returned validation status "VS_NEEDSNEWMETADATA"
I have developed an SSIS package which extracts and creates 5 flat files and finally using Process Extraction task zip the folder. On my Dev environment everything is working fine but when I am moving to SIT and UAT, not able to set up jobs dynamically by importing XMLConfig file.I created variables and assigned values but still it doesnt take.Below are varaibles I created for flat file destination, Arguments and Working Directory (for zipping)On UAT when I go to SQLAgentJobs to set, import .dtsx file, XML config file....the new values doesnt appear. why ?DataSource is taking always dev location....why ? How can I set it up to take dynamic values what I mentioned in config file ?
View 14 Replies View RelatedHi friends,
I have a text box with n number of text.
I want to set the font weight of each text in the textbox dynamically..
For eg.. suppose the text of the textbox is "Hello Friends", then i need "Hello Friends" as output.
Is there any way to accomplish this in SQL Reporting Service.
Any help will be appreciated. Its critical.
Please help me out ASAP.
I am having SP which gives, two result sets. The columns which are coming from result sets are also dynamic.
i.e. some time 5 columns and some time 10 columns.
Now I want to load this output into 2 different tables on daily basis. This would be truncate/delete table and load again.
Now my problem is that as I am not sure about columns, Is it possible to create table(Physical Table) depends on output of SP, and after load data into it.
During each load we can drop table, No issue and we can handle this through SSIS Package.
Hi,
There is a table exists in a database, I have to write a stored procedure to create the same table in different database, with the same column name and field. This should be done in runtime. Is it possible. The table will be passed as a parameter to the stored procedure.
I have a need to create a database, and then populate it. However, thecode below doesn't work as I hoped it might (it creates the table inthe "master" database, which is Not A Good Thing). I know already(thanks Tony!) that if you use Dynamic SQL for the USE command, thenthe subsequent operations need to be Dynamic SQL as well, which is apity since there are over 11,000 lines of it and I don't really fancydebugging it!Does anyone have a cunning plan? In a nutshell, I would like to beable to:1. Create a new database with a derived name - as in the case below aname based on the year, though it might be month as well.2. Create and populate tables in this new database.These operations would ideally be running from a scheduled job.Any thoughts?TIAEdward====================================USE MASTERDECLARE @DBName VARCHAR(123)SET @DBName = 'MyTest_' + CAST((Year(Getdate())) AS Varchar)if not exists(select dbid from master.dbo.sysdatabases where name =@DBName)exec('CREATE DATABASE ' + @DBName)elseraiserror('Database already exists.',3, 1)EXEC ('USE ' + @DBName)if exists (select * from dbo.sysobjects where id =object_id(N'[dbo].[TestTable]') and OBJECTPROPERTY(id, N'IsUserTable')= 1)drop table [dbo].[TestTable]GOCREATE TABLE [dbo].[TestTable] ([TestID] [int] NOT NULL ,[Description] [varchar] (50) COLLATE Latin1_General_CI_AS NULL) ON [PRIMARY]GO
View 3 Replies View RelatedHi,
how to dynamically create columns for a table
Hi,
I need to dynamically create a script component (by code).
Is it possible?
Thanks ahead,
Eran
hi,
Can you guys please give me the steps of how to create a destination file dynamically. What i mean is for example i want to get everything from a table and ssis should create a file distination according to today's date and save it in a specific folder.
thanks
I have a parameter in my report with 2 options.
1."Less than Jan 1 2007"
2."Greater than Jan 1 2007"
According to the option selected, I should have the query look like:
1."Less than Jan 1 2007"
select * from table_name
where open_date < Jan 1 2007
or
2."Greater than Jan 1 2007"
select * from table_name
where open_date > Jan 1 2007
Please let me know how can I acheive this.I'm Using ORACLE database
Log-in access web-page.When page is accessed a stored procedure is executed that will provide a list of all the RA(Problem -- when a new building is added, this stored procedure must be opened and syntax added)..Results are stored in a drop down list on the page.Create a table holds list of all buildings something similar to this
Create Table [dbo].[BuildingInformation] (
ID INT IDENTITY NOT NULL,
BuildingName varchar(50),
RAName varchar(50)
[code]....
Alter the stored procedure so that when it is run it will iterate the BuildingInformation table and compare the value in BuildingInformation to the values in the field BuildingName of the view [RA]. If all buildings contained in the table [BuildingInformation] exist in the view [RA] nothing will change. If their are values in the [BuildingInformation] table that do not exist in the view [RA], the view [RA] will be dropped and recreate with a 'Union ALL' for each building in the table
I' m using Microsoft Reporting. Is there a way to create Report .rdlc file Dynamically on the fly? I' m using VB.net.
Any help can be appreciated.
Thanks
Mythili
how do you load the multiple flat files to into destination dynamically?
View 9 Replies View RelatedHi,
I have a dts package that imports a number of text files into a SQL Server 2000 database table. The package has been set up to accept a text file with a row delimiter of carriage return and line feed ({CR}{LF}). Some of the text files I receive only have a line feed ({LF}) as the row delimiter and the dts package fails the file. Is there an activex script i can use that will scan the file and change the row delimiter as required?
i was going to use the filesystemobject which allows me to read a line at a time, however the Readline method doesn't read the new line character. The text files are too big to read into one variable and then do a replace.
Any help would be appreciated
I am using SQL Server 2000 SP3, Windows Server 2000 and Windows XP Professional. All systems are fully patched
Regards Justin
Hi all
I hope someone can help me on this issue:
I am designing dynamical reports, that are abel to run in four different languages - so far everything works just fine for me (I select my headers and labels from a database). Only problem is the prompt text for report parameters. How can I set this text dynamically?
E.g. I have a parameter where the user selects a storage - this prompt should be "Storage: " in the English report and "Lager: " in the Danish report. I have absolutely no idea how to fix this.
I am greatful for any help.
Hi ,
I have a table (source file) where all sourse file(its all text file) name and location stored. i have a dts packeage, through this package all the sourse file will upload. So I need to dynamically change the source file name in dts packages.
How i can do it??????
Can any body plz help me out!
Thnaks
SAndipan
Good Afternoon,
So this one has been bugging me for a while and I am ready to punt...
Is it possible to dynamically create a text file destination in SSIS and then pump the results of a query stored in a variable to this text file?
So my package looks like this
1) SQL task that pulls back a list of tables to be exported
2) For Each Loop ADO enum that passes the table name to a SQL Task that builds the select...ie select * from <DTS.Variables()>
3) Data flow task that sets the command from variable from step 2
4) Text File destinaiton that is built using a varable as the connectionstring
I am delaying validation in steps3 and 4 above without any luck...basically I am curious if i can even do what I am thinking I should be able to do here...I get as far as getting metadata errors because SSIS can't seem to handle dynamically filling the pipeline with the columns from the variable/SQL statement.
Am I missing something? Is this possible?!
Thanks in advance.
Dave
I am having problem with 'TOP @pageSize'. It doesn't work, but if I replace it by 'TOP 5' or 'TOP 6' etc., then the stored procedure runs without errors.
Can someone please tell me how I could use @pageSize here so that it dynamically determines the 'n' of 'TOP n' ?
ALTER PROCEDURE dbo.spGetNextPageRecords
(
@pageSize int,
@previousMaxId int
)
AS
/* SET NOCOUNT ON */
SELECT Top @pageSize ProductId, ProductName
FROM Products
WHERE (ProductID > @previousMaxId) order by ProductId
RETURN