Storing A Dynamic File Name Table
Apr 17, 2014
I have a file that is automatically generated by an external process that will always have the same name + a date stamp.I will say test_(Datestamp).txt
what i am trying to do is use sql to bulk insert this file but i will not know the full file name as the date stamp is also includes time. What my script is doing is taking this file everyday storing its contents temporarily in a table where i then use a trigger to edit and repopulate the table and kick out a new file that another external process uses.
View 4 Replies
ADVERTISEMENT
Apr 6, 2015
I have a dynamic sql which uses Pivot and returns "technically" variable no. of columns.
Is there a way to store the dynamic sql's output in to a temp table? I don't want to create a temp table with the structure of the output and limit no. of columns hence changing the SP every time I get new Pivot column!!
View 3 Replies
View Related
Jul 16, 2002
Is it possible to store a .wav file in a SQL Server table? I have looked in books online and on the web... to no avail. Any info would help.
Thanks
Mike
View 1 Replies
View Related
Jan 14, 2002
Hi ... I need to read rows from a large source file , check if the data selected already exists in the destination and if it does, upate the destination , else insert a new row . Now i could use a sp, but that means i have to call it for each row in the text file ...
any ideeas on what kind of package can be created /
cheers
benjamin
View 1 Replies
View Related
Dec 29, 2006
Hello there,I just want to ask if storing data in dbase is much better than storing it in the file system? Because for one, i am currenlty developing my thesis which uploads a blob.doc file to a web server (currently i'm using the localhost of ASP.NET) then retrieves it from the local hostAlso i want to know if im right at this, the localhost of ASP.NET is the same as the one of a natural web server on the net? Because i'm just thinking of uploading and downloading the files from a web server. Although our thesis defense didn't require us to really upload it on the net, we were advised to use a localhost on our PC's. I'll be just using my local server Is it ok to just use a web server for storing files than a database?
View 6 Replies
View Related
Mar 2, 2014
I am trying to create an ssis package with dynamic csv file as output. and out format contains query output.
sample file name:
Unique identifier + query output + systemdate();
The expression is looking like this.
@[User::FilePath] + @[User::FileName] + ".CSV"
-- user filepath is a variable from ssis package. File name is the output from SQL query. using script task i have assigned the values to @[User::FileName] .
When I debugged the script task the value getting properly but same variable am using for Flafile destination. but its not working.
View 3 Replies
View Related
Sep 26, 2006
How would I import a dynamic file name into a known table?I have the file name in a variable. So what object in SSIS do I use to do this. Thanks
View 2 Replies
View Related
Apr 13, 2007
Hi All,
I am really in a great trouble. My requirement is quite complex, as I feel, it may be quite simple for some of you.
Here is my problem -
Actually I am doing a project, where client has a specific requirement. i.e. he want's to build a query on runtime for selecting particular record he wants, so that we have provided a user interface where he can select any datasource e.g. SQL, Oracle, Excel etc. He also specifies the database name. He is displayed all the tables and columns under those tables. He selects columns from those table boxes and write the query he wants, with where clause, if required.
I am saving these query in a table, storing column names in another table where we map those oringinal column names with columns of another table, wehre we want to store actual result set of the prepared query by the user.
for examaple ..
if user preapare query like - 'SELECT CUST_ID, CUST_NAME FROM CUSTOMER
WHERE CUST_ID = 10'
In case of above query I will store CUST_ID in column COL1 of table TAB1 and CUST_NAME in COL2 of table TAB1, like that we have such fifty columns in that table. Now question is here every thing is dynamic data provider, database, tables, columns and where clause, then how can get the result set out of those queries and store that query output in the that storage table with columns col1, col2 and so on, upto 50 columns.
Please help me on this, as it is so urgent.
I will be very much thankful to you people.
Thanks & regards,
Praveen Kadam
View 3 Replies
View Related
Aug 30, 2007
Hello all:
Here is a sample query:
DECLARE @KEYID NVARCHAR (50) ; SET @KEYID = '1074958'
DECLARE @ENTITY NVARCHAR (100); SET @ENTITY = 'HouseDimension'
DECLARE @KeyCol NVARCHAR(50);
SET @KeyCol = (SELECT LEFT(@ENTITY, (SELECT CHARINDEX( 'DIM', @ENTITY) -1)) )+ 'Key'
DECLARE @KeyValue NVARCHAR (1000)
SET @KeyValue = 'SELECT '+ @KeyCol + ' FROM HouseManagementFact WHERE HouseKey = ' + @KEYID +
' GROUP BY ' + @KeyCol + ' HAVING SUM(TotalClaimCount) > 0 OR SUM(HouseCount) > 0 '
The value resulting from Executing @KeyValue is an integer.
I want to store this value in a new variable say @VAR2
When I do this
DECLARE @VAR2 INT
SET @VAR2 = execute sp_executesql @KeyValue
its giving me an error.
can somebody let me know the correct form of storing the value resulting from @KeyValue in some variable ?
View 3 Replies
View Related
Jun 11, 2015
I have a requirement to load bulk of csv files to sql table. some times, some columns could not come in csv file(some times 100 columns and some times 80 cloumns). That time the package is getting failed. How to create a table dynamically based on csv file structure.
View 8 Replies
View Related
Nov 21, 2007
Is there any method to store the MP3 file in database. And also I want free text search in the table? Please help me its important
Thanks in advance
View 3 Replies
View Related
May 26, 1999
HI,
We are using a VB component which utilises a FILE DSN to connect to SQL but we have noticed that when using SQL Authentication in setting up the DSN in the control panel the password does not get saved. So the only way we can connect is to have a user with no password i.e blank.
we only noticed this when we tried to use a user other than sa which happened to have a password that is blank.
Has anybody else found a solution to this problem ?
View 1 Replies
View Related
May 26, 2004
How can i store a word file in my database table..
View 3 Replies
View Related
Apr 21, 2008
Hello,
In SQL 2005, what is the best way to take a text file and store it in a table field, then later extract that file to a directory with original name and format intact?
Thanks!
--PhB
View 1 Replies
View Related
Nov 5, 2007
How can I store the big files in the SQL Database, like MP3 files or an image or an video file?
View 3 Replies
View Related
Jun 23, 2004
I have records with 50 plus fields of data.I was thinking of writing all the fields data into a XML file and then store it in a SQL server database for retrieval later on.Is there anyway i can go about doing this?
View 6 Replies
View Related
Oct 2, 2001
How do I store a sound/wave file in SQL Server. Does SQL Server have the facility to store sound files in the database, if so what data type should be used and functions related to it.
Thanks
Paul
View 1 Replies
View Related
Jan 16, 2008
Does anyone know whats wrong with my code?
~Nothing wrong with my code, code is correct!
I let it run and it keeps saying that the file does not exist. Seem as though, the path of my file doesn't exist! Any suggestions on how to program the path of my file so I can be able to retreive it such as open the file and access the file?
CODE:
Imports System
Imports System.Data
Imports System.Data.Sql
Imports System.Data.SqlClient
Imports System.Data.SqlTypes
Imports System.Web
Imports System.Configuration
Imports System.Reflection
Imports System.Security.Permissions.FileIOPermission
Imports System.IO
Imports System.Collections
Imports System.Collections.Generic
Imports System.Windows.Forms
Imports System.Security.Permissions
Imports System.Windows.Forms.Form
'<Assembly: PermissionSetAttribute(SecurityAction.RequestMinimum, Name:="Local Intranet")>
'<Assembly: PermissionSetAttribute(SecurityAction.RequestOptional, Unrestricted:=True)>
Public Class Form1
Inherits Form
Private Sub Button1_Click(ByVal sender As Object, ByVal e As System.EventArgs) Handles Button1.Click
Dim SQL As String
Dim rawData() As Byte
Dim RowsAffected As Integer
Dim filename As String
filename = "H:My Documentsabc.pdf"
Try
If System.IO.File.Exists(filename) Then
Dim filePermission As New System.Security.Permissions.FileIOPermission(System.Security.Permissions.FileIOPermissionAccess.Read, System.Security.AccessControl.AccessControlActions.View, filename)
filePermission.Demand()
Else
MessageBox.Show("file does not exist", "File", MessageBoxButtons.OK, MessageBoxIcon.Information)
End If
Dim fs As FileStream
fs = New FileStream(filename, FileMode.Open, FileAccess.Read)
Dim FileSize = Convert.ToInt32(fs.Length)
rawData = New Byte(FileSize) {}
fs.Read(rawData, 0, FileSize)
SQL = String.Format("INSERT INTO test_file_save(FILE_ID, FILE_NAME, FILE_SIZE, CREATE_FILE) VALUES({0}, {1}, {2}, {3})", _
1, filename, FileSize, rawData)
Using Conn As New SqlConnection("server=1234;" _
& "uid=F_User;" _
& "pwd=password;" _
& "database=Database")
Using Cmd As New SqlCommand("sp_InsertPDF", Conn)
Cmd.CommandType = CommandType.StoredProcedure
Cmd.Parameters.Add(New SqlParameter("@FILE_ID", 1))
Cmd.Parameters.Add(New SqlParameter("@FILE_NAME", filename))
Cmd.Parameters.Add(New SqlParameter("@FILE_SIZE", FileSize))
Cmd.Parameters.Add(New SqlParameter("@CREATE_FILE", rawData))
fs.Close()
Conn.Open()
RowsAffected = Cmd.ExecuteNonQuery()
End Using
End Using
MessageBox.Show("File Inserted into database successfully!", _
"Success!", MessageBoxButtons.OK, MessageBoxIcon.Asterisk)
Catch ex As Exception
MessageBox.Show("There was an error: " & ex.Message, "Error", _
MessageBoxButtons.OK, MessageBoxIcon.Error)
End Try
End Sub
End Class
View 5 Replies
View Related
Feb 20, 2006
I have a package that processes a flat file, which is generated by a separate system each weekday around 3:30AM. I would like my package to store the timestamp of the input file and compare it to the timestamp of the next day's file. If they are the same, then the package can exit without reprocessing a particular file again.
So, my first question is whether a package variable can be updated so that its value will persist from one run to the next?
Second, what is the syntax to retrieve the file's timestamp, either directly from the file connection manager or within a script task?
Thanks,
Phil
View 1 Replies
View Related
Nov 17, 2006
I am trying to create a file DSN to link a password protected access satabase to SQL server anywhere backend. I create the DSN using Microsoft Access Driver, Select my access database, go to advanced properties and type in the database password (user ID is admin). However, everytime I change records in the SQL front end, I am prompted to enter the Access database password. It doesn't seem to be storing it.
Any suggestions?
View 1 Replies
View Related
Dec 27, 2000
hi all...
i want to store the output of an sql query as a text file.this is for my project i am doing in java.can u help me how to do this..it is very urgent
thanking u
View 2 Replies
View Related
Sep 13, 2004
Hi,
i am using command files(.cmd) and osql utility to conneft to a database.
following is the requirement.
I have a query "SELECT top 1 au_fname FROM pubs..authors".
Can i store the value of au_fname in the command file?
Is there any work around for this.
Any help is appreciated.
Thanks.... :)
View 1 Replies
View Related
Sep 29, 2015
I have to use sql cmd and run diagnostic queries.
I need to run multiple dmvs as a batch file and storing the dmv result to some place.
View 9 Replies
View Related
May 27, 2008
The error I recieve is as follows:
"[Flat File Destination [13]] Error: Data conversion failed. The data conversion for column "SDATA" returned status value 4 and status text "Text was truncated or one or more characters had no match in the target code page.". "
"[Flat File Destination [13]] Error: Cannot copy or convert flat file data for column "SDATA". "
"[DTS.Pipeline] Error: SSIS Error Code DTS_E_PROCESSINPUTFAILED. The ProcessInput method on component "Flat File Destination" (13) failed with error code 0xC02020A0. The identified component returned an error from the ProcessInput method. The error is specific to the component, but the error is fatal and will cause the Data Flow task to stop running. There may be error messages posted before this with more information about the failure. "
"[DTS.Pipeline] Error: SSIS Error Code DTS_E_THREADFAILED. Thread "WorkThread0" has exited with error code 0xC02020A0. There may be error messages posted before this with more information on why the thread has exited. "
"[DataReader Source [207]] Error: The component "DataReader Source" (207) was unable to process the data. "
"[DTS.Pipeline] Error: SSIS Error Code DTS_E_PRIMEOUTPUTFAILED. The PrimeOutput method on component "DataReader Source" (207) returned error code 0xC02090F5. The component returned a failure code when the pipeline engine called PrimeOutput(). The meaning of the failure code is defined by the component, but the error is fatal and the pipeline stopped executing. There may be error messages posted before this with more information about the failure. "
"[DTS.Pipeline] Error: SSIS Error Code DTS_E_THREADFAILED. Thread "SourceThread0" has exited with error code 0xC0047038. There may be error messages posted before this with more information on why the thread has exited. "
I am selecting data from an OBDC then copying it to a text file an I always get this error, but when I change the destination to a excel file it works perfectly. But the whole point of the package is to copy to a text file.
Please could you help me, your replies will be greatly appreciated.
View 5 Replies
View Related
Dec 21, 2005
I am using FileUpload control in ASP.net 2.0 to upload files and store them into SQL server database as an image. I am fine with MS office files, image files and etc. We have Product Center (Proe) engineering software to configure parts and the files generated through this software have PLT extension when I store these files, the file type is Plian/text. I am using Fileupload.PostedFile.Content to get the file type. How can I store PLT, TIF files in SQL server? Please help.
View 12 Replies
View Related
Mar 5, 2007
I am looking for some microsoft recommendation on storing images in SQL Server Vs File System. Here is what our requirement is.
Currently we are using SQL Server 2000 with a VB6(COM+) & ASP front end. Our architecture involves Log Shipping & Replication. We use log shipping for our DR site and Replication to replicate to Datawarehouse & DB2 enviornment. I have been trying to research this topic and have come up with opinion favoring both sides mostly file system but nobody seems to point any Microsoft recommendation.
With our application we are looking for Transactions, backups, Log shipping & replication with these images. I have read MS implementation of TerraServer and there they recommend storing them in SQL Server if above is required. (https://www.microsoft.com/technet/prodtechnol/sql/2000/reskit/part3/c1161.mspx?mfr=true)
Can somebody please point me to some Microsoft recommendation and what does SQL Server 2005 offers in this aspect.
Thanks
View 1 Replies
View Related
Oct 10, 2006
Dear all I create this html file on the fly in my asp.net application abd what i would like to do is to store it inside my sql2005 database. What would be the best way?The html file itself is not really big. Probably not more then 600 - 800 characters most. I was thinking the text type fields of the database and then when retreiving it dump it inside in a file and save the file with html extention. Are there any better sugestions? Thank you for your time
View 3 Replies
View Related
Mar 4, 2008
Hello,
We have some queries that are long and intensive. We have thought about running the queries and storing the data in a text file for lookup from our website.
Example: Our online store only displays items that are in stock so when a user selects a category a query runs and grabs only items that are in stock and then displays them. There could be thousands of items the query needs to sort through before displaying the items that are in stock. What if we ran this query once every hours an stored the results in a txt file? The asp page would then go to the text file to grab the results instead of having to run the query every time a user selects a category. Will this speed up the site by not having to query every time? Would this be a correct way to eliminate queries that run thousands of times a day?
thanks
Andy
View 2 Replies
View Related
Jul 24, 2015
Need to know how I can get the dynamic filename created in the FlatFile destination for insert into a package audit table?
Scenario: Have created a package that successfully outputs Dynamiclly named flat files { Format: C:Test’Comms_File_’ + ‘User::FileNumber’+’_’+Date +’.txt’
E.g.: Comms_File_1_20150724.txt, Comms_File_2_20150724.txt etc} using Foreach Loop Container :
* Enumerator Set to: “Foreach ADO Enumerator” with the ADO object source variable selected to identify how many total loop iterations there are i.e. Let’s say 4 thus 4 files to be created
*Variable Mappings : added the User::FileNumber – indicates which file number current loop iteration is i.e. 1,2,3,4
For the DataFlow task have a OLDBSource and a FlatFile Destination where Flat File ConnectionString is set up as:
@[User::Output_Path] + "Comms_File"+ @[User:: FileNumber] +"_" + replace((DT_WSTR, 10) (DT_DBDATE) GETDATE(),"-","")+ ".txt"
All this successfully creates these 4 files:
Comms_File_1_20150724.txt, Comms_File_2_20150724.txt, Comms_File_3_20150724.txt, Comms_File_4_20150724.txt
Now the QUESTION is how do I get these filenames as I need to insert them into a DB Audittable. The audit table looks like this:
CREATE TABLE dbo.MMMAudit
(
AuditID INT IDENTITY(1, 1) NOT NULL,
PackageName VARCHAR(100) NULL,
FileName VARCHAR(100) NULL,
LoadTime DATETIME NULL,
NumberofRecords INT NULL
)
To save the Filename & how many records in each file in our Audit Table, am using an Execute SQL Task and configuring it as this:
Execute SQL Task
Parameter mapping - Mapped the User Variable (RecordsInserted) and System Variable( PackageName) to Insert statement as shown below
SQLStatement: INSERT INTO [dbo].[MMMAudit] (
PackageName,NumerofRecords,LoadTime)
(?,?.GETDATE)
Again this all works terrific & populates the dbo.MMMAudit table as shown below BUT I also need to insert the respsctive file name – How do I do that?
AuditID PackageName FileName NumberOfRecords
1 MMM NULL 12
2 MMM NULL 23
3 MMM NULL 14
4 MMM NULL 1
View 2 Replies
View Related
Oct 9, 2006
I am having an issue with the File System Task.
I was wondering if there is a way to 'Move File' with the File System Task inside of a For Each Loop container but to dynamically set the Destination path variable.
Currently, this is what I have:
FileDestinationPath variable - set to C:TestFiles
FileSourcePath variable - set to C:TestFiles
FileNameAndLocation variable - set to blank
For Each Loop Container Iterates through a folder C:TestFiles that has .txt files in it with dates in the file name. Ex: Test_09142006.txt. Sets the file path (fully qualified) to the Variable Mapping FileNameAndLocation.
Script Task (within For Each Loop, first step) Sets the FileDestinationPath to the correct dated folder within C:TestFiles. For example, if the text files I want to move are for the 14th of September, it takes FileDestinationPath and appends the date folder to the end of it. The text files have a date in the file name (test_09142006.txt) and I am picking this apart (from FileNameAndLocation in the For Each Loop) to get the folder date. (dts.Variables(User::FileDestinationPath?).Value = dts.Variables(User::FileDestinationPath?).Value & ? Month & _? & Day & _? & Year & ?) which gives me C:TestFiles 9_14_2006?.
File System Task (within For Each Loop, second step) This is where the action is supposed to occur. I want it to take the FileDestinationPath and move the FileNameAndLocation file (from the For Loop) into this folder for each run.
Now as for my problem. I want this package to run everyday but it has to set the FileDestinationPath variable dynamically according to that days date. Basically, how do I get this to work since I cant hard code the destination path variable from the start? I have the DestinationVariable on the File System Task set to the FileDestinationPath variable, after the script task builds it. However, using FileNameAndLocation as the SourceVariable on my File System Task tells me that the Variable FileNameAndLocation? is used as a source or destination and is empty.?
Let me know if I need to clarify further...I may be missing something very simple. Any help would be greatly appreciated!
View 10 Replies
View Related
Aug 9, 2007
I want to store a Zip file as a BLOB, but I get an error:
"File 'C:<path of mdf file> ' appears to have been truncated by the operating system. Expected size is 2560KB but actual size is 1536KB "
whenever the BLOB exceeds 1MB.
Any suggestions? How can I store larger .ZIP files to the Database?
I am using MS SQL 2005 Express and the data type that I gave for the column to store Blob is "varbinary(MAX)".
I am inserting byte array in to this field.
It works fine for a zip file less than 1MB but as soon as the zipped file size increases beyond 1MB, lots of error pop-up and then the database is not readable. Its says that, the data in the data base may have been corrupted...
View 5 Replies
View Related
Feb 27, 2008
Hi All,
I have a source files folder where the files generated everyday.
My goal is pick the latest file and copy this single file to another folder.
I used the Foreach loop container and got the latest file and stored the file name to a varible i.e. LatestFile
Then i want to use the File System Task to copy this to the destination.
On the beginning, I could not setup the Latestfile since I don't its name then, so when I setup the Source Connection property of the File system task, it is not allowed to leave the SourceVarible as blank!
Any suggestion?
Thanks
Micor
View 3 Replies
View Related
Mar 28, 2006
Hi,
we have one requirement to run the package daily basis.
The package should run at specific time on that day.
we are using windows schedular for that.
we will have one new flatfile everyday.
Is there any process to attach this file to flat file source dynamically?
The requirement is,
The flat file should be able to read the new flatfile everyday.
we have no option change it manually, the flatfile source should have to take the file automatically at that time.
So that it can take that flatfile and load it into database table.
View 1 Replies
View Related
