How To Extract Data From LDAP And Then Import It Into SQL Database (for Quicker Retrieval)
Apr 21, 2008
Hi Everyone,
Am a third year student doing work placement.
Could anyone please give me clues on how to go about extracting data from a LDAP and then into an SQL database?
1 A defined subset of data is to be extracted from GDS on a nightly basis,
2 Then imported into a SQL database for quick & easy retrieval.
3 A web interface is required to present data retrieved from the SQL database.
I will appreciate every assistance.
Regards
Lidiolo
View 6 Replies
ADVERTISEMENT
Aug 31, 2006
Hi all,
Having some fun with an excel import.
I have a excel sheet that has data that needs to be UpSert'ed into 2
different tables. In addition, I need to use a value in the spreadsheet
to determine the PK from a reference data table, for one of the UpSert
oprations.
That is all working now.
The thing I'm struggling with is something I am sure is quite simple -
but I'm not seeing a solution from attempts, googling or BOL.
2 of the columns I receive have either nothing, or X in them. The columns they go into are defined as BIT, NOT NULL.
So, in SQL it would be something relatively simple like:
CASE
When IsAvailable = 'X' then 1
When IsAvailable is null then 1
ELSE 0
end
But I don't know how to do this to data that was in a spreadsheet, and
now is a resultset being handed from a task to another task.
to outline my current solution:
---- table 1 = this all works -------------
Excel Source --> MultiCast (For Table 1)-->Data conversion for
table1-->:Sort for Table1--> Merge Join for table 1 (left Outer
join) as 'left' leg
Table1 Source --> Sort Table1 --> Merge Join for table 1 (left Outer join) as 'right' leg
Merge Join for table 1 --> Conditional Lplit for table1
Conditional Split for table1 (table1 source PK is null) -->Insert Into Table1 Destination
Conditional Split for table1 (table1 source PK is not null) -->Update Table1 OLE DB Command
---- table 2 = this needs to be able to convert X/NULL to BIT -------------
MultiCast (For Table 2)-->Copy Column for Table2 -->Data
Conversion for Table 2-->table3 lookup to get FK-->Sort for
Table2 merge-->Merge Join for table 2 (left outer join) as 'left' leg
Table2 Source --> Sort Table2 --> Merge Join for table 2 (left outer join) as 'right' leg
Merge Join for table 2 --> Conditional split for table 2
Conditional split for table 2(table2 source PK in null) -->insert into table 2
Conditional split for table 2(table2 source PK in not null) -->update table 2 ole db command
-----------------------------------------
Now, if I correct the spreadsheet to have 0's and 1's in the two
column, then the process above works. But I cannot (yet) force business
to do that.
If tried to use SQL Command for the excel source, but there is limited
functionality on the command - I cannot do SQL coalese, isnull or case
statements, which would allow me to resolve that data at source.
I've tried to use derived columns to alter the columns. I think that
the REPLACE (IsAvailable, VariableContainingX,VariableContaining1)
might work to change X's to 1, but that doesn't resolve the NULL issue.
I've tried to use a script component to handle the conversion - which
REALLY feels like a bad way to do this - the .Net script is wrote was:
-------------.net script code-------------
Imports System
Imports System.Data
Imports System.Math
Imports Microsoft.SqlServer.Dts.Pipeline.Wrapper
Imports Microsoft.SqlServer.Dts.Runtime.Wrapper
Imports Microsoft.SqlServer.Dts
Public Class ScriptMain
Inherits UserComponent
Public Overrides Sub Input0_ProcessInputRow(ByVal Row As Input0Buffer)
If Not (Row.EndOfRowset) Then
If
(Row.IsDotComVanEnabled_IsNull) Or (Row.IsDotComVanEnabled.Equals("X"))
Then
Row.IsDotComVanEnabled = "1"
End If
If
(Row.IsStoreCollectionEnabled_IsNull) Or
(Row.IsStoreCollectionEnabled.Contains("X")) Then
Row.IsStoreCollectionEnabled = "1"
End If
Row.NextRow()
End If
End Sub
End Class
------------------------------------------
I also tried it like this:
----------------1st draft .Net script code--------
Imports System
Imports System.Data
Imports System.Math
Imports Microsoft.SqlServer.Dts.Pipeline.Wrapper
Imports Microsoft.SqlServer.Dts.Runtime.Wrapper
Imports Microsoft.SqlServer.Dts
Public Class ScriptMain
Inherits UserComponent
Public Overrides Sub Input0_ProcessInputRow(ByVal Row As Input0Buffer)
If
(Row.IsDotComVanEnabled_IsNull) Or (Row.IsDotComVanEnabled.Equals("X"))
Then
Row.IsDotComVanEnabled = "1"
End If
If
(Row.IsStoreCollectionEnabled_IsNull) Or
(Row.IsStoreCollectionEnabled.Contains("X")) Then
Row.IsStoreCollectionEnabled = "1"
End If
End Sub
End Class
-------------------------------------------------
Those both threw the following error;
-------------error information-----------------
Script Component has encountered an exception in user code:
PROJECT NAME: ScriptComponent_a333140d269b413bb1bddef390da7e16
Object reference not set to an instance of an object.
at ScriptComponent_a333140d269b413bb1bddef390da7e16.ScriptMain.Input0_ProcessInputRow(Input0Buffer Row)
at ScriptComponent_a333140d269b413bb1bddef390da7e16.UserComponent.Input0_ProcessInput(Input0Buffer Buffer)
at
xScriptComponent_a333140d269b413bb1bddef390da7e16.UserComponent.ProcessInput(Int32
InputID, PipelineBuffer Buffer)
at Microsoft.SqlServer.Dts.Pipeline.ScriptComponentHost.ProcessInput(Int32 inputID, PipelineBuffer buffer)
-----------------------------------------------
Any ideas?
View 4 Replies
View Related
Jul 20, 2005
Hi,I wonder if anyone can help with the following on a fee paying basisfor the the design or development of some type of script or utility orpiece of code to do the following work.I wish to retrieve some data from an online database that is in thepublic domain. The online database has a search facility that matchesthe entry (name) in a search box then returns a screen stating that amatch has been found or not found.If a match is found there is a button to click that proceeds to thescreen containing the data which is simply two names. It is these twonames that I wish to retrieve and store them in something like a textfile where they are associated with the original entry (name) used inthe search box.I have a list of the entries for the search box that can be suppliedin sample format for testing as a columnar text file or commadelimited file or a spreadsheet.. I would need the procedure toprocess the list of search entries, retrieve the data then move on tothe next one in the list. Obviously, if a match was not found then theprocess would need to move on the the next entry in the list withperhaps a message saying "No Match" until the entire list wasprocessed.The PC I hope to run the process on is a Windows XP machine and if Ineed to purchase any particular software that is necessary for the jobthen I am quite willing to do so. Also, as I pointed out above I wouldpay for the work to be done.I hope that I have explained the above OK and that I have posted it tothe correct newsgroup(s). If it is not in the correct newsgroup Iwould be grateful if anyone could point me in the right direction.RegardsDave Gibson
View 1 Replies
View Related
Dec 12, 2006
This is something that should not be so difficult, but it is proving to be very cumbersome.
I need to query an LDAP database on a Linux server FROM a sql server, namely sql server 2005.
I have found bits and pieces, but can€™t seem to put it all together yet €“ it should be easy, but it€™s proving to be very difficult.
It seems like you can either link an LDAP server through sp_addlinkedserver OR through enterprise manager/sql server studio, you can create a LINKED SERVER.
The problem is HOW to do this.
Here are the parameters for the ldap server:
Server name: serverabc
Ldap info (which I don€™t know how to label) is: 'ou=Hosts,dc=mc,dc=vanderbilt,dc=edu'.
I was able to somehow get this to work last week, but now it€™s not working - it didn't return any data, as i don't have any ldap hierarchy specified, but at least it came back with "command completed successfully":
SELECT * FROM OPENQUERY( [serverabc],
'SELECT * FROM ''LDAP://serverabcu/ OU=Hosts,DC=mc,DC=vanderbilt,DC=edu''
')
With all the ldap usage out there, this is really frustrating not being able to find out how to do this.
Any help is greatly appreciated.
Thanks
View 3 Replies
View Related
Jul 20, 2005
Hi,Anybody have an idea of copying data from tables of a database to anotherdatabase. It should be a choice to select all tables, single table orseveral tables.For them knowing Oracle it is possible to do it with 'exp', where you canchoose to script the database with or without data. So I am trying to getalike to MSSQL. The job is to unload data from a database with onestructure to another database with another structure.Thanks in advanceBest regardsTom Frank
View 4 Replies
View Related
Sep 29, 2014
I'm trying to extract data from our database for the number of phone calls our reps are doing.
In counting the calls I only want to include up to 3 calls to the same customer (field name is CompanyID) per day - anything more than this is ignored.
The query at the moment is something like:
SELECT COUNT(CallID) AS CallCount FROM Sales_Calls WHERE CallDate >= '2014-09-01' AND CallDate <= '2014-09-30' AND RepID = 1
Using MSSQL 2012.
View 13 Replies
View Related
Sep 25, 2007
I have to extract data from 5 different oracle databasee with same schema.This will be scheduled job.Can someone guide me.
View 1 Replies
View Related
Sep 25, 2007
I have to extract data from 5 different oracle databases with same schema.This will be scheduled job.Can someone guide me.
View 1 Replies
View Related
Feb 27, 2007
Hi,
My task is simple, I want to use the execute sql task editor to grab a value from a database in Access and put it in a variable. The connection is via ODBC and the access database is protected by a password.
I've done all the preliminary stuff such as running profiler to make sure that the package is getting the call to the database, setting up the ResultSet to be "single row" in the general tab, mapped the Result Set correctly, but nothing works. I get the same error every time.
This is my sql command:
select count(FingerPrintID) as FingerPrint
from Employee
Result Set is set up like this:
Result Name: FingerPrint ; Variable Name: User:: varDataset
Here is the error I get:
Error: 0xC002F309 at Execute SQL Task, Execute SQL Task: An error occurred while assigning a value to variable "varDataset": "Value does not fall within the expected range.".
My variable is set up as a Int16.
Please help!!!!!
If you could provide step by step example's that would really make my day.
Thanks,
Lawrence
View 7 Replies
View Related
Sep 25, 2007
Please guide me urgently how to extract data in SSIS from 10 identical oracle database into 1 sql server database.
There is a table which list all the 10 databases.
View 2 Replies
View Related
Oct 31, 2006
Good afternoon SQL dudes Does anyone have any experience of extracting data from IBM's UniData (http://en.wikipedia.org/wiki/UniData) (or any post-relational Pick nested relational multi-valued relational database) into a SQL Server?More info here (http://www.rpbourret.com/xml/ProdsXMLEnabled.htm), here (http://www.pick-ware.co.uk/) and here (http://en.wikipedia.org/wiki/Pick_operating_system) I don't (which is why I am asking) but I could imagine it being a right bugger. No need for detailed or technical info - I have no more info at this stage - just wondered if anyone has any similar experience. Super duper, thank you SQL troopers :)
View 2 Replies
View Related
Jul 24, 2015
Why does it take me 4 hours to set up an SSIS package that I can run from a SQL job to extract data from a SQL database to and Excel workbook. Shouldn't this be easy to do with 2 Microsoft products? Writing the query to extract the data takes 10 minutes, the rest of this process should take less than that.Â
I should be able to create a new job that runs my query (I can actually do that) and saves the data to an Excel workbook. Why can't I do that?
View 3 Replies
View Related
Oct 12, 2007
Hi
I need to export data from an LDAP-compliant data source, to a SQL Server database. Is this possible with SQL Server 2005 Integration Services? The Import and Export Wizard
does not appear to support LDAP in the Data Source drop down list.
Thanks
Alan
View 4 Replies
View Related
Oct 12, 2007
Hi
I need to export data from an LDAP-compliant data source, to a SQL Server database. Is this possible with SQL Server 2005 Integration Services? The Import and Export Wizard does not appear to support LDAP in the Data Source drop down list.
Thanks
Alan
View 8 Replies
View Related
May 7, 2008
hi lads,
what i need to do for my project are as following:
a mobile user send data via GPRS to SQL server Database. then i need to have a method to detect while a particular table is being inserted. and then extract data from table construct a report dynamiclly. what should i do to achieve this goal? e.g. window application, store procedure or trigger ?
PS : client side is a mobile application developed using MCL. i don't in which way he will send all data to SQL server Database, so what i need to do is to monitor new data inserting.
2) how to auto generate and print report directly after record been inserted into the table ? Do i need to import report web service API ? if yes, which one? or i can use other methods e.g. predefine report control view in my window application, turn off pop-up menu while printing a report(I guess)
thanks
View 2 Replies
View Related
Jun 23, 2015
Looking for sample ETL package to extract data from SQL Sever Database and load into Oracle Database using SQL SERVER INTEGRATION SERVICES 2008. The requirement is for full load and incremental load both.
View 5 Replies
View Related
Jan 7, 2008
i m having a huge problem! how to retrieve data from notepad files using asp.net and store the info in fields in MS-Access/Sql db? Plz help !!
View 1 Replies
View Related
Oct 2, 2014
I have a MySql Database i need to extract data from based on matching info in a local SQL Server database,
I will ultimately need to cycle through 20 of these MySql databases , but the query below is taking 1 minute and 20 plus seconds...Just for one..
Is there a simple tweak that would speed it up?
USE [CAS]
GO
DECLARE @PhoneDB varchar(max),
@SQL NVARCHAR(MAX);
set @PhoneDB = '[PHONEDB_PHI]';
set @SQL = 'SELECT
[Code]....
View 4 Replies
View Related
Jul 12, 2004
Hi,
I wanted to retrieve all the databases present in an SQL Server and put them in an Dropdown list.
Then retrieve all tables of the selected database
and finally retrieve all data from the selected table (This i know)
How do i accomplish the above two.
Regards
Vijay R
View 3 Replies
View Related
Nov 16, 2007
Code Block
string commandString = "SELECT Id,Name FROM [DatabaseTable];";
using (SqlConnection conn = new SqlConnection(connectionString))
{
SqlCommand cmd = new SqlCommand(commandString, conn);
conn.Open();
IDataReader rdr = cmd.ExecuteReader();
IList ids = new List();
IList names = new List();
while (rdr.Read())
{
ids.Add((int)rdr[0]);
names.Add((String)rdr[1]);
}
rdr.Close()
}
What i want to know is if there is any better way of obtaining the 'id' and 'name' data
values then just assuming that id is the first returned object and name is the second returned object in each reader record.
ie. Is there any way to retrieve a specific field from the reader in a Dictionary-type manner...?
Code Block
while(rdr.Read())
{
ids.Add( (int)rdr["Id"] );
names.Add( (String)rdr["Name"] );
}
View 6 Replies
View Related
Aug 31, 2006
Hai All
How to insert and delete records from a table in the remote database?"
I want it to be done through a tool that used LDAP Servers.
I want tools for the LDAP .
Thank u Advance
Regards
Sunil
aboutknowledge@gmail.com
View 2 Replies
View Related
Oct 29, 2007
Hi All,
I currently have the problem that I have to write some data into a SUN Directory Server 5.2 LDAP directory.
Does anyone know how I can do this. I already found some articles in this forum that provide solutions how to access
the active directory - but how can I access a none Microsoft LDAP Server?
Is there any way to use the OLE DB Destination or do I have to implement my own LDAP adapter in VB.net?
Thanks a lot for help.
View 1 Replies
View Related
Jul 27, 2006
Hi,
I'm using ASP.NET 1.1, SQL Server 2000 Server:
I followed the ASP.NET 1.1 Starter Kit's Commerce application and applied the same principles it had written the code to retrieve data to my web application I created. For example I've written this Function in a class to return a sqldatareader:
Public Function GetAdvanceSearch(ByVal s As String, ByVal Ext As Integer, ByVal fdate As DateTime, ByVal tdate As DateTime) As SqlDataReader
Dim oDrAdSearch As SqlDataReaderDim oCmdGetSearch As New SqlCommand("spAdvanceSearch", oComConn)
With oCmdGetSearch .CommandType = CommandType.StoredProcedure .Parameters.Add(New SqlParameter("@DialNo", SqlDbType.VarChar)).Value = s .Parameters.Add(New SqlParameter("@FDate", SqlDbType.DateTime)).Value = fdate .Parameters.Add(New SqlParameter("@TDate", SqlDbType.DateTime)).Value = tdate .Parameters.Add(New SqlParameter("@Ext", SqlDbType.Int)).Value = ExtEnd With
oComConn.Open()oDrAdSearch = oCmdGetSearch.ExecuteReader(CommandBehavior.CloseConnection)
If oDrAdSearch.HasRows Then Return oDrAdSearchElse Return NothingEnd If
End Function
And When I'm calling this function I do write in this way (assuming that this function is in a class called "Calls"):
Dim objCalls as New Calls
DataGrid1.DataSource = objCalls.GetAdvanceSearch(<PARAMS.......>)DataGrid1.Databind
My application is a Telephone Call Recording System and could expect vast amount of data. Averagely, a month may produce approximately 50,000 records or more. So while querying through my web application for a month, the application itself either gets stuck or the retrieval speed gets drastically slow. However I'm using Datareaders for every querying scenario. My Web application is hosted in a Windows 2000 Server and accessed via Local Network or IntraNet.
What are the ways I could make this retrieval more speedier and efficient? I would like to hear from anyone who have come across this problem and anyone who could help me on this.
Thanks in Advance. Looking forward for a reply from some one.
View 3 Replies
View Related
Mar 29, 2001
Is there a stored proceduire that grabs all the data from a specified table and places it in a file.
View 1 Replies
View Related
May 4, 2006
Hello all,
I'm stumped on how to solve this question so I figured i'd ask the community. As a warning i'm not sure best how to describe my situation so i'll try and give as much detail as I can.
First in table A, I have two columns that already have data in them that are numeric (Col1, Col2). Also, in table A I have two more columns that are going to derive their data based of a complicated data retrieveal routine (Col3, Col4). So my table structure looks something like this:
Code:
Table A
Col1 Col2 Col3 Col4
20 20 NULL NULL
(Where Col3, and Col4 are going to be populated based off the routine)
The Data for Col3 and Col4 is in an excel spreadsheet that i'd like to convert into a table for MSSQL. However i'm not sure how to do this because in the spreadsheet there is a lookup routine (that i'm trying to copy to MSSQL code, i'll show that in a minute) that generates its values based off data in the X / Y columns, so something like this:
Code:
_|1__|2___|3__
0|0 |0 |0
1|1 |15 |25
So when 2 is met, and 1 is met, they would equal '15'. No arithmetic involved, just simply matching up the X / Y and pulling the data.
My question is, how do I create tables out of this, so my lookup routines can get the values as a result of matching X / Y? (2, 1 = 15)???
The excel routine is this:
=IF(VLOOKUP(F66,'Appendix A'!A5:K56,MATCH(F68,'Appendix A'!A4:K4,1),TRUE)>F46/12*0.125,F46/12*0.125,(VLOOKUP(F66,'Appendix A'!A5:K56,MATCH(F68,'Appendix A'!A4:K4,1),TRUE)))
Thanks!
View 3 Replies
View Related
Apr 11, 2007
Hello,
i am getting a hard time in minimizing time for data retrieval, over SQL Server DataBase. My DataBase consist of 2 tables. One of the table has more than 10000 entries and other table with more than 10 million entries. I have used SQLNative Client for connecting to data base and my goal was to find a value from the 1st table and search it out in 2nd table. The result is more than hundered thousand rows. Now the problem is: the time it took for retrieving those rows is much slower approx. 12 minutes. Can this time be cut down. I have used SQLClient connection to make sure it is accessing SQL server on a direct access base.
Also, i am using
SqlClient::SqlDataReader
for reading, getting rows returned by the my query.
Please help me out.
Thanx.
R. T.
View 4 Replies
View Related
Jun 3, 2008
Hi All,
I am working on an application to retrieve data to the windows ce 3.0 handheld from the Sql server database. I am able to retrieve the data from the server using RDA pull method and able to see the data on the local handheld database. To display the retrieved data in the dialog to the user, it requires to query the local databse and get the information and then display the information, which is taking some time.
My question is, to reduce the time and improve the performance , instead of pulling the data to the local table, is there any way to pull the data and have it in memory and display the details to the user?
To develop the above application, I used some of the code to pull the data from the server from the sample application C:Program FilesMicrosoft SQL Server CE 2.0SampleseVCeVCReplRdaHPC. I am developing the application for windows ce 3.0 device using eVC 3.0
thanks
pyd.
View 4 Replies
View Related
Feb 22, 2007
Hi,
I have a summary table like this
Field1
Field2
Field3
Field4
Field5
AAA11
value1
value2
value3
value4
value5
AAB23
value6
value7
value8
value9
value10
BCD14
value11
value12
value13
value14
value15
GFD12
value16
value17
value18
value19
value20
SDL25
value21
value22
value23
value24
value25
AUD56
value26
value27
value28
value29
value30
BER11
value31
value32
value33
value34
value35
Columns are obviously fixed, but not rows.
I want to show this data using lables and SqlDataReader for report purpose like;
Label1.text=dr("value16").toString( )
Label2.text=dr("value28").toString( )
Label3.text=dr("value31").toString( ) etc
Do you have any idea how i can do it or am I approaching it in the wrong way????
Thanks.
Michelle
View 1 Replies
View Related
Feb 16, 2007
A call to a stored procedure completes in 13 seconds when ran from within SQL Server Management Studio. A report whose data source uses the same stored procedure can take as long as 10 minutes to run. The same parameter values are used in both the direct call and the report. The execution log says almost all of that time is spent on data retrieval. How could that be? What might be the cause?
View 14 Replies
View Related
Feb 27, 2008
Hi there,
I was wondering if someone could propose a solution for the following scenario:
TimeID column would have values from 1 to 6 and rows will be inserted only for those timeIDs where we have Data value as well. While retrieving data we would like to have all timeID range returned with explicitly specifying NULL for missing Data column. Please see below to better understand the situation.
create table test
(
TimeID INT,
Data INT
)
INSERT INTO test VALUES(1, 100)
INSERT INTO test VALUES(2, 180)
INSERT INTO test VALUES(4, 550)
INSERT INTO test VALUES(6, 120)
select * from test
1 100
2 180
4 550
6 120
Desired resultset
1 100
2 180
3 NULL
4 550
5 NULL
6 120
...
View 14 Replies
View Related
May 28, 2015
I have below DB structure in MSSQL for a small application which follow relational approach. Data retrieval (for Hostels) will need several Join, may be Key-Value approach where data retrieval will be fast.
Hostels
------------
HostelId,
Name,
Address,
CategotyId,
SubCategoryId,
FoodCategoryId,
LandLordId
Data:
1 H1 Address1 1 1 2 20
2 H2 Address2 1 2 2 21
3 H3 Address3 2 2 1 17
Category
----------
CategoryId,
CategoryName
[code]...
View 10 Replies
View Related
Jun 13, 2012
I've been asked to build a scripter and database for one of our automated sync reporters we have in place. I understand that I will have to script it out to get the information from our sync server(red hat box) into SQLServer, and then script it to output the daily reports. My biggest question is, how would I be able to import the data from the box into my db? I've tried the import/export wizard, and it just doesn't work for what I need. I'm running SQL Server 2008 R2.
View 2 Replies
View Related
Mar 5, 2015
I have a DB with a few related tables. I need to set up an automated import of a text file to completely replace an existing table data. There will be no translation involved, but a re-map Looking around, a lot of words on Integration services and then using a stored procedure to schedule the package.
The question is, is this the preferred method, and any pointers to any posting that would give me a good overview of the steps needed.
View 2 Replies
View Related
