Foxpro .net Odbc Data Provider Misidentifying Strings As Unicode
Nov 23, 2006
Since my foxpro OLE driver has been rendered useless by service pack 1 for sql server 2005 I am forced to use the .net data provider for odbc.
I am importing a number of tables.. each time I add the DataReader Source to the dataflow and connected it to the OLE DB Destination I get a load of the good old "cannot convert between unicode and non-unicode string data types" errors...
So I'm having to do derived column transforms, for each and every column that it coughs up on.
Using expressions like (DT_STR,30,1252)receivedby to convert the "recievedby" column to a DT_STR,
Some of these tables have 100 string columns.. so I'm getting a bit sick of the drudgery of adding all these derivations...
Is there any way to tell this provider to stop deciding that the strings in the foxpro tables are unicode?
We've installed the Oracle provider for OLE DB on SQL Server 2005, which has the default collation (SQL_Latin1_General_CP1_CI_AS), and we've created a linked server for the Oracle 9.2.0.5 database, which has AL32UTF8 as the database character set. We can successfully insert strings into VARCHAR2 columns on Oracle from SQL Server via EXEC SP_EXECUTESQL('INSERT OPENQUERY(...) VALUES(...)') -- as long as the strings (whether selected from NVARCHAR columns on SQL Server or specified as literals with the N prefix during testing) only contain Windows-1252 characters.
If the SQL statement contains a character above U+00FF, the string on the Oracle side is incorrectly/doubly encoded; there are nearly (but not exactly) 4 bytes per character instead of the 1 or 2 you'd expect from ASCII/Latin-1 characters encoded as UTF-8.
We've tried reconfiguring the linked server: collation compatible = false, use remote collation = true, and collation name = Latin1_General_BIN2. But that had no effect.
I am building a data warehouse. Some of the data comes from an AS 400 EPR system. I used the OLEDB connector when first pulling the data into SQL Server doing simple import data from table option. That worked great for getting the initial data load into SQL Server and creating the base SQL Server tables although it was excruciatingly slow (that was probably due to the transport from the AS 400).
Now, I need to get new records that are added to the AS400 side of things on a daily basis. For that, I was trying to use the OLEDB AS400 connector. However, I found that the OLEDB connector wouldn't work when I was trying to specify an SQL Statement for what to get; i.e., a simple query like Select * from TWLDAT.STKT where BYSDAT >= '2005-01-27' would simply not work. Found articles here explaining that it is probably a problem on the AS400 side of things and where people recommended using an ADO ODBC data reader source for this type of thing. So, I'm trying to implement that. However, I have a huge problem with it.
The original tables that got created were mapped to use NVARCHAR fields for character data. When the ADO ODBC data reader source accesses the AS400 data, it insists on interpreting the string type fields as being unicode strings and giving it a data type of DT_WSTR when what I need it to have is a plain old DT_STR data type. When the strings are interpreted as unicode strings, they cannot be converted in a way that allows the NVARCHAR fields to be filled with the data. The exact error message I get for all the fields that should wind up being nvarchar fields is as follows:
Column "BYStOK" cannot convert between unicode and non-unicode string data types.
Okay, so I try to change the data types in the ADO ODBC data reader to be plain DT_STR data types and I cannot do so.
Does anyone have any idea why the ADO ODBC data reader source insists on interpreting the string data coming from the AS 400 as unicode string data or why it refuses to allow that to be changed to DT_STR data type?
Thanks in advance for any info. By the way, if there is a better way than the ADO ODBC data source to get at this data when I need to specify an SQL command, I would love to hear about it. Not wild about using ODBC in the OLEDB age.
I am using OleDbMicrosoft OLE DB Rpovider for Visual FoxPro for my Data Source Connection and it includes the deleted records from the Visual FoxPro database, how can it have it ignore the deleted records?
Im quite interested to see if any one is using Microsoft OLE DB Provider for Visual FoxPro 9.0 in an SSIS package. Is it possible to use this for free table directory like you can with the ODBC driver. ????
thanks kindly
My basic proccess is as follows: The source foxpro database has hundreds of dbf files ---the SSIS procress is as follows
1: Copy only required dbf files and/or matching .fpt files over to local drive on SSIS box(we only need 10 files)
use MS Visual Foxpro ODBC driver (free table option) and set up a system DSN--dataflow tab---datareader source--and away you go
SELECT * from tableeMass where Name=N'???' In the above select statement in microsoft SQL server 2005 ,I only want to select all the rows whose name is '???' a Gee'z string(Gee'z is set of Ethiopian characters) it is a unicode string(???=gebru in english,the reason why it is question mark is the characters are gee'z).When we execute the select statement, it brings all the rows instead of the rows whose name is '???'. why? the data type of column Name is nvarchar(50).would you please help me in tackling the problem. please reply me soon,I really need it badly soon for my project of Localized database development. thanks inadvance. gebru
I have an ODBC connection setup for Oralce RDB. This works fine in say MSAccess. Now I would like to add a SQL Server Linked Server to this ODBC driver (it is for RDB not regular Oracle), but I am not able to get this to work. What is the Provider String and Location / Catalog for using this type of ODBC? I setup DSN as 'name', Provider String as 'UID=user;PWD=pass;' and it seems it tries to connect but gives an error. "%SQL-F-SCHNOTDEF, Schema USER is not defined"
Hi, I have 2 separate clients experiencing this problem, Both systems are 1.Win 2000 Server with Small Business Server 2000 (SP4) 2.SQL Server 2000 (SP2 upgraded to SP4, SP4) 3.Exchange 4.IIS etc
We use SQL Server to connect to FoxPro 2.6 Tables.
We are using OPENROWSET as follows:
Select * from openrowset('VFPOLEDB.1','C:Client Data';;,'Select * from cm')
We are now getting this error: Server: Msg 7330, Level 16, State 2, Line 1 Could not fetch a row from OLE DB provider 'vfpoleDB.1'. OLE DB error trace [OLE/DB Provider 'vfpoleDB.1' IRowset::GetNextRows returned 0x80040155].
The statement worked up until around 20 days ago, which made us think that it may be windows updates/sps/hotfixes, but i have installed the above on a test machine and installed all windows updates and the test machine works AOK.
I have 2 separate clients experiencing this problem,
Both systems are
Win 2000 Server with Small Business Server 2000 (SP4) SQL Server 2000 (SP2 upgraded to SP4, SP4) Exchange IIS etc
We use SQL Server to connect to FoxPro 2.6 Tables.
We are using OPENROWSET as follows:
Select * from openrowset('VFPOLEDB.1','C:Client Data';;,'Select * from cm')
We are now getting this error:
Server: Msg 7330, Level 16, State 2, Line 1 Could not fetch a row from OLE DB provider 'vfpoleDB.1'. OLE DB error trace [OLE/DB Provider 'vfpoleDB.1' IRowset::GetNextRows returned 0x80040155].
The statement worked up until around 20 days ago, which made us think that it may be windows updates/sps/hotfixes, but i have installed the above on a test machine and installed all windows updates and the test machine works AOK.
Here is the situation, please let me know if you have any tips:..TXT files in a share at \fooSPROCS run daily parses of many things, including data on that share. Theother day, we encountered rows in the TXT files which looked like:column1Row1data,column2Row1datacolumn1Row2data,column2Row2data...etc..However, column2 was about 6000 bytes of unicode. We are bulk insertinginto a table specifying nvarchar(4000). When it encounters high unicoderows, it throws a truncation error (16).We really need information contained in the first 200 bytes of the string incolumn2. However, the errors are causing the calling SPROC to abort.Please let me know if you have any suggestions on workarounds for thissituation. Ideally, we would only Bulk Insert a sub-section of column2 ifpossible.Thanks!/Ty
I like to know if anyone of you out there can help me solve this issue. I need to generate an SQL Query to access two different FOXPro (databases) connections.
Example, Connection 1 - has a DSN setup of CMSBM and it sits in the path of D:/CMS/CMSBM/= and the table is bmboml15 Connection 2 - has a DSN setup of CMSMA and it sits in the path of D:/CMS/CMSMA/= and the table is maprodl15
I am coding in ASP.NET using C#. I have tried the command like:
SELECT codeno, link, type FROM bmboml15 IN CMSBM INNER JOIN maprodl15 IN CMSMA ON bmboml15.codeno = maprodl15.codeno.
Obviously the above query is not a working one and a very poor construction.
I am following the SSIS overview video- URL...I have a flat file that i want to import the contents onto a SQL database.I created a Dataflow task, source file and oledb destination.I am getting the folliwung error -"column "A" cannot convert between unicode and non-unicode string data types".in the origin file the data type is coming as string[DT_STR] and in the destination object it is coming as "Unicode string [DT_WSTR]"I used a data conversion object in between, dosent works very well
I have an SSIS package that pulls data from a MYSQL DB (Using RSSBus for Salesforce in SSIS to accomplish this). Most of the columns are loading properly, but I have many columns that I need to convert.
I have been using the Data Conversion dataflow task in SSIS to convert the rows.
I have 2 data conversions that work on most of the columns, but the DESCRIPTION column continues to return an error saying "Cannot convert between unicode and non-unicode types", regardless of what I choose on the Data Conversion task. So, basically I want to dump this column data into a SQL table with NVARCHAR datatypes. Here is what I am doing in my SSIS package...
1) Grab subset of data from SOURCE 2) Converts to TEXTSTREAM. (Data Conversion) 3) Converts to STRING. (Data Conversion) 4) Load Destination table. (OLE DB Destination)
I have also tried to simply convert the values to STRING, but that doesn't work either.
So, I have 2 Data Conversions working here that process most of the data correctly. What I can do to load the DESCRIPTION column?
I have a report that accessed a Visual FoxPro 6.0 database via ODBC. Since I upgraded to Visual FoxPro 9.0, now I am getting the error:
Query Server Error DMS-E-RBI_TABLE The table or view <table name> was not found in the dictionary
I've verified that the right path is setup and I've tried installing about every driver and none of them are working. I created a new database in 9.0 and the I can access it fine. Also, there are old tables that are accessible in the same path, just not the ones I need. Any ideas would be helpful..
I've had some great headaches with SSIS this morning, which I have managed to get a workarounds for, but I'm not happy with them so I've come to ask for advice.
Basically, I am exporting data from an SQL Server database into an Excel spreadsheet and hitting issues with unicode and non-unicode data types.
For example, I have a column that is char(6) and have added a data conversion step to the data flow, which converts it to type DT_WSTR and then everything works!
However, this seems like a completely un-neccessary step as I should be able to do the conversion in T-SQL - but no matter what I try I keep getting the same problem.
SELECT Cast(employee_number As nvarchar(255)) As [employee_number] FROM employee WHERE forename = 'george'
ErrorValidation error. details: 1 [1123]: Column "employee_number" cannot convert between unicode and non-unicode string data types.
I know I have a solution (read: workaround) but I really don't want to do this everytime!
I have an Excel Source component hooked to an OLE DB Destination component in my SSIS 2005 Data Flow Task. After I mapped the excel columns to the OLE DB table columns i get these errors below. I noticed that for the first error, the Excel Field format (when you mouse over the column name in the mappings section in OLE DB component) is of type [DTWSTR] and the corresponding SQL field from my SQL table that it's mapping to is of type [DT_STR] when mousing over that field in the mappings in the properties of my OLE DB component. All table fields in SQL Server for the table I'm inserting into are of type varchar.
Error at Data Flow Task [OLE DB Destination [27]]: Columns "Commission Agency" and "CommissionAgency" cannot convert between unicode and non-unicode string data types.
Error at Data Flow Task [OLE DB Destination [27]]: Column "Product" cannot convert between unicode and non-unicode string data types.
Error at Data Flow Task [OLE DB Destination [27]]: Columns "Officer Code" and "OfficerCode" cannot convert between unicode and non-unicode string data types.
Error at Data Flow Task [OLE DB Destination [27]]: Columns "Agency Name" and "AgencyName" cannot convert between unicode and non-unicode string data types.
Error at Data Flow Task [OLE DB Destination [27]]: Columns "Agency Id" and "AgencyID" cannot convert between unicode and non-unicode string data types.
Error at Data Flow Task [OLE DB Destination [27]]: Columns "Tran Code" and "TranCode" cannot convert between unicode and non-unicode string data types.
Error at Data Flow Task [OLE DB Destination [27]]: Columns "User Id" and "UserID" cannot convert between unicode and non-unicode string data types.
Error at Data Flow Task [OLE DB Destination [27]]: Columns "Acct Number" and "AccountNumber" cannot convert between unicode and non-unicode string data types.
Error at Data Flow Task [DTS.Pipeline]: "component "OLE DB Destination" (27)" failed validation and returned validation status "VS_ISBROKEN".
Error at Data Flow Task [DTS.Pipeline]: One or more component failed validation.
Error at Data Flow Task: There were errors during task validation.
I use Visual Studio's, integration project to load XML file into SQL Server. In the XML file, i have defined collumns as string. When i try to load XML file with parts defined in scheme as string, i get an error "cannot convert between unicode and non-unicode string data type.
Destinated collumns in SQL are defined as varchar and char.
I have spent countless number of hours trying to solve the issue, but to no vail. My problem is SSIS throws "cannot convert between unicode and non-unicode string data types" when i am try to transform data from DB2 to SQL Server 2005. And please note, i tried all possibilities like changing the destination field which is in SQL Server 2005 to nvarchar and also text. But so far no help. And i also looked at previous posts which did not help me either.
I have a table that has arabic, japanese, utf8 and ascii strings in it. When I try and rerieve those strings using the windows odbc driver, the ascii and since byte utf8 characters come back fine, the arabic and japanese are returning as single byte characters. I read on microsofts page that odbc versions before 3.7 are considered non-unicode. When I look at the version of the odbc administrator it says 3.5. Is there a newer version of the sqlserver odbc driver or windows odbc administrator that I would need?
Is there some way to specify what the client charset is? I'm using SQLGetData with a target type of SQL_C_WCHAR.
Hi all, we are now planning to upgrade our application from anon-unicode version to a unicode version. The application's backend isa SQL Server 2000 SP3.The concern is, existing business data are stored using collation"Chinese_PRC_CI_AS", i.e. Simplified Chinese. So I thought we need toextract these data out to the new SQL Server which is using Unicode (Iassume it means converting them to nchar, nvarchar type of fields for Idon't enough information from the application side, or is there ageneral unicode collation that will make even char and varchar types tostore data as Unicode?).The problem is what's the best and most efficient way to do this dataconversion?bcp? DTS? or others?thanks a lot
SQL Native Client 2005.90.3042.00 ODBC driver does not return SQL_CVT_DOUBLE as part of the mask from SQLGetInfo for SQL_CONVERT_WCHAR, SQL_CONVERT_WVARCHAR even though driver actually supports converting nchar and nvarchar to SQL_DOUBLE. For example,
Due to some problems, we have to reformat Our Windows NT4.0 Server. The SQL Server 2000 has several ODBC Configured as "User DSN" and "System DSN" .
Question: Is there any way that we could copy these User DSN and System DSN Config Files so that after Rebuilding Server AND Database, we could Re-aaply/ Re-create these ODBC DSN specifically "System DSN" as they are More Number.
Due to some problems, we have to reformat Our Windows NT4.0 Server. The SQL Server 2000 has several ODBC Configured as "User DSN" and "System DSN" .
Question: Is there any way that we could copy these User DSN and System DSN Config Files so that after Rebuilding Server AND Database, we could Re-aaply/ Re-create these ODBC DSN specifically "System DSN" as they are More Number.
Hi,I created a linked server for MS SQL Server 2005 Express to an Oracledatabase using the OLE DB Provider for ODBC. My ODBC Source uses theMicrosoft ODBC for Oracle driver.I'm using the OLE DB Provider for ODBC instead of the Oracle OLE DBproviders because those don't handle Oracle's Numeric Data Type well.When I set this up yesterday it worked fine. However I log in today andattempt to run sp_tables_exec against my linked server and get thefollowingOLE DB provider "MSDASQL" for linked server "WHATEVER" returned message"[Microsoft][ODBC Driver Manager] Driver's SQLSetConnectAttr failed".OLE DB provider "MSDASQL" for linked server "WHATEVER" returned message"[Microsoft][ODBC Driver Manager] Driver's SQLSetConnectAttr failed".OLE DB provider "MSDASQL" for linked server "WHATEVER" returned message"[Microsoft][ODBC driver for Oracle][Oracle]".Msg 7303, Level 16, State 1, Procedure sp_tables_ex, Line 41Cannot initialize the data source object of OLE DB provider "MSDASQL"for linked server "WHATEVER".I can tnsping the server, and I have no trouble connecting with theOracle OLE DB providerI try to set up ODBC logging but the log file is blank -- same withattempting to set up logging for Oracle's network client. I've beendealing with crap like this the last 4 days, something works one day,then the next nothing (I've done nothing to change the configurationbetween these outages).Can anyone help me?ThanksFrustrated
I'm not able to see microsoft ole db provider for odbc drivers on sql server 2005. Anyone know which driver I should install to see it? I'm trying to create a linked server from as400 to SS2K5.
I cant find any odbc provider from my linked server.(SQL 2005) I am using MDAC 2.8 SP1 ON WINDOWS XP SP2 on my machine. What can i do to get odbc provider..?
I finally made data transfer between AS400 and SQL2005 work by using Microsoft OLE DB provider for DB2. I have tried other methods, some do not have build-in destination, some have unicode conversion problem. The only issue I have with this method is the peformance. For example, 300,000 rows load from AS400 take only 3 minutes with the ODBC connection in DTS, but more than 5 minutes with the microsoft OLD DB provider for DB2 in SSIS.
Does anyone have the same issue or figured out any tweaking that may speed up the transfer?
I think this might be a good place to ask the following question.
I am writing the error handling code for my data access layer for a web application. I am using the Enterprise Library Data Access Application Block. Although this supports generic database connections, I realized that I need to handle errors specific to each database type. Microsoft SQL is the only database type I am using for now, so I am using a try...catch (SqlException e).
In testing my code, I intentionally changed the instance name in web.config to a name that does not exist. I get the very popular error 26 - Error Locating Server/Instance Specified. This is returned as a SqlException, but the SqlError.Number property is set to -1.
Am I getting "-1" because the provider hasn't actually connected to SQL yet, so it doesn't have an actual SQL error number? Can I assume that (SqlError.Number == -1) is always a fatal, provider-level connection exception? Will the provider ever use another SqlError.Number of its own? Or do all numbers besides -1 come from the SQL sysmessages table?. Is there a comprehensive list of what exceptions might be raised by the SqlClient provider, including #26?
The reason for all the questions is that in a web application, I want to prevent the end-user from seeing the "real" exception if it has to do with configuration errors. However, maybe there are other errors that the user should see and handle? It's hard to know without a full list of SqlClient provider errors, along with the SqlError.Number that each error maps to.
