Using SQL Server 2005 Public Keys In VB.Net

I'm creating an asymmetric key in SQL Server:

CREATE ASYMMETRIC KEY MY_KEY
WITH ALGORITHM = RSA_512
ENCRYPTION BY PASSWORD = 'password1@'

I'm then taking the public key...

SELECT public_key
FROM sys.asymmetric_keys
WHERE [name] = 'MY_KEY'

I'm trying to import this into VB.Net using RSACryptoServiceProvider.

Dim RSA As New
Security.Cryptography.RSACryptoServiceProvider(512)
RSA.ImportCspBlob(KeyBlobByteArray)

I have to add the following line to get it to not give me errors.
(Why?) But the data looks right when I look at the exponet and modulus
of the parameters that are exported and then immediately imported.
Something is obviously being thrown away when I do this, but I don't
know what. It's probably the key to the whole problem.

RSA.ImportParameters(RSA.ExportParameters(False))

I encrypt the message...

EncryptedMessageByteArray = RSA.Encrypt(MessageByteArray,
False)

And in SQL server I try to decrypt the message and I get NULL.

SET @DecryptedMessage =
DecryptByAsymKey (
AsymKey_ID('MY_KEY') ,
@EncryptedMessage ,
N'password1@')

I've noticed that the encrypted array is completely different in VB
than when I encrypt it in SQL (obviously), but I can't figure out how
to configure to the RSACryptoServiceProvider to match what SQL Server
is doing under the hood.

Any ideas?
Thanks,
Jerrill