3

I am trying to communicate a numerical matrix operations library like LAPACK with any DBMS.

Is it possible to send/receive complete matrices as binary or as a direct memory pointers to process them (it will be something like: The Outside library processes data stored in DBMS, then it computes some huge matrix stuff and then via memory block or a binary DBMS get the result from library)?

The main purpose is speed and avoid passing through a flat file, and last but not least, use library toefficiently do some operations DBMS are not designed to.

* Is it possible that Oracle, SQL Server, MySQL support this technique?.

3 Answers 3

2

Oracle has support for matrix operations, including LAPACK compatible commands.

Have a look at the Oracle package UTL_NLA to give you a starting point. I think you're better off looking at either using the database tools, or leaving the database behind entirely - the way you seem to want to approach is, well, not really an RDBMS.

0

No, not possible, not sensible. DB data is transactional, and doing tansactional shared memory is living hell. Definitely not "handling pointers around". Possible, but you then require the client libraries to be VERY good / under control OR run a lot of overhead, and that means no go again.

Seems like you pretty much have requirements that dont match. RDBMS, standard products, are not optimized for your type of work.

The BEST you could possibly do is:

  • Store the files in BLOBS.
  • Use SQL Server
  • Use a FILE based BLOB store

THen you could access the blobs via file share whould should allow memory mapped files. Not export - they ARE files. Every commit makes a new copy with a differ5ent name. You get the names in sql from the database (it is a GUID).

That possibly is the most efficient way to deal with this.

3
  • So, any other way to let some huge processing like statistic analysis outside DBMS, because some things can not be done inside DBMS?
    – edgarmtze
    Mar 18, 2011 at 7:02
  • One question, Do you have an example? just to follow your idea of BLOBS
    – edgarmtze
    Mar 18, 2011 at 7:04
  • Ah, no - but the documentation has. technet.microsoft.com/en-us/library/bb933993.aspx has the starting point (FILESTREAM option). WIth an add in (free) this storage can happen on a sparate server. This is then called "Remote Blob Storage".
    – TomTom
    Mar 18, 2011 at 7:09
0

You could possibly store a cache copy of the data locally to avoid another transfer if it hasn't changed since the last access. That would be coded totally outside of the DBMS.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .