id,summary,reporter,owner,description,type,status,priority,resolution,keywords,cc,topic,difficulty,mentor
1598,Improve/rewrite HDBC backends,gbaz,,"cf http://www.haskell.org/pipermail/haskell-cafe/2011-February/089535.html

Although there are a number of issues with HDBC backends, the main one is that nearly everything goes through strings, even when drivers offer support for proper wire formats. This makes large inserts of selects much slower than necessary, and consume much more space than necessary as well. Along with this, there's no implementation of batched inserts. And finally, there's BLOB support.

Additionally, the backends could be simplified. That is to say, that when HDBC was produced, there were no seperate bindings packages for most databases. So the backends include the bindings themselves. But now there are seperate packages that just provide simple bindings to e.g. sqlite and postgres. It would be very good to switch over to those libraries where possible, and where such packages don't exist to split out, e.g., the direct ODBC bindings to a separate package as well. This would help maintainability and future development.


HDBC remains by far the most used Haskell database package, and now that John Goerzen has announced plans to switch to BSD licensing, there's no obstacle to its use across the board except for it's current limitations. It would be far better for the Haskell community to bring HDBC up to snuff as a single intermediate tool for accessing multiple database backends than to continue down the path of a profusion of competing single-db bindings that higher-level libraries are forced to access individually.

Major points from the above email reproduced below:

  1. I have no Windows development platform.  I can't test the releases on Windows.  Many Windows users don't have the skill to diagnose problems.  These problems do eventually get fixed when a Windows user with that skill comes along -- and I appreciate their efforts very much! -- but it takes longer than it ought to.

  2. The ODBC documentation is monumentally terrible, and the API is perhaps only majestically terrible, and it is not 100% clear to me all the time that I have it right.  A seasoned ODBC person would be ideal here.

  3. Issues exist with transferring binary data and BLOBs to/from at  least PostgreSQL databases and perhaps others.  There appear to be bugs in the backend for this, but BLOB support in the API may need to be developed as well.

  4. Although the API supports optimizations for inserting many rows at once and precompiled queries, most backends to not yet take advantage of these optimization.

  5. I have received dueling patches for whether foreign imports should be marked ""safe"" or ""unsafe"" on various backends.  There seems to be disagreement in the community about this one.

  6. Many interactions with database backends take place using a String when a more native type could be used for efficiency.  This project maybe rather complex given varying types of column data in a database -- what it expects for bound parameters and what it returns.  The API support is there for it though.
",proposed-project,new,good,,"hdbc, bindings",,Bindings,1 person Summer,not-accepted