Mac OS X / Linux.

The Haskell OpenAL 1.3.1.1 build is broken: something goes wrong in the autoconf hell, and the result is that the library builds, but does not work (because configure gets some magic constants, like AL_TRUE, wrong). Though for some mysterious reasons it seems to work on Linux. Anyway, this is how you can fix it.
1. \$ darcs get http://darcs.haskell.org/packages/OpenAL
\$ cd OpenAL

(the tarball on Hackage won't work, as it misses the file aclocal.m4)
2. Open the file configure.ac with your favourite text editor, and paste the following five lines at somewhere around the 82th line:
#if defined(HAVE_AL_AL_H)
#include <AL/al.h>
#elif defined(HAVE_OPENAL_AL_H)
#include <OpenAL/al.h>
#endif
3. \$ autoreconf
\$ cabal install
At this point you should have a working OpenAL build.

Windows.

Unfortunately, the OpenAL binding uses the UNIX tools called "autoconf" and friends, which are not really compatible with the Windows environment. Additionally, it has the same problem as above. So here is what to do:
1. Install MinGW and MSys from http://www.mingw.org
2. Install OpenAL and the OpenAL SDK from here.
The default directory it puts the SDK is C:\Program Files\OpenAL SDK 1.1.
3. Create a new directory for the headers so that the path does not contain spaces, for example C:\OpenAL; copy the content of the directory C:\Program Files\OpenAL SDK 1.1\include to the subdirectory C:\OpenAL\AL. Similarly, copy the files from C:\Program Files\OpenAL SDK 1.1\lib to the subdirectory C:\OpenAL\lib.
4. Set the environmental variables C_INCLUDE_PATH to C:\OpenAL and LIBRARY_PATH to C:\OpenAL\lib using Control Panel / System / whatever.
5. Download the tarball from hackage and extract it to somewhere, say to C:\OpenAL\OpenAL-1.3.1.1
6. Replace the file C:\OpenAL\OpenAL-1.3.1.1\configure with the fixed one I prepared on OSX, because it is utterly impossible to make the whole autoconf stuff work on Windows...
7. Start MSys and type
\$ cd /c/OpenAL/OpenAL-1.3.1.1
\$ cabal install
Congratulations for your patience!