[Initial import of FastPackedString from darcs, with bug fixes from h4sh Don Stewart **20050825033507] [Added cbits to include-dirs for hugs John Goerzen **20050825132539] [Fixes for compilation with Hugs John Goerzen **20050825134613] [Also hacked fps.cabal.no-mmap to work with hugs John Goerzen **20050825134643] [Correct misleading docs about the PackedString type. John Goerzen **20050825135826] [Added some Haddock docs John Goerzen **20050825135853] [Correct some syntax for Haddock John Goerzen **20050825135931] [Noted that FastPackedString.hs is not available on Hugs John Goerzen **20050825141138] [Slightly nicer output in testsuite Don Stewart **20050826043117] [Clean up FastPackedString. Don Stewart **20050826043209 * Add haddocks to those functions with List cousins * Implement initPS and tailPS following David Roundy's suggestion -- no more substrPS. * Add back filterPS, unwordsPS * Unsafe functions now named as such: unsafeConcatLenPS * Rename: psniceq -> eqPS pscmp -> comparePS * Some functions were equivalent to list functions, but had confusing names. Now they follow their list versions, namely: findPS -> elemIndexPS wfindPS -> elemIndexWord8PS findWhenPS -> findIndexPS ] [clarify behaviour of findIndexPS. Add back darcs versions of lines/unlines (commented out for now) Don Stewart **20050826053349] [More tests Don Stewart **20050826053421] [New implementation of comparePS, based on memcmp. 1/5 space usage, 96% faster Don Stewart **20050827061615] [More tests Don Stewart **20050827062900] [Add comparePS short circuit for empty strings Don Stewart **20050827062907] [Warning police Don Stewart **20050827065946] [Add -Wall -Werror Don Stewart **20050827065959] [Add QuickCheck tests Don Stewart **20050827085102] [Missing case in takePS, spotted by QuickCheck Don Stewart **20050827103425] [Lots more quickchecks Don Stewart **20050827105959] [Allow slightly longer for the foldr/foldl tests to run dons@cse.unsw.edu.au**20050827110623] [More tests for compare Don Stewart **20050827121700] [Implement elemPS using memchr, via elemIndex Don Stewart **20050827121714] [Clean up properties Don Stewart **20050827125021] [Simpler data for char generation. Make tests a bit faster Don Stewart **20050828031447] [Use a C loop to implement reversePS, 99% less space use. and almost too fast Don Stewart **20050828031510] [Mergeo Don Stewart **20050828031635] [More tests Don Stewart **20050828032658] [Use memcpy to do consPS. Don Stewart **20050828032709] [Implement sortPS, using qsort(3). Add quickchecks for it too Don Stewart **20050828042958] [More QC properties for sort Don Stewart **20050828044242] [Seem that it is a peek after munmap causing segfaults on openbsd Don Stewart **20050828070216] [Implement findIndex and findIndices Don Stewart **20050829014643] [And quickcheck them Don Stewart **20050829014708] [find, and test it Don Stewart **20050829015347] [Make readme more accurate dons@cse.unsw.edu.au**20050829044113] [Rename most functions to 'PS'-less versions. Also import patch from darcs while we're at it Don Stewart **20050829121419] [Add extensions field, makes haddock work Don Stewart **20050830030221] [Implement `intersperse', QC it, clean up haddocks Don Stewart **20050830034723] [More efficient version of unlines Don Stewart **20050830041645] [simple implementation of transpose Don Stewart **20050830043925] [Implement and QC maximum, minimum and snoc Don Stewart **20050830050906] [Implement and QC `all' Don Stewart **20050830052436] [Unpacked version of foldr, 2x faster, 1/3 the space Don Stewart **20050830053523] [And write a packed version of foldl. 2x faster, 1/3 space Don Stewart **20050830054338] [Implement concatMap, cleaner versions of any, all Don Stewart **20050830055424] [More properties for foldl, foldr Don Stewart **20050830055830] [Implement a packed version of filter. 85% less space useage. 80% faster Don Stewart **20050830063022] [Misc cleanups Don Stewart **20050830063636] [Add some more complexity docs, index -> (!!) Don Stewart **20050830065204] [More QC tests Don Stewart **20050830071140] [foldl1, foldr1, head1, tail1 Don Stewart **20050830071159] [Implement elemIndices Don Stewart **20050830071716] [Wibble Don Stewart **20050830071800] [clean ups, revert (!!)/index pathc Don Stewart **20050830072731] [slightly more canonical names for C functions Don Stewart **20050830122619] [haddockise, improve or cleanup more of the extension functions Don Stewart **20050830122640] [QC properties for dropSpace, breakSpace, spanEnd, breakOn and split Don Stewart **20050830122700] [Wibble to docs Don Stewart **20050831002852] [whitespace only Don Stewart **20050831011406] [Clean up implementation of hash :: PS -> Int32 Don Stewart **20050831011422] [Haddockise breakFirst/breakLast Don Stewart **20050831012947] [Make it build. Add QC property for breakFirst Don Stewart **20050831024641] [Better haddocks for breakFirst Don Stewart **20050831024939] [Document breakLast and provide QC properties Don Stewart **20050831030032] [Wibble dons@cse.unsw.edu.au**20050831074413] [Typo in build instructions Don Stewart **20050831083656] [QC breakLast Don Stewart **20050831111934] [Finish cleanups Don Stewart **20050831124452] [Kill conv_to/from_hex. Should be implemented in Haskell Don Stewart **20050831124503] [More QC properties Don Stewart **20050831124527] [make haddock happy Don Stewart **20050831130713] [Final round of un-*PS-ifying Don Stewart **20050901011213] [mmapFilePS -> P.mmapFile Don Stewart **20050901012429] [Wibble in comment Don Stewart **20050901012454] [Add QC for words', lines', unlines' Don Stewart **20050901023904] [Change license to BSD-style Don Stewart **20050901023919] [wibbles Don Stewart **20050901023943] [Better comments for head1, tail1. Don Stewart **20050901025502] [Implement and QC dropSpaceEnd. Like dropSpace, but from the end of the string Don Stewart **20050901033023] [Implement unfoldr for PackedStrings Don Stewart **20050901040952] [Add QC property for unfoldr Don Stewart **20050901041002] [Typo in comment Don Stewart **20050901041256] [Add complexity details to unfoldr's comments Don Stewart **20050901041705] [More complexity tags Don Stewart **20050901042337] [Typo Don Stewart **20050901042428] [More typos in comments Don Stewart **20050901042516] [improve docs Don Stewart **20050901042656] [Better pack/unpack. About half the space, 2-3 times faster Don Stewart **20050901053716] [Begin collecting performance figures on FastPackedString Don Stewart **20050901053754] [fix hint on how to import FastPackedString. spotted by stefanw dons@cse.unsw.edu.au**20050901080524] [wibble dons@cse.unsw.edu.au**20050901090556] [20% less space usage in map Don Stewart **20050901132707] [performance measurements versus PackedString Don Stewart **20050901135829] [Changed PackedString type to `Packed'. Avoids name clash with Data.PackedString Don Stewart **20050902011829] [Decide that `FastString' (a la GHC) is a better name than `Packed' Don Stewart **20050902013355] [wibble in QC Don Stewart **20050902013415] [Improve pack. withCString -> useAsCString Don Stewart **20050902052011] [More profiling Don Stewart **20050902052046] [Compiling via GHC needs -fglasgow-exts for unboxed operators dons@cse.unsw.edu.au**20050902052850] [Documentation update and two functions for raw access to the string pointer. lemmih@gmail.com**20050904035843] [Implement packAddress :: Addr# -> FastString Don Stewart **20050904054954] [basic test for packAddress in QC Don Stewart **20050904055006] [Non-copying version of packAddress, idea from Lemmih Don Stewart **20050904060510] [Implement unsafeFinalise, explicitly request an object if finalised. Idea from Lemmih Don Stewart **20050904061826] [fix warning pointed out by haddock dons@cse.unsw.edu.au**20050904063145] [Implement unsafePackAddress :: Int -> Addr# -> FastString, allowing for *O(1)* construction of FastStrings (if they're known statically)0 Don Stewart **20050905041003] [Fix mergeo Don Stewart **20050905041316] [typo in comment Don Stewart **20050905043437] [Add packCString, packCStringLen. Rename mallocCString2FastString as packMallocCString Don Stewart **20050905052651] [typo in comment only dons@cse.unsw.edu.au**20050905053924] [More haddocks Don Stewart **20050908065002] [typos in comments Don Stewart **20050911015759] [TAG version 0.1 Don Stewart **20050912022724] [More complexity annotations Don Stewart **20050912043931] [explicit export of $GM4 to GHC invocation Ketil Malde **20050912120714] [better m4 default in logpp Ketil Malde **20050912120746] [Raw access to Word8's. lemmih@gmail.com**20050916165438] [hGetNonBlocking lemmih@gmail.com**20051002123234] [Missing -fglasgow-exts in .no-mmap file Don Stewart **20051019000838] [Add toForeignPtr, fromForeignPtr Don Stewart **20051106235334] [Make sure to null terminate when we malloc, so unsafeUseAsCString works Don Stewart **20051115035112] [Add error checking to Data.FastPackedString.pack Samuel Bronson **20051121172657] [Add RULE to make pack more efficient on string literals Samuel Bronson **20051121172736] [Add shabang line to Setup.hs Samuel Bronson **20051121172903] [Add example/test program to make sure my RULE works Samuel Bronson **20051121174525] [Implement, and add QC tests for, isPrefixOf and isSuffixOf Don Stewart **20051124044221] [FPS is actually not GPLed anymore, it is BSDed. Samuel Bronson **20051206023355] [small typos in comments Ketil Malde **20051207102735] [Add Description field Don Stewart **20051214235723] [handle merge-o Don Stewart **20051215003801] [new APIs from Pugs: idx, lineIdxs, packChar, elems **20060101143125] [Added more documentation to betweenLines. bjorn@bringert.net**20060106165240] [Changed FastFastString to FastPackedString in some error messages. bjorn@bringert.net**20060108175003] [Include index and string length in index and indexWord8 error messages. bjorn@bringert.net**20060108222920] [Added copy function. bjorn@bringert.net**20060108235229] [Added QuickCheck property for copy. bjorn@bringert.net**20060109002059] [Added inits and tails, with QuickCheck properties. bjorn@bringert.net**20060109003522] [Added isSubstringOf, findSubstring and findSubstrings, with QuickCheck property. Implemented using the Knuth-Morris-Pratt algorithm. bjorn@bringert.net**20060109011953] [Some clean-up of findSubstrings. bjorn@bringert.net**20060109013331] [Splitting lines with CRLF Einar Karttunen **20060328151210] [Add replicate. been hanging around in hmp3 for a while Don Stewart **20060411144704] [add some new functions. and use inlinePerformIO, courtesy Simon M Don Stewart **20060412030158] [Faster map, factor out cbits, and general cleanup Don Stewart **20060417095904] [mergeo Don Stewart **20060417100836] [Make no-mmap the default Don Stewart **20060419024751] [Tweak some functions, based on benchmarks Don Stewart **20060419063652] [tweak Don Stewart **20060419063817] [benchmark tweaks Don Stewart **20060419064040] [typo in cabal file Don Stewart **20060419065130] [And also test Data.PackedString while we're here Don Stewart **20060419070928] [and flush Don Stewart **20060419072949] [more flushing Don Stewart **20060419073239] [Some tests won't run in the benchmark dons@cse.unsw.edu.au**20060419081025] [benchmark results dons@cse.unsw.edu.au**20060419081135] [TAG 0.2 Don Stewart **20060419081343] [And bump version Don Stewart **20060419082504] [And benchmark against previous version Don Stewart **20060419085241] [comment only Don Stewart **20060419125726] [Replace C maximum and minimum with fast Haskell versions Don Stewart **20060420020735] [Also run benchmarks by default Don Stewart **20060420021054] [BSD license everything. Remove some more cbits Don Stewart **20060420025410] [Comments, and clean up a warning Don Stewart **20060420031029] [Use plusPtr, instead of advancePtr, as Bulat suggested Don Stewart **20060420031410] [update benchmark results with non-cbit code Don Stewart **20060420033051] [Squash space leak in findIndicies Don Stewart **20060420034707] [typo in STRICT1 macro Don Stewart **20060420040526] [Update bench times with new fast elemIndicies Don Stewart **20060420040722] [mention speed of box the benchmarking was done on Don Stewart **20060420040926] [Add SCC tags to Bench.hs, so we can optimise on space too Don Stewart **20060420044927] [More efficient tails and much better elemIndices on space and tiem Don Stewart **20060420044951] [Add filterChar, a space and time efficient version of filter . (==), + properties Don Stewart **20060420050451] [elemIndices benches updated Don Stewart **20060420050704] [Tweak Bench.hs Don Stewart **20060420051107] [1G stress test results dons@cse.unsw.edu.au**20060420062731] [and results for 0.5G strs Don Stewart **20060420073336] [typo Don Stewart **20060420094023] [USE_MMAP should wrap some cbits too Don Stewart **20060421011242] [And guard the .h file , just for good measure Don Stewart **20060421011824] [Restore fast C versions of maximum and minimum. But make them optional Don Stewart **20060421013527] [+unsafeIndex, and Simon's 'split', which has nicer algorithmic properties Don Stewart **20060421030048] [Merge Simon's QC tests with our own. Combine all testing into 'make' target Don Stewart **20060421030116] [hmm. can't use -Defs in cbits, for current cabal, at least dons@cse.unsw.edu.au**20060421030744] [Clean up todo lsit Don Stewart **20060421031659] [More emphasis on Word8. Don Stewart **20060421033424] [More docs Don Stewart **20060421041321] [Bump version Don Stewart **20060421041452] [1 less todo Don Stewart **20060421042936] [TAG 0.3 Don Stewart **20060421043046] [doc tweaks Don Stewart **20060421043915] [wibbles Don Stewart **20060421082343] [Return rest of string in readInt Don Stewart **20060421085814] [And update qc test Don Stewart **20060421091758] [Benchmark for ghc 6.6 Don Stewart **20060421095136] [Fix compiling for GHC 6.5 Einar Karttunen **20060421195626] [HEADS UP: Module renamed: Data.ByteString Don Stewart **20060422024429] [getLine, getContents, faster elemIndices and lineIndices Don Stewart **20060422061624] [more benchmarks, more properties Don Stewart **20060422061733] [Merge splitWith and breakAll. splitWith is faster. Hence encode words as tokens isSpace Don Stewart **20060422065012] [Update speed Don Stewart **20060422070220] [Faster version of 'split' Don Stewart **20060422085154] [Update timings Don Stewart **20060422085442] [Reexport ByteString through FastPackedString. Solves versioning issues. Idea from Einar Don Stewart **20060423005939] [Add join2, a specialised join. And a ByteString getArgs (!) Don Stewart **20060423023544] [Docs only Don Stewart **20060423025655] [Add wc -l benchmark Don Stewart **20060423025710] [Export unsafeIndexWord8. Sometimes useful Don Stewart **20060423040324] [wibble Don Stewart **20060423042226] [Add bytestring putStr and putStrLn. Just synonyms Don Stewart **20060423051548] [Add zip, zipWith, unzip + QC properties Don Stewart **20060423054354] [Benchmark zip and zipWith Don Stewart **20060423055438] [Add revcomp benchmark Don Stewart **20060423055539] [Comments and cleanup Don Stewart **20060423062333] [5x faster replicate. Uses memset Don Stewart **20060423063530] [More benchmarks Don Stewart **20060423064340] [Most of the todos are done Don Stewart **20060423064744] [Bump version Don Stewart **20060423065714] [benchmark Don Stewart **20060423070704] [TAG 0.4 Don Stewart **20060423070723] [And also make a legacy synonym for FastString. Einar's idea again Don Stewart **20060423073431] [Move options back into pragma. Helps standalone use. Don Stewart **20060423075245] [Docs, and 'construct' is now called 'packCStringFinalizer' Don Stewart **20060424003842] [Add notElem + QC property Don Stewart **20060424025706] [Add filterNotChar == filter . (/=) , but 5x faster Don Stewart **20060424033759] [whitespace only Don Stewart **20060424043117] [space leak squashed in elemIndexLast Don Stewart **20060424131937] [Much more efficient breakLast Don Stewart **20060424134408] [Add bench case for elemIndexLast Don Stewart **20060424135143] [HEADS UP : split Word8 and Char between Data.ByteString and Data.PackedString.Latin1 Don Stewart **20060425091116] [HEADS UP: the split is now : Data.ByteString and Data.ByteString.Char, the latter provides Char ops (and exports pure ByteString ops too) Don Stewart **20060425101818] [typo Don Stewart **20060425103920] [typo Don Stewart **20060425104255] [complexity wrong in packChar/Byte Don Stewart **20060425104610] [Also export IO functions through Data.ByteString.Char Don Stewart **20060425105514] [typo Don Stewart **20060425110851] [filterChar * and friends aren't using predicates, pointed out by Duncan Don Stewart **20060425115829] [typo Don Stewart **20060425120200] [typo Don Stewart **20060425121952] [Inline pragmas speed up some things Don Stewart **20060425132505] [complexity annotations Don Stewart **20060425133212] [tweak inlines, based on empirical studies Don Stewart **20060425134509] [Add some #ifdefs for GHC specific things Einar Karttunen **20060426002759] [HEADS UP: Data.ByteString.Char -> Data.ByteString.Latin1. All functions needing Char.* now in Latin1. Don Stewart **20060425235642] [Better comments. 100% faster isSuffixOf (only look at the end of the string), and 30% faster append (don't use concat, just memcpy) Don Stewart **20060426003608] [Also move readInt* into Latin1.hs. It drops spaces. Don Stewart **20060426004640] [wibble Don Stewart **20060426010715] [Can't append in one memcpy. Bad on memory pressure Don Stewart **20060426011330] [Update bench mark times Don Stewart **20060426013638] [Benchmakr Char and Word8 layer simultaneously Don Stewart **20060426020127] [No need for separate Byte column. they run at the same speed Don Stewart **20060426020925] [strictness on split doubles speed Don Stewart **20060426021819] [comment only Don Stewart **20060426023708] [comment Don Stewart **20060426024224] [wibble Don Stewart **20060426024611] [Ok. Finally. John, Einar and I all prefer ByteString.Char8. So that's what it is. Don't make any claims about latin1 or encodings or nought. Don Stewart **20060426030318] [Implement count :: Word8 -> ByteString -> Int. Makes for 1 liner filterChar, and 3 x faster too. Idea from Ketil Don Stewart **20060426083502] [Export packAddress* via Char8 too Don Stewart **20060427003740] [Use Foreign.Concurrent.newForeignPtr whenever using GHC. Means ByteString now runs interpreted in ghci successfully Don Stewart **20060427003802] [Add QC properties for (w2c . c2w) == id , and (c2w . w2c) == id Don Stewart **20060427010353] [comment only Don Stewart **20060427012656] [A more elegant 'count' Don Stewart **20060427015642] [Use memcmp for compareBytes. It really is 4x faster now I check Don Stewart **20060427021219] [And use faster (?) Char8 pack, after consultation with Einar Don Stewart **20060427025726] [update benchmarks. looks good Don Stewart **20060427031105] [Use Ketil's idea for a custom Word8 isSapce Don Stewart **20060427032939] [point free Char8 makes the code simpler Don Stewart **20060427034243] [comment only Don Stewart **20060427034758] [TAG 0.5 Don Stewart **20060427035021] [bump version Don Stewart **20060427035312] [comment only Don Stewart **20060427041949] [hGetNonBlock is glasgow-specific Don Stewart **20060427061120] [Fix portability to hugs. Only a few cppism and a pattern type annotation had crept in. Don Stewart **20060427063052] [Make Quick.hs hugs friendly Don Stewart **20060427065021] [comment on pack's performance Don Stewart **20060427112537] [space only Don Stewart **20060428020949] [Fix minor bug in splitWith on hugs only Don Stewart **20060428021443] [Use Simon's qc framework. more amenable to testsuite error diffs Don Stewart **20060428021549] [typo Don Stewart **20060428022009] [comment Don Stewart **20060428064219] [AudreyT's copyCStringLen patch Don Stewart **20060428130726] [Add cbits version of count, around 30% over memchr version Don Stewart **20060429012016] [typo Don Stewart **20060429024852] [Fix indenting for YHC Don Stewart **20060429024911] [Unnecessary header import Don Stewart **20060429025206] [HEADS UP: remove mmapFile for now. Not portable enough. Don Stewart **20060429034429 Enable cbits by default for hugs and ghc This patch attempts to make fps 0.6 as portable as possible, after some experience trying to build with yhc. Redundant imports are gone. Cbits are on by default. mmapFile is removed for the time being (its both ghc and unix dependent). Hugs happily builds with the fast cbits code ] [Notes about porting to yhc and nhc Don Stewart **20060429042836] [comments only Don Stewart **20060429061359] [Fix rounding-pasto error causing string truncation in hGetLine Don Stewart **20060429081731] [Add spellcheck test from packedstring Don Stewart **20060429083026] [More info on running the testsuite Don Stewart **20060430031928] [Fix foldr1 in Data.ByteString and Data.ByteString.Char8 sjanssen@cse.unl.edu**20060430081629] [More foldl1/foldr1 properties, thanks to sjanssen Don Stewart **20060430083124] [Add group and groupBy. Suggested by conversation between sjanssen and petekaz on #haskell Don Stewart **20060501015116] [And quickcheck and benchmark them Don Stewart **20060501015139] [Another benchmark Don Stewart **20060501025339] [Another groupBy test. Suggested by sjanssen Don Stewart **20060501025628] [Bit faster if we filter before we map Don Stewart **20060501030102] [Fix groupBy to match Data.List.groupBy. sjanssen@cse.unl.edu**20060501064204] [Add -fglasgow-exts to OPTIONS_GHC Einar Karttunen **20060502163719] [comments only Don Stewart **20060503015657] [Migrate to counting sort. sjanssen@cse.unl.edu**20060503050107 Data.ByteString.sort used C's qsort(), which is O(n log n). The new algorithm is O(n), and is faster for strings larger than approximately thirty bytes. We also reduce our dependency on cbits! ] [Benchmarks for the new sort. sjanssen@cse.unl.edu**20060503050652] [couple of wibbles. and remove the stdlib header. not needed at all now Don Stewart **20060503054653] [sort goes from 13.9s to 0.16s over 20M. update benchmark numbers Don Stewart **20060503060334] [todo is empty now Don Stewart **20060503060504] [have linesort run in 'make' under tests/ too Don Stewart **20060503061209] [clean up makefile Don Stewart **20060504050127] [Array fusion for maps. Don Stewart **20060504084243 We can now fuse map f (map g (map h ... ) into map (f . g . h ..) More functional array fusion to come. ] [Add test for map fusion Don Stewart **20060504084955] [A better fusion test Don Stewart **20060504090949] [Add gloop, a generic bytestring loop operator, and another RULE Don Stewart **20060504123839] [simpler not elem Don Stewart **20060504131356] [Some short cut cases in eq and cmp. Suggested by dcoutts, ndm and musasabi Don Stewart **20060504225902] [Full array fusion for pipelines of maps and filters Don Stewart **20060505041200] [Add array fusion version of foldl. Don Stewart **20060505054353] [Fix build on hugs Don Stewart **20060505060041] [Oh. oops. \a0 should be in the isSpace function. spotted by QuickCheck Don Stewart **20060506012031] [comment only Don Stewart **20060506012546] [generalise some tests Don Stewart **20060506012556] [Use simple, 3x faster concat. Plus QC properties. Suggested by sjanssen and dcoutts Don Stewart **20060506022858] [Do realloc on the Haskell heap. And add a really tough stress test Don Stewart **20060506030134] [RULE to rewrite length . lines -> count Don Stewart **20060506031818] [Misc wibbles Don Stewart **20060506031830] [wibble Don Stewart **20060506033142] [Remove some dead code Don Stewart **20060506044855] [dcoutt's packByte bug squashed Don Stewart **20060506055931 With inlinePerformIO, ghc head was compiling: packByte 255 `compare` packByte 127 into case mallocByteString 2 of $ ForeignPtr f internals -> $ case writeWord8OffAddr# f 0 255 of _ -> $ case writeWord8OffAddr# f 0 127 of _ -> case eqAddr# f f of $ False -> case compare (GHC.Prim.plusAddr# f 0) $ (GHC.Prim.plusAddr# f 0) which is rather stunning. unsafePerformIO seems to prevent whatever magic inlining was leading to this. Only affected the head. ] [Squish orphan rules warning Don Stewart **20060506060240] [Add inline/unsafe bug test Don Stewart **20060506070213] [steal some more properties from the ndp quickchecks Don Stewart **20060506125651] [TAG 0.6 Don Stewart **20060506130044] [Start on fps 0.7 Don Stewart **20060506130626] [untab Don Stewart **20060506132823] [Much improved find and findIndex. Idea from sjanssen. + QC properties Don Stewart **20060507032014] [filterF is fast enough now we can defined filterNotByte in terms of it Don Stewart **20060507041405] [Fix import syntax for hugs. From sven panne Don Stewart **20060508001436] [Fix all uses for Int that should be CInt or CSize in ffi imports. Spotted by Igloo, dcoutts Don Stewart **20060508004014] [update timings Don Stewart **20060508022038] [Import nicer loop/loop fusion rule from ghc-ndp Don Stewart **20060508060941] [Fix stack leak in split on > 60M strings Don Stewart **20060508073607] [Try same fix for stack overflow in elemIndices Don Stewart **20060508075013] [Give unlines a chance to handle 0.5G Don Stewart **20060508080838] [Help out splitWith a bit Don Stewart **20060508081934] [Fix for picky CPP Einar Karttunen **20060508163628] [hide INLINE [1] from non-ghc compilers. pointed out by Einar Don Stewart **20060509020903] [Critical INLINE on words exposes splitWith and halves running time Don Stewart **20060509021450] [Better results on big arrays Don Stewart **20060509025506] [Reorder memory writes for better cache locality. Duncan Coutts **20060509123553] [Surely the error function should not be inlined. Duncan Coutts **20060509123629] [Generalise the type of unfoldrN Duncan Coutts **20060509132809 The type of unfoldrN was overly constrained: unfoldrN :: Int -> (Word8 -> Maybe (Word8, Word8)) -> Word8 -> ByteString if we compare that to unfoldr: unfoldr :: (b -> Maybe (a, b)) -> b -> [a] So we can generalise unfoldrN to this type: unfoldrN :: Int -> (a -> Maybe (Word8, a)) -> a -> ByteString and something similar for the .Char8 version. If people really do want to use it a lot with Word8/Char then perhaps we should add a specialise pragma. ] [merge-o dons@cse.unsw.edu.au**20060510020759] [Add foldl', and thus a fusion rule for length . {map,filter,fold}, that avoids creating an array at all if the end of the pipeline is a 'length' reduction Don Stewart **20060510032640] [Missing RULES pragma. doh Don Stewart **20060510033718] [An Int constraint on the fusion rule can't hurt Don Stewart **20060510035619] [clearer RULE names Don Stewart **20060510040126] [Test length/loop fusion Don Stewart **20060510040652] [Disable length/loop fusion for the time begin. Too many allocs Don Stewart **20060510042155] [Partial implementation of Data.ByteString.Lazy Duncan Coutts **20060507160506 The crux of this version is this lazy representation: data ByteString = LPS [P.ByteString] that is a lazy list of normal strict packed byte string. There is some redundancy in the representation (zero-length ByteStrings in the list) so we chose the data type invariant to be that list is either empty or consists of non-null ByteStrings. That is: > invariant (LPS []) = True > invariant (LPS xs) = L.all (not . P.null) xs If we don't use this invariant then various functions have several more tests to do. I think tt keeps things simpler to eliminate the redundancy. So all functions need to preserve this invariant. Conversion functions will need to check this (which can be done lazily) or we can provide unsafe versions where the proof obligation is on the user. One issue that's not quite clear is what the laziness guarantees should be (affects things like cons and concat) and should we do anything to coalase small chunks (particularly affects concat, concatMap etc) since small chunks are going to generally reduce performance. We probably want to optimise for the case where chunks are fairly large, eg 4k or bigger. Several functions are O(n/c) where n is the total length and c is the average chunk size. If we cannot heuristicly maintain chunk size then perhaps ew should provide an explicit rechunking function that coalacses chunks that are below a certain size. ] [More ByteString.Lazy updates Duncan Coutts **20060510172654 Needs P.findIndexOrEnd expored from ByteString, or needs to use a different implementation in a couple places. ] [group, join & indexing functions for .Lazy Duncan Coutts **20060510235409] [Factor error functions & strings. Duncan Coutts **20060510235512] [Export a couple of things needed by Lazy.hs and Char8.hs Don Stewart **20060511042955] [-Wall police Don Stewart **20060511045101] [More blurb on Lazy.hs Don Stewart **20060511045537] [Bug in foldr (it was folding from the left) spotted by QuickCheck Don Stewart **20060511060207] [Handle negative indices to take correctly. Spotted by QuickCheck Don Stewart **20060511062002] [Add Lazy.filter Don Stewart **20060511063408] [whitespace only Don Stewart **20060511065737] [Add QuickCheck properties for Data.ByteString.Lazy. Try runhaskell Lazy.hs in tests/ Don Stewart **20060511065747] [Fix deriving Show for hugs, now QC/Lazy tests pass on hugs too Don Stewart **20060511070313] [Also compile the QC/Lazy tests. Good for your stress Don Stewart **20060511070654] [A better Arbitrary instance Don Stewart **20060511091357] [Change a couple more Int to CInt. Fixes compare on 64bit arches. Duncan Coutts **20060511093426] [Fix reverse and reverse QC check Duncan Coutts **20060511101719] [Optimise foldr1 foldl1 cases by relying on invariant Duncan Coutts **20060511101851] [More Lazy updates. Duncan Coutts **20060511220530] [Test a smaller range of Word8 in Lazy,hs, hoping to hit pivot elements more often. Clean up Quick.hs Don Stewart **20060512014938] [Comment out splitWith until we work out how to do it. Don Stewart **20060512020522] [QC properties for find, findIndex/find, elem, notElem, filterByte, filterNotByte Don Stewart **20060512021743] [HEADS UP: Add lazy IO functions. Haskell breaks the 4G barrier! Don Stewart **20060512033330] [Add IO with tunable chunk size Don Stewart **20060512034507] [Add a Lazy.hs IO benchmark Don Stewart **20060512035522] [make haddock happy Don Stewart **20060512042921] [Fix Int/Int64 types in Lazy tests Duncan Coutts **20060512073653] [Fix and tidy split & slitWith Duncan Coutts **20060512091352] [Add invariant checking and prop_split to QC tests of Lazy Duncan Coutts **20060512091532] [Do our own realloc, and thus generate() can use the Haskell heap. Makes abotu 10% for reduction functions that use generate Don Stewart **20060513020613] [whitespace Don Stewart **20060513020655] [Tweak test Don Stewart **20060513020912] [Fix type of snoc in docs Duncan Coutts **20060512190305] [Add an alternative framework for QCing the Lazy module Duncan Coutts **20060512190359] [Add some zipping ops to .Lazy Duncan Coutts **20060512190724] [Increase default chunksize to 128k, assuming everyone's cache is at least this big. Don Stewart **20060513030736] [More useful lazyio.hs Don Stewart **20060513030758] [Handle invariant failures in init and concatMap Don Stewart **20060513043417] [Add new tests to testsuite Don Stewart **20060513043442] [new realloc helps timings Don Stewart **20060513051208] [Add isPrefixOf to Lazy Josef Svenningsson **20060513180238] [Add QC test for isPrefixOf Don Stewart **20060514014058] [-Wall police. Manually fuse filter (not.null) . map f. More comments Don Stewart **20060514051427] [more tests Don Stewart **20060514141605] [more tests Don Stewart **20060514144058] [Bump down default chunk size to 64k. Should be enough room for other programs Don Stewart **20060515001807] [dead code elimination Don Stewart **20060515020037] [Add 'make plot' target in tests/ that draws pretty graphs of Lazy vs Vanilla bytestrings Don Stewart **20060515042823] [Disable group/groupBy. There's a bug. Spotted by ADEpt. I dislike this function anyway. Too inefficient ;) Don Stewart **20060515071048] [Make "mycheck" generate different randoms each run. Moved test helpers common to both testsuites to separate module. Dmitry Astapov **20060515071310] [Merge ADEpts patch dons@cse.unsw.edu.au**20060515071948] [Missing --make in tests/ Don Stewart **20060516002907] [Probably fix for group/groupBy Duncan Coutts **20060515140638] [change code to make comment unnecessary Duncan Coutts **20060515143438] [Remove special cases from split/splitWith. Duncan Coutts **20060515143541 This improves the performance of L.split. For some reason L.splitWith is much slower than P.spltWith where as L.split is only marginally slower than P.split. ] [Add group & groupBy to benchmark Duncan Coutts **20060515143929 The lazy version performs reasonably well. ] [disable groupBy for now. Note bug in groupBy (/=) when chunk size is 1 byte Don Stewart **20060516005346] [-Wall police, and tweak Lazy.hs Don Stewart **20060516005659] [Reorder Bench.hs slightly Don Stewart **20060516010117] [Tweak the graph output slightly Don Stewart **20060516014949] [Fix sily copy'n'paste bug in Lazy.groupBy Duncan Coutts **20060516042455] [Fix -Wall groupBy warnings and re-enable groupBy benchmark Duncan Coutts **20060516051812] [mapF -> map', filterF -> filter' Don Stewart **20060516113411] [tweak tests Don Stewart **20060516113433] [Comment about the shift/peek::Word32 approach being slower Don Stewart **20060516115733] [Move -fno-warn-orphans into OPTIONS pragma Don Stewart **20060516120917] [Tweak bench output Don Stewart **20060516120929] [Handle n <= 0 cases for unfoldr and replicate. Spotted by QC Don Stewart **20060516124622] [Handle n < 0 in drop and splitAt. Spotted by QC. Don Stewart **20060516131511] [Lots more QC tests using the new framework Don Stewart **20060516132636] [Add 'make fast' test target, just runs the QC tests 4 ways Don Stewart **20060516134450] [bug in comment Don Stewart **20060516142027] [More QuickChecks Don Stewart **20060517012419] [More messing in the tests/ Don Stewart **20060517015513] [INLINE on Lazy.find,findIndex squashes whatever was happening, and they now run at ByteString speeds Don Stewart **20060517022434] [INLINE also fixes splitWith Don Stewart **20060517023405] [Since we can't fuse Lazy.map or Lazy.filter, use the fast P.map' and P.filter' Don Stewart **20060517024325] [Clarify performance of lazy, now we've squashed the leaks Don Stewart **20060517025107] [Add lazy foldl', with QC and benchmarks Don Stewart **20060517040826] [Comment out unimplemented things Don Stewart **20060517070425] [Note that groupBy is still broken on defaultChunkSize=1 Don Stewart **20060517071948] [Fix Lazy.groupBy again Duncan Coutts **20060517170250 The problem was happening for operators that are not equivalence relations eg (<). One bug was that it was using the operator the wrong way wround. The more subtle problem was that it was using the final element in the group to compare against rather than the first as Data.List.groupBy does. For example groupBy (<) [0,10,5] = [[0,10,5]] rather than [[0,10],[5]] this is because we compare 0 & 5 rather than 10 & 5 since the first element in the group is the one that is used as the representative element. ] [Whitespace changes Duncan Coutts **20060517172315] [Add unsafeTake and unsafeDrop Duncan Coutts **20060517231908 These versions assume the n is in the bounds of the bytestring, saving two comparison tests. Then use them in varous places where we think this holds. These cases need double checking (and there are a few remaining internal uses of take / drop that might be possible to convert). Not exported for the moment. ] [Eliminate special case in findIndex since it's handled anyway. Duncan Coutts **20060517232249] [More effecient findIndexOrEnd based on the impl of findIndex Duncan Coutts **20060517232425] [copyCString* in IO, and fix buglet in splitAt (degenerate cases the wrong way around) Don Stewart **20060518022054] [assertion test wrong way around. spotted by hugs (!) Don Stewart **20060518025118] [Use readFile, it'll avoid realloc issues Don Stewart **20060518055949] [Comment on when to use readFile over hGetContents Don Stewart **20060518060305] [Export unsafeTake and unsafeDrop sjanssen@cse.unl.edu**20060518054546] [Make Lazy.replicate more efficient for large inputs sjanssen@cse.unl.edu**20060518055354] [Add lazy hGetLines to Data.ByteString Don Stewart **20060518063928] [Add test for hGetLines. A faster, lazier wc -l Don Stewart **20060518064516] [Comment Don Stewart **20060518064704] [tweak Don Stewart **20060518072339] [Fix invariant violation in new replicate Don Stewart **20060518080636] [asthetic tweaks to bench code, also add B.findIndexOrEnd Duncan Coutts **20060518083425] [Use Lazy.replicate in Lazy.filterByte Don Stewart **20060518091539] [Make sure to optimise the cbits Don Stewart **20060518135255] [Add files to help us compare the api of list vs. fps Duncan Coutts **20060518154230] [Add iterate, repeat & cycle to .Lazy Duncan Coutts **20060518155146 These are more infinite list functions using sjanssen's trick. Also add a smaller chunk size for use with these functions to make it a bit less wasteful of space (currently 4k vs 64k). ] [Handle merge dons@cse.unsw.edu.au**20060519010318] [And make Duncan's patches type check dons@cse.unsw.edu.au**20060519010800] [Add foldl1' Don Stewart **20060519015308] [More Properties.hs Don Stewart **20060519015328] [Update api diff Don Stewart **20060519015343] [Make the api easier to diff (try vimdiff *.api) Don Stewart **20060519020218] [Comment fix. sjanssen@cse.unl.edu**20060518202323] [Data.ByteString.Internal sjanssen@cse.unl.edu**20060519025815 Contains functions which might be useful in several modules, but aren't appropriate to expose to users of ByteString. For example: inlinePerformIO; various C functions. ] [Add fuseable scanl, scanl1 + properties Don Stewart **20060519034124] [Clean up api list Don Stewart **20060519040944] [export scanEFL Don Stewart **20060519041004] [Merge Quick.hs into Properties.hs, the grand, unified properties list Don Stewart **20060519041016] [And clean up makefile. Add make fast target, for a quick, quick check Don Stewart **20060519042246] [And make Interal.hs hugs-friendly Don Stewart **20060519044211] [Update readme to reflect new test structure Don Stewart **20060519045703] [Comment on the inappropriatness of concatMap on ByteStrings Don Stewart **20060519051513] [Spotted another chance to use unsafeTake,Drop (in groupBy) Don Stewart **20060519082040] [Sadly cc-options get applied to .hs files too which will break things. Duncan Coutts **20060519094916 Such aggressive gcc flags are only appopriate for the fpstring.c module. Incidentally, -optc-O2 will also break on some arches due to mangler issues. ] [Add Lazy.interact Duncan Coutts **20060519095122] [Use List function rather than inlined definition. Duncan Coutts **20060519095231 (And this time it type-checks!) ] [Hide Prelude.interact.. Don Stewart **20060519102523] [Tweak api Don Stewart **20060520031245] [Move c2w, w2c into Internal, now Data.ByteString.Lazy.Char8 needs them too Don Stewart **20060520031418] [Move more Char8 things into Internal.hs Don Stewart **20060520032232] [Add Data.ByteString.Lazy.Char8 Don Stewart **20060520035726] [Whitespace only Don Stewart **20060520035945] [Plot with boxes. Clearer Don Stewart **20060521003938] [Add SCC pragmas so make prof keeps working Don Stewart **20060521041626] [Minor doc tidyup. Use haddock markup better. Duncan Coutts **20060523004209 Remove untrue comment about CStrings in .Lazy. Note dons as author of .Lazy too. ] [Simplify the join() implementation. Spotted by Duncan. Don Stewart **20060523010031] [Add a TODO list of things duncan and I talked about on irc Don Stewart **20060523010550] [More stuff to do Don Stewart **20060523012624] [Abolish elems. It's name implied it was unpack, but its type didn't. it made no sense Don Stewart **20060523022409] [wibble in todo list Don Stewart **20060523022906] [In the search for a more orthogonal api, we kill breakFirst/breakLast, which were of dubious value. Don Stewart **20060523035734] [elemIndexLast -> elemIndexEnd Don Stewart **20060523040051] [pack{Byte,Char} -> singleton. As per fptools convention Don Stewart **20060523040432] [Start a bit on the null terminated string property Don Stewart **20060523040714] [Tweak todo Don Stewart **20060523044403] [Reorder the export lists to better match the Data.List api Duncan Coutts **20060523153941 If this seems a good idea, I'll do it for the .Char8 modules too. ] [Comment only. sjanssen@cse.unl.edu**20060523023434] [Another comment. sjanssen@cse.unl.edu**20060523032625] [Add unfoldr to ByteString and .Char8 sjanssen@cse.unl.edu**20060523223229 A preliminary implementation of unfoldr. Things that still need to be done: 1. Tests for unfoldr. 2. unfoldr currently reallocates a new buffer twice the size of the old one, dcoutts has suggested an approach using concat. We need to compare the performance of these two approaches. 3. unfoldr for .Lazy. ] [Merge dcoutt's and sjanssens's patch dons@cse.unsw.edu.au**20060524004748] [Change the implementation of the unfoldr(N) functions. Duncan Coutts **20060524044732 Use a more compact implementation for unfoldrN and change it's behaviour to only return Just in the case that it actually 'overflowed' the N, so the boundary case of unfolding exactly N gives Nothing. This also helps with maintaining the invariant in .Lazy Implement unfoldrr and Lazy.unfoldr in terms of unfoldrN. Use fibonacci growth for the chunk size in unfoldr and simple quadratic growth for Lazy.unfoldr and in the latter case an upper bound on the chunk size. upper ] [Doh! revert change to type used during debugging. Duncan Coutts **20060524051437] [Implement mapAccumL and reimplement mapIndexed using loopU Duncan Coutts **20060524045956 Not fully tested yet. ] [Implement coalescing in cons Duncan Coutts **20060524050156 Not perf tested yet. The coalescing threshold needs testing to find the ideal value. ] [Rearange export lists for the .Char8 modules Duncan Coutts **20060524050404 and minor tidyups for the exprt lists of the other two modules. ] [instance Monoid ByteString sjanssen@cse.unl.edu**20060524054938] [Swap the result tuple in loopU Duncan Coutts **20060524160046 because it's obviously the wrong way round! ;-) Currently we have: loopU :: (acc -> Word8 -> (acc, Maybe Word8)) -> acc -> ByteString -> (ByteString, acc) which has the result of the per-elem map/fold/filter func returning (acc, Maybe Word8) but the overall result as (ByteString, acc) which is reversed. So we also have to swap loopArr, loopAcc and loopSndAcc. At the same time I'm generalising their types so that we'll be able to use them (and the rules that apply to them) to [ByteString] as well as just ByteString. ] [Add loopU for lazy ByteStrings Duncan Coutts **20060524161022 This reuses much of the fusion machinery from the ByteString module so we only need to add loopU and the loop/loop fusion RULE. ] [Use loopU to make several ByteString.Lazy funcions fusable Duncan Coutts **20060524161837 This is really just an experiment, though the QC tests still seem to be working, and the fuse test indicates that the fusion RULES are working. ] [export mapAccumEFL, mapIndexEFL for use in .Lazy Duncan Coutts **20060524231532] [Add Lazy.inits and tails, including QC tests Duncan Coutts **20060524231616] [Add Lazy.foldl1' Duncan Coutts **20060524232019 And fix docs for foldl' ] [Documentation fix. Lemmih **20060524190752] [Remove the api files, we know what's missing now Don Stewart **20060525012327] [Disable coalescing cons. Its too strict. Add QC test for diverging cons. Don Stewart **20060525015607] [Add properties for unfoldr, mapAccumL and foldl1' Don Stewart **20060525032520] [Make hugs happy Don Stewart **20060525032757] [Further simplify the instances for ModeledBy Don Stewart **20060525035021] [Dont need undecidable instances, just overlapping ones Don Stewart **20060525042349] [Lots of commentary on how the model checking quickchecks works Don Stewart **20060525043308] [unfoldr is done. push of todo list Don Stewart **20060525043836] [Use the NatTrans class from Gofer to write a generic Model instance for types of kind * -> * Don Stewart **20060525052016] [Note that foldl . map with lazy fusion seems to run slower. foldl'.map seems fine though Don Stewart **20060525055458] [Export some more fusion utilities, so we can QC them Don Stewart **20060525061026] [Convert all RULES into QC properties, and check them Don Stewart **20060525061044] [And QC tests reveal my hacking lines/count rule was wrong if there's no \n in the string. Disable it. Don Stewart **20060525061635] [Make hugs happy Don Stewart **20060525061936] [Add breakEnd, (== spanEnd (not.p)). Like break, but from the end of the string Don Stewart **20060525093410] [Move the ByteString representation and low-level bits to a Base module Duncan Coutts **20060525143852 Also move the array fusion code. Patch up the import/exports for the other modules. ] [Make fuseEFL use a StrictPair for much better fusion performance Duncan Coutts **20060525144903 With this change the fusion of fold . map runs about 40 times quicker since the extra strictness allows ghc to avoid allocting lazy pairs. ] [Prevent fuseEFL from getting inlined too early. Duncan Coutts **20060525145121 This is a patch from ndp upstream. ] [Trivial doc fix/tidy. Duncan Coutts **20060525145229] [Do not need -fffi on any module except Internal Duncan Coutts **20060525145447] [Re-enable all the fusion tests. Duncan Coutts **20060525145718 Yay, fusion is no longer slower. ] [Update the fusion properties Duncan Coutts **20060525151151] [Add (commented out) wrapper/loop version of loopU Duncan Coutts **20060525152156 including the loop/loop wrapper elimination RULE ] [wibbles Don Stewart **20060526021755] [Abolish .Internal. Move fusion stuff into .Fusion. And internal stuff into .Base Don Stewart **20060526041812] [comment wibbles Don Stewart **20060526042106] [More extensive use of strict pairs, from the ndp branch Don Stewart **20060526052624] [Make hugs happy Don Stewart **20060526054110] [Reinstante coalescing cons with a bigger warning about its strictness Duncan Coutts **20060526101927 Also remove lazy cons QC test. Fix untrue comment on replicate. ] [Tiny doc addition Duncan Coutts **20060526144642] [Cleanup of existing Fusion code and wdges of new code for a new method. Duncan Coutts **20060526144708 Rename NoAL -> NoAcc. Remove the noAL value, just use the constructor everywhere. rename type EFL to AccEFL and add several other special cases. Don't parameterise AccEFL by the in and out types, it's always Word8 Since we don't have so many 'Word8's all over the place we don't need the type W = Word8 short hand any more. Huge new chunk of code for an alternative loop fusion system. It uses first wrapper elimination to get several loops running in sequence and allocating just one memory block for them all. Then within that we fuse adjacent compatible loops (eg up/up, down/down) But it much more detailed than that because we've got cases for not just up and down loops but also loops that can run in either direction (ie they don't use an accumulator) and two important special cases of that: map and filter. This gives a lot of combinations of loops that can be fused. 23 to be exact giving rise to 23 fusion RULES. Currently the fusion is not as reliable as it could be but we believe that the RRULES matcher in the current version of ghc will deal with it better. See the analysis in the file for details. ] [Enable and export the alternative array fusion system. Duncan Coutts **20060526181130 It's not being used yet in ByteString but it can be tested seperately. It needs a very recent version of ghc-6.5 (May 25 or later) for the RULES matching to work reliably. ] [Add a couple TODO items to think about Duncan Coutts **20060526194348] [Typo in comment Don Stewart **20060527011017] [Typo fix. Maxime Henrion **20060524164903] [Heads up. unsafe* now moved to .Base Don Stewart **20060527015915] [Make haddock happy Don Stewart **20060527020535] [Move rules and loopU for lazy strings into Fusion.hs too Don Stewart **20060527025612] [Simplify type of loopL Don Stewart **20060527030204] [Remove duplicated RULES Duncan Coutts **20060527231603] [Fix nasty bug in doDownLoop Duncan Coutts **20060527231907 Copy'n'paste bug. I wasn't calculating the new size of the array correctly which caused the array copying code to segfault. ] [Remove big useless comment. Duncan Coutts **20060527233207 We've debugged that problem now. ] [Update type of copyCString,copyCStringLen in export list comment Duncan Coutts **20060528020241] [Make ByteString use the new fusion framework Duncan Coutts **20060528020835 That is the V2 fusion framework. ] [Fix to use the V2 fusion api properly Duncan Coutts **20060528021852 (hangover from testing the V3 api) ] [Another TODO item Duncan Coutts **20060528024227] [Wibble. And fix bug in fuseMapMapEFL, spotted by QuickCheck Don Stewart **20060528065843] [Add QC properties for all v2 rules. Disable down/* */down rules for now Don Stewart **20060528070516 All fused down rules are wrong at the moment, I think due to a bug in doDownLoop, when filtering. When filtering we end up with a shorter dest buffer, but shorter from the start, not the end. When doing down filtering: [abcdefg] -> [...defg] So then, when the wrapper reallocates, it tries as usual to realloc from the front of the string. Leading to a result: [...d] This doesn't happen when we sequenceLoops down though. So disable all down fusion for now, until this is resolved. We may need to return the start offset of the string to loopWrapper, instead of relying on an offset of 0, when reallocating. Added test program, down-fuse.hs, testing the down/down fusion rule, that reveals this bug (try it in ghci, for example). ] [Split up the list of tests to make QCing quicker Duncan Coutts **20060528162409 Also add 'module Main where' so everything is exported which also makes QCing individual tests or groups of easier because we can do it in GHCi. ] [Fix down/down loop fusion bug Duncan Coutts **20060528163337 We have to keep track of the offset as well as just the length of the part of th dest array that has been filled in. ] [Don't use inlinePerformIO for loopWrapper due to danger of sharing Duncan Coutts **20060528163438 We really really don't want the dest array to be shared between two loops so use unsafePerformIO rather than inlinePerformIO. We're really depending pretty heavily on GHC optimiser semantics here. This fixes all the QC down loop fusion properties when compiled with -O. ] [Specify type for 'run', makes hugs happy Don Stewart **20060529004534] [Use unsafePerformIO on all functions that allocate. Hopefully avoids sharing issues Don Stewart **20060529010222] [Fix bug in Lazy.scanl (bizarrely, imo, Haskell's scan returns a list one element longer than the input list) Don Stewart **20060529024043] [Three (!) pastos in scanr, scanr1. QuickCheck people! Don Stewart **20060529032438] [. Add tests that fusion is working in up/down/map/filter cases. Don Stewart **20060529034333 . Clean up QuickCheckUtils a bit. . Add some commentary to the rules All looks good. Loops written with up/down/filter/map are fusing correctly. Now, we don't seem to have any NoAcc loops (since we've got map and filter instead). Can we then throw away the noAcc rules? This would simplify things. One issue was raised. Haskell's scanr/scanl won't fuse in certain cases, since they return a buffer 1 larger than their input. Do we break the semantics and behave like a parallel scan would, or do we just live with this? Where now? Performance testing with the head, with a view to discarding map' and filter'. Try writing a few more things using up/down loops. ] [Wibbles/ cleanup in Fusion.hs. Add a marginally faster doMapLoop (only carry one offset) Don Stewart **20060529050554] [Add length/loop* fusion, with quickchecked properties. Worth ~10% speedup Don Stewart **20060529050628] [makefile wibble Don Stewart **20060529050929] [wibbles Don Stewart **20060529051351] [wibbles Don Stewart **20060529051536] [typo dons@cse.unsw.edu.au**20060529052205] [whitespace only Don Stewart **20060529052756] [wibble dons@cse.unsw.edu.au**20060529055835] [Add minimum/maximum fusion too. But disable for now, its still at least 50% down on the C functions Don Stewart **20060529061354] [Actually, reenable rules for maximum/minimum. Only around 10% worse in stable. Should be ok in the head. Don Stewart **20060529062420] [Add properties for wrapper elimination and associativity of sequence Don Stewart **20060529064500] [Generate random up/down loops, for tougher testing of associativity and wraper elim Don Stewart **20060529070631] [whitespace, and don't flip args in fuseMapMap. Don Stewart **20060529122839] [Add fusion properties for all noAcc forms Don Stewart **20060529133540] [missing cpp pragma in tests/ dons@cse.unsw.edu.au**20060529134853] [And make QC properties for map/map fusion match the flipped args Don Stewart **20060529140203] [Lazy.hGetNonBlocking Lemmih **20060529234435] [wibble Don Stewart **20060530001407] [Add a fusion bench mark. Great results Don Stewart **20060530014141] [And add some big pipelines to illustrate the O(n) versus O(1) overhead fusion gives us Don Stewart **20060530015139] [Add cpp to switch between v1 and v2 fusion. Enables us to run benchmarks both ways Don Stewart **20060530034713] [Tweak copyright on Lazy.hs. the university of glasgow wasn't involved. Don Stewart **20060530082440] [notice 1 extra byte was allocated on empty and on singleton Don Stewart **20060530095147] [Comments on what happens in empty string cases for head, tail etc. Prompted by ndm Don Stewart **20060531014945] [cpp in QuickCheckUtils.hs to handle missing functor instances Don Stewart **20060531031309] [add make hugs target Don Stewart **20060531031328] [Add mallocByteStringWith. Don Stewart **20060531031553 Previously, mallocByteString would just poke a null byte, and return a buffer. This is bad for cache behaviour, particularly in .Lazy, as we'd be touching the last cache line, then switching back to the first to fill it. This patches introduces mallocByteStringWith: mallocByteStringWith :: Int -> (Ptr Word8 -> IO Int) -> IO ByteString mallocByteStringWith l f = do fp <- mallocForeignPtrArray (l+1) m <- withForeignPtr fp $ \p -> do n <- f p poke (p `plusPtr` n) (0::Word8) -- i.e. touch the last cache line last return n return $ PS fp 0 m So now we fill the buffer linearly. A side effect is that 'create' is now: create l f = unsafePerformIO $ mallocByteStringWith l (\p -> f p >> return l) Another benefit is that we need only one 'withForeignPtr' to create a bytestring, instead of 2. ] [generate -> createAndResize. Old name was silly. Don Stewart **20060531033310] [tweak makefile for wc dons@cse.unsw.edu.au**20060531034357] [comments and todos, only Don Stewart **20060531034051] [Wrong test in hGetN, spotted by bringert Don Stewart **20060531035856] [clean up notNull functions Don Stewart **20060531054900] [Changed all functions in Lazy and Lazy.Char8 which deal with indices and lengths to use Int64. (All except mapIndexed that is, I leave the magic to dons). bjorn@bringert.net**20060531062119] [A couple TODO isses to think about Duncan Coutts **20060531014024] [Use mallocForeignPtrBytes rather than mallocForeignPtrArray Duncan Coutts **20060531131551 Since we're always using bytes anyway, saves te comopiler having to specialise the sizeOf (undefined :: Word8). ] [Add assertions to null & length Duncan Coutts **20060531171146 Assert that the length is not negative. In the case that the assertion is disabled, consider negative lengths as null. ] [Use bracket in readFile, just like we do for writeFile Duncan Coutts **20060531171616] [Add assertaions to unsafeHead, unsafeTail and unsafeIndex Duncan Coutts **20060531172117] [make hGetN and hGetNonBlockingN strict Duncan Coutts **20060531172656 we can't do lazy IO except for hGetContents which reads the whole file and can semi-close the handle. also make hGetNonBlockingN take a maximum size so it matches the version from Data.ByteString. ] [Merge dons@cse.unsw.edu.au**20060531235148] [Remove pointless special case from hPut Duncan Coutts **20060531225027] [put all the foreign imports in IO rather than pure Duncan Coutts **20060531232644 This makes the unsafeness more explicit. We were mostly using them in an IO context anyway. Where we do need to use them in a pure context I've added unsafePerformIO ] [Reorgaise and rename low level ByteString generators Duncan Coutts **20060531234711 rename create to unsafeCreate since it's not in IO rename mallocByteStringWith to create rename createAndResize to createAndTrim add createAndTrim' as a generalisation remove mallocByteString ] [Fix type of pack in hugs Don Stewart **20060601002041] [remove skipIndex. Use one of the elimination forms instead Don Stewart **20060601005423] [Add appendFile for both strict and lazy Duncan Coutts **20060601004358] [whitespace, wibble Don Stewart **20060601005915] [Cleanup QCs to make them more presentable Don Stewart **20060601055945] [Comments on the difficulty of moving compareN/eqN into a single class Don Stewart **20060601073756] [Update sum test Don Stewart **20060601134737] [Test described on haskell-cafe@ Don Stewart **20060602023554] [Also export appendFile via Char8.hs Don Stewart **20060602024722] [Add cpp to switch between the 4 forms of fusion possibilities we currently have. Makes benchmarks much easier to generate Don Stewart **20060602033740] [Bench wibbles Don Stewart **20060602033820] [todo item. Don Stewart **20060602033842] [make it possible to do fusion profiling too Don Stewart **20060603024545] [Add more strictness in the results of several functions Duncan Coutts **20060602214417 Not sure if it actually help but it looks sensible. ] [Make the code stle for head consistent with tail Duncan Coutts **20060602214522] [Make readChunks strict in a more obvious way Duncan Coutts **20060602214637] [Add more strictness in the results of a few Char8 functions Duncan Coutts **20060602214756] [Add experimental Haskell impls of of readInt for Char8 and Lazy.Char8 Duncan Coutts **20060602215242 Seems to work but not fully QC'ed (indeed no QC tests added yet) However it gets the same results for the sumfile test so it's probably ok... ] [Experimental reimplementation of Lazy.Char8.lines for improved performance Duncan Coutts **20060602220044 Not fully QC'ed but does work with sumfile. Quite a bit faster but significantly more complex (ie a page long rathe than 5 lines) ] [QuickChecks for new readInt impl. And comments, -Wall police Don Stewart **20060603030319] [Remove half-completed, commented out readint Don Stewart **20060603031118] [Add QC properties for lines Don Stewart **20060603032703] [wibbles Don Stewart **20060603034618] [HEADS UP: remove null termination pseudo-property Don Stewart **20060603034653 Previously, certain "fresh" bytestrings would be allocated with a null byte at the end. They could then be used, without a copy, in C, or with a small number of 'unsafe*' operation. The problem with this scheme was that the property wasn't preserved when using substring operations, so was very difficult to reason about (the property wasn't reflected in either the type, or the functions used). Another problem, pointed out by Bjorn B., was that it messes with the lazy chunking code, where our neat cache chunks get allocated as 2^k + 1 -- which we suspect has performance issues. In the end, due to the difficulties of working out when there was a null byte, it was only me using this property, in hmp3. So, we scrap it now, and ask that you use useAsCString when you need the null byte (or else try a `snoc` 0). ] [some todo issues dealt with Don Stewart **20060603040030] [Remove cpp in tests, makes hugs happy Don Stewart **20060603041308] [tweak readme Don Stewart **20060603041325] [make it easier to experimentally benchmark different chunk sizes Don Stewart **20060603061139] [tweak Don Stewart **20060603072106] [Have 'darcs test' run a quick version of the QCs Don Stewart **20060603074219] [Use -optc-O1 in .Lazy*, -optc-O2 breaks with gcc 4.0.4. Spotted by Audrey Don Stewart **20060603092200] [Add support for tuned foreign pointer implementation Don Stewart **20060604072133 For large numbers of bytestrings, we should see a few percent speed up, and a few percent less space allocated, once the tuned foreign pointer patch makes it into ghc. This patch adds fps support for that patch (though you currently need -DFAST_FOREIGN_PTR to enable it, along with a custom ghc build). ] [investigated, patched, done with foreign ptr tuning Don Stewart **20060604073757] [Remove test that depends on TH. Rarely used. Not maintained Don Stewart **20060604074500] [Update testsuite instructions Don Stewart **20060604075225] [update list of things that are run on 'make everything' Don Stewart **20060604075506] [More readme tweaks Don Stewart **20060604075802] [wibble Don Stewart **20060604110147] [Start documenting how to do this handle flushing/copying trick Don Stewart **20060604114812] [couple of todo/comments only Don Stewart **20060605114632] [some more todos Don Stewart **20060605121941] [Run each benchmark several times Duncan Coutts **20060605233858 Requires changing the 'F' stuff so that it doesn't memoise. Rearange the fusion tests into up,down and noAcc groups. Currently run each test 11 times so we can use 10 runs (after discarding the first run) ] [todos Don Stewart **20060608013620] [add better hPutStrLn, avoids locking the handle twice on small strings. Suggested by Bulat Don Stewart **20060608020152] [Comment on the include-dirs relative path issue Don Stewart **20060609005913] [unsafe* shouldn't be exported from Char8 Don Stewart **20060609014206] [hGetLines should be exported via char8 too Don Stewart **20060609072106] [Prepare for merging back into bsae/ Don Stewart **20060609075426] [faster 'darcs check' Don Stewart **20060609075557] [make hugs happy Don Stewart **20060609081657] [Prefix C calls in fpstring.[hc] with fps_ audreyt@audreyt.org**20060611033535] [resync Lazy.Char8 and Lazy Don Stewart **20060611043208] [and fix import Prelude hiding list Don Stewart **20060611043626] [Add model checking property for mapAccumR Don Stewart **20060611043650] [more export wibbles Don Stewart **20060611043844] [wibble Don Stewart **20060611043950] [comments in .cabal file Don Stewart **20060611044125] [note that bench is temporarily broken. Don Stewart **20060611045257] [wibble in test dons@cse.unsw.edu.au**20060611063243] [wibble dons@cse.unsw.edu.au**20060611082431] [update docs Don Stewart **20060611140028] [doc fix Don Stewart **20060611140341] [Add target 'prop-opt', after uncovering as-yet-unfound bug in -O -frules-on Don Stewart **20060612083832] [Also export appendFile via Lazy.Char8 Duncan Coutts **20060602093505] [Dirty the cache before each run Duncan Coutts **20060612094302 and reorder the GC & delay ] [Fix copy'n'pasto in SCC name Duncan Coutts **20060612094418] [Re-export appendFile and import hGetNonBlocking(N). Duncan Coutts **20060612095625] [move unsafeUseAsCString* into Base.hs. Sync Char8 and normal ByteStrings Don Stewart **20060615055924] [Add zipWith', a speclialised zipWith. And QC properties for the zip family Don Stewart **20060615070057] [and add a zipwith specialisation test Don Stewart **20060615070801] [missing #ifdef __GLASGOW_HASKELL__ in Lazy.Char8 Don Stewart **20060615080527] [use just -O2 Don Stewart **20060615085333] [no -optc-O2, breaks too often Don Stewart **20060616070944] [better zipwith spec test. more notes Don Stewart **20060621033651] [Disable unpack/build fusion for now. It interacts badly with new rules stuff in ghc 6.5 Don Stewart **20060702084231] [Fix unpackFoldr bug, twas the seq in the acc which breaks with rules Don Stewart **20060703094346] [Further comment on seq and build/foldr Don Stewart **20060706033352] [portable call to time. Add unpack/head/build/seq test Don Stewart **20060707041419] [Fix bench. make runbench now works again, using the new dirty cache system Don Stewart **20060707054002] [fromIntegral missing in test Don Stewart **20060707070304] [missing import in test Don Stewart **20060707083708] [some things done on the todo list Don Stewart **20060708042037] [doc Don Stewart **20060714040333] [better dirtyCache, idea from roman Don Stewart **20060714040553] [O(1) useAsCStringLen, idea from BulatZ Don Stewart **20060714040816] [Clarify comments regarding *AsCString functions Don Stewart **20060714072025] [Handle TextMode properly on windows. Thanks to BulatZ Don Stewart **20060714072351] [tighten import list Don Stewart **20060714072819] [openBinaryFile/openFile distinction Don Stewart **20060716023918 On windows we need to think carefully about \r\n handling. So, the solution: Data.ByteString -- all Word8 operations, has to provide unfiltered access to bytes (in case we're doing some binary hacking). So it uses openBinaryFile Data.ByteString.Char8 -- implies some encoding layer. it uses openFile ] [And use openFile/openBinaryFile in .Lazy stuff too Don Stewart **20060716025041] [-optc-O1 is no more required (because we don't use -optc-O2 by default) Bulat Ziganshin **20060715100841] [Simple implementation of getArgs for non-GHC platforms Bulat Ziganshin **20060715110850] [hGetLines now use hIsEOF instead of catching all exceptions Bulat Ziganshin **20060715123823] [ByteString.Char8 - removed now unnecessary #if_GHCs Bulat Ziganshin **20060715125506] ['interact' for strict ByteString Bulat Ziganshin **20060715131354] [mallocByteString = mallocForeignPtrBytes | mallocPlainForeignPtrBytes Bulat Ziganshin **20060715150109] [Merge changes Don Stewart **20060716031315] [Add hugs versions of some IO functions, merged from BulatZ's patches Don Stewart **20060716035424] [Simplified and unified lazy hGetContentsN/hGetN/hGetNonBlockingN Bulat Ziganshin **20060715100013] [wibble in Makefile for clean Don Stewart **20060716040359] [Faster record test Don Stewart **20060716040423] [Merge-o Don Stewart **20060716040900] [hugs wibble Don Stewart **20060716041730] [missing import for hugs Don Stewart **20060716041935] [todo, fix groupBy (in ghci only?) Don Stewart **20060716052031] [Add test for groupby issue Don Stewart **20060717025941] [testify Don Stewart **20060717033704] [run unpackand groupby tests each 'make everything' run dons@cse.unsw.edu.au**20060717044934] [Lazy.Char8 was exporting interact from Prelude rather than Lazy Spencer Janssen **20060724200507] [TAG 0.7 Don Stewart **20060804090650] [notes Don Stewart **20060804093009] [add some missing functions to Char8 Don Stewart **20060804093043] [this is now the 0.8 branch Don Stewart **20060804093255] [remember that we still have to fix groupBy Don Stewart **20060804095000] [Encourage users to use -funbox-strict-fields Don Stewart **20060807042123] [comment out some tests we don't use now Don Stewart **20060812042504] [Sync 0.8 and streams api Don Stewart **20060812042515] [wibble Don Stewart **20060812042544] [wibbles Don Stewart **20060812042644] [Add lazy-as-available byte strings agl@imperialviolet.org**20060813194125 Lazy reading of network sockets is fantastic, but often you need to process data as it's sent. Currently you can either set the block size to 1 byte or you have to wait for an entire block to be read. This adds the (terribly named) hGetSomeContentsN, which returns the contents of a handle, lazily, but in a timely fashion. ] [comment out lengthU rules, now there's no lengthU dons@cse.unsw.edu.au**20060814002542] [Merge dons@cse.unsw.edu.au**20060814003049] [lazy init isn't O(1), I think it should be O(n/c) Don Stewart **20060819042149] [Rewrite some lazy consumers so they don't cause space leaks Duncan Coutts **20060814020004] [Merge from the unstable branch. API mostly, and hGetSome story. Add fast 'empty()' while I'm here Don Stewart **20060822133026] [no more findIndexOrEnd exported Don Stewart **20060822135201] [no more joinWithByte Don Stewart **20060822135841] [remove some bogus things done better with rules Don Stewart **20060822142305] [fix Properties.hs Don Stewart **20060822143005] [hide hGet*N functions in .Lazy Don Stewart **20060822144158] [no more tokens Don Stewart **20060822144710] [inline words, don't export hGet*N from .Lazy.Char8 Don Stewart **20060822145337] [no more tokens QC Don Stewart **20060822145445] [No, I can't just merge createAndTrim straight from unstable Don Stewart **20060822151838] [good bye packWith/unpackWith Don Stewart **20060822153959] [rule for join -> joinByte Don Stewart **20060823071407] [rules for break/breakByte and span/spanByte Don Stewart **20060823073808] [propeties for the new rules Don Stewart **20060823075135] [add rules for break isSpace -> breakSpace Don Stewart **20060823075814] [Add rule for specialise dropWhile isSpace -> dropSpace Don Stewart **20060823080532] [fix build for hugs Don Stewart **20060823090227] [more hugs fixes Don Stewart **20060823090814] [fusion wibbles Don Stewart **20060824040553] [update Bench.hs to new api Don Stewart **20060824042140] [hide internal module from haddock in Data.ByteString Don Stewart **20060828010842] [wibbe Don Stewart **20060905020458] [hide LPS constructor. move lazy bytestring defn in to .Base Don Stewart **20060907082102] [Fix testsuite Don Stewart **20060907082136] [add fromChunks/toChunks Don Stewart **20060907083254] [Fix type an implementation of toChunks. Duncan Coutts **20060907182429 It should be returing a list of chunks! :-) Spotted by int-e. ] [add readInteger for *.Char8, and add appropriate quickcheck properties Bertram Felgenhauer **20060907210243] [remove leftover strictness annotations. they seemed to hurt more than help. Bertram Felgenhauer **20060907212129] [remove tokens and mapAccumR from .Lazy Don Stewart **20060909044423] [and another tokens occurence Don Stewart **20060909044658] [fix hugs Don Stewart **20060909045538] [fix hugs nice and good Don Stewart **20060909050020] [Fix lazyness of take, drop & splitAt. Duncan Coutts **20061004111822 Spotted by Einar Karttunen. Thanks to him and to Bertram Felgenhauer for explaining the problem and the fix to me. ] [make cons create 16 byte chunks instead of 17 byte chunks for lazy butestrings Bertram Felgenhauer **20061007141231] [export the right 'cycle' Don Stewart **20061110014621] [import portability fixes from Ross's base commits Don Stewart **20061110021347] [remove a slighly suspicious use of unsafeCoerce# Don Stewart **20061120115933] [double check blank lines (since QC isn't generating them for us) Don Stewart **20061125064030] [portability: give alternate import modules for nhc98 Malcolm.Wallace@cs.york.ac.uk**20061116112452] [Workaround for import resolution bug in nhc98. Malcolm.Wallace@cs.york.ac.uk**20061116112651 Where there are multiple renamed imports: import X as P import Y as P import Z as P and they all export or re-export the same entity e, nhc98 does not seem to be able to recognise that P.e is a unique entity, despite X.e, Y.e, and Z.e all referring to the same thing. This patch just introduces an extra module name import X as S so that S.e is resolvable. ] [workaround nhc98 import resolution bug for another module Malcolm.Wallace@cs.york.ac.uk**20061116113410] [wibbles to fix ghc build Don Stewart **20061203033218] [Setup.lhs Don Stewart **20061203034405] [Fixups for building with nhc98 - inadvertently missed this file earlier. Malcolm.Wallace@cs.york.ac.uk**20061204120912] [In teh testsuite, always pass the same GHC options, via ${GHCFLAGS} Ian Lynagh **20070124150146] [Test the library in-place rather than whatever is installed Ian Lynagh **20070124151123] [Add a unit tests file, a test that append is lazy in the tail, and make it so. Ian Lynagh **20070124160043 Append was looking at the tails to see whether or not it was [], which forced evaluation of the tail in situations where that is undesirable. ] [Add lazybuild test Ian Lynagh **20070124161853] [Hide build noice when running tests Ian Lynagh **20070124162017] [HEADS UP: Change CString api Don Stewart **20070130024408 Formerly, the CString api was very unsafe. The old api consisted of: Zero-copying, efficient. Unsafe if you mutate the C string, in the case of packMallocCString, if you run the finalisers twice: packCString :: CString -> ByteString packCStringLen :: CStringLen -> ByteString packMallocCString :: CString -> ByteString useAsCStringLen :: ByteString -> (CStringLen -> IO a) -> IO a Safe. Copies the ByteString: copyCString :: CString -> IO ByteString copyCStringLen :: CStringLen -> IO ByteString useAsCString :: ByteString -> (CString -> IO a) -> IO a Note that the default functions, packCString et al are fundamentally unsafe. By modifying the CString (or Ptr) passed to them, it is possible to mutate the ByteString, breaking referential transparency for pure Haskell code. First step, move these functions into IO, and tag those that are unsafe with the name 'unsafe'. Secondly, all unsafe* functions are moved into Data.ByteString.Base. The API is also sanitised a bit, becoming: Data.ByteString: packCString :: CString -> IO ByteString packCStringLen :: CStringLen -> IO ByteString useAsCString :: ByteString -> (CString -> IO a) -> IO a useAsCStringLen :: ByteString -> (CStringLen -> IO a) -> IO a Safe, always copies to and from the CString*. Dangerous, efficient api, suitable for constant CStrings only (and may have other guarantees, such as null termination): unsafeUseAsCString :: ByteString -> (CString -> IO a) -> IO a unsafeUseAsCStringLen :: ByteString -> (CStringLen -> IO a) -> IO a unsafePackCString :: CString -> IO ByteString unsafePackCStringLen :: CStringLen -> IO ByteString unsafePackMallocCString :: CString -> IO ByteString Currently untested, though the code was only suffled around. All QC tests pass, but they don't exercise the CString api. ] [add iavor's CString test Don Stewart **20070130025217] [remove filterNotByte, and its rules. Its a rather pointless rule (filter -> filterNotByte -> filter) Don Stewart **20070205020018] [Make fromForeignPtr take the start so it is truly the inverse of toForeignPtr Ian Lynagh **20070202154627] [transforming Don Stewart **20070212004317] [adhering to agile principles, requiring a boot-in-place to run the tests on each commit is removed.. Don Stewart **20070212004501] [no boot-in-place Don Stewart **20070213065256] [comment only Don Stewart **20070213065321] [Make the lazy cons lazy, and add a cons' for when you want it to be strict Ian Lynagh **20070201140835] [Correct docs for Data.ByteString.Lazy.Char8.cons Ian Lynagh **20070201155054] [Add cons' to Lazy.Char8 too Ian Lynagh **20070201155643] [Add complexity to Lazy.cons' docs Ian Lynagh **20070201155906] [Define headTail :: ByteString -> Maybe (Word8, ByteString) Ian Lynagh **20070201161048] [headTail -> uncons dons@cse.unsw.edu.au**20070215034944] [Makefile build system for nhc98 Malcolm.Wallace@cs.york.ac.uk**20061124142035] [need things from cbits directory for nhc98 build Malcolm.Wallace@cs.york.ac.uk**20061204120154] [fps package needs extra stack to build (+ add D.BS.Char8) Malcolm.Wallace@cs.york.ac.uk**20061208103626] [Implement byteswapping, used for endian fiddling. Malcolm.Wallace@cs.york.ac.uk**20070227163230 If the ByteString is going to end up being interpreted as a binary representation of a sequence of 4-byte quantities (e.g. Float), then one might need to change the endianness (byte ordering) of the representation. This patch adds the API call byteswap :: ByteString -> ByteString for such applications. It assumes the bytestring will be a multiple of 4 in size. (If not, the last n<4 bytes will be untouched.) It also assumes (without checking) that the start of the bytestring has the correct alignment. ] [nhc now has hGetBuf/hPutBuf Don Stewart **20070301094400] [append on lazy bytestrings is O(n/c) (just the inital spine is copied0 Don Stewart **20070312122615] [Fix types of foreign imports Ian Lynagh **20070403194713] [Remove incorrectly typed commented out foreign imports Ian Lynagh **20070403194812] [old nhc98 Makefiles now obsolete Malcolm.Wallace@cs.york.ac.uk**20070525134324] [nhc98 needs extra stack to build for profiling Malcolm.Wallace@cs.york.ac.uk**20070531141202] [Change package name to bytestring Duncan Coutts **20070620163522 and bump version number, and add myself as another author and maintainer ] [Expect to use ghc-6.7 so no need for -DSLOW_FOREIGN_PTR Duncan Coutts **20070620163724 and -funbox-strict-fields was never needed in the first place. ] [also rename fps.cabal -> bytestring.cabal Duncan Coutts **20070620171054] [Rename .Base module to .Internal Duncan Coutts **20070620172814] [Move definition of empty from Internal to main ByteString module Duncan Coutts **20070620173526 I can't recall why we moved it there in the first place, but it's not necessary any more. ] [Split .Internal module into .Unsafe and .Lazy.Internal Duncan Coutts **20070621172546 The Unsafe module exports parts o the public API but that do unchecked things like unsafe indexing, or things with CStrings that have side conditions. But it doesn't really expose the internal representation, for example it does not give away the fact that internally it uses ForeignPtrs. The Internal module now just exports really internal things including the representation, low level construction functions and utilities. The representation of lazy ByteStrings is moved into it's own Internal module which means we can call it ByteString without having to use a type alias. That should make hugs and/or nhc a bit happier. ] [We no longer use instance of type synonym extension Duncan Coutts **20070622050544] [Change useAsCString so it does not use C malloc and free Duncan Coutts **20070622062652 it was unnecessary ] [Fix use of free finaliser in hGetContents Duncan Coutts **20070622062918 The memory was allocated with mallocBytes so it must be freed with free or finalizerFree from the Foreign.Marshal.Alloc module. There is no guarantee that this is the same as the C free function. ] [Eliminate newForeignFreePtr Duncan Coutts **20070622063301 just inline its only use. ] [No longer need to ffi-import C's malloc and free Duncan Coutts **20070622063504] [Use the (re|m)allocBytes rather than (re|m)allocArray functions Duncan Coutts **20070622063810 It's simpler and more direct, we are working with bytes here, not arrays. And remove unused imports. ] [Remove byteswap function Duncan Coutts **20070623025303 It should not have been added in the first place. It's a fine function to have but not in the basic ByteString module which we should try and limit to the list api. ] [Implement intersperse for lazy bytestring Duncan Coutts **20070623025446] [Remove unnecessary import Duncan Coutts **20070623025517] [update some todos Don Stewart **20070623033120] [Rename join to intercalate to match the standard Data.List Duncan Coutts **20070623043717 And implement Lazy.Char8 version of intersperse ] [Don't include undocumented functions into haddoc docs Duncan Coutts **20070623043955 Use the #prune haddock module annotation. Move 'join' in export list to a section with no header so we don't get an empty section where join used to be. ] [Move some lazy ByteString internals into the .Lazy.Internal module Duncan Coutts **20070623050245 This therefore exposes them in case people need them (eg binary, zlib). It now exports the chunk size constants and the LPS data type invariant and data abstraction functions. ] [Add Data.ByteString.Fusion to the cabal file Ian Lynagh **20070728214511] [Update copyright holders and add more metadata to .cabal file Duncan Coutts **20070724173946] [Sort really does need allocaArray not allocaBytes Duncan Coutts **20070827001615 Since it's an array of CSize not Word Since it's an array of CSize not Word8. Fixes segfault. ] [Fix Lazy.lines "foo\n\nbar" Duncan Coutts **20070827152345] [Add a dep on array Ian Lynagh **20070828225749] [Add a prologue.txt Ian Lynagh **20070829171829] [Fix inlining of intercalate Duncan Coutts **20070829132247 join was just the deprecated alias for the new intercalate ] [Add isInfixOf as an alias of isSubstringOf Duncan Coutts **20070829132329 For api compatability with Data.List ] [Update the tests to work with module reorganisation Duncan Coutts **20070829133401] [Add Lazy.mapAccumR and reimplement mapAccumL Duncan Coutts **20070829133508 including tests ] [Implement Lazy.groupBy Duncan Coutts **20070829135113 Seems to pass the QC test. ] [Rarange modules in .cabal file Duncan Coutts **20070829152421 Hide the .Fusion module so we can change it later without changing the public api. Also, install the .h file. ] [Few updates to the README Duncan Coutts **20070829152547] [Remove hacks that are no longer necessary Duncan Coutts **20070829152603 The simplifier no longer discards these I think. ] [Add commented out exports for functions we're missing Duncan Coutts **20070829152825 And change the formatting of some exports and docs of exported bits. ] [Add partition for strict and lazy Duncan Coutts **20070829153129 Uses a simple implementation, we can optimise later. Getting the api addition in now is the priority. ] [-Wall fix: don't import unsed thing Duncan Coutts **20070829153233] [Export isInfixOf, alias for isSubstringOf Duncan Coutts **20070829153303] [Add Lazy.hPutStr as compatibility alias for hPut Duncan Coutts **20070829154934 To match api of strict module. ] [Add naive implementations of Lazy.unzip and isSuffixOf Duncan Coutts **20070829155615] [Don't run the testsuite on commit, as it doesn't work Ian Lynagh **20070905160854] [Move the fpstring.h to an include directory. Ian Lynagh **20070905160916] [TAG 2007-09-05 Ian Lynagh **20070905212625] [spell nhc98-options correctly Malcolm.Wallace@cs.york.ac.uk**20070906151641] [Remove reference in docs to non-existant function tokens Duncan Coutts **20070906180647] [Remove one indirection in the representation of lazy ByteString Duncan Coutts **20070906181142 Instead of: newtype ByteString = LPS [S.ByteString] we have: data ByteString = Empty | Chunk {-# UNPACK #-} !S.ByteString ByteString That is, the strict ByteString element is unpacked into the Chunk constructor so it's now just one pointer to get to the data rather than having to look up the S.ByteString cons element and then follow the pointer to the data. Combined with SpecConstr we should be able to get some good speed improvements. ] [fix for Hugs: import internal newForeignPtr_ Ross Paterson **20070906221622 Don't know how nhc98 will handle this. ] [Export Fusion module, just for the sake of the testsuite Duncan Coutts **20070906224412] [TAG ghc-6.8 branched 2007-09-03 Ian Lynagh **20070903155552] [Add a boring file Ian Lynagh **20070913204646] [Use new style syntax in .cabal file and use configurations for ghc-6.4.2 Duncan Coutts **20070928223356 We previously had a comment saying what to change to get it to build with older ghc versions. Now we can do it automatically with configurations. ] [Haddock section header fixes, spelling, and trivial whitespace bits Duncan Coutts **20070928231755 Section names and content should be more consistent between the modules now. ] [Export isInfixOf from Data.ByteString.Char8 Duncan Coutts **20070928232026 It got added to Data.ByteString as an alias for isSubstringOf because isInfixOf got added to Data.List ] [Note that the ByteString searching api is about to be replaced Duncan Coutts **20070928232147 So add a deprecated note to that effect to encourage people not to use them. Besides, the current implementation is slower than a naive search. ] [Make it build with ghc-6.4.2 Duncan Coutts **20070928223330] [extra , at end of line not accepted by the head Don Stewart **20070929200952] [Comment out Cabal version check Ian Lynagh **20071005150733 This was causing the build to fail as Cabal didn't know its own version number. ] [bytestring head depends on array Don Stewart **20071006132955]