Datapack Stream Wrapper Commited!
Written on October 9, 2008 by Mike Gerwitz
FINALLY! It feels so good to finally commit the wrapper to SVN. It was pretty annoying to get working - it is perhaps the most complicated script I have had to work on. It took a lot of planning and rewrites - it was a learn-as-you-go thing for me. Thankfully, this rewrite is fairly good and fast.
Included with the wrapper is a test script: tools/tests/datapack.php. This script will test most aspects of the system and help demonstrate the speeds. Run the script a few times to get a good idea of the average speeds. The first run may take a bit to complete, depending on the speed of your system and your server load - 10-30 seconds, probably. After that, the random date generated is cached and the script should run, hopefully, in under a few seconds.
Unfortunatly,the wrapper is not yet complete - but it is operational and usable. The todo list is included in the file header and is as follows:
- Verify written data by checking bytes written rather than FALSE
- Support multiple keypack file operations (multiple opens)
- Cache next fragment position within previous descriptor (maybe…it’d speed up read times but slow writes, so perhaps we could have the OPTION to do this…)
- Wait for lock rather than return error (unless timeout)
- 0×03 mark fragments as empty for other blocks to use?
- Use constants for $block_frags indexes…less confusing and less trouble should we decide to modify it later
- After working on this for so long, I want to get to work on other things; so more performance optimization can wait. But, optimize.
- Negative seeks are not yet supported
- “r+” mode is not yet properly implemented
- When appending, system seeks to last fragment by default; it does not check for available space in previous fragments
- get_frag_bytes may report more bytes than the file contains - why?
- convert() is a little messy; maybe add a function to switch between blocks?
- Check for free space before converting/defragmenting
- You receive no warning if you’re writing to an already opened keypack
If you have any free time on your hands and feel like trying to fix any of those, go right ahead 
So..there you have it. I’ll be posting a link to the generated documentation and hopefully begin working on the language system tonight.
Posted in Development
