Written on
November 7, 2008
by
Mike Gerwitz
WebKernel has been modified to support namespaces. The two namespaces are currently webkernel, which will house all main functions and classes, and webkernel::cli, which contains the CLI functions. I had originally included a webkernel::daemon namespace, but the daemon uses too many functions and classes within the webkernel namespace, so it was dropped for ease of coding.
I’ve introduced the System classes, combining the logger and other core systems for the future into a single variable: $_webkernel. All functions have also lost their wk_ or wkd_ prefixes, constants as well, and global constants are prefixed with “WEBKERNEL_” to ensure there are never any name conflicts.
Finally (I’m sure there are other changes, but I didn’t keep track of the smaller ones - you can check out SVN), I’ve added a configuration constant to control the datapack stream wrapper protocol. By default, it is “wkdp”. This is done to ensure that, should there ever be a conflict, it may be easily modified without consequence.
I will now continue work on localization and the datapack and language command-line tools. I was busy so the process of making the necessary changes took longer than I wanted.
Posted in Development 
No Comments »
Written on
October 27, 2008
by
Mike Gerwitz
Well, despite the lack of updates and visible code modifications in SVN (aside from the last commit which included a great deal of late changes), a lot as been going on. One of these days I’ll get to uploading my notes as I write them so you guys can see what’s going on. I now have over 70 pages of them.
Because of my decision to move to PHP 5.3, namespaces are an obvious thing to implement. However, a day after coming up with the system, it was announced that the PHP namespace separator would be changed from the static operator “::” to a backslash “\”! I’m not too happy with the change (dmitry was pretty much the only opposer - I’m with him), but I do understand the need for it. And it was better than most of the alternatives. But it will take a lot of getting used to and I’m going to hate using it. You can read up on this via the following links:
http://developers.slashdot.org/article.pl?sid=08/10/26/1610259
http://news.php.net/php.internals/41374
A patch is available for PHP 5.3, but not PHP 6. As I am testing the code in both at the moment, I will wait for the patch to become live in the snapshots before converting to the new operator - which sucks, as I’ve just begun implementing namespaces and am nowhere near complete. I’ve changed function names and such so many times…this is getting tedious. But it’s necessary.
So the idea is to drop all of the “wk” prefixes. Chances are, in the future, a project may come up with the same prefix. So instead, a “webkernel” namespace will be used (if you are so inclined, in your script, you may “use” the namespace as “wk”). Where the prefix is required (very rarely, in certain configuration constants that require expressions, like paths), the full project name “WEBKERNEL” is used, to prevent conflicts with other projects in the future.
To solve the variable problem, there now exists one single variable rather than many for the various objects - $webkernel. This variable will contain the other objects (while this design is still being given some though, it will be something similar to $webkernel->user->lang(). Methodss will provide shortcuts to commonly used functions in other objects.
The system will also be ensured to work and integrate properly with any scripts - existing or new. Thus so many changes to prevent conflicts. For example, say you wanted to include WebKernel functionality in WordPress to help integrate it. You would include one file to initialize WebKernel and all its functions. The method would be the same for new scripts. You may then use the namespaces as you please.
Finally, as I use VIM as my editing tool, I am creating a syntax file (php-wk) as I go (an expansion of the default php file) to properly highlight WebKernel’s functions, classes, constants, etc. I’ll upload it when it is more useful. It also includes some missing PHP core functions that have been left out.
That’s enough information for now. This blog primarily exists as a record and for Google to index, as I haven’t really advertised the project yet and I assume I have few to no readers.
Posted in Development, Planning No Comments »
Written on
October 23, 2008
by
Mike Gerwitz
A lot’s been going on since the last entry (after a short break). The first I’d like to mention is the progress regarding localization.
I’ve begun work creating language packs and integrating them. However, while doing so, I had to implement unlink() functionality within the datapack (language packs use the datapack wrapper). This broke some code. Therefore, I’m working on fixing that up (and working on some minor optimizations), and will continue with localization after.
The process has spawned two new tools - the “lang” and “dped” tools. The lang tool is a command-line administration tool (which may be used in place of the web interface even when it is created) to create and modify language packs. It will also be used to aid the translation process.
The dped tool I created because debugging datapacks is the biggest pain in the ass I’ve ever seen. Working with easily corrupted binary data (as in, my code has to be perfect, because if it’s one byte off in writing, it may corrupt a lot of data) is not fun. The tool will allow the creation and manipulation of the datapacks. It will also display detailed information, such as header information, descriptors, and even allow the direct modification of descriptors. The tool may also be used to read/write to the datapack.
Perhaps the biggest change is the decision to go with PHP 5.3. It took many months of thought, on and off, and I finally decided within the past couple hours to go with it. Some may think this is a bad decision. I’ll try to outline some of the reasons why I’m doing this:
- PHP 6 is on the way. It will contain many new features that would be unwise to continue without - it’s the future of PHP. PHP 5.3 incorporates many of these new features, such as namespaces. Web hosts will not have much of a problem upgrading from 5.2 to 5.3 - its’ a minor upgrade (though a major leap in PHP). However, it will take some time to move to PHP 6. Therefore, we’ll be able to use many of the features, instead of sacrificing so many because of web hosts.
- Namespaces are a major selling point. Currently, I’m prefixing everything with “wk_”. It works, but what if something comes along with the same prefix (or what if something already exists)? It’s also a tedious method. Namespaces will solve this problem. Not only that, but it will allow the overloading of core functions. This makes the system even more customizable. This also means unicode support may be feasable, should I have the time (unless someone else takes that up) in PHP 5.3. There would then be very little difference between WebKernel in PHP 5.3 and 6. In the long run, this would be an excellent decision.
- Most major web hosts support PHP 5 and keep it up to date. Upgrading to PHP 5.3 shouldn’t take too long. If it does - so what? WebKernel is still far off. PHP 6 may be released by the time it’s fully complete (I do not know its release date - I’m just considering it). So that gives web hosts plenty of time to upgrade.
- No dealing with revision numbers (5.2.X) for PHP 5.2 to determine compatibility. 5.3 will work - period.
- Late static binding is a necessity to ensure classes can be overloaded properly.
- OpenSSL support for OpenID, which I plan to encorporate as one (of many) authentication options, sometime in the future.
- getopt() will work on all OS’s now. While I do not endorse nor will I test or maintain WebKernel on Windows systems, this means I can get rid of my command-line option parser and use PHP’s core function.
I am still revewing changes in PHP 5.3, but they just keep getting better. I have looked over the majority of the changes and must say I am very impressed - excited, even - about the direction it is going in. I hope that some day PHP will evolve to be a C-like langauge with more advanced features such as pointers, compiler directives, varibable type declarations, etc. I understand PHP wants to keep it simple, but they should provide the option for the more skilled developers. But PHP is evolving and I’m very happy to be sticking with it. I’ll cross my fingers for PHP 7.
Posted in Development, Planning No Comments »
Written on
October 12, 2008
by
Mike Gerwitz
Well, it seems that PHP 6’s unicode support breaks the datapack. So I’ll have to spend some time getting it to work properly (mostly a tedious task of using binary strings rather than the default unicode). This does mean, however, that in order for the project to be backwards-compatible, PHP 5.2.1 is required (that is when the binary cast was added). However, I will provide a tool to remove all the casts from all files to support 5.1 as before. Eventually.
Unicode support definatly complicates the project a great deal. Let’s just hope PHP 6 doesn’t change too much before it’s release. I’d hate to do all this work for nothing. But it’s a chance I’ll have to take - I can’t put off PHP 6 support until the last minute. That’d be hell to fix.
Edit: Well, I had almost given up due to a bug in fread() in PHP 6. When reading past the EOF, it returns FALSE, rather than an empty string, as it does in PHP 5. After a fair deal of research and experimenting (when I upload my notes you can see my shameful thought process) I managed to identify it and provide an alternative (and perhaps speedier) solution.
The datapack wrapper didn’t take much work at all - just a few binary casts here and there. Not the bunch I expected. The reason for it is simply because - it’s a wrapper. Users calling fwrite() and such can deal with the casts - they have to, or PHP bitches at them. I’m not saying the datapack is entirely ready and prepared - it’s not. I haven’t even tested it yet with unicode strings. But it does work as originally intended. I’ll worry about the details later.
Anyway, I’ll commit the changes after I fix up a few things - primarily add the new exceptions I’ve been too lazy to add. I also changed the convert function around a little - instead of accepting a $new_file argument, it simple generates it own, deletes the old file for you, and renames the new, converted pack, to the old filename.
Posted in Development No Comments »
Written on
October 11, 2008
by
Mike Gerwitz
To save myself the trouble later, I’ve decided to begin researching more into unicode in PHP, ensuring it is fully PHP 5 and PHP 6 compatible (at this point, unicode will still be limited to PHP 6, unless I figure out a speedy alternative).
The problem with unicode in PHP 5 is that…it’s a bit of work. I’d have to create a number of functions to handle unicode strings. But that would make the PHP 6 code messy and less efficient. PHP 6 converts the normal, core functions to handle unicode. So substr(), for example, will properly, handling unicode strings in PHP 6. So creating a separate function to use in place of the core functions so PHP 5 can have unicode support would be a poor decision. That function, in PHP 6, would have to forward straight back to the core function. Unnecessary overhead.
So, the language system will take a bit longer. I want to get this right the first time.
I’ll also be taking time to figure out the best way to design the project for PHP 6, with backwards-compatibility for PHP 5. I’m not talking “eh, it works” - I mean, work completely, with a few extra features for PHP 6 (one being unicode support). PHP 6 won’t be on web hosts for a while, so I need to ensure WebKernel runs flawlessly in both PHP 5.1 and PHP 6.
Posted in Development, Planning No Comments »
Written on
October 10, 2008
by
Mike Gerwitz
After trying to think of a more creative name, I decided that it’s really not necessary. “WebKernel” sounds professional enough, is to the point, memorable (at least for developers), and still has the “open source” feel many applications have - simple, not necessarily creative. Though I do think the name is creative. Thanks, Landon 
So, the code has been converted (yet again, with the help of a VIM script I wrote and some grep commands), to contain the new name and variable/constant/function/class/etc prefixes (WK). The next commit, which will include the beginnings of the language system, will reflect those changes.
I will put up the documentation (generated with phpDocumentor) after the next commit.
Posted in Development No Comments »
Written on
October 9, 2008
by
Mike Gerwitz
FINALLY! It feels so good to finally commit the wrapper to SVN. It was pretty annoying to get working - it is perhaps the most complicated script I have had to work on. It took a lot of planning and rewrites - it was a learn-as-you-go thing for me. Thankfully, this rewrite is fairly good and fast.
Included with the wrapper is a test script: tools/tests/datapack.php. This script will test most aspects of the system and help demonstrate the speeds. Run the script a few times to get a good idea of the average speeds. The first run may take a bit to complete, depending on the speed of your system and your server load - 10-30 seconds, probably. After that, the random date generated is cached and the script should run, hopefully, in under a few seconds.
Unfortunatly,the wrapper is not yet complete - but it is operational and usable. The todo list is included in the file header and is as follows:
- Verify written data by checking bytes written rather than FALSE
- Support multiple keypack file operations (multiple opens)
- Cache next fragment position within previous descriptor (maybe…it’d speed up read times but slow writes, so perhaps we could have the OPTION to do this…)
- Wait for lock rather than return error (unless timeout)
- 0×03 mark fragments as empty for other blocks to use?
- Use constants for $block_frags indexes…less confusing and less trouble should we decide to modify it later
- After working on this for so long, I want to get to work on other things; so more performance optimization can wait. But, optimize.
- Negative seeks are not yet supported
- “r+” mode is not yet properly implemented
- When appending, system seeks to last fragment by default; it does not check for available space in previous fragments
- get_frag_bytes may report more bytes than the file contains - why?
- convert() is a little messy; maybe add a function to switch between blocks?
- Check for free space before converting/defragmenting
- You receive no warning if you’re writing to an already opened keypack
If you have any free time on your hands and feel like trying to fix any of those, go right ahead
So..there you have it. I’ll be posting a link to the generated documentation and hopefully begin working on the language system tonight.
Posted in Development No Comments »
Written on
October 8, 2008
by
Mike Gerwitz
Well, a few days after coming up with the name and modifying the code to reflect it (WebCORE), I find that Apple has already taken it - a component of WebKit. To avoid future conflicts, I’ve decided to change it.
I’m not going to worry about it until after the Datapack’s completion - I’m still coming up with ideas. Many of them were already taken. A friend of mine had suggested “WebKernel” as an alternate name, and I like it. So, if I can’t think of anything else, I may go with that.
Posted in Planning No Comments »
Written on
October 8, 2008
by
Mike Gerwitz
The datapack stream wrapper is moving along much more quickly than I had thought. It’s an incredible relief to see this finally pulling together. I have the majority of the bugs sorted out and it is currently in working order. I’m making some final changes before I commit it to the repository and move on. I’m leaving myself a todo list, which I will post here (it is also at the head of the file and will be in the documentation). I am not worried about finishing the wrapper 100% for now. I just want to get it working.
So, in future months, I will finish it up and make the final optimizations, hopefully with the help of an expert in the field of filesystems or flat-file storage to help make it run more quickly. Despite how many times I’ve essentially re-written it from scratch in the past, the code could use a bit of a clean up as well. But hey, that’s why it’s ALPHA, right?
So, I hope to finish it up within the next 48 hours. I want to get WebCORE rolling and in its ALPHA stage soon. MyCustomBB has been waiting a long time and I want something to show for these years of planning and on-and-off work.
Posted in Development No Comments »
Written on
October 4, 2008
by
Mike Gerwitz
Alright, now for a status update on WebCORE. All WebCORE posts will remain in the MyCustomBB blog for convenience until the project gets further on and is separated into its own site.
With the error systems complete and logging ready, I want to get into localization. Before I do that, I want to take the time to finish up the datapack wrapper I stopped working on a number of months ago. It’s not far from finished - it’s just a pain in the ass (especially converting all the exceptions to the new error system). Then the language packs can use the wrapper for storage, so I can avoid rewriting the system later.
So, the near-future goals:
- Finish up datapack stream wrapper
- Add localization (language pack) support and move all strings to langauge packs
- Create a script to add and modify language packs so we can avoid the admin panel scene for the time being. The functions will be created in such a way that they may be used both by the command-line script and the admin panel, saving time later on (and providing administrators with multiple options).
The language system will be a piece of cake compared to the datapack. I’m hoping to finish these up sometime soon. I can’t say for sure how long the datapack will take - if I work at it, hopefully no more than a week. The language system shouldn’t take more than a few days to get the basic structure down. Script may add to the time. I’ll try my best to stick to it, but again - this is done in my spare time.
After that, I may begin reconstruction on the template system. Much of the code may be re-used, but I’ll still be re-writing it from scratch to comply with the new project standards and to optimize it as much as I can for the time being.
Posted in Development, Planning No Comments »
