This release refactors and rearchitects the core objects of the system using diamond inheritance of the interfaces so that you can cast between the defining schema names for the objects and the current project's schema names. All of the methods of the new interfaces rely on the defining schema names, but you can safely cast the returned objects to your current schema names.
The goal of this effort was to make it easier to write custom GUI components that rely on the defining schema names for object interfaces, so that you should be able to just reference the jars that provide those custom GUI components, wire them to your projects GUI factories, and automatically use the customized screen components.
It isn't perfect, but it's the best I could do to address what I saw as a glaring flaw with the code base. Of course, the downside is that you may have to do more typecasting in some of your code. There are always trade-offs when designing an architecture.
Another benefit of the new approach is that it is made very clear that you are using the objects and interfaces from a referenced schema/project, an issue which was concerning me as some people might have taken the approach of ignoring the license requirements of the referenced schemas because the code gets re-manufactured into their project. Now there is a clear delineation between the referenced objects and the current project, preventing anyone from pretending they "didn't know" they had to abide by the licenses of referenced projects.
MSS Code Factory 2.3.12992 is the preview of 2.3 Service Pack 1. I'm going to refresh and build all of the 2.4 sub-projects before I release it as SP1, just in case there are any issues with the new rules that I've missed. I don't expect that to be the case, though.
I was going to skip releasing this batch of code as a service pack due to the drastic refactoring and re-engineering of the core objects, but then I realized that the inputs to the system have not changed, so it really is still a 2.3 code base.
After SP1 is released, I'll work on migrating the engine code itself to 2.4 and use it to manufacture 2.5. Once 2.5 is up and running, it will be used to manufacture the 2.6 projects, and then I'll have completed the "loop" of running MSS Code Factory on the code that the previous release produces. I "eat my own dog food" with this project.
There was more debugging required than I expected, but CFDbTest 2.4 now passes it's regression tests for PostgreSQL. I won't be doing full regression tests of the other databases because the only thing that changed in the DbIO layers is the name of the buffer and key objects, not the actual JDBC code.
In order to get CFDbTest 2.4 to build successfully, I had to modify the core objects such that diamond inheritance is used by the interface specifications for the Obj and EditObj, but full re-implementation of the methods is done by the schema's objects rather than relying on inheritance from the DefSchema implementations.
More than ever, you need to write your code to the interfaces rather than relying on the specific object implementations that are providing them.
The build for CFCrm 2.4 now completes successfully.
The Swing prototype GUI now builds for CFCrm 2.4, as well as the main for the HTTP client GUI.
That leaves the two JEE web servers to be updated yet, then I'll be ready to shift over to CFDbTest 2.4, refresh it, and test the code that has been produced.
Note that the EltJTabbedPanes (at a minimum) coerce to the top level schema's objects for accessing the sub-element lists. That means that those panes are not portable to other schema interfaces. My goal all along has been to generalize the interfaces used by the Swing prototype GUI as much as possible so you can do custom windows based on the schema your objects are defined by (the DefSchemaName), and use those windows with any client's GUI with minimal effort or work. Alas, I haven't quite achieved plug and play portability in this arena, and I'm not sure whether I could. I'd need some way of incorporating all the referencing schema interfaces into a schema's object definitions, not just the direct incorporation of DefSchema objects in this schema.
The four XMsg layers (XMsg, XMsgRqst, XMsgRspn, and XMsgClient) now build clean for CFCrm 2.4.
The SAP/Sybase ASE layer now builds clean for CFCrm 2.4, and the SAX Loader mains for each of the databases also build clean. Next up are the XMsg layers.
The PostgreSQL layer now builds clean for CFCrm 2.4.
The Oracle layer now builds clean for CFCrm 2.4.
The MySql layer now builds clean for CFCrm 2.4.
The MSSql layer now builds cleanly for CFCrm 2.4. I am no longer going to do the builds for CFSecurity and CFInternet, as the CFCrm build catches all the mistakes and errors in the rules that the first two, as well as a few others.
When I'm done with this massive restructuring/refactoring effort, then I'll rebuild all of the applications and retest CFDbTest 2.4 prior to issuing SP1.
The DB/2 LUW layer now builds cleanly for CFSecurity, CFInternet, and CFCrm 2.4. Next up is the MSSql layer.
The SAX XML parsers build clean for CFSecurity, CFInternet, and CFCrm 2.4.
Next up is the DB/2 integration layer.
The Buff, HBuff, PKey, HPKey, and various index key buffers are now only created by their defining schema. The bring-forwards from the imported projects have been removed for CFInternet 2.4 and CFCrm 2.4. There were a few additional rule changes to adjust to the loss of the objects, but everything is in the same build state that it was before: failing on the XML SAX parsers.
The base, Obj, Ram, and MssCF layers now build successfully for each of CFSecurity, CFInternet, and CFCrm 2.4. Only CFSecurity 2.4 builds completely, of course -- both CFInternet and CFCrm fail on the next layer: the XML SAX parser.
2015.08.18 MSS Code Factory 2.3.12964 Anything which can be automated, will be automated
"Anything which can be automated, will be automated"
-- Every lazy BOFH out there
As a long time Linux/Unix user, everything gets automated down to little batch scripts or repeated command-line buffers. My day revolves around this little ditty:
ant clean; ManufactureCFSecurity24.bash ; ant
ant clean; ManufactureCFInternet24.bash ; ant
vi rule files and fix another batch of build errors
ant clean; ManufactureCFInternet24.bash ; ant
vi rule files and fix another batch of build errors
lather, rinse, repeat
But it beats the heck out of the old days. A complete cycle takes under 10 minutes with a CFSecurity refresh, and about 3-4 minutes for a CFInternet cycle without doing a CFSecurity refresh.
So things move at a fair clip, fix wise. It doesn't take long to refactor code and work it through to a clean, interface-driven build. I should have been building to the interfaces all along. Consider this much-needed cleanup.
This release refreshes and refactors the base and Obj layers, producing builds of those two jars for CFInternet 2.4 as well as a full build of CFSecurity 2.4. The builds now fail on the MssCF code. I'll tackle that next.
It was a grueling over-24-hours session to get the Obj layer to build for CFInternet without breaking CFSecurity.
Next I'll need to correct the Ram implementations for CFInternet and then do a regression-test run for CFSecurity and make sure I don't break it along the way.
I'm still working on getting the core layers to build properly at least two layers deep. Then there are all the supporting objects which will have to change from SchemaName to DefSchemaName specifications in a number of locations. It's going to be a lot of work getting this to a clean building state for CFInternet. And there will be more work to be done for supporting CFCrm, as that adds another layer of indirection.
This checkpoint release sees CFSecurity 2.4 building cleanly again.
There are still numerous issues with CFInternet 2.4 that need to be addressed before I'm ready to proceed with layering CFCrm 2.4 on top of CFSecurity and CFInternet.
I know the CFCrm third layer will break another batch of rules. But once that third layer clean-builds, the fundamental objects of the system will have been properly propagated amongst the build jars of the various schema projects.
The Scope reference and HasScope binding are used to properly populate the referenced schema names so that source code can be built. Or at least a build attempted for CFInternet 2.4. It still won't clean compile; it is no where near doing so.
This is, therefore, a "work in progress" release for demonstration of the new code, not for use in production applications.
I changed the implemementation of popto to support a direct short-cut implementation of "popto Object", but that doesn't want to work properly. So I'm in the midst of debugging a "Scope" hard-coded reference and a "HasScope" binding.
In order to test my workarounds, I need a fixed build with the corrections to the "popto Object" implementation of special-case "always pop" handling.
If you perform a popto Object, you are now force-popped one level up the stack rather than going into an infinite loop.
CFSecurity 2.4 now builds cleanly with the code manufactured by 2.3.12952.
That isn't to say that it's going to produce valid code for CFInternet 2.4 and the subsequent sub-projects, but it's a start.
The core packages defined by the lowest layer of the class hierarchy are now properly bound and propagated throughout the object model source code produced. The goal of this architectural shift is to make it easier to adapt custom forms to a generic data object interface rather than relying on any particular SchemaDef specification.
For example, the code can use $DefSchemaName$ to obtain the current DefSchemaName for output.
This was a lot of work. But right now it clean compiles so I'm checking it in and calling it a "night" at 19h00 the next day after starting on this refactoring exercise.
After each page/request is processed, the page handler thread invokes minimizeMemory() to reduce the memory footprint of the session. This should load balance quite nicely on multicore machines with a lot of memory to act as web service nodes.
You may have noticed I haven't done any MSS Code Factory updates in a while. That is because I've only got one piece left to the puzzle that I can think of, so I'm taking my time getting around to it.
When I do get around to it, I expect it to require a burst of energy for all of 2-3 days, after which I'll be ready to release 2.3 SP1.
Which will, at last, be the end of the MSS Code Factory work itself save for any bug fixes that are required in the future.
It has taken me many long years from concept to fruition, but it will finally be done this year. The core algorithms were prototyped way back in 1998. I took breaks from it for months at a time. I made many a false start, and often ran into roadblocks that led to the end of the 1.x phases of development.
But persistence paid off. I eventually found my way around all the roadblocks, cobbled, tweaked, and hacked my way to a modular system, and got things to their current state.
What I'm proudest of is the sheer performance of the resulting code base. The factory produces code at a rate of thousands of lines per second, which is far faster than I ever dreamed it would be. When I started, I was hoping for something that I could run in a day, not in under ten minutes!
I'm going to miss this project. It has been a big part of my life for a very long time. But all things do eventually come to an end.
It would seem that some of the behaviour I was seeing a month or two ago was due to the Oracle JDK, which incorrectly optimizes some of the probes performed by the CFCore code to resolve rules. In particular, MSS Code Factory was not correctly resolving rules when expanding references. Instead of searching for the GenDef and ScopeDef of the CFContext, it was using the GenDef where the expansion rule had been found, causing it to be unable to find the subsequent expansions of the remainder of the reference rule.
I've spent the past couple of days combing through the code trying to find the cause of that behaviour, and found nothing. So today I decided to start pruning the "workaround" rules I'd coded for the "any" rule cartridge, and discovered that the problem does not occur with OpenJDK 8. Nor had it been occuring with OpenJDK 7.
I'm not interested in working around Oracle bugs. Let Oracle fix their own crap.
Rebuilt and repackaged with OpenJDK 8 versions of CFLib and CFCore.
The version number has been updated, but no other code has changed in CFCore. It is primarily a rebuild with OpenJDK 8.
The refactored CFTip code is now debugged along with the XMsg server enhancements that were made to support the CFTip changes. You can now either specify all of the server key parameters to use a predefined key for your XMsg server, or define none of them in order to use a dynamic key.
This release has been rebuilt with OpenJDK 8.
If no log has been configured for a parser, exception reporting now defaults to writing to System.err. The debugging of the refactored CFTipServerInfo has been completed as well.
This release has been rebuilt with OpenJDK 8.
The SAX core parser objects had some null pointer exceptions being produced because they weren't making sure that getLog() was returning a non-null log handle.
The CFTipServerInfo object was not properly unwrapping the server public key from a Base64 encoded value before attempting to turn it into a public key instance.
The $SchemaName$XMsgSrvWar initialization has been reworked such that you are no longer required to specify server key information. If all of the server key parameters for the server are present, they are used to retrieve the server keys and they are initialized once and only once during application initialization. If you don't specify any of them, then a dynamically generated key is used instead, but again, it is only initialized once. If you only specify some of the server key configuration parameters, exceptions are thrown during the server initialization process.
While the code clean compiles, it is completely untested at this point. It's 03h30, and it's time for some sleep...
The CFTipServerInfo object now incorporates the public and private key attributes of the server, and the CFTipClientHandler and CFTipEnvelopeHandler now use those accessors and methods. This makes it easier to ensure that the server's private key information is only set once by the initialization of a server, rather than constantly resetting the string representation of the server's public key because the server info associated with a CFTipEnvelopeHandler was schizophrenically keeping part of it's information in the envelope instead of the server info.
There were some defects in the initial production release, which have been corrected. In particular, the HTTP client was throwing an exception due to an array-index out of bounds error while storing the list of server URLs.
I was also uncomfortable with the code used to initialize the server keys for the XMsgSrvWar, so I switched the code over to using a JCEKS key store with server configuration parameters for locating the key store, the key store's password, the name/alias of the server key, and the password for the server key. The server no longer generates a dynamic key as a result, though the CFTip code does still support that mode of operation, should you prefer to use it in custom code.
Apologies for the bad release.
The new setServerKeys() method is required in order to have sessions use a common server key pair configured for the XMsgSrvWar servers.
This re-release of the production 2.3.12930 code reworks the way license headers are produced such that the copyright and license of the originating project are respected. You do not get to take over copyright on a piece of code simply by manufacturing it -- the original author retains copyright and their license applies to that code, not your project's license.
This re-release of the production 2.3.12927 code changes the license of the CFSecurity 2.4, CFInternet 2.4, and CFCrm 2.4 projects to an Apache V2 license from the BSD 3-Clause license, as I've learned that the BSD 3-Clause is not compatible with GPLv3 licensing.
That wasn't an issue for my code because I owned all the projects, but it would have been an issue for anyone else seeking to author GPLv3 projects with MSS Code Factory. Note that Apache V2 is not compatible with Apache V2 licensing, so you may not use MSS Code Factory to produce GPLv2 projects, as all manufactured projects extend at least CFSecurity 2.4.
This is the production release of MSS Code Factory 2.3.
This is an untested build of the 2.3 code. Use at your own risk.
CFBam 2.3 has been manufactured and built for use by MSS Code Factory 2.3.
The testing of the new ServerProc, ServerObjFunc, and ServerListFunc model specification has been completed.
There was a problem with adjusting the enable state of the custom instance menu items in the new GUI code, and there was also an issue with ObjFunc XMsgRqst handlers trying to marshall null buffers.
The custom GUI enhancements just boiled down to some menu items and message dialogs, so it didn't take all that long. It isn't what I'd call robust testing, but it's enough to make sure everything works.
Running the tests comes next. If the tests pass, I'll just update this web page. If it fails, you can expect more updates.
Sample server configuration files for Tomcat 8 have been added.
The framework for the custom GUI enhancements has been added, and the run scripts have been corrected as well. Maybe the custom GUI work won't take as long as I'd been thinking, seeing as all I intend to add are some menu items to the ReqFullRange and Table windows for launching some simple parameter entry forms for the custom server methods, and display the results returned in message dialogs. It's not like my sample code does anything really useful that has to be displayed to the user, so I may as well keep it as simple as possible.
The implementation of the enhancements to support ServerProc, ServerObjFunc, and ServerListFunc specifications is now complete and ready for testing.
I'll need to do some custom GUI coding for CFDbTest 2.3 in order to be able to actually do the testing, but that should only take a few days to code.
The XMsg ServerProc support is now ready for testing. I'll have to code some custom GUI enhancements for CFDbTest 2.3 before I can test it, though.
The response handlers now support [get/set]DataChanged() in order to implement the ServerProc code.
The Obj layer for the ServerProc implementations has been coded and clean-builds.
The initial test code for the lowest layer of the ServerProc implementation has been coded and clean compiles.
This is a baseline build of the CFDbTest 2.3 model before the rules for implementing the ServerProc specifications are added to MSS Code Factory 2.2. When the final release of 2.2 is ready for upversioning to 2.3, you'll be able to compare this tag's source code with the last of the CFDbTest 2.3 enhancements to see what new code gets created to implement ServerProc specifications.
The code clean builds, of course, though I'm not bothering to test it as there is no real difference between 2.2 and 2.3 at this point.
I need to add some manufacture rules for the web.xml files, but other than that the initially manufactured code for CFSecurity 2.3 built just fine once I copied around the libraries and build scripts from 2.2. I now consider 2.2 fully migrated, tested, and ready for enhancement.
There were a couple of changes needed for the CFCore 2.3 model because I use the code embedding features of MSS Code Factory to glue together the custom code with the manufactured code base. I'll check in the model changes shortly. But we're ready for a test build of CFSecurity 2.3 as soon as I'm done with that! Woohoo!
There weren't any significant changes to CFLib during the migration process, just redirection the package names from v2_1 to v2_3.