MSS Code Factory 2.2 Release History

2015.06.26 Programmer's Notes

I've decided against including any more functionality for the initial 2.3 release. The history based synchronization will not require changes to the core engine, as all the information I need for authoring the rules to support it already exist in the 2.2 release.

I will therefore begin working on shifting the code base forward to 2.3 and issuing an initial production release of 2.3.

The history synchronization will appear in a service pack at some point in the future.

2015.06.26 MSS Code Factory 2.2.12921 XMsg support for ServerProc, ServerObjFunc, and ServerListFunc passed testing

The testing of the new ServerProc, ServerObjFunc, and ServerListFunc model specification has been completed. It is now ready for use.

I am debating whether to implement history based client synchronization for the 2.3 release or not. I didn't spend a lot of time on the enhancements so far, so I think it would be kind of a "thin" release to push the current code as 2.3. I'd like to have more functionality in the next production release than I've added since 2.1.

2015.06.26 MSS Code Factory 2.2.12919 XMsgRqst ObjFunc implementation corrected

The XMsgRqst ObjFunc implementation had been trying to marshall a null object response, which was causing Null reference exceptions.

2015.06.25 MSS Code Factory 2.2.12914 XMsg support for ServerProc, ServerObjFunc, and ServerListFunc ready for testing

The implementation of the enhancements to support ServerProc, ServerObjFunc, and ServerListFunc specifications is now complete and ready for testing.

I'll need to do some custom GUI coding for CFDbTest 2.3 in order to be able to actually do the testing, but that should only take a few days to code. I'm not in any rush to do so, though. One thing I have plenty of is time. Disability retirement may have shitty pay, but the lack of user and customer pressure to get things done on a schedule is glorious!

2015.06.25 MSS Code Factory 2.2.12913 XMsgRspn response processing now supports ServerObjFunc and ServerListFunc

With the completion of the XMsgRspn enhancements, I only have to enhance the XMsgClient layer, then I'll be in a position to write tests for the new code.

2015.06.24 MSS Code Factory 2.2.12912 XMsgRqst request processing now supports ServerObjFunc and ServerListFunc

The $schemaname$xmsgrqst$version$ package now clean compiles for CFDbTest 2.3.

That leaves the enhancements for XMsgRspn and XMsgClient to be done. XMsgRspn is the next up...

2015.06.24 MSS Code Factory 2.2.12911 XMsg message formatting layer enhanced to support ServerObjFunc and ServerListFunc

The ServerObjFunc and ServerListFunc message formatters are now supported by the $schemaname$xmsg$version$ package/jar.

2015.06.24 MSS Code Factory 2.2.12910 XMsg XSDs enhanced to support ServerObjFunc and ServerListFunc

The request and response XSDs for the XMsg layers have been enhanced with message definitions for the ServerObjFunc and ServerListFunc model specifications.

2015.06.24 MSS Code Factory 2.2.12909 ServerObjFunc and ServerListFunc implemented for core and persistance layers

The ServerObjFunc and ServerListFunc specifications are now supported by the core and database persistance layers, and have been clean compiled for CFDbTest 2.3.

The CFDbTest 2.3 model has been updated to correct numerous defects in the sample specifications of the ServerObjFuncs and ServerListFuncs associated with the Table object.

Next I'll need to enhance the XMsg layers to support the new specifications. This build produces invalid code for those layers, because the rules don't recognize the new model elements yet.

2015.06.24 MSS Code Factory 2.2.12908 ServerObjFunc and ServerListFunc test data added to CFDbTest 2.3 model

The CFDbTest 2.3 model has been updated to add class and instance based ServerObjFunc and ServerListFunc specifications to the Table object. As a result, you can't manufacture CFDbTest 2.3 with this build of MSS Code Factory 2.3, so it's only useful to download it if you want to get a preview/example of the use of the new elements.

Note that the intent is to have the methods returning instances and arrays of the Buff specifications for objects, not the actual objects themselves, in order to maintain consistency with the rest of the lower-level database interface specifications. So while you write your code in terms of the Obj layer, you need to extract the buffers from those objects using obj.getBuff() in order to return them from your JavaMethodBody specifications.

It's also worth noting that the order of the elements is going to retained in transmission, so you can do things like give special meanings to element 0..n of your result array.

2015.06.23 MSS Code Factory 2.2.12907 Engine enhanced to support ServerListFunc and ServerObjFunc elements

The SAX parser of MSS Code Factory has been enhanced to support the additional ServerListFunc and ServerObjFunc elements. The XSD specification has been updated accordingly.

Next up is modifying the CFDbTest 2.3 model to incorporate test data for the new elements, and then to update the rule base to manufacture the code for the new objects required by the specifications.

Note that the new code has not been tested yet.

Just to put things in perspective, CFBam 2.1 comprises 13,692,749 lines of code, while CFBam 2.2 weighs in at 15,205,452 lines, a difference of 1,512,703 lines. MSS Code Factory 2.1 itself weighs in at 365,546 lines, and 2.2 is 368,415 lines -- an additional 2,869 lines. The grand total of the new code is 1,515,572 lines. I started working on the enhancements 2015.06.06 or 17 days ago, so that works out to 89,151 lines per day, 756 lines per hour, or 12 lines per minute -- if I was working 24/7 for 17 days straight without so much as a bathroom break.

2015.06.23 MSS Code Factory 2.2.12906 Reworked ServerProc support for new CFBam object hierarchy

The code for MSS Code Factory 2.2 has been reworked to support the new class hierarchy as defined by the latest version of CFBam 2.2.

The new ServerObjFunc and ServerListFunc specifications are not supported yet, but a run for CFDbTest 2.3 produces output identical to the pre-refactoring version of the code.

2015.06.23 MSS Code Factory CFBam 2.2.12905 Refactored ServerProc support

CFBam 2.2 has been refactored to add support for ServerObjFunc and ServerListFunc specifications in addition to the existing ServerProcs.

2015.06.22 MSS Code Factory 2.2.12897 XMsg ServerProc support ready for testing

The XMsg ServerProc support is now ready for testing. I'll have to code some custom GUI enhancements for CFDbTest 2.3 before I can test it, though. I don't expect any problems, because most of the code is just copy-paste-rename from the existing code. Still, there is always the possibility of a bug, so if you download this release and use the ServerProc enhancements, you do so at your own risk.

2015.06.22 MSS Code Factory 2.2.12896 XMsgClient ServerProc layer implemented

The XMsgClient layer now issues the RqstTableServerProc messages and interprets the passes back the RspnServerProcExecuted.DataChanged values from the ServerProc stubs for the tables.

That leaves the XMsgRqst layer to be coded yet, then the framework of the ServerProc implementation will be complete and ready for exercising in code. I'll have to do some GUI customization for CFDbTest 2.3 in order to invoke the trivial ServerProcs I've defined, though, and I'll want to implement some ServerProc bodies that actually do something other than returning true/false blindly.

2015.06.22 MSS Code Factory 2.2.12894 XMsgRspn ServerProcExecuted message handled

The RspnServerProcExecuted XML messages are now handled by the XMsgRspn layer.

Next up is the XMsgClient layer.

2015.06.22 MSS Code Factory 2.2.12893 XMsg formatting layer for ServerProcs coded

The XMsg formatting layers for preparing the request and response messages for ServerProc communications have been coded and clean compiled for CFDbTest 2.3.

I'm going to deal with the XMsgRspn layer next.

2015.06.16 Programmer's Notes

I've realized I can code up another enhancement: ServerFunc, with the same argument signatures as a ServerProc, except that it would return an instance or list of objects, similar to the way the Read invocations do.

That would take care of about 80% of the usual server-side procedures and functions that one would need to code an application. The only remaining feature would be to come up with a way of passing *lists* of parameters or objects. Right now I only let you pass atomic data types arguments based on types defined for the schema.

It's also worth noting that I don't actually pass object *instances* for the Server access, only the primary keys of the objects involved. You can't count on the client to have current data -- you want to use the latest version of the object from the server cluster's database instead. (Passing objects themselves is a common mistake in other XML-based remote invocation technologies as far as I'm concerned. Why bother passing an object if you know you're going to have to pull a fresh copy to check if it's *valid* anyhow?)

I'm also debating whether I should allow arguments that are object instances, or if I should keep forcing you to unbundle the primary key attributes of the objects as individual attributes in order to specify argument objects to the remote invocations. Sure it would be more convenient to be able to pass an object argument, but what I do now is workable, as there can be no automation of the server proc/func invocations anyhow -- their intent and meaning is completely unfathomable to the rule base, so I can't infer anything useful.

Were I to try to infer argument forms in the GUI for the server procs, then it would be useful to be able to specify object arguments, because I could fire up a Picker window to let you select the argument object. But I don't think it's really worthwhile to do so, because the prototype GUI is only meant as a framework and for demos, not for deployment.

2015.06.16 MSS Code Factory 2.2.12892 ServerProc XSD enhancements defined

The XSD enhancements for the XMsg Rqst/Rspn XSDs have been coded and test-manufactured.

2015.06.14 MSS Code Factory 2.2.12890 Obj layer implementation of ServerProcs coded and built

The Obj layer over the underlying persistant storage layers for the ServerProcs has been coded and built. Now all I have left to do is the "fun stuff" of the XMsg layer enhancements -- those are currently NotImplementedYet stubs.

2015.06.13 MSS Code Factory 2.2.12888 Low level implementations of ServerProcs coded and built

The sample ServerProc test code in CFDbTest 2.3 has been manufactured and built.

The XMsgClient layer is just NotImplementedYet stubs, of course -- that will get coded much later.

Next on my plate is wiring the Obj layer to the underlying low-level ServerProc code. That will be a combination of instance methods and TableObj methods for the "class" methods (you need to have an actual object in order to resolve the SchemaObj that has to be passed as the first argument of the ServerProc method interface, so they're not really "class" methods in the usual sense; they just don't have an object instance associated with their invocation.)

2015.06.13 MSS Code Factory 2.2.12887 Low level interfaces for ServerProcs defined

The low level interfaces of the table objects for the new ServerProcs have been defined. As a result, the code produced by this release does not compile, because the implementations of those interfaces have not been specified yet.

2015.06.12 MSS Code Factory 2.2.12885 CFDbTest 2.3 model enhanced with sample ServerProcs

The CFDbTest 2.3 model has been enhanced with four sample ServerProc specifications. Two each for class and instance procedures, one of which has no arguments and one of which takes a required and an optional Int16 argument. This test data won't exercise the rules that are to be defined in as full a fashion as I'd like, but it's good enough to get started.

The updated model parses successfully, and has been used to manufacture CFDbTest 2.3, which has been test-compiled and will be packaged and posted shortly.

2015.06.11 MSS Code Factory 2.2.12884 Support for new ServerProc and Param elements coded

The XSD for the 2.2 BAM syntax has been enhanced with the element definitions for the ServerProc and Param elements, the SAX parser enhanced accordingly, and the additional custom verbs wired to the GEL runtime. The enhanced engine is now ready to accept the new models and rules for testing.

2015.06.10 MSS Code Factory 2.2.12883 Ready to begin customization enhancements for new features

The 2.2 version of MSS Code Factory is now ready to begin the customization and enhancements necessary to support the definition and processing of the ServerProc and Param elements. First I'll have to work on the parser enhancements, of course.

2015.06.10 MSS Code Factory CFBam 2.2.12882 ServerProc.Body renamed ServerProc.JProcBody

In keeping with the other code customization expansion attributes, the ServerProc body has been renamed to JProcBody. After all, other languages will be supported some day.

2015.06.09 MSS Code Factory 2.2.12880 New features built into underlying CFBam 2.2

The new features for the ServerProc and Param objects have been built into the underlying CFBam 2.2 code. They have not been wired into the parser or rules yet, of course. That will take some time. Ok, a lot of time...

2015.06.09 MSS Code Factory CFBam 2.2.12879 Bug corrected in new features

There was an error in the rules for the lowest level of a PopDep for MSS Code Factory 2.1 which produced a compile error for CFBam 2.2. The bug has been corrected.

2015.06.09 MSS Code Factory CFBam 2.2.12877 Enhanced to support new features

CFBam 2.2 has been enhanced to support the new ServerProc and Param features that will be needed by MSS Code Factory 2.2 and 2.3. Older code has also been cleaned from the source tree.

2015.06.06 MSS Code Factory 2.2.12874 CFCore 2.3 model corrected

There were some occurances of v2_1 that had to be changed to v2_3 in the custom code segments of the CFCore 2.3 model. I am now ready for setting up a test build of CFSecurity 2.3 as soon as I remanufacture it and migrate the build scripts and such from 2.2.

2015.06.06 MSS Code Factory 2.2.12871 Initial test run passed

An initial test manufacturing of CFSecurity 2.3 has been performed and ran successfully. Note that you can't build the code produced by this release yet, as I haven't migrated CFLib and CFCore from 2.1 to 2.3.

2015.06.06 MSS Code Factory 2.2.12870 Initial build

You can't run this build because I haven't migrated any of the rules or models yet, but it's a clean initial migration from 2.1 to 2.2, ready for me to start working on it.

2015.06.05 MSS Code Factory CFAll 2.2.12869 Rebuilt with JDK 8

All of the 2.2 projects have been rebuilt with JDK 8.

2015.06.01 Programmer's Notes

I will be focusing future development on the 2.2 code base, adding some new features to the implementation compared to 2.1.

Once I get 2.2 running, I'll be migrating the rule base to 2.2 and getting it running to produce code for 2.3 first of all.

Then I'll focus on modifying the model to allow the definition of server methods with an element body comprised of the Java code to be used as the implementation body of the method on the server side of the code. The methods are made part of the core Table interface and implementation base. They are not implemented as instance members, because I need to be able to remote them through the lower level server interface, not the application programmer interface.

I am not going to support Schema methods, only table methods. But you can always do something relatively creative like adding your methods to a Tenant or Cluster for your project, as those objects always exist in the server runtime.

In order to create the 2.3 code from the 2.2 runtime I'll be testing, I'll have to create CFLib and CFCore 2.3, as 2.3 code expects to run with 2.3 libraries, not 2.1 libraries. I don't expect this to be anything more than a copy-edit exercise.

I won't worry about running the 2.3 code base until I'm done my work with 2.3, but I will be doing builds regularly. That's why I need the core libraries ported to the next release.

Note that 2.2 is an internal development release. You should wait for 2.3 before trying out it's features if you can. I intend to leave the 2.1 release as the default download until any patch service packs have to be issued. If I make it to the point that the 2.2 and 2.3 code are producing a viable implementation of remote server method invocations, I'll get 2.3 running, snapshot the 2.2 rule base forward, and do a validation run of 2.4 code (i.e. the same model as 2.3 and 2.2, but produced by 2.3 instead of 2.2.) Once 2.3 is producing output identical to 2.2, I can issue a production release of 2.3, and let it take over for 2.1 as the default download.

2015.06.01 MSS Code Factory CFAll 2.2.12865 Build of 2.2 projects updated and refreshed

With Service Pack 2 out the door, it was time to turn my attention to some niggling details in the 2.2 build processes that were annoying me.

I've automated the packaging of the WAR files via the ant build.xml scripts, and done the builds with the modified scripts.

I've also added PostgreSQL and MySQL run scripts for the SAX loaders to the projects, and a script for running the HTTP Swing prototype GUI client.

There may be bugs in the scripts or the WAR packaging. I'm much too lazy to test it all. :P

2015.05.31 MSS Code Factory CFAll 2.2.12864 Build of 2.2 projects manufactured by 2.1.12863 SP2 complete

All of the 2.2 projects have been refreshed by MSS Code Factory 2.1.12863 SP2, rebuilt, and repackaged.

Since 2015.04.16, 53,833 lines of code have been added to the 2.2 projects.

Over the course of 46 days, that works out to an average of 1,170 lines per day. Not my most productive by far, as I didn't even average 2,000 lines a day (which is what I expect of myself doing hand coding.) But I did have a very relaxing work-period. I really only put in about 2-3 weeks of effort. And those were only full days half the time.

Project Prior Total Lines New Total Lines Lines Added
CFCore 2.2 310,906 310,906 0
CFSecurity 2.2 1,046,206 1,047,870 1,664
CFInternet 2.2 1,563,561 1,566,156 2,595
CFCrm 2.2 2,173,005 2,176,705 3,700
CFAcc 2.2 2,695,873 2,700,065 4,192
CFAsterisk 2.2 2,043,599 2,046,780 3,181
CFBam 2.2 14,944,104 14,965,273 21,169
CFDbTest 2.2 7,258,138 7,268,925 10,787
CFEnSyntax 2.2 1,830,918 1,833,756 2,838
CFFreeSwitch 2.2 2,616,583 2,620,290 3,707
TOTAL 36,482,893 36,536,726 53,833

2015.05.31 MSS Code Factory CFSecurity, CFDbTest 2.2.12862 Completed debugging removal of Cluster Add/Delete from GUI

The testing for the removal of support for the Add/Delete operations on Cluster objects has been completed.

2015.05.30 MSS Code Factory CFSecurity, CFDbTest 2.2.12860 Cluster Add/Delete removed from GUI

The Add/Delete support for Clusters has been removed from the Swing prototype GUI. You can't create arbitrary clusters. There is only the special "system" cluster and the default cluster this set of compute service nodes implements.

There was a missing "Type" suffix in the XSD for XMsg response documents, which has been corrected.

I still need to remove the "Delete Cluster..." menu item from the View/Edit forms.

2015.05.30 MSS Code Factory CFSecurity, CFDbTest 2.2.12857 Rebuilt with CFLib 2.1.12855

This rebuild and repackaging with CFLib 2.1.12855 will prevent some bad data from occuring when encoding extended charcters of UTF-8 import data.

2015.05.29 MSS CodeFactory CFSecurity, CFDbTest 2.2.12854 Remote file imports now work

The remote file imports now work with the latest fixes to CFLib's XML utilities. I'm still not happy with the way I encode XML strings. Surely there is an API for doing that already instead of having to maintain my own broken code.

2015.05.28 MSS Code Factory CFSecurity 2.2.12849 Refreshed and repackaged with CFLib 2.1.12847

The correction applied to XML string conversion in CFLib 2.1.12847 should address the issue I was having with sending a request to the CFDbTest 2.2 server from the CFSecurity 2.2 web client GUI.

2015.05.28 MSS Code Factory CFSecurity 2.2.12846 Added code to load file contents for import

The file contents have to be loaded before they can actually be imported. Now I can start testing.

2015.05.28 MSS Code Factory CFDbTest 2.2.12844 Clean build and packaging

CFDbTest 2.2 now clean builds and is ready for use in running a web service on behalf of a CFSecurity 2.2 debug client.

2015.05.28 MSS Code Factory CFSecurity 2.2.12843 Clean build and packaging

CFSecurity now clean builds and is ready for use in testing a file import client against a CFDbTest web server.

2015.05.28 MSS Code Factory CFDbTest 2.2.12841 FileImport window ready for testing

The FileImport window is ready for testing.

2015.05.28 MSS Code Factory CFDbTest 2.2.12839 Clean build

With that last defect corrected, CFDbTest 2.2 now clean builds with the fileImport() support.

2015.05.28 MSS Code Factory CFDbTest 2.2.12837 fileImport() signature now returns string log contents

The log contents resulting from the import parse are now returned by the various fileImport() implementations. The XMsgRspnHandler has seen the addition of a LogContents attribute which is used to capture the output provided by a RspnFileLoaded message.

I'm ready to start working on the import window itself.

2015.05.28 MSS Code Factory CFDbTest 2.2.12835 Groundwork for client-side fileImport() requests added

The core objects and interfaces have all been coded to support fileImport() requests from the client. Next I need to work on the FileImport GUI form, and then wire it up to the GUI itself.

2015.05.27 MSS Code Factory CFDbTest 2.2.12832 Regression test of PostgreSQL SAX Loader passed

The regression testing and corrections for the PostgreSQL SAX Loader has been regresssion tested and now loads the cluster and tenant information as specified through the updated command line argument parsing.

2015.05.27 MSS Code Factory CFDbTest 2.2.12830 SAX Loaders now restrict Cluster and Tenant arguments

The SAX Loader's Cluster argument is now restricted to "system" or "default", and the Tenant argument subsequently restricted for a "system" Cluster as well (the "system" Cluster only contains one Tenant, the "system" Tenant.) The Tenant must exist within the Cluster prior to running a loader session for that Tenant; you cannot create a Tenant with this implementation of a SAX Loader.

Loading the client configuration object now clears the buffer variables for re-resolution by the next invocation of their getters.

The client configuration file is refreshed/reloaded when opening a login window.

2015.05.26 MSS Code Factory CFDbTest 2.2.12828 20-entry Server URL history supported in GUI

See MSS Code Factory 2.1.12827 notes for details.

2015.05.19 MSS Code Factory 2.2.12826 Login form now uses a Cluster combo box

The Swing GUI prototype login form now uses a combo box to let you choose between the default and system clusters. If you choose "system", the Tenant is forced to system as well and made read-only. If you choose "default", you are allowed to specify a Tenant.

2015.05.15 MSS Code Factory CFDbTest 2.2.12824 Leaking web server connections fixed

I've added a SchemaPooledObj which derives from SchemaObj, but does not disconnect the backing store during a logout. This prevents the leaking connections that I was seeing with the web services. CFDbTest has been used to exercise and verify the new code.

2015.05.15 MSS Code Factory CFDbTest 2.2.12822 HTTP Client works!

The HTTP client works! See MSS Code Factory 2.1.12821 release notes for details.

2015.05.01 MSS Code Factory CFDbTest 2.2.12819 X(ml)Msg layer now specifies correct dialect

The dialect document for an X(ml)Msg response now properly identifies the document associated with the request, rather than defaulting to the hard-coded schema definition of the server. This will allow clients of downlevel projects (e.g. CFSecurity) to talk to a server that imports or references their schema definition.

2015.04.29 MSS Code Factory CFDbTest 2.2.12817 Throwables are now always formatted by the request Envelope handler

The $SchemaName$XMsgRqstHandler has been updated to implement ICFLibThrowableFormatter, and to wire itself to the CFLibEnvelopeHandler that it is associated with whenever the envelope handler is specified. There are no changes required to higher level code such as the database-specific mains or the HTTP request processing page handler. There should no longer be unreported exceptions producing empty responses except for bad security key handshake data. But as long as you're relying on the CFLib implementation of the handshake, you should be safe from having to deal with empty responses.

2015.04.16 MSS Code Factory CFAll 2.2.12813 Updated licensing information

The formerly Apache 2.0 licensed projects are now licensed under the BSD 3-Clause license so there are no issues with extending and modifying them as part of manufactured projects. The other projects have been updated to credit the BSD source code as required.

Since 2015.02.01, 3,287,736 new lines of code have been added over the course of 75 days, bringing the total up to 36,482,893 lines. That is an average of 43,836 lines per day, 1,826 lines per hour, or 30 lines per minute.

An upload of the source .zips and installer .zips weighs in at over 1.2 GB.

Project Prior Total Lines New Total Lines Lines Added
CFCore 2.2 291,454 310,906 19,452
CFSecurity 2.2 925,969 1,046,206 120,237
CFInternet 2.2 1,400,627 1,563,561 162,934
CFCrm 2.2 1,959,260 2,173,005 213,745
CFAcc 2.2 2,413,455 2,695,873 282,418
CFAsterisk 2.2 1,819,764 2,043,599 223,835
CFBam 2.2 13,786,105 14,944,104 1,157,999
CFDbTest 2.2 6,626,405 7,258,138 631,733
CFEnSyntax 2.2 1,620,706 1,830,918 210,212
CFFreeSwitch 2.2 2,351,412 2,616,583 265,171
TOTAL 33,195,157 36,482,893 3,287,736

2015.04.15 MSS Code Factory CFAll 2.2.12810 Refreshed by 2.1.12808 Service Pack 1

Running under Ubuntu 14.04.2 on a P4 3.8GHz single-core system with 4GB of DDR2-800 memory and a 7200 RPM hard drive, it took 23.5 minutes to manufacture all of the 2.2 project source code.

Since 2015.02.01, 571,464 new lines of code have been added over the course of 74 days, for an average of 7,722 lines per day, 321 lines per hour, or 5 lines per minute.

Project Prior Total Lines New Total Lines Lines Added
CFCore 2.2 291,454 294,846 3,392
CFSecurity 2.2 925,969 961,806 35,837
CFInternet 2.2 1,400,627 1,441,753 41,126
CFCrm 2.2 1,959,260 2,006,605 47,345
CFAcc 2.2 2,413,455 2,466,093 52,638
CFAsterisk 2.2 1,819,764 1,866,699 46,935
CFBam 2.2 13,786,105 13,942,200 156,095
CFDbTest 2.2 6,626,405 6,718,198 91,793
CFEnSyntax 2.2 1,620,706 1,667,018 46,312
CFFreeSwitch 2.2 2,351,412 2,401,403 49,991
TOTAL 33,195,157 33,766,621 571,464

2015.04.13 MSS Code Factory CFDbTest 2.2.12809 Test suite for Service Pack 1

This is the test suite used to exercise MSS Code Factory 2.1.12808 Service Pack 1.

2015.04.13 MSS Code Factory CFDbTest 2.2.12807 Corrected Base64 encoding

The code now uses "new String( Base64.encode( val ) )" instead of "Base64.encode( val ).toString()".

2015.04.12 MSS Code Factory CFDbTest 2.2.12805 The Swing Protype GUI for PostgreSQL supports CFTip

The Swing GUI prototype for PostgreSQL now works with the CFTip processing layers from CFLib 2.1.12803. You can log in to the database, log back out, and log in again. A quick run-through of the navigation windows works as well.

2015.04.12 MSS Code Factory CFDbTest 2.2.12802 CFTip login processing coded

The login processing has been reworked to support CFTip. The code is almost ready for testing at this point; I think I still need to query the server information as part of the Swing GUI prototype login form processing in order to be able to issue a login request.

2015.04.07 MSS Code Factory CFDbTest 2.2.12799 Security Manager now initializes cluster correctly

The security manager was not coded quite right for initializing the system cluster and system password (there was a nested system session problem.) This has been corrected, and the CFDbTest 2.2 Security Manager and XMsg Request WARs now function properly.

2015.03.29 MSS Code Factory CFDbTest 2.2.12793 Web interface for submitting CFTIPEnvelope requests debugged

The CFTIPEnvelope Request submission form has been debugged and now send plain text exception identification and traceback information when there is an error processing the LoginRequest or AppRequest Payload, such as a decryption error.

The ServerInfoRequest is properly responded to with a text/plain document comprising of an XML response message suitable for processing. This is the only plain-text response the server normally produces; any other occurances indicate exception handling was invoked by the server, and that the exception message in the response text should be displayed to the user.

The build now expect commons-io-2.4.jar to be in the runtime path for all executables, and CFLib has been updated to version 2.1.12791.

2015.03.28 MSS Code Factory CFDbTest 2.2.12790 XMsgRequestXml page now expects a CFTipEnvelope XML document

Instead of expecting a base-64 encoded blob, the XMsgRequestXml processing page now expects the submitted request to be a CFTipEnvelope XML message, which subsequently contains a base-64 encoded message body being submitted.

The raw response from the processing of the message is passed back to the client as the plain text response to the POST of the form data for processing.

GET just returns a user interface form for doing test data submissions. The client web interface implementation will never be doing GET processing, only POST submission of message bodies for processing.

2015.03.28 MSS Code Factory 2.2.12788 XMsgRequestXml processing page ready for testing

The LoginRequest and AppRequest pages have been consolidated into a single Request page that instantiates a CFTipEnvelopeHandler for the client session, relying on it to process the pages after converting the MessageBody back from base-64 encoded data to a byte array and from thence into a String to be processed by the EnvelopeHandler with the results passed back to the client as text/plain data (a base-64 encoded data block encrypted with the client-generated AES256 session key specified by the login request to the server.)

2015.03.27 MSS Code Factory CFDbTest 2.2.12786 There are now separate XMsg LoginRequest and AppRequest forms

The LoginRequest and AppRequest forms now redirect to each other based on the state of the Authorization.

I haven't gotten anywhere near as far as decrypting the code block with the appropriate key information, as I haven't stored the client session key information in the server yet. I need to actually implement the new login XMsg request/response processing and propagate down to the database layer as appropriate.

2015.03.27 MSS Code Factory CFDbTest 2.2.12784 Stubbed in XMsg Request WAR

The $lower SchemaName$xmsgsrvwar$PackedVersion$ project source tree has been added to the manufactured project JEE support. See the 2.1.12783 project notes for details.

2015.03.26 MSS Code Factory CFDbTest 2.2.12782 Security Manager refreshed

The Security Manager web interface is now in sync with the code as implemented by the PostgreSQL Swing Prototype GUI.

2015.08.19 MSS Code Factory CFDbTest 2.2.12780 Public device key displayed for copy-paste

The public device key is now displayed after it is retrieved from the keystore so that you can copy-paste the device information to the web forms of the Security Manager.

2015.08.19 MSS Code Factory CFDbTest 2.2.12778 Use keytool to create keys to be loaded for device

The public and private keys for the configured device name are now loaded from the JCEKS key store, with prompting for passwords as appropriate. The forms to prompt for the passwords are auto-launched when the application starts, and exit the application if they are cancelled out of. You must know the passwords for the keystore and the device key in order to use the application at all.

I did have to give up on implementing custom code to create self-signed client device certificates. You'll have to use the Java "keytool" to create RSA 2048 certificates (self signed is fine.) Don't forget to specify "-storetype jceks" and to make sure you're working with the keystore specified in the client configuration file. The name of the device should be used as the alias for the key you create.

I do still auto-create the keystore itself, though if the key is not found in a newly created keystore, you're given instructions on how to create a self-signed RSA 2048 key and the application exits.

2015.03.13 MSS Code Factory CFDbTest 2.2.12776 KeyStore now properly initialized

The form conversation for logins now uses the client configuration file data to open or create the KeyStore as appropriate. If you cancel out of opening or creating the key store, the application exits so that you can modify the client configuration file by hand (yes, I should have forms for editing the attributes of the client configuration file, but it is only a prototype, not a full-featured application.)

2015.03.10 MSS Code Factory CFDbTest 2.2.12774 PostgreSQL GUI prototype now initializes ServerInfo

The ServerInfo is now properly initialized during startup of the GUI. It takes several seconds for the server public-private key pair to be generated, so a message is written to the console while that initialization is happening, after which the GUI screens get displayed.

2015.03.08 MSS Code Factory CFDbTest 2.2.12771 Client configuration file wired for PostgreSQL GUI

The PostgreSQL GUI client prototype now properly initializes the client configuration file and sets a reference to it in the Swing schema object so that it can be referenced. See the 2.1.12770 release notes for more information.

2015.02.07 MSS Code Factory CFDbTest 2.2.12769 Swing PostgreSQL client stitches together security objects

The security objects provided by CFLib's Tip package are now integrated with the Swing prototype GUI for PostgreSQL, though they are not properly initialized yet and there is quite a lot of work remaining before I can even think about testing any of the code.

I've also removed the XMsg Loaders from the project.

2015.02.05 MSS Code Factory CFDbTest 2.2.12767 Clean build with client side of CFTip

The X(ml)Msg Client handler has been updated to use the CFTipClientHandler provided as provided by CFLib 2.1.12763. The code does not work at this point because the server side still needs to be wired together, and because the login processing does not handshake the security keys yet.

2015.02.02 MSS Code Factory CFDbTest 2.2.12761 Added SysCluster via CFSecurity 2.2

The SysCluster object is a singleton that is a component of one and only one Cluster -- the one that will be created and initialized as part of the system password initialization in the Security Manager web interface.

The sample configuration for Tomcat now has an example configuration for the SSL support, and the Security Manager WAR web.xml now specifies that it must be mounted using https.

A seperate WAR will be created for the XMsgRequest servlet in the future, and will be mounted over http, as the bodies of the requests will be using CFTip to ensure the privacy of their contents.

2015.02.02 MSS Code Factory CFDbTest 2.2.12759 Corrected defect in Add Device page

The Security Manager Add Device page was not properly initializing the Cipher instance. However, seeing as the client code for providing a source for the copy-paste of the client's public key has not been coded yet, the point is moot -- you could only successfully add a device if you created some custom code.

2015.02.01 MSS Code Factory CF All 2.2.12757 Removed BouncyCastle from all 2.2 projects

It took 45 minutes to manufacture all of the Code Factory 2.2 projects, stripping away the BouncyCastle requirement. That works out to 12,294 lines per second. Not too bad for a creaky old 3.8GHz P4 with a paltry 4GB of RAM.

Since 2015.01.03, an additional 323,298 lines of code have been added during the intervening 29 days, for a new code rate of 11,148 lines per day on average.

Project Prior Total Lines New Total Lines Lines Added
CFCore 2.2 292,121 291,454 -667
CFSecurity 2.2 912,465 925,969 13,504
CFInternet 2.2 1,375,574 1,400,627 25,053
CFCrm 2.2 1,919,238 1,959,260 40,022
CFAcc 2.2 2,356,170 2,413,455 57,285
CFAsterisk 2.2 1,780,925 1,819,764 39,511
CFBam 2.2 13,706,253 13,786,105 79,852
CFDbTest 2.2 6,628,639 6,626,405 -2,234
CFEnSyntax 2.2 1,594,193 1,620,706 26,513
CFFreeSwitch 2.2 2,306,281 2,351,412 45,131
TOTAL 32,871,859 33,195,157 323,298

2015.02.01 MSS Code Factory CFDbTest 2.2.12756 Remove dependancy on BouncyCastle

BouncyCastle has been stripped from the WAR installation requirements.

2015.02.01 MSS Code Factory CFDbTest 2.2.12754 Replaced anchors with JavaScript onclick actions for Security Manager

I was using an obsolete form of button links in an effort to avoid JavaScript dependencies for the Security Manager web interface, but that syntax is no longer supported by Firefox/Iceweasel, so it has been converted to onclick JavaScript events.

2015.01.22 MSS Code Factory CFDbTest 2.2.12750 Manufactured Security Manager interface tested

The manufactured version of the Security Manager web interface has been tested, and the hand written code has been removed from the project.

2015.01.12 MSS Code Factory CFDbTest 2.2.12746 Passwords are now double-hashed

With the changes made to the encoding of passwords, the intent is that a client application will do the following, and send the resulting string to the server as the password:

MessageDigest msgDigest = MessageDigest.getInstance( "SHA-512" );
msgDigest.update( password.getBytes( "UTF-8" ) );
byte[] hash = msgDigest.digest();
byte[] encodedHash = Base64.encodeBase64( hash );
String hashedPassword = new String( encodedHash );

The server receives this encoded password string, and does the following:

byte[] encodedHash = Base64.decodeBase64( hashedPassword );
MessageDigest msgDigest = MessageDigest.getInstance( "SHA-512" );
msgDigest.update( encodedHash );
byte[] hash = msgDigest.digest();
String encodedAndHashedPassword = new String( hash );

The resulting string can then be compared to the database value. The purpose to this double encoding is so that after initially creating and setting a password, the client application can change the password and ensure that the server has never seen a version of the password that wasn't hashed, thereby obfuscating the user's password completely from the server.

Granted, few people are paranoid enough to care, but I wanted to take care of the wishes of those who do care. Of course, if you reset the password and enter one through the HTML Security Manager forms, the server sees your plain-text password again, so you'd want to use an application to change it again, but that degree of paranoia is left up to the user -- I'm not going to force you to change the password through the application interface.

2015.01.10 MSS Code Factory CFDbTest 2.2.12745 Database scripts corrected, CFSecurity.SecUser updated

The database creation scripts have all been tested with the changes made to CFSecurity 2.2, and the Security Manager prototype has been updated to incorporate the changes to SecUser.

There are no longer separate Html and App passwords, as it's my intent to double-hash the passwords in the Security Manager forms (TBD.) That way the client applications can hash the password entered by the user to protect it from interception until I have all of the encryption code working.

2015.01.03 MSS Code Factory CFDbTest 2.2.12743 EMail encryption supported but broken

I've got the EMail encryption happening if you set the server variable CFDbTestSmtpEncryptEMails to "true" (set it to "false" otherwise), but the email body being produced by Bouncy Castle doesn't want to decrypt properly. I'm going to set that on a shelf for now; there may be bugs in Bouncy Castle. I'll know better when I start implementing the client-server communications, as that will be using Bouncy Castle on both ends of the link.

For now, you should just disable email encryption.

2015.01.03 MSS Code Factory CFAll 2.2.12742 Life, The Universe, and Everything

It took 45 minutes for MSS Code Factory 2.1.12736 to refresh all of the 2.2 projects, producing 32,871,859 lines of code for an average rate of 12,174 lines per second.

Five hours since starting that ManufactureCFAll22.bash run at around 20h20 this evening, I've built all the projects, packaged their installers and source distributions, tagged the releases in git, and am about to start the uploads to SourceForge for distribution.

Project Prior Total Lines New Total Lines Lines Added
CFCore 2.2 0 292,121 292,121
CFSecurity 2.2 0 912,465 912,465
CFInternet 2.2 0 1,375,574 1,375,574
CFCrm 2.2 0 1,919,238 1,919,238
CFAcc 2.2 0 2,356,170 2,356,170
CFAsterisk 2.2 0 1,780,925 1,780,925
CFBam 2.2 0 13,706,253 13,706,253
CFDbTest 2.2 0 6,628,639 6,628,639
CFEnSyntax 2.2 0 1,594,193 1,594,193
CFFreeSwitch 2.2 0 2,306,281 2,306,281
TOTAL 0 32,871,859 32,871,859

2015.01.02 MSS Code Factory CFDbTest 2.2.12741 EMail decrypted, but still unreadable

I reworked my approach to encrypting emails rather dramatically a few times over the past few days. I'm content that I'm using Bouncy Castle correctly now, as Enigmail is claiming that its decrypted the email.

However, Enigmail does not display the message contents. And if I try to copy-paste decrypt the message with GPG, it gets errors. It may be that Enigmail is masking the errors and silently refusing to display the content.

Regardless, I've made a lot of progress towards encrypted emails. It doesn't work yet, but it's coming along.

2015.01.02 MSS Code Factory CFDbTest 2.2.12740 EMail encrypted, but unreadable

The email for a password reset request is now encrypted when re-sent, but the received email doesn't decrypt properly. It prompts for the correct key for the decryption, and it *looks* ok, but it doesn't work yet.

2015.01.01 MSS Code Factory CFDbTest 2.2.12739 Device public keys must now be valid PGP keys

You can now copy a public key from the Thunderbird/Enigmail key manager to the clipboard, and paste that clipboard into the Public Key field of the Add New Device form. It will be parsed and validated by loading it into a null key store before it is allowed to be saved to the database.

Only PGP keys as supported by will be allowed. But with the use of Thunderbird and Enigmail on top of the core GPG implementation, a fairly easy to install and configure user interface results.

I'll leave it up to you to search for the installation instructions for GPG, Thunderbird EMail, and the Enigmail add-on for using PGP keys with Thunderbird.

Once you have your private key configured and tested in Thunderbird/Enigmail, all you need to do is copy the public to the clipboard and paste it in the form.

Next I want to figure out how to send an encrypted email with an HTML email body. I believe S/MIME is what I need to look at next. The first place it should be applied is to the ResetHtmlPasswordRequest emails. I don't want just anybody to be able to intercept an email and reset someone's password. That's always bugged me about web enterprises -- they send unencrypted emails all over the place.

2014.01.01 Programmer's Notes

From what I've been reading, what I want to do is create a null key store, which can then load the text of the public key provided by the user through the Security Manager web interface.

The first instance to create is a null-keyed SecDevice bound to the "web" device name. Once this instance is presumed to exist, it can become an attribute of the SecSession, defaulting to "web" for the HTTPS-interfaced clients and the Security Manager web interface itself, relying instead on the HTTPS protocol configuration to secure the responses in transit.

I'm going to presume that the public key provided was produced by a Thunderbird installation with Enigmail installed and properly configured to provide secured email for the user's email account, and that the public key was produced by exporting an ASCII-armoured public key from the Thunderbird/Enigmail menus.

After loading the public key/ASCII-armoured text, you would need to resolve the loaded key in the null key store, parse out any device name associated with the key, and save the public key in association with the loaded device name as appropriate (a create or an update of a SecDevice instance.)

If the user specified a different device name, presume that they've copied the private key to that device's Java Keystore for the user, and create a SecDevice binding the new name to the same public key string/ASCII-armoured text.

After that, implement a new login API for the manufactured XMsgRqst interface to replace the one which currenlty expects a database-recognized client-server username authentication on a database-provider basis. The new interface will expect a username/email address, a password, and a device name.

On the client-side, when a login is attempted for those parameters, the client keystore will be probed for "username - devicename" as a private key provider. If the client keystore locates the private key, it will then step through a user-guided unlocking of the private key so it can be used to decrypt the message sent by the server and encoded by the public key.

On the server-side, when a login request is made for a specified device, the public key from the authenticating user's matching SecDevice entry is calculated and added to the Session data with the key $SchemaName$SecDevicePublicKey.

Whenever a secured response is to be sent to the client, all code then resolves the $SchemaName$SecDevicePublicKey rather than re-locating and unlocking the key.

When the user logs out, the $SchemaName$SecDevicePublicKey is cleared to null.

2014.01.01 MSS Code Factory CFDbTest 2.2.12738 Force user to create a default device

The first thing a user has to do after verifying their email address is to create a default device.

Right now the public key you specify for the devices are just unvalidated strings. I need to figure out how to convert the parameter string/text into a Java-useful crypto key to be used for sending responses to the client.

The client in turn will hold the private key for that public key in its Java Wallet, and will provide the appropriate forms and interfaces for opening the wallet, retrieving the private key, unlocking the private key with a password, and holding on to the result for use in decrypting the responses sent by the server.

The next step is going to be adding the crypto-key upload/conversion and validation for the public keys. What I want is to be able to export a Thunderbird Enigmail public key for the specified email and host name combination to an ASCII-armoured text file, which you then copy-paste the contents of into the device creation form's public key field.

If the uploaded key can't be parsed by the server, it will be rejected as an invalid key. Other than that, I don't care what level of security the user wants to use for their key -- that's their decision, not mine. I'll support anything Java supports.

Don't forget to download and install the Java Mail library into your Tomcat server's library directory, or you won't be able to run the provided WAR file.

2015.01.01 MSS Code Factory CFDbTest 2.2.12737 HTML Password Reset requests implemented

The emails and forms for the HTML Password Reset conversation have been added to the prototype Security Manager web interface.

2014.12.31 MSS Code Factory 2.2.12735 EMail confirmation lifecycle coded and tested

The EMail confirmation lifecycle changes have been applied to the Security Manager web interface. Now when you register a new account, an email with a unique UUID is sent to you. If you try to use the login before that link has been clicked, the email is re-sent. If you try to create an account for that same email id, the email is re-sent.

Only after you have confirmed your email address are you allowed to log in to the Security Manager.

2014.12.31 MSS Code Factory CFDbTest 2.2.12733 Migrated 2.1 prototype Security Manager Web Interface

The Security Manager Web Interface prototype code has been migrated to 2.2, the database refreshed with the new attributes mentioned in the 2.1 release history, the code all built, tested, and debugged. At this point, 2.2 is ready to take over for the 2.1 development stream for the web interface.

The migration and refresh were done in less time than it took MSS Code Factory 2.0 to refresh two sets of database install scripts on the speedy laptop. Such is the power of 2.1, which did the refresh of 2.2 in under 8 minutes.

2014.12.22 MSS Code Factory CFDbTest 2.2.12728 Latest code as manufactured by MSS Code Factory 2.1.12725

This is the latest code as manufactured by MSS Code Factory 2.1.12725 with the addition of CFCore 2.1.12727.

MSS Code Factory 2.1 is about to go production...

2014.12.14 MSS Code Factory CFDbTest 2.2.12701 Latest code as manufactured by MSS Code Factory 2.1.12700

This is just a test build; it doesn't get run. It's just to exercise MSS Code Factory 2.1, not for regression testing at this time.

2014.12.08 MSS Code Factory CFDbTest 2.2.12688 Latest code as manufactured by MSS Code Factory 2.1.12687

This is the lastest code produced by the 2.1 development version of MSS Code Factory. With this clean build and some visual inspection, I'm now willing to sign off on 2.0 SP1.

2014.12.06 MSS Code Factory CFDbTest 2.2.12678 Initial release, regression tests pass for MySQL

I'm only going to run the tests for MySQL. The point was to verify that the manufactured code by MSS Code Factory 2.1 is equivalent to that of 2.0. That's been accomplished. I'll be setting aside the 2.2 code for now until I need to test some new functionality that is *only* produced by the 2.1 code base, not 2.0.