I've decided against including any more functionality for the initial 2.3 release. The history based synchronization will not require changes to the core engine, as all the information I need for authoring the rules to support it already exist in the 2.2 release.
I will therefore begin working on shifting the code base forward to 2.3 and issuing an initial production release of 2.3.
The history synchronization will appear in a service pack at some point in the future.
The testing of the new ServerProc, ServerObjFunc, and ServerListFunc model specification has been completed. It is now ready for use.
I am debating whether to implement history based client synchronization for the 2.3 release or not. I didn't spend a lot of time on the enhancements so far, so I think it would be kind of a "thin" release to push the current code as 2.3. I'd like to have more functionality in the next production release than I've added since 2.1.
The XMsgRqst ObjFunc implementation had been trying to marshall a null object response, which was causing Null reference exceptions.
The implementation of the enhancements to support ServerProc, ServerObjFunc, and ServerListFunc specifications is now complete and ready for testing.
I'll need to do some custom GUI coding for CFDbTest 2.3 in order to be able to actually do the testing, but that should only take a few days to code. I'm not in any rush to do so, though. One thing I have plenty of is time. Disability retirement may have shitty pay, but the lack of user and customer pressure to get things done on a schedule is glorious!
With the completion of the XMsgRspn enhancements, I only have to enhance the XMsgClient layer, then I'll be in a position to write tests for the new code.
The $schemaname$xmsgrqst$version$ package now clean compiles for CFDbTest 2.3.
That leaves the enhancements for XMsgRspn and XMsgClient to be done. XMsgRspn is the next up...
The ServerObjFunc and ServerListFunc message formatters are now supported by the $schemaname$xmsg$version$ package/jar.
The request and response XSDs for the XMsg layers have been enhanced with message definitions for the ServerObjFunc and ServerListFunc model specifications.
The ServerObjFunc and ServerListFunc specifications are now supported by the core and database persistance layers, and have been clean compiled for CFDbTest 2.3.
The CFDbTest 2.3 model has been updated to correct numerous defects in the sample specifications of the ServerObjFuncs and ServerListFuncs associated with the Table object.
Next I'll need to enhance the XMsg layers to support the new specifications. This build produces invalid code for those layers, because the rules don't recognize the new model elements yet.
The CFDbTest 2.3 model has been updated to add class and instance based ServerObjFunc and ServerListFunc specifications to the Table object. As a result, you can't manufacture CFDbTest 2.3 with this build of MSS Code Factory 2.3, so it's only useful to download it if you want to get a preview/example of the use of the new elements.
Note that the intent is to have the methods returning instances and arrays of the Buff specifications for objects, not the actual objects themselves, in order to maintain consistency with the rest of the lower-level database interface specifications. So while you write your code in terms of the Obj layer, you need to extract the buffers from those objects using obj.getBuff() in order to return them from your JavaMethodBody specifications.
It's also worth noting that the order of the elements is going to retained in transmission, so you can do things like give special meanings to element 0..n of your result array.
The SAX parser of MSS Code Factory has been enhanced to support the additional ServerListFunc and ServerObjFunc elements. The XSD specification has been updated accordingly.
Next up is modifying the CFDbTest 2.3 model to incorporate test data for the new elements, and then to update the rule base to manufacture the code for the new objects required by the specifications.
Note that the new code has not been tested yet.
Just to put things in perspective, CFBam 2.1 comprises 13,692,749 lines of code, while CFBam 2.2 weighs in at 15,205,452 lines, a difference of 1,512,703 lines. MSS Code Factory 2.1 itself weighs in at 365,546 lines, and 2.2 is 368,415 lines -- an additional 2,869 lines. The grand total of the new code is 1,515,572 lines. I started working on the enhancements 2015.06.06 or 17 days ago, so that works out to 89,151 lines per day, 756 lines per hour, or 12 lines per minute -- if I was working 24/7 for 17 days straight without so much as a bathroom break.
The code for MSS Code Factory 2.2 has been reworked to support the new class hierarchy as defined by the latest version of CFBam 2.2.
The new ServerObjFunc and ServerListFunc specifications are not supported yet, but a run for CFDbTest 2.3 produces output identical to the pre-refactoring version of the code.
CFBam 2.2 has been refactored to add support for ServerObjFunc and ServerListFunc specifications in addition to the existing ServerProcs.
The XMsg ServerProc support is now ready for testing. I'll have to code some custom GUI enhancements for CFDbTest 2.3 before I can test it, though. I don't expect any problems, because most of the code is just copy-paste-rename from the existing code. Still, there is always the possibility of a bug, so if you download this release and use the ServerProc enhancements, you do so at your own risk.
The XMsgClient layer now issues the RqstTableServerProc messages and interprets the passes back the RspnServerProcExecuted.DataChanged values from the ServerProc stubs for the tables.
That leaves the XMsgRqst layer to be coded yet, then the framework of the ServerProc implementation will be complete and ready for exercising in code. I'll have to do some GUI customization for CFDbTest 2.3 in order to invoke the trivial ServerProcs I've defined, though, and I'll want to implement some ServerProc bodies that actually do something other than returning true/false blindly.
The RspnServerProcExecuted XML messages are now handled by the XMsgRspn layer.
Next up is the XMsgClient layer.
The XMsg formatting layers for preparing the request and response messages for ServerProc communications have been coded and clean compiled for CFDbTest 2.3.
I'm going to deal with the XMsgRspn layer next.
I've realized I can code up another enhancement: ServerFunc, with the same argument signatures as a ServerProc, except that it would return an instance or list of objects, similar to the way the Read invocations do.
That would take care of about 80% of the usual server-side procedures and functions that one would need to code an application. The only remaining feature would be to come up with a way of passing *lists* of parameters or objects. Right now I only let you pass atomic data types arguments based on types defined for the schema.
It's also worth noting that I don't actually pass object *instances* for the Server access, only the primary keys of the objects involved. You can't count on the client to have current data -- you want to use the latest version of the object from the server cluster's database instead. (Passing objects themselves is a common mistake in other XML-based remote invocation technologies as far as I'm concerned. Why bother passing an object if you know you're going to have to pull a fresh copy to check if it's *valid* anyhow?)
I'm also debating whether I should allow arguments that are object instances, or if I should keep forcing you to unbundle the primary key attributes of the objects as individual attributes in order to specify argument objects to the remote invocations. Sure it would be more convenient to be able to pass an object argument, but what I do now is workable, as there can be no automation of the server proc/func invocations anyhow -- their intent and meaning is completely unfathomable to the rule base, so I can't infer anything useful.
Were I to try to infer argument forms in the GUI for the server procs, then it would be useful to be able to specify object arguments, because I could fire up a Picker window to let you select the argument object. But I don't think it's really worthwhile to do so, because the prototype GUI is only meant as a framework and for demos, not for deployment.
The XSD enhancements for the XMsg Rqst/Rspn XSDs have been coded and test-manufactured.
The Obj layer over the underlying persistant storage layers for the ServerProcs has been coded and built. Now all I have left to do is the "fun stuff" of the XMsg layer enhancements -- those are currently NotImplementedYet stubs.
The sample ServerProc test code in CFDbTest 2.3 has been manufactured and built.
The XMsgClient layer is just NotImplementedYet stubs, of course -- that will get coded much later.
Next on my plate is wiring the Obj layer to the underlying low-level ServerProc code. That will be a combination of instance methods and TableObj methods for the "class" methods (you need to have an actual object in order to resolve the SchemaObj that has to be passed as the first argument of the ServerProc method interface, so they're not really "class" methods in the usual sense; they just don't have an object instance associated with their invocation.)
The low level interfaces of the table objects for the new ServerProcs have been defined. As a result, the code produced by this release does not compile, because the implementations of those interfaces have not been specified yet.
The CFDbTest 2.3 model has been enhanced with four sample ServerProc specifications. Two each for class and instance procedures, one of which has no arguments and one of which takes a required and an optional Int16 argument. This test data won't exercise the rules that are to be defined in as full a fashion as I'd like, but it's good enough to get started.
The updated model parses successfully, and has been used to manufacture CFDbTest 2.3, which has been test-compiled and will be packaged and posted shortly.
The XSD for the 2.2 BAM syntax has been enhanced with the element definitions for the ServerProc and Param elements, the SAX parser enhanced accordingly, and the additional custom verbs wired to the GEL runtime. The enhanced engine is now ready to accept the new models and rules for testing.
The 2.2 version of MSS Code Factory is now ready to begin the customization and enhancements necessary to support the definition and processing of the ServerProc and Param elements. First I'll have to work on the parser enhancements, of course.
In keeping with the other code customization expansion attributes, the ServerProc body has been renamed to JProcBody. After all, other languages will be supported some day.
The new features for the ServerProc and Param objects have been built into the underlying CFBam 2.2 code. They have not been wired into the parser or rules yet, of course. That will take some time. Ok, a lot of time...
There was an error in the rules for the lowest level of a PopDep for MSS Code Factory 2.1 which produced a compile error for CFBam 2.2. The bug has been corrected.
CFBam 2.2 has been enhanced to support the new ServerProc and Param features that will be needed by MSS Code Factory 2.2 and 2.3. Older code has also been cleaned from the source tree.
There were some occurances of v2_1 that had to be changed to v2_3 in the custom code segments of the CFCore 2.3 model. I am now ready for setting up a test build of CFSecurity 2.3 as soon as I remanufacture it and migrate the build scripts and such from 2.2.
An initial test manufacturing of CFSecurity 2.3 has been performed and ran successfully. Note that you can't build the code produced by this release yet, as I haven't migrated CFLib and CFCore from 2.1 to 2.3.
You can't run this build because I haven't migrated any of the rules or models yet, but it's a clean initial migration from 2.1 to 2.2, ready for me to start working on it.
All of the 2.2 projects have been rebuilt with JDK 8.
I will be focusing future development on the 2.2 code base, adding some new features to the implementation compared to 2.1.
Once I get 2.2 running, I'll be migrating the rule base to 2.2 and getting it running to produce code for 2.3 first of all.
Then I'll focus on modifying the model to allow the definition of server methods with an element body comprised of the Java code to be used as the implementation body of the method on the server side of the code. The methods are made part of the core Table interface and implementation base. They are not implemented as instance members, because I need to be able to remote them through the lower level server interface, not the application programmer interface.
I am not going to support Schema methods, only table methods. But you can always do something relatively creative like adding your methods to a Tenant or Cluster for your project, as those objects always exist in the server runtime.
In order to create the 2.3 code from the 2.2 runtime I'll be testing, I'll have to create CFLib and CFCore 2.3, as 2.3 code expects to run with 2.3 libraries, not 2.1 libraries. I don't expect this to be anything more than a copy-edit exercise.
I won't worry about running the 2.3 code base until I'm done my work with 2.3, but I will be doing builds regularly. That's why I need the core libraries ported to the next release.
Note that 2.2 is an internal development release. You should wait for 2.3 before trying out it's features if you can. I intend to leave the 2.1 release as the default download until any patch service packs have to be issued. If I make it to the point that the 2.2 and 2.3 code are producing a viable implementation of remote server method invocations, I'll get 2.3 running, snapshot the 2.2 rule base forward, and do a validation run of 2.4 code (i.e. the same model as 2.3 and 2.2, but produced by 2.3 instead of 2.2.) Once 2.3 is producing output identical to 2.2, I can issue a production release of 2.3, and let it take over for 2.1 as the default download.
With Service Pack 2 out the door, it was time to turn my attention to some niggling details in the 2.2 build processes that were annoying me.
I've automated the packaging of the WAR files via the ant build.xml scripts, and done the builds with the modified scripts.
I've also added PostgreSQL and MySQL run scripts for the SAX loaders to the projects, and a script for running the HTTP Swing prototype GUI client.
There may be bugs in the scripts or the WAR packaging. I'm much too lazy to test it all. :P
All of the 2.2 projects have been refreshed by MSS Code Factory 2.1.12863 SP2, rebuilt, and repackaged.
Since 2015.04.16, 53,833 lines of code have been added to the 2.2 projects.
Over the course of 46 days, that works out to an average of 1,170 lines per day. Not my most productive by far, as I didn't even average 2,000 lines a day (which is what I expect of myself doing hand coding.) But I did have a very relaxing work-period. I really only put in about 2-3 weeks of effort. And those were only full days half the time.
|Project||Prior Total Lines||New Total Lines||Lines Added|
The testing for the removal of support for the Add/Delete operations on Cluster objects has been completed.
The Add/Delete support for Clusters has been removed from the Swing prototype GUI. You can't create arbitrary clusters. There is only the special "system" cluster and the default cluster this set of compute service nodes implements.
There was a missing "Type" suffix in the XSD for XMsg response documents, which has been corrected.
I still need to remove the "Delete Cluster..." menu item from the View/Edit forms.
This rebuild and repackaging with CFLib 2.1.12855 will prevent some bad data from occuring when encoding extended charcters of UTF-8 import data.
The remote file imports now work with the latest fixes to CFLib's XML utilities. I'm still not happy with the way I encode XML strings. Surely there is an API for doing that already instead of having to maintain my own broken code.
The correction applied to XML string conversion in CFLib 2.1.12847 should address the issue I was having with sending a request to the CFDbTest 2.2 server from the CFSecurity 2.2 web client GUI.
The file contents have to be loaded before they can actually be imported. Now I can start testing.
CFDbTest 2.2 now clean builds and is ready for use in running a web service on behalf of a CFSecurity 2.2 debug client.
CFSecurity now clean builds and is ready for use in testing a file import client against a CFDbTest web server.
The FileImport window is ready for testing.
With that last defect corrected, CFDbTest 2.2 now clean builds with the fileImport() support.
The log contents resulting from the import parse are now returned by the various fileImport() implementations. The XMsgRspnHandler has seen the addition of a LogContents attribute which is used to capture the output provided by a RspnFileLoaded message.
I'm ready to start working on the import window itself.
The core objects and interfaces have all been coded to support fileImport() requests from the client. Next I need to work on the FileImport GUI form, and then wire it up to the GUI itself.
The regression testing and corrections for the PostgreSQL SAX Loader has been regresssion tested and now loads the cluster and tenant information as specified through the updated command line argument parsing.
The SAX Loader's Cluster argument is now restricted to "system" or "default", and the Tenant argument subsequently restricted for a "system" Cluster as well (the "system" Cluster only contains one Tenant, the "system" Tenant.) The Tenant must exist within the Cluster prior to running a loader session for that Tenant; you cannot create a Tenant with this implementation of a SAX Loader.
Loading the client configuration object now clears the buffer variables for re-resolution by the next invocation of their getters.
The client configuration file is refreshed/reloaded when opening a login window.
See MSS Code Factory 2.1.12827 notes for details.
The Swing GUI prototype login form now uses a combo box to let you choose between the default and system clusters. If you choose "system", the Tenant is forced to system as well and made read-only. If you choose "default", you are allowed to specify a Tenant.
I've added a SchemaPooledObj which derives from SchemaObj, but does not disconnect the backing store during a logout. This prevents the leaking connections that I was seeing with the web services. CFDbTest has been used to exercise and verify the new code.
The HTTP client works! See MSS Code Factory 2.1.12821 release notes for details.
The dialect document for an X(ml)Msg response now properly identifies the document associated with the request, rather than defaulting to the hard-coded schema definition of the server. This will allow clients of downlevel projects (e.g. CFSecurity) to talk to a server that imports or references their schema definition.
The $SchemaName$XMsgRqstHandler has been updated to implement ICFLibThrowableFormatter, and to wire itself to the CFLibEnvelopeHandler that it is associated with whenever the envelope handler is specified. There are no changes required to higher level code such as the database-specific mains or the HTTP request processing page handler. There should no longer be unreported exceptions producing empty responses except for bad security key handshake data. But as long as you're relying on the CFLib implementation of the handshake, you should be safe from having to deal with empty responses.
The formerly Apache 2.0 licensed projects are now licensed under the BSD 3-Clause license so there are no issues with extending and modifying them as part of manufactured projects. The other projects have been updated to credit the BSD source code as required.
Since 2015.02.01, 3,287,736 new lines of code have been added over the course of 75 days, bringing the total up to 36,482,893 lines. That is an average of 43,836 lines per day, 1,826 lines per hour, or 30 lines per minute.
An upload of the source .zips and installer .zips weighs in at over 1.2 GB.
|Project||Prior Total Lines||New Total Lines||Lines Added|
Running under Ubuntu 14.04.2 on a P4 3.8GHz single-core system with 4GB of DDR2-800 memory and a 7200 RPM hard drive, it took 23.5 minutes to manufacture all of the 2.2 project source code.
Since 2015.02.01, 571,464 new lines of code have been added over the course of 74 days, for an average of 7,722 lines per day, 321 lines per hour, or 5 lines per minute.
|Project||Prior Total Lines||New Total Lines||Lines Added|
This is the test suite used to exercise MSS Code Factory 2.1.12808 Service Pack 1.
The code now uses "new String( Base64.encode( val ) )" instead of "Base64.encode( val ).toString()".
The Swing GUI prototype for PostgreSQL now works with the CFTip processing layers from CFLib 2.1.12803. You can log in to the database, log back out, and log in again. A quick run-through of the navigation windows works as well.
The login processing has been reworked to support CFTip. The code is almost ready for testing at this point; I think I still need to query the server information as part of the Swing GUI prototype login form processing in order to be able to issue a login request.
The security manager was not coded quite right for initializing the system cluster and system password (there was a nested system session problem.) This has been corrected, and the CFDbTest 2.2 Security Manager and XMsg Request WARs now function properly.
The CFTIPEnvelope Request submission form has been debugged and now send plain text exception identification and traceback information when there is an error processing the LoginRequest or AppRequest Payload, such as a decryption error.
The ServerInfoRequest is properly responded to with a text/plain document comprising of an XML response message suitable for processing. This is the only plain-text response the server normally produces; any other occurances indicate exception handling was invoked by the server, and that the exception message in the response text should be displayed to the user.
The build now expect commons-io-2.4.jar to be in the runtime path for all executables, and CFLib has been updated to version 2.1.12791.
Instead of expecting a base-64 encoded blob, the XMsgRequestXml processing page now expects the submitted request to be a CFTipEnvelope XML message, which subsequently contains a base-64 encoded message body being submitted.
The raw response from the processing of the message is passed back to the client as the plain text response to the POST of the form data for processing.
GET just returns a user interface form for doing test data submissions. The client web interface implementation will never be doing GET processing, only POST submission of message bodies for processing.
The LoginRequest and AppRequest pages have been consolidated into a single Request page that instantiates a CFTipEnvelopeHandler for the client session, relying on it to process the pages after converting the MessageBody back from base-64 encoded data to a byte array and from thence into a String to be processed by the EnvelopeHandler with the results passed back to the client as text/plain data (a base-64 encoded data block encrypted with the client-generated AES256 session key specified by the login request to the server.)
The LoginRequest and AppRequest forms now redirect to each other based on the state of the Authorization.
I haven't gotten anywhere near as far as decrypting the code block with the appropriate key information, as I haven't stored the client session key information in the server yet. I need to actually implement the new login XMsg request/response processing and propagate down to the database layer as appropriate.
The $lower SchemaName$xmsgsrvwar$PackedVersion$ project source tree has been added to the manufactured project JEE support. See the 2.1.12783 project notes for details.
The Security Manager web interface is now in sync with the code as implemented by the PostgreSQL Swing Prototype GUI.
The public device key is now displayed after it is retrieved from the keystore so that you can copy-paste the device information to the web forms of the Security Manager.
The public and private keys for the configured device name are now loaded from the JCEKS key store, with prompting for passwords as appropriate. The forms to prompt for the passwords are auto-launched when the application starts, and exit the application if they are cancelled out of. You must know the passwords for the keystore and the device key in order to use the application at all.
I did have to give up on implementing custom code to create self-signed client device certificates. You'll have to use the Java "keytool" to create RSA 2048 certificates (self signed is fine.) Don't forget to specify "-storetype jceks" and to make sure you're working with the keystore specified in the client configuration file. The name of the device should be used as the alias for the key you create.
I do still auto-create the keystore itself, though if the key is not found in a newly created keystore, you're given instructions on how to create a self-signed RSA 2048 key and the application exits.
The form conversation for logins now uses the client configuration file data to open or create the KeyStore as appropriate. If you cancel out of opening or creating the key store, the application exits so that you can modify the client configuration file by hand (yes, I should have forms for editing the attributes of the client configuration file, but it is only a prototype, not a full-featured application.)
The ServerInfo is now properly initialized during startup of the GUI. It takes several seconds for the server public-private key pair to be generated, so a message is written to the console while that initialization is happening, after which the GUI screens get displayed.
The PostgreSQL GUI client prototype now properly initializes the client configuration file and sets a reference to it in the Swing schema object so that it can be referenced. See the 2.1.12770 release notes for more information.
The security objects provided by CFLib's Tip package are now integrated with the Swing prototype GUI for PostgreSQL, though they are not properly initialized yet and there is quite a lot of work remaining before I can even think about testing any of the code.
I've also removed the XMsg Loaders from the project.
The X(ml)Msg Client handler has been updated to use the CFTipClientHandler provided as provided by CFLib 2.1.12763. The code does not work at this point because the server side still needs to be wired together, and because the login processing does not handshake the security keys yet.
The SysCluster object is a singleton that is a component of one and only one Cluster -- the one that will be created and initialized as part of the system password initialization in the Security Manager web interface.
The sample configuration for Tomcat now has an example configuration for the SSL support, and the Security Manager WAR web.xml now specifies that it must be mounted using https.
A seperate WAR will be created for the XMsgRequest servlet in the future, and will be mounted over http, as the bodies of the requests will be using CFTip to ensure the privacy of their contents.
The Security Manager Add Device page was not properly initializing the Cipher instance. However, seeing as the client code for providing a source for the copy-paste of the client's public key has not been coded yet, the point is moot -- you could only successfully add a device if you created some custom code.
It took 45 minutes to manufacture all of the Code Factory 2.2 projects, stripping away the BouncyCastle requirement. That works out to 12,294 lines per second. Not too bad for a creaky old 3.8GHz P4 with a paltry 4GB of RAM.
Since 2015.01.03, an additional 323,298 lines of code have been added during the intervening 29 days, for a new code rate of 11,148 lines per day on average.
|Project||Prior Total Lines||New Total Lines||Lines Added|
BouncyCastle has been stripped from the WAR installation requirements.
The manufactured version of the Security Manager web interface has been tested, and the hand written code has been removed from the project.
With the changes made to the encoding of passwords, the intent is that a client application will do the following, and send the resulting string to the server as the password:
MessageDigest msgDigest = MessageDigest.getInstance( "SHA-512" );
msgDigest.update( password.getBytes( "UTF-8" ) );
byte hash = msgDigest.digest();
byte encodedHash = Base64.encodeBase64( hash );
String hashedPassword = new String( encodedHash );
The server receives this encoded password string, and does the following:
byte encodedHash = Base64.decodeBase64( hashedPassword );
MessageDigest msgDigest = MessageDigest.getInstance( "SHA-512" );
msgDigest.update( encodedHash );
byte hash = msgDigest.digest();
String encodedAndHashedPassword = new String( hash );
The resulting string can then be compared to the database value. The purpose to this double encoding is so that after initially creating and setting a password, the client application can change the password and ensure that the server has never seen a version of the password that wasn't hashed, thereby obfuscating the user's password completely from the server.
Granted, few people are paranoid enough to care, but I wanted to take care of the wishes of those who do care. Of course, if you reset the password and enter one through the HTML Security Manager forms, the server sees your plain-text password again, so you'd want to use an application to change it again, but that degree of paranoia is left up to the user -- I'm not going to force you to change the password through the application interface.
The database creation scripts have all been tested with the changes made to CFSecurity 2.2, and the Security Manager prototype has been updated to incorporate the changes to SecUser.
There are no longer separate Html and App passwords, as it's my intent to double-hash the passwords in the Security Manager forms (TBD.) That way the client applications can hash the password entered by the user to protect it from interception until I have all of the encryption code working.
I've got the EMail encryption happening if you set the server variable CFDbTestSmtpEncryptEMails to "true" (set it to "false" otherwise), but the email body being produced by Bouncy Castle doesn't want to decrypt properly. I'm going to set that on a shelf for now; there may be bugs in Bouncy Castle. I'll know better when I start implementing the client-server communications, as that will be using Bouncy Castle on both ends of the link.
For now, you should just disable email encryption.
It took 45 minutes for MSS Code Factory 2.1.12736 to refresh all of the 2.2 projects, producing 32,871,859 lines of code for an average rate of 12,174 lines per second.
Five hours since starting that ManufactureCFAll22.bash run at around 20h20 this evening, I've built all the projects, packaged their installers and source distributions, tagged the releases in git, and am about to start the uploads to SourceForge for distribution.
|Project||Prior Total Lines||New Total Lines||Lines Added|
I reworked my approach to encrypting emails rather dramatically a few times over the past few days. I'm content that I'm using Bouncy Castle correctly now, as Enigmail is claiming that its decrypted the email.
However, Enigmail does not display the message contents. And if I try to copy-paste decrypt the message with GPG, it gets errors. It may be that Enigmail is masking the errors and silently refusing to display the content.
Regardless, I've made a lot of progress towards encrypted emails. It doesn't work yet, but it's coming along.
The email for a password reset request is now encrypted when re-sent, but the received email doesn't decrypt properly. It prompts for the correct key for the decryption, and it *looks* ok, but it doesn't work yet.
You can now copy a public key from the Thunderbird/Enigmail key manager to the clipboard, and paste that clipboard into the Public Key field of the Add New Device form. It will be parsed and validated by loading it into a null key store before it is allowed to be saved to the database.
Only PGP keys as supported by BouncyCastle.org will be allowed. But with the use of Thunderbird and Enigmail on top of the core GPG implementation, a fairly easy to install and configure user interface results.
I'll leave it up to you to search for the installation instructions for GPG, Thunderbird EMail, and the Enigmail add-on for using PGP keys with Thunderbird.
Once you have your private key configured and tested in Thunderbird/Enigmail, all you need to do is copy the public to the clipboard and paste it in the form.
Next I want to figure out how to send an encrypted email with an HTML email body. I believe S/MIME is what I need to look at next. The first place it should be applied is to the ResetHtmlPasswordRequest emails. I don't want just anybody to be able to intercept an email and reset someone's password. That's always bugged me about web enterprises -- they send unencrypted emails all over the place.
From what I've been reading, what I want to do is create a null key store, which can then load the text of the public key provided by the user through the Security Manager web interface.
The first instance to create is a null-keyed SecDevice bound to the "web" device name. Once this instance is presumed to exist, it can become an attribute of the SecSession, defaulting to "web" for the HTTPS-interfaced clients and the Security Manager web interface itself, relying instead on the HTTPS protocol configuration to secure the responses in transit.
I'm going to presume that the public key provided was produced by a Thunderbird installation with Enigmail installed and properly configured to provide secured email for the user's email account, and that the public key was produced by exporting an ASCII-armoured public key from the Thunderbird/Enigmail menus.
After loading the public key/ASCII-armoured text, you would need to resolve the loaded key in the null key store, parse out any device name associated with the key, and save the public key in association with the loaded device name as appropriate (a create or an update of a SecDevice instance.)
If the user specified a different device name, presume that they've copied the private key to that device's Java Keystore for the user, and create a SecDevice binding the new name to the same public key string/ASCII-armoured text.
After that, implement a new login API for the manufactured XMsgRqst interface to replace the one which currenlty expects a database-recognized client-server username authentication on a database-provider basis. The new interface will expect a username/email address, a password, and a device name.
On the client-side, when a login is attempted for those parameters, the client keystore will be probed for "username - devicename" as a private key provider. If the client keystore locates the private key, it will then step through a user-guided unlocking of the private key so it can be used to decrypt the message sent by the server and encoded by the public key.
On the server-side, when a login request is made for a specified device, the public key from the authenticating user's matching SecDevice entry is calculated and added to the Session data with the key $SchemaName$SecDevicePublicKey.
Whenever a secured response is to be sent to the client, all code then resolves the $SchemaName$SecDevicePublicKey rather than re-locating and unlocking the key.
When the user logs out, the $SchemaName$SecDevicePublicKey is cleared to null.
The first thing a user has to do after verifying their email address is to create a default device.
Right now the public key you specify for the devices are just unvalidated strings. I need to figure out how to convert the parameter string/text into a Java-useful crypto key to be used for sending responses to the client.
The client in turn will hold the private key for that public key in its Java Wallet, and will provide the appropriate forms and interfaces for opening the wallet, retrieving the private key, unlocking the private key with a password, and holding on to the result for use in decrypting the responses sent by the server.
The next step is going to be adding the crypto-key upload/conversion and validation for the public keys. What I want is to be able to export a Thunderbird Enigmail public key for the specified email and host name combination to an ASCII-armoured text file, which you then copy-paste the contents of into the device creation form's public key field.
If the uploaded key can't be parsed by the server, it will be rejected as an invalid key. Other than that, I don't care what level of security the user wants to use for their key -- that's their decision, not mine. I'll support anything Java supports.
Don't forget to download and install the Java Mail library into your Tomcat server's library directory, or you won't be able to run the provided WAR file.
The emails and forms for the HTML Password Reset conversation have been added to the prototype Security Manager web interface.
The EMail confirmation lifecycle changes have been applied to the Security Manager web interface. Now when you register a new account, an email with a unique UUID is sent to you. If you try to use the login before that link has been clicked, the email is re-sent. If you try to create an account for that same email id, the email is re-sent.
Only after you have confirmed your email address are you allowed to log in to the Security Manager.
The Security Manager Web Interface prototype code has been migrated to 2.2, the database refreshed with the new attributes mentioned in the 2.1 release history, the code all built, tested, and debugged. At this point, 2.2 is ready to take over for the 2.1 development stream for the web interface.
The migration and refresh were done in less time than it took MSS Code Factory 2.0 to refresh two sets of database install scripts on the speedy laptop. Such is the power of 2.1, which did the refresh of 2.2 in under 8 minutes.
This is the latest code as manufactured by MSS Code Factory 2.1.12725 with the addition of CFCore 2.1.12727.
MSS Code Factory 2.1 is about to go production...
This is just a test build; it doesn't get run. It's just to exercise MSS Code Factory 2.1, not for regression testing at this time.
This is the lastest code produced by the 2.1 development version of MSS Code Factory. With this clean build and some visual inspection, I'm now willing to sign off on 2.0 SP1.
I'm only going to run the tests for MySQL. The point was to verify that the manufactured code by MSS Code Factory 2.1 is equivalent to that of 2.0. That's been accomplished. I'll be setting aside the 2.2 code for now until I need to test some new functionality that is *only* produced by the 2.1 code base, not 2.0.