The C++14 RAM storage layer clean compiles and packages without any unresolved symbols, but it will leak like a sieve as I've done virtually nothing about memory management, and the Java code base I started with doesn't deal with such things.
The RAM effort started on 2017-09-08, nine days ago. So for that period, I've averaged 608200 new lines of code per day, 25341 per hour, or 422 lines per minute. Not bad productivity, eh?
|Pct. Increase 29.6%|
The CFMessageLogWrapper was using the wrong signature for getBackLog(), resulting in link errors in the projects that use it.
The C++14 XML library was leaking like a sieve. That has been corrected.
Oddly enough, I hadn't brought the glue layer for BLOBs down to CFLib in the Java incarnation, because the APIs for doing data conversions were so easy to use. But the same is not true of C++, so I'll go the extra mile and make BLOBs full-class CFLibXmlUtil citizens. Blobs are implemented as vectors of BYTE.
The SAX2 XML Parsers provided by the [schema]xml[version] sub-projects have been ported from Java 8 with Xerces-J to C++14 with Xerces-C for Ubuntu/Debian amd64 platforms.
All code for this release requires CFLib 2.10.14094.
Work on this feature started 2017-08-27, nine days ago. The new lines of code were produced at an average of 151195 lines per day, or 6299 lines per hour of each 24 hour day.
|Project||Lines of C++14 Code|
The Xerces components have to be directly inherited. You therefore do not explicitly implement the ContentHandler interface in subsequent code.
The XmlCore objects have been coded and clean compile. As with all CFLib code, it is completely untested at this point in time.
The new objects are CFLibXmlCore, CFLibXmlCoreContext, CFLibXmlCoreElementHandler, CFLibCoreIso8859Encoder, CFLibXmlCoreParser, CFLibXmlCoreSaxparser, and ICFLibXmlCoreContextFactory.
C++14 now supports diamond inheritance of the models. CFUniverse won't be built, though, as that takes a couple of days and it really is a "for future implementation" project long after I've debugged the code (I want to have one server for all my projects save for Mark II, which has to use a Java server because of OpenNLP and WordNet.)
|Project||Lines of C++14 Code|
The Java implementations of the projects now provide diamond inheritance of the interfaces for the objects and tables so that you can cast them to any inherited project's interface safely, not just the corresponding DefSchema objects. DefSchemas are now largely irrelevant for Java; the way they're done has changed.
The corresponding C++14 changes and builds will be done some other day, so CFUniverse has no changes today and will not be uploaded.
All of the 2.10 projects have been clean built. Despite the complaints from lintian, they do specify a copyright as per dh_make requirements.
I won't be building CFUniverse again any time soon...
time ./PackageCFSecurityObj.bash 14084 real 31m47.830s user 22m48.396s sys 4m21.840s time ./PackageCFInternetObj.bash 14084 real 34m20.794s user 27m34.820s sys 2m58.216s time ./PackageCFCrmObj.bash 14084 real 30m43.694s user 25m29.868s sys 2m38.608s time ./PackageCFDbTestObj.bash 14084 real 350m43.929s (Sleeping while it ran.) user 195m59.664s sys 29m34.404s time ./PackageCFAccountingObj.bash 14084 real 93m12.509s user 78m23.872s sys 9m45.648s time ./PackageCFAsteriskObj.bash 14084 real 39m15.937s user 31m24.812s sys 4m36.448s time ./PackageCFEngObj.bash 14084 real 126m24.019s user 106m42.192s sys 12m29.380s time ./PackageMarkIIObj.bash 14084 real 207m6.695s user 175m25.212s sys 24m4.932s time ./PackageCFFreeSwitchObj.bash 14084 real 61m33.783s user 50m50.696s sys 6m41.272s time ./PackageCFBamObj.bash 14084 real 447m26.095s user 396m39.120s sys 35m59.428s time ./PackageCFUniverseObj.bash 14084 real 2554m43.563s user 2387m8.848s sys 143m28.748s
I decided to add some features before I embark on the build of CFUniverse, which will probably take over a day as it is over six million lines of code.
The other 2.10 projects have all been built and uploaded to the SourceForge download page.
All of that clean compiling code was produced in under 3 months, starting from zero lines of C++14 code.
|Project (* is packaged)||.hpp/.cpp lines|
time ./PackageCFSecurityObj.bash 14081 real 29m51.601s user 23m36.492s sys 3m40.604s time ./PackageCFInternetObj.bash 14081 real 41m24.393s user 27m52.664s sys 3m21.412s time ./PackageCFCrmObj.bash 14081 real 31m5.056s user 25m53.440s sys 3m2.492s time ./PackageCFDbTestObj.bash 14081 real 228m48.362s user 199m50.608s sys 19m35.200s time ./PackageCFAccountingObj.bash 14081 real 93m23.979s user 79m20.064s sys 9m59.400s time ./PackageCFAsteriskObj.bash 14081 real 41m17.221s user 31m20.024s sys 4m7.212s time ./PackageCFEngObj.bash 14081 real 133m31.288s (I didn't notice the password prompt for a while.) user 105m31.344s sys 14m17.184s time ./PackageMarkIIObj.bash 14081 real 263m39.351s (Fell asleep.) user 174m26.536s sys 22m56.676s time ./PackageCFFreeSwitchObj.bash 14081 real 67m16.107s user 51m0.360s sys 7m16.052s time ./PackageCFBamObj.bash 14081 real 462m46.108s user 404m19.604s sys 43m39.756s
The implementsClassCode( value ) method will be used to properly implement the qualified names.
CFDbTest, CFCrm, CFInternet, and CFSecurity 2.10 all clean compile and package now, with no unresolved symbols reported by shlibdeps.
There is no reason not to expect the remaining projects to clean compile as well, but I want to post a release before I make that multi-day attempt.
Since 2017-06-01 I have produced the following C++14 code. That is a total of 79 days, for an average of 195137 lines of code per day, 8130 lines per hour (presuming 24x7), or 135 lines per minute.
|Project (* is packaged)||.hpp/.cpp lines|
time ./PackageCFSecurityObj.bash 14078 real 30m17.909s user 24m25.784s sys 3m14.052s time ./PackageCFInternetObj.bash 14078 real 33m58.188s user 28m3.104s sys 3m32.972s time ./PackageCFCrmObj.bash 14078 real 32m7.648s user 26m9.992s sys 3m44.040s time ./PackageCFDbTestObj.bash 14078 real 234m47.877s user 206m55.528s sys 20m7.104s
The Range, Overflow, and Underflow exceptions all support 16, 32, and 64 bit unsigned integers now.
As per usual, there are issues with optional vs. required arguments in the manufactured code, which require dual method signatures, one for a reference and one for a pointer.
CFSecurity, CFInternet, and CFCrm are now all packaged cleanly. Those are the foundation packages upon which all MSS Code Factory manufactured projects are built.
CFLib now has a hash function for a vector of BYTE (which is used for BLOBs.)
I finally got the issues with CFInternet 2.10 resolved. Next I'll deal with CFCrm's use of Blobs.
CFSecurity 2.10 C++14 Obj layer has 298165 lines of .cpp and .hpp files time ./PackageCFSecurityObj.bash 14070 real 31m54.168s user 23m3.852s sys 5m21.592s CFInternet 2.10 C++14 Obj layer has 267136 lines of .cpp and .hpp files time ./PackageCFInternetObj.bash 14070 real 35m53.837s user 27m58.228s sys 4m54.092s
CFInternet 2.0 compiles, but there are still Debian packaging errors, so no binaries will be posted for CFInternet with this release, just for CFSecurity. I am posting a CFInternet source update, though.
time ./PackageCFSecurityObj.bash 14068 real 31m14.513s user 23m32.956s sys 4m4.644s
There have been a lot of changes to the code today, but CFInternet still isn't willing to build. But I'm calling it a day. Uploads to do, and then the night is mine.
time ./PackageCFSecurity210Obj.bash 14065 real 31m40.136s user 23m12.600s sys 4m46.064s
The joins in the Obj and EditObj implementations now use the Reference methods for optional target arguments, and the regular attribute accessors for required arguments. CFInternet 2.10 still doesn't build.
References have replaced pointers in most cases.
The Reference methods have been added to the Obj layer. That leaves one more step to implement, then I can test my solution to the latest CFInternet build problem. In a nutshell, C++ doesn't have automatic variable wrapping like Java does.
The rules have been updated to expect CFLib 2.10.14056, which allows the code to add the Reference methods to the attribute getters. The Reference methods return a const pointer to the value if the value is not null, or a NULL pointer if the value is null. This will take care of emulating the Java atomic type wrappers.
So far I've implemented the Reference methods for the base objects. Next I need to move up a layer and implement them for the Obj layer.
time ./PackageCFSecurity210.bash 14058 real 30m23.603s user 23m17.148s sys 3m30.856s
The nullable wrappers all have getReference() methods that return pointers to the data now. If the value is NULL, they return a NULL pointer.
CFSecurity 2.10 has been refreshed to use the lastest CFLib 2.10.14053.
time ./PackageCFSecurityObj210.bash 14055 real 32m27.310s user 22m7.592s sys 6m20.056s
Going crazy with the const specifiers, trying to make CFLib methods as tolerant of data constraints as possible.
The getValue() accessors now return const values. A niggling detail that was giving me grief.
There were a couple of cases where a const was returned by existing methods, causing an invalid conversion error (-fpermissive) when the arguments were passed to CFLibXmlUtil's formatting routines. Rather than fight it, I modified CFLibXmlUtil.
time ./PackageCFLib210.bash 14051 real 2m58.280s user 1m54.076s sys 0m36.432s
CFSecurity 2.10's C++14 code builds without unresolved shlibdep symbols. CFInternet 2.10 doesn't quite build yet.
time ./PackageCFSecurityObj210.bash real 31m54.330s user 22m4.976s sys 6m15.800s
The getClassCode() method of an ICFLibAnyObj is now a const accessor.
CFSecurity should build now that the Makefile.am has been corrected. I'm not going to wait to issue the refresh, though. The Java code remains unchanged since 2.10.14045, so it was not rebuilt, just repackaged.
The Java builds have all been refreshed, with repackaged downloads on their way to the servers shortly.
CFSecurity's C++14 implementation builds successfully.
It will be a while before I know if CFInternet C++14 builds or not. It is close if not there yet.
The implementation of Debhelper has been updated such that my old approach to installing header files no longer works. I've found a fix while working on the CFSecurity 2.10 builds using the new manufactured GNU/Debian build scripts, and have applied the same kind of changes to CFLib 2.10 so that it will build from source again.
Life, the Universe, and Everything...
CFSecurity builds successfully with the manufactured Debian build scripts.
time ./PackageCFSecurityObj.bash 14041 real 30m14.618s user 21m42.224s sys 4m59.948s time BuildCFAll210.bash real 9m8.091s user 32m35.332s sys 1m4.816s
The migration from CFAccounting 1.2 to MSS Code Factory CFAccounting 2.10 has been completed.
CFAccounting is being moved back into the 2.10 project family. The custom code hasn't been migrated yet; only the manufactured code and most of the scripts have been converted so far.
There were bugs in the narrowed relationship implementations for years and years and years and years...
MSS Code Factory 2.9.14030 has been used to remanufacture all of the 2.10 projects. It took a little work due to the changes to CFInternet before the 2.10 projects would compile again, but I got 'er done.
I'm taking a little bit of a break as I await parts for a new computer -- a nice, fast Ryzen 5 job. Parts will be trickling in for the next week or three before I have it put together and will be resuming coding. Now that a new machine is on the horizon, I don't have the patience to put up with the long builds on my creaky old laptop.
The C++14 code for all of the 2.10 projects have been refreshed by MSS Code Factory 2.9.14022.
The Blob data is now passed around as a vector of BYTE.
That is it for the initial coding of the base and obj layers.
SchemaObj has been coded and split into .cpp/.hpp implementation files.
TableObj has been coded and split into .cpp/.hpp implementation files.
All of the base object operators have been modified to take const arguments.
getObjQualifier has been corrected to take a string pointer as its first argument, not a string reference. The value has to be nullable.
The signature of getNamedObject() is supposed to take a string pointer as its first argument, not a string reference. I'm surprised I wasn't getting more complaints from dh_shlibdeps about this.
EditObj has been properly implemented as pairs of .hpp/.cpp files.
I've decided I want to map the primitive parse trees of OpenNLP to the grammar-based parse trees of CFEng (which is based on Oxford rules, if I recall correctly.)
In the long run, I'd also like a C++14 tool chain that can populate the CFEng structure from an evaluation of English text. Then I can go either route into the database to parse sentences for subsequent evaluation.
Then comes the long, long, LONG term fun stuff -- extracting meaning from the grammatically structured text, both on the sentence level and on a higher associative level of a conversation per archived message posting. Maybe eventually I'll want to tie together conversations on an associative basis as well, to get "focus groups" of knowledge about given subjects.
CFEng.Sentence has been renamed CFEng.Sent, with a global patterm match substitution of "Sent" for "Sentence" in the model file. This will result in some bizarre help text for now, but I don't really care much at this time. I'll clean it up later.
The C++14 code has been manufactured by MSS Code Factory 2.9.14005, and packaged with an updated source distribution .zip file. The builds are not being refreshed, as the Java code remains unchanged since the last build. I've been entirely focused on C++14 lately.
The Obj has been implemented. There are major changes throughout the code base.
The run with 4GB of CAS11 memory ran in under half an hour:
real 27m41.104s user 22m55.704s sys 1m26.524s
Surprisingly enough, the run with 8GB of CAS9 memory took longer:
real 33m20.575s user 27m8.560s sys 1m42.916s
Inconsistencies similar to those of the use of the time_points were discovered for the uuid_t, so I implemented the same solution: making all references use pointers.
Being inconsistent between the methods of required and optional attributes caused problems for code equivalency, so I opted to go with a full-scale shift from object references to object pointers for the time points.
I missed the const specifications on getClassCode() and getGenDefName(). My bad.
There are too many cases where I need a mutable object in order to evaluate toString(), so I made the assumption that the same issues would affect things like getObjName(), and made all of the methods apply to mutable objects.
The public static strings S_EMPTY, S_TRUE, and S_FALSE have been added to CFLib.
I decided to treat the object attributes as normal string objects instead of const references, because someone has to be able to destruct the instance, and that only happens with non-const objects.
I still wasn't satisfied with the interface that was specified.
The ICFLibAnyObj specification has been changed in accordance with the way they it is actually used and implemented by the C++14 code.
With the implementation of SchemaPool, the base layer of code is complete.
The base Schema has been implemented.
Only the Schema remains to be coded for the base layer, then it is on to the Obj layer.
I've installed my replacement hard drive, reinstalled the OS, restored the packages I had installed on the system, and restored my projects. I've rebuilt everything that is important to me to verify the recovered system, and all is good.
I've learned a few tricks since the last time I had a hard drive go bad on me. This time it only took four hours to go from fresh hard drive to functional system instead of 2-3 days.
Now when the mood strikes me, I can get back to coding.
By the way, a 7200 RPM drive vs. a 5400 RPM drive only saves about 5 minutes on what was a 30 minute C++14 build, but saves roughly 20% of the time for the build of the Java 2.10 projects.
I have some CAS 9 memory on back-order as well, which will replace the CAS 11 that is currently installed. It will be interesting to see what that does to my build times as well.
Perhaps I'll be able to live with this laptop longer than I'd expected. I certainly don't have any more cause to worry about it just dying on me, as I had been recently. It was the age of the hard drive that was worrying me, as it gets rather heavily abused and was roughly 8 years old, suffering near daily heavy abuse.
The ConfigurationFile has been coded and clean compiles.
In compliance with the reworked version of CFLib 2.10, the code has been switched from using std::u16string to std::string, and the CFLib conversion routines that used to exist have been eliminated.
I've also removed the "using namespace std" statements, and instead explicitly refer to the std namespace as per normal C++14 coding conventions.
It would seem that the standard for C++14 is to use std::string() to contain UTF-8 strings. The only reason I'd been using std::u16string() is because Java uses UTF-16 internally. But it doesn't matter to me, so I'll go with "normal" coding conventions instead.
I've never used the Cursors, so I removed all support for them from the code.
The .cpp files for the DefaultFactory implementations have been created and clean-compile.
All of the manufactured code is now in the "codefactory" namespace.
The object interfaces have all been coded and clean compile as well.
The IEditObj interfaces are now specified and clean-compile.
CFSecurity 2.10.13974 is now packaged as libcfsecurity by the Debian debhelper packaging system.
IObj has been coded and clean compiles.
The base interfaces have been specified and implemented. ISchemaSchema.hpp is incomplete, though.
The ISchemaTableTable.hpp and ISchemaTableTable.cpp files clean compile.
The foundation objects are now fully-fleshed and ready to participate in C++14 standard containers.
That is 189,539 lines of C++14 code in about a month. Not bad productivity, eh?
The assignment operator overloads have been corrected to return *this instead of void.
The warnings from the build have been cleaned up and toString() has been implemented for Buff and HBuff.
The C++14 code produced so far by MSS Code Factory clean compiles and links into a library. Only the headers included by the .cpp files so far get evaluated; I know there are errors in the other ones.
I need instance pointers from getUTCGregorianCalendar(), not temporaries.
Perhaps a quarter of the code now compiles before I encounter problems in the assignment operators that replace the set() methods.
The UUID hash has been overloaded with a version that takes a reference to a uuid_t instead of a const pointer.
The CFLib hash methods have been fleshed out to allow for a variety of invocation styles.
The build is very close to succeeding. There are still issues with the hashCode() methods.
The get time method needs to return a pointer to a time, not a temporary object reference.
CFLib now has a uuidClone() method and one for allocating a new timestamp of the current system time.
I still need to initialize the specified time value; this code doesn't work yet.
CFLib now has a uuidClone() method and one for allocating a new timestamp of the current system time.
I still need to initialize the specified time value; this code doesn't work yet.
Many build errors have been fixed and the Buff.cpp files almost commpile, but not yet.
I think I've got the headers and implementations for the Buff, HBuff, PKey, HPKey, and IndexKey all straightened out and ready to do some build testing.
The HBuff.cpp implementations have been split off.
The buffer object has been split. The header for HBuff is ready to be split next. Then I'll have to deal with the index keys.
The skeletons for the SchemaTablePKey, SchemaHPKey, and SchemaTableHPKey all look good for now. Some of them even load without errors in Eclipse, though some insist that std::u16string isn't defined. Go figure.
All of the 2.10 projects have been remanufactured by MSS Code Factory 2.9.13932, rebuild using the latest CFCore 2.9 release, repackaged, and the archives updated to include the not-building-yet C++14 source code to date.
I've only been concerned with the code produced for the ClusterPKey so far, but it looks good.
The packaging names for CFLib no longer follow the MSS Code Factory naming conventions, but instead follow the default naming conventions for Debian Linux development and packaging.
I need a convenient way of initializing UUID members.
Eventually I want to manufacture these new files.
There are now subdirectory Makefile.am files for cfsecurity and cfsecurityobj.
All you can really do with them is "make install-HEADERS" as part of a deployment.
I realized you need a Nullable wrapper for booleans as well.
The interfaces and object specifications have been migrated from Java 8 to an initial C++14 skeleton, whose bodies comprise the original Java 8 code. This code is no where near ready for build testing.
Behold the first manufacturing run of C++14 code for MSS Code Factory... :)
Don't use new and then return a dereferenced variable; use proper temp variables. I don't know if it matters, but I was concerned about potential memory leaks with the old code. Now the formatters and string converters follow normal C++14 coding conventions, so I know there won't be a problem.
The wrappers have been reworked as Nullable objects, including a base Nullable class.
The idea is to have instances of the Nullable objects as permanent members of the manufactured code objects, rather than allocating and deallocating little objects all the time.
Then member accessors will bring forward the use of those objects as follows:
const NullableType& getAttribute() const type_t getAttributeValue() const bool isAttributeNull() const void setAttributeNull() void setAttributeValue( type_t value )
Playing with pointers is clumsy and crude, so I knocked together some Java-style basic type wrappers. They aren't as rich as Java's, but they do what I actually use in my code: construct with a value and get the value.
The corrections to the model saving for the editor have been migrated forward from 2.9.
The 2.9 enhancements that support the C++14 code have been brought forward from CFBam 2.9.
The member attributes for the Cpp/Hpp equivalents to the J(ava) code customizations have been added to CFBam 2.10, but the corresponding custom code has not been added yet.
Note that as will CFBam 2.9, the "bootstrapBindings()" method has blown the Java limits and has to be manually split into two separate methods before it can be compiled. I haven't done that for 2.10 yet, as I want to focus on the 2.9 custom code before I worry about building 2.10; I just wanted to get the skeleton of the new code checked in. This code will not compile as a result.
Instead of implementing equals() and compareTo(), C++14 requires that you overload the member operators <, <=, == !=, >= and >. Similarly, a template instantiation approach is used to inject hash implementations into the std:: namespace. This is going to bulk up my code a lot, but it will get rid of the dynamic_cast invocations I'd been using in the prototype code so far.
Several sections of code have been reworked to allow for the latest CFLib changes. The code that has been migrated to date all clean compiles.
The ICFLibAnyObj interface now returns std::u16string instead of a pointer for the toString() method, as every object is required to respond to toString() with some sort of value. Returning NULLs is not acceptable, and so has been prevented entirely.
CFSecurityHPKey.cpp has been coded and clean-compiles.
The code depends on libuuid1, not uuid.
The CFSecurityCluster*.cpp implementations have been migrated.
At this point in time, I'm focused on CPP Proto, which is prototyping a migration of the Java foundation layer code to C++14 for Debian-based amd64 Linux, with just enough of the obj layer interfaces to be able to build the foundation layer.
Once that is complete, I'll take a step back and enhance MSS Code Factory 2.9 to support the additional tags required for the Cpp and Hpp equivalents to the Java attributes of the SchemaObj and Table definitions in a Business Application Model.
I don't look forward to enhancing the code for the editor, though. That is going to take a while, even with substitutions by Eclipse editors. Maybe I'll just use vi to do the editing instead; it is a pretty simple copy-paste-substitute pattern to be followed, just in a number of segments of code. I'm going to presume a 1:1 match between Java and Hpp/Cpp tags for now.
Then it is time to work on the rules for producing the foundation layer code, including makefile support if I can.
All of the interfaces defined for cfsecurityobj have had their headers migrated. The implementations (.cpp files) haven't been migrated yet.
The Obj.hpp and EditObj.hpp interface headers in cfsecurityobj have been migrated, and are now properly specified as pure virtual interfaces.
The TableObj.hpp and SchemaObj.hpp interfaces have not been migrated yet.
The headers now use int[16/32/64]t for attributes of the objects, but not for more generic interfaces like compareTo(). Generic types are only used for system and infrastructure APIs now, not application data.
CFSecurityAuthorization has been migrated, which required an initial migration of the Obj and EditObj headers from the second layer of code (cfsecurityobj.)
The interface headers for the foundation layer compile cleanly now.
Slowly but surely I will work my way through this pile of code. Man, this sure has been a reminder of how much grunt work is involved in C++ coding...
The header files for the interfaces of the lowest level code, cfsecurity, have been migrated. At least I think they have.
Massive changes, but I'm still not even done migrating the headers from the Java source, never mind tackling the .cpp files.
Use the dh_make "-p" option instead of moving directories around. That also makes it easier to establish directory references from other projects.
The prototype code has been switched over to using the new CFLib code. It is no where near building, of course, but I've started on the migration to using the new code.
All of the functionality required for CFLib has now been coded, though none of it has been tested. At this point, the APIs and interfaces have been stabilized, and only additional functionality should be added rather than subtracting from the current code base or modifying it substantially.
The CFLibXmlUtil::formatTimestamp() method is now used in the messaging for the Overflow, Underflow, and Range exceptions instead of printing the raw seconds since epoch.
Next I need to deal with my timezone information. I've got the settings retrieved, but I need to figure out how to translate that string into a timezone offset value using standard Linux facilities. I don't care about Windows. This is NOT intended to be cross-platform code. I write for Linux. I don't care about any other platforms.
I noticed that many of the other packages I've worked with return instances instead of pointers when allocating new objects. This semantic seems clear and obvious in retrospect, as Java has a one syntax fits all approach to references, pointers, and instances.
My presumption is that instances returned/allocated within a method are automatically destructed when that method loses scope. You have to "catch" the new instance with a pointer dereference if you want to keep it beyond the scope of the method.
The semantics of the XmlUtil parsers is NOT changing, however -- they continue to return pointers to objects so that they can indicate an empty parse by returning a NULL. The semantic of the formatters has been changed to always return a temporary string instance instead of an instance you have to manage, though.
Getters also return constant pointers so that the caller does not own the reference, and can not alter or destroy it due to the use of the const semantic. I thought about changing those to const references, but I realized that doesn't fit with the way the manufactured code for MSS Code Factory works -- it is heavily reliant on pointers that can be late-resolved, which means there is (at least temporarily) no instance in existence to reference.
I went through all the headers and their corresponding implementations, looking for "orphaned" methods that didn't have matching declarations, or which were declared but not implemented. In some cases, there was just a mismatch of getters and setters corrected, and there were a lot of missing destructors added as well.
I am now comfortable that I have no missing implementations for any of the members or methods of my declared C++14 interface.
There are also no unresolved symbols reported any more.
The instance getter methods are now tagged with "const" so that you can get the attributes of a const reference, but not set the attributes. As Java has no concept of a const object reference, there was no support for this syntax in the original Java code.
Next I want to go through the interfaces and implementations and look for any methods which have been defined but not implemented.
The namespace has been changed to cflib for the sake of consistency with the directory name the include files are found in, the name of the installed libraries, etc. Everything shall be just so, shall it not?
There is a source archive created by the build process that is not a complete set of everything included in the full project source archive. It really should only be used for the sake of providing debug symbols/source code to display when tracing code.
Just a couple of things that were bugging me.
Simply running a "./PackageCFLib210 nnnnn" will clean the builds, move the directories, edit the version tagged files, and run a clean build with production of the .deb package as its ultimate goal.
I love Linux. You can automate just about anything if you're good with command line tools.
The proper version numbers for the shared libraries are now used, although the build system produces shared libraries named "*.so.0.0.0" as the primary copy of the library, to which all other versions reference by symlink.
Similarly, I've overridden the rule from the default Makefile in my Makefile.am to copy the libraries using version numbers instead, and to use spawned shells to establish the symlinks.
The header files have all been switched over to using a "#include <cflib/CFLibBlah.hpp>" syntax.
The main .deb file for libcflib-dev looks like it is a reasonable size for the compiled debug code with symbols included. A stripped file would be roughly 1/10th the size.
The .dsc seems rather small, though. I don't think it is including all the source code, only the headers.
This is the work in progress on getting proper integration with the debian build process. I'm still having trouble getting the shlibs packaged for distribution, but other than that, I seem to have resolved all of my problems.
I am so close I can taste it. But it is late and I'm going to bed.
A .deb file gets produced, but it is too small. It should be 4MB or larger, the same as the installer was. I still have more work to do. In the meantime, you can build the source yourself.
The deb helper build prepares the code and everything, but it fails when trying to package the results for distribution, and I haven't figured out why yet.
The Deb helper skeleton has been filled out, and you can now follow the procedures up to and including a debuild. However, I'm not sure where it ends up putting the built files or the packaged .deb. Maybe I'm still missing steps. More reading will happen shortly.
The dh_make produced skeleton of debian configuration files has been added to the project.
I'll need to modify them accordingly, and continue to work my way through the Debian packaging tool chain. Eventually the "PackageCFLib210" script will take the source tree from source to built and packaged, ready for distribution.
Then I'll finally have a template project for "How To Build A Debian Library" that I can use for my various component libraries of a manufactured project. I want complete integration of the C++, GNU, and Debian tool chains for the manufactured code.
The debug build has been packaged for distribution.
I want full debug information for the GNU tool chain.
The GNU auto tools have successfully produced a shared library in the .libs subdirectory of the build.
The Makefile syntax errors have been resolved, though I'm not entirely sure how/why, but I'm not arguing with progress.
You can now do the usual steps to produce a Makefile from the repository source files, but the Makefile produced has some lines around 480 or so that have a prefix of a space instead of a tab. Manually edit to correct that, and the build still fails.
But it is attempting to build code. I'm that much farther along than I was this morning.
The version numbers used by the various scripts and utilities have been synchronized to 2.10.13863.
The initial GNU auto tool scripts have been added to the project. You can do an "autoreconf -fmi", an "autoconf", an "aclocal", an "automake --add-missing-files", and then an "automake" that does nothing further.
I need to work my way through the auto tool documentation to figure out how to get the tools to package a shared debug library for distribution. I'm hoping the tools like my 2.10.release version number for the library.
The CFLib 2.10 source tree for C++14 has been split off from the C++ prototype project as the standalone project net-sourceforge-MSSCodeFactory-CFLib-2-10.
CFLib 2.10 is the C++14 migration of CFLib 2.9 for Java 8. This a pre-production release for internal use only; use at your own risk.
All source is built using the GCC 6 tool chain on Ubuntu Linux 17.4 amd64 with near-default compiler options. As far as I can recall, only one special flag was added to the Eclipse build rules to support initialization of constant statics for classes and to enable virtual inheritance throughout the code base.
This is the alpha release of the CFLib 2.10 code migrated to C++14 from Java 8.
The migration of CFLibXmlUtil is complete. I need to look into the Java code and see if there were any other utility or support classes I missed that I need to migrate sooner rather than later. But this may well be the alpha release of CFLib 2.10 for C++14.
The const void* Blob methods have been implemented, leaving the XML string parsers. You don't normally need the XML string parsers because the SAX parser processing does the XML string translation for you.
The uuid_t methods have been implemented, along with a wealth of additional type-specific parsers. That leaves the XML string parsers, and the Blob methods. I need to find a base64 encoding/decoding solution...
Only the Blob and Uuid parsers and formatters remain to be coded for CFLibXmlUtil. Then I will be done this big class migration.
The majority of the CFLibXmlUtil methods have been migrated to C++14.
There are still the parser and formatter methods for uuid_t, float, double, and mpfr::mpreal types to be implemented. Once those remaining methods are implemented, I believe CFLibXmlUtil will be complete.
I'll have to give the code a good run-through to make sure I haven't omitted the implementation of any methods.
Two more CFLibXmlUtil methods have been migrated to C++14.
Over half of the methods for CFLibXmlUtil have been migrated to C++14.
This coordinated release is a clean build of the CFLibXmlUtil functionality so far. You can not access most of the methods of CFLibXmlUtil.hpp yet because the implementations are commented out until I get around to migrating them.
I was able to get the system environment into UTC time, save the local timezone, initialize the globals for CFLib related to time, and implement parseDate(), parseTime(), and parseTimestamp() for CFLibXmlUtil without resorting to the Boost libraries. As dealing with Gregorian dates had been my only concern, I no longer need Boost and will be removing it from my system.
The build is only clean because I've decided to comment out a bunch of stuff with #ifdef 0 in CFLibXmlUtil.cpp.
CFLibXmlUtil has been sketch-migrated from Java to C++14. I believe the header is ok, but there is still a lot of work even to get the method arguments straight for the implementations.
There is now a debug library existing on my system for CFLib 2.10 as compiled and packaged by Eclipse CDT.
I believe the successful production of a shared library also means you've resolved all of the symbols in the library. i.e. There are no missing method implementations or constant initializations.
Several more exceptions have been migrated, and the destructors for all the exceptions are at least stubbed as empty methods.
The source for the migration of the Java CFLib 2.9 source to the C++14 CFLib 2.10 library has been appropriately renamed.
There is no way I can see to migrate these classes, as they rely on Java's ability to query the class that defines an object. C++14 uses a different technique, where you attempt a dynamic cast and a NULL result means the cast couldn't be completed.
There is no way to dynamically specify the target class of a dynamic cast in C++14, so you just can't get there from here.
Fortunately there were no uses of either class directly in CFBam 2.10, so I was able to get rid of the classes. If they turn out to be used by the internals of some part of CFLib or CFCore, I'll deal with it in some other fashion.
The CollisionDetected exception has been tweaked, and the EmptyArgument and InvalidArgument exceptions have been coded.
CFLibCollisionDetectedException, CFLibDbException, and CFLibDbUtil have been migrated. That leaves 18 more objects to deal with on the initial migration of the CFLib exception framework and essential utilities.
The initial migration of the Overflow, Range, and Underflow exceptions has been completed. In the future, I'll add support for construction with int8_t, uint8_t, uint16_t, uint32_t, and uint64_t arguments to flesh out the type support for the "normal" set of C++14 integer types.
I just continue to make progress with whipping the code for CFLib into shape.
There is still a long way to go.
The CFLib class headers have been reworked to allow for memory management by switching most instances of const std::u16string* to const std::u16string&
The CFLib interfaces now specify virtual destructors, and the string members are now owned instances of the objects, as are any ICFLibAnyObj instances that were passed as arguments to them.
In cases where const ICFLibAnyObj* had been passed to the exception constructors, the const has been removed. The exception implementation takes ownership of the object and will destroy it accordingly; once the constructor returns, the caller should not destroy the instance that was passed as a construction argument.
In the case of strings, a copy of the argument string is taken, so the arguments to the exception constructors are still designated as const, indicating that the caller is still responsible for destroying that instance.
I've also removed the CFLibBigDecimalUtil.hpp file. I'm not ready to start migrating utilities unless they're used by the implementation of the exceptions.
Next I need to refresh the .cpp files to add implementations of the destructors, take copies of argument strings, and destroy the string copies and ICFLibAnyObj* members in the destructor implementations.
Java also assumes all objects are virtual, so I start right from the core of ICFLibAnyObj. That is where I gather the "common functionality" I expect all my object implementations to provide, which is based in the most part on the Java Object contract.
I coded a couple more CFSecurityCluster objects and interfaces.
All objects produced by MSS Code Factory will respond to the extended interface of ICFLibAnyObj compared to the Java interface.
Java already provides the additional methods as part of the generic Object contract, but C++ has no such niceties.
Well, at least one object clean compiles now. It isn't a small one, either.
I'm thinking of making the buffers derive from ICFLibAnyObj, and having them always return false when asked if they are an instance of a class, and to return nulls for the navigation methods because buffers don't know anything about objects.
The same would apply to the keys.
If HBuff and HPKey aren't descending from ICFLibAnyObj, they need to do so as well. Except that HBuff returns the historical resolutions of the containership hierarchy, which may result in null resolutions of current data if an instance has been deleted.
That way the compareTo() and equals() methods would be valid for those data types.
Then I can add the compareTo(), equals(), hashCode(), and toString() templates to the ICFLibAnyObj interface.
The source all clean compiles, but Eclipse CDT won't turn it into a shared library for some reason. I may have to just write my own makefile. No big deal. It isn't like it would be the first time.
Of the pieces of CFLib that I've ported to C++14 so far, the CFLibFiringList.cpp file is the only one with an error. All of the headers are clean. But I've thought about it, and I can't think of a way around this problem.
The exception hierarchy and the factory constructor for exception instances has been ported to C++14.
Just a checkpoint of the work in progress for CFLib 2.10.
I've started working on migrating CFLib in the cfsecurity210 project for now; later I'll move the code to the separate library I have in Eclipse and figure out how to reference a library from another project with CDT.
The .cpp skeletons have been copied from the originating Java 8 source, and decorated with the includes and namespace references. A lot of work has been done, though much remains before I'll have a clean build.
The headers now compile. Remember I am not doing a full migration of CFSecurity by hand. I've migrated a set of core objects, and will now proceed with the C++ implementation of those core objects.
The Java String references have been replaced by the C++14 equivalent, std::basic_string<char16_t>.
UUID has been replaced by uuid_t.
The CPPProto project will be a hand written partial migration of the CFSecurity Java implementation to C++14 for AMD64 Linux systems with the GCC 6 tool chain.
Once I am satisfied with the prototype, I'll start working on rules for MSS Code Factory 2.9 to produce the core object libraries and a thread-safe Ram implementation.
The extensions .hpp/.cpp are the best option for file names for C++14 nowadays. There was a time when I used .hxx/.cxx, but I think it was the Microsoft tool chain that liked that. I'm not targeting Windows at all. Either implement standards or bugger off.
The hashmap of primary-keyed objects would be of shared_ptr references, owning the objects. The index maps would use weak_ptrs, as would the collections contained by the objects themselves. The objects would have a shared_ptr reference to the collection members themselves. I haven't decided who owns the PKey instances yet. I may need to implement a detach() method if I haven't already done so in Java. I forget whether I did or not.
forget() is different -- it deallocates the member that is in the hashmap of primary-keyed objects, so that none of the shared_ptr references can be resolved. When a sharted_ptr is unresolved, the read() for the non-null instance is performed automatically to create a cached copy of the referenced instance.
I'd need to seriously restructure the code for the core cache, to deal with allocation and deallocation properly. I think I can isolate the deallocation to forget() with some careful code structuring.
For the TableObj implementations, the code would look something like:
HashMap<PKey,std::shared_ptr<Object>>, and the indexes would be
The TableObj is responsible for allocating and deallocating copies of the Objects PKey member for it's collection member keys.
The Ram database thread-safe as well, by grabbing mutexes over the members of the RamXxxTableObj.cpp implementations via the RamXxxTableObj.hpp interfaces.
I do not want to thread lock over any finer granularity than a table, as Ram databases are only intended for very simple applications, like editors.
There will be no ACID considerations for the Ram implementation in C++ any more than there are for the Java version. I just want to be able to make the C++ code thread-safe.
C++ on a 64-bit platform will allow me to escape the bounds of the 32-bit JVM, which is a limitation right now.
I can't implement a Java version of CFUniverse, which is to be the conglomerate of all MSS Code Factory 2.10 projects except for CFDbTest, including CFAccounting and Mark II. I want a *global* server image that you run specialized client interfaces against.
There is no need to rework the databases, only the database interfaces. I need the Ram implementation first.
In the long run, I want to fully automate the manufacturing and build processes under a C++ implementation. I'm even considering creating an MSS Code Factory 2.10 as a C++14 implementation. We shall see.
I know I'll need to modify MSS Code Factory itself to define the C++ implementation and interface customizations in a similar fashion to the JavaXxx attributes of the Table and SchemaDef specifications. They will be names CppXxx, though there may be fewer or more customizations available.
I'm hoping to emulate the Java code structure as much as possible, with one C++14 library per package, though of course the directory structure for the cplus rule base and implementations will be different from the Java versions, as you don't specify full ownerships with C++ code. C++ just has the .cpp files in the package directory, and an include subdirectory with the .hpp files.
This is going to be a fun year, I think. This is going to take a long time to learn what I need to know about modern C++ and implement the prototype, never mind coding the rule base.
There was still a lowercase table name being used by the delete scripts in the ClearDep implemenation. This has been corrected, and verified through painful testing with the Mark II Chatbot.
There was a problem with the MySQL rule set for MSS Code Factory, which has been corrected (join tables were being specified with lowercase names instead of mixed case.)
The source code for the MSS Code Factory 2.10 projects has been successfully extracted from the GitHub repositories into a clean directory tree, remanufactured by MSS Code Factory 2.9.13823, clean built, and repackaged as a refresh deployment.
It turns out that commons logging doesn't seem to like Log4J 2.x.
All of the databases should now have code that supports the self-referencing DelDeps.
You should also be able to specify self-referencing ClearDeps, with the same restriction that only the terminal node of the dependency chain can be self-referencing.
As far as I know, only the Mark II project actually makes use of this new feature at this time.
The MySQL connector has been upgraded, and all of the relevent scripts in the 2.10 bin path have been updated to reference the new jars, as have the build.xml scripts.
MySQL 5.7 syntax is now supported and has been verified under Ubuntu 16.10.
The DelDep relationship chain can now terminate with a self-referencing relation, so that you can build trees of nodes. I don't think it will work with subclass hierarchies, though, only flat tables. This feature is only used by the Mark II project at this time, not the 2.10 projects.
The SAX RAM loaders will now execute properly because they initialize the security information for the connection to the RAM instance. The requirement has changed -- you must always use connect( "system", "system", "system", "system" ) for RAM databases.
The copyright headers have been updated for 2017.
The build and packaging scripts have been corrected to include the XML package and the SAX RAM Loader package.
All of the projects now support RAM database imports of the standard loader files of the manufactured SAX XML parser. This code was manufactured by MSS Code Factory 2.9.13795.
The adjusted RAM implementation has the same login/logout and transaction semantics as a persistent store now, though it cannot actually roll back failed transactions.
You can now let the security forms reference a RAM schema instead of an XMsgClient, and after validating your access to the application's security keys, you are automatically logged in as system/system against the RAM database without providing a user name and password.
All of the 2.10 projects have been remanufactured, rebuilt, and repacked using the code produced by MSS Code Factory 2.9.13789. Although the bugs corrected by 2.9.13789 should not affect the 2.10 code base, I like to remanufacture the entire set of code produced by the tool when a major defect is corrected, just in case.
CFLib and CFCore are now released under an Apache V2 license, so the 2.10 projects have been rebuilt and repackaged to leverage the relicensed code.
All of the 2.10 projects are now under the Apache V2 license.
I realized that while I am a GPL enthusiast, most of the third party code I rely on is under the Apache license. I felt the 2.10 projects would find wider acceptance with the new license.
The Apache jars have been updated to the latest versions, and CFLib 2.9.13778 and CFCore 2.9.13779 were used to build and package this latest release.
The packaging scripts have been corrected, and the builds repackaged. The naming of the HTTP jars had changed.
MSS Code Factory CFAsterisk, CFBam, CFDbTest, CFEng, and CFFreeSwitch are now distributed under a GPLv2.0 license instead of the former LGPLv3 license. As the sole contributor to the maintenance of the code base, I can make this license change without having to get agreement from a developer community.
After some consideration, I realized that the LGPLv3 attempts to restrict the use of code in cloud environments. The cloud is a fact of life. Any attempt to restrict the use of code in the cloud is asinine.
All of the projects are now built and shipped with the LGPLv2 versions of CFLib and CFCore as well.
I intend to explore the information available through the WordNet 3.0 database with the 3.1 files overlaid on top of the 3.0 core. You will find the extracted files in the bin/data/WordNet-3.0 directory of the installer and source tree.
The initial skeletal model of the English parse objects has been completed and is ready for review and experimental use.
This is a clean compiling version of the core objects for CFEng. There never will be a persistence layer for the parse objects, so only the core jar and the Ram storage jar are produced.
This is the first cut of the English syntax model, CFEng. All I can really say for it is that it manufactures code. It is not a valid model for doing development with yet.
I'll use the in-memory storage and services to build the parser. This is the "knowledge structure" that the parser component will be trying to build in-memory as its "understanding" of a piece of text.
All of the third party libraries have been updated to the latest versions as delivered/supported by Ubuntu 16.04 LTS.
I need to create a fresh work environment from the git repositories. I've extracted the git repositories and updated their configurations, including wiring a new SSH key from this new development installation. Keys should be refreshed instead of brought forwards from time to time.
The initial goal of this project is a command line utility which lets you specify the old and new models to be compared for creating diff scripts. I can load entirely separate copies of the old and new versions of a database model (Business Application Model.) In fact, it will be quite trivial to do, though I'll obviously have to make some assumptions about prev-next/old-new argument ordering for the command line API.
That will give me two in-memory images of entirely separate models. So now I need to compare and diff the the two models, tying them together with optional attributes referencing the "DefiningXxx" of the current object. There would be a GetDefiningXxx()/SetDefiningXxx(Obj value) accessor pair for the defining object, which is a reference from the new model to the old model that it is defined by in the comparison. There are "GetPrevXxxAaaa" accessors used to expose the previous Xxx attributes specified. You aren't expected to use Setters through these APIs, so only Getters are wanted -- there are no SetPrevXxxAaaa( Obj value ) accessors. There should be un-decorated GetXxxAaaa() getters on the object that retrieve the new/current values of the attributes in question.
Once all of the old objects and attributes have been mapped by setting the DefiningXxx of all the objects between the compared models, you are left with a list of the objects that have been removed from the old system in the new model, and a list of new objects that have been added to the new model that weren't in the old system.
Creating the drop scripts and create scripts for the database changes alone becomes a relatively trivial exercise.
Another change is I need to add a database version string singleton to be stored indicating the current release level of a given database instance.
I'll also need to add a command line argument for specifying the output directory where the new database scripts are to be written. I don't want to make assumptions about the naming, as I'm providing flexibility with the inputs so I don't want to get stubborn about outputs now.
There are no changes to the code when remanufacturing with the Oracle 8 JDK. This contrasts with earlier results from a few years back, when Oracle's JDK was producing invalid results for manufacturing runs. The bug has been fixed during the intervening years; Oracle's JDK now runs for all known test cases.
Rebuilt with Oracle Java 8 JDK for Linux 64-bit instead of OpenJDK 8.
Remanufactured by MSS Code Factory 2.9.13739 and rebuilt by OpenJDK 1.8.0_91 using CFLib 2.9.13737 and CFCore 2.9.13738.
I decided I was abusing PageData for CFCrm 2.10 just because it was easy to do, so I reverted back to non-paged data for those tables. Paging is a clumsy user interface that should only be used when absolutely necessary.
Contact lists and their components do not grow without end like accounting data does, so it makes more sense for people to split up their contacts into multiple lists if they are having performance problems than it does to force them to click a "more data" button over and over during normal use of the application(s).
The latest build as manufactured by MSS Code Factory 2.9.13731-SP2 provides full support for page data by the JavaFX ListPaes, PickerPanes, PickerForms, and FinderForms.
Development will now focus exclusively on Code Factory Accounting 1.2, with any bug fixes to MSS Code Factory itself rolled out as they are encountered. However, all of the new code has been tested, so I am not expecting any bug fixes in the near future. But then again, what programmer expects bugs in their code?
The PickerPage and PickerForm now support page data for JavaFX.
The common lookups defined by CFSecurity and CFInternet no longer enable page data, so all of the databases for the 2.10 projects need to be recreated. This version of the 2.10 applications is not compatible with earlier database models.
The ListPane objects for JavaFX now automatically implement page data support.
There was an occurance of a reference to the 2.7 CFBam schema in the CFBam Editor's save/export code. This has been corrected to reference the 2.10 version of the schema.
There were focus propagation errors in the JavaFX ViewEditPane code which have been corrected. This was preventing the display from being properly refreshed after certain data changes were applied.
The latest code as manufactured by MSS Code Factory 2.9.13721 Service Pack 1 adds support for the PageData="true" attribute of a model's tables. See the notes for the 2.9 releases for details.
With the completion of the XMsg enhancements for the page data methods, the new code is now ready to be tested.
All of the databases now have the code in place to invoke the new stored procedures for paging data.
I still have to do the Oracle JDBC enhancements for the new stored procedures, but the code has been written and clean-compiles for DB/2 LUW, MySql, PostgreSQL, and SAP/Sybase ASE.
All support for Microsoft SQL Server has been removed from all of the projects because I have no way to test that database at this time, so I can not have it on the official list of supported vendors any more.
All of the 2.10 projects now clean build despite the changes I'd made to the Java implementation at the foundation of the system, and despite the significant number of CFSecurity, CFInternet, and CFCrm table objects had PageData="true" added to their specifications. I wanted all of those big fat lookup and security tables to be pageable. It is unlikely anyone really wants to looks at that raw data, but I make it possible to do so without hitting the network for a full set of umpteen records every time they open a record browser over a global lookup.
From here on in, all 2.10 projects are affected by changes to the page data implementation, because even CFSecurity has tables with page data enabled.
The PostgreSQL JDBC implementation now defines the new prepared statement variables for the new page data queries, and cleans them up as part of the releasePreparedStatements() implementation for the table.
The Obj layer enhancements have been completed and now support the new data paging interfaces. The back end implementations are still not-implemented-yet stubs, but the supporting logic has been coded.
Each of the persistence layers implements a set of not-implemented-yet stubs for the new paging interface methods. The interface specification itself should remain stable while I flesh out the implementations.
The Oracle version of the page data read procedures install without errors, as verified through CFAccounting 1.2.
The SAP/Sybase ASE page data queries now restrict their result sets to 25 rows.
The SAP/Sybase ASE version of the page data read procedures install without errors, as verified through CFAccounting 1.2.
The MySql verson of the page data read procedures install without errors, as verified through CFAccounting 1.2.
The DB/2 LUW version of the page data read procedures install without errors.
The new PostgreSQL data paging stored procedures install successfully now.
The new PostgreSQL stored procedures are ready for a test install. They have been wired to the crprocs_schema.[bat|bash] scripts.
The new sp_page_dbtablename[_all|_by_suffix] stored procedures have been coded for PostgreSQL, but not installed to the database nor executed yet.
The PostgreSQL rules for the crsp_page_dbtablename_all.pgsql file make use of the new GEL spread statement, and a manual code review has been done on that stored procedure. Note that the new procedures are not wired in to the bash scripts yet, so they don't actually get created.
All of the database stored procedures now sort the duplicate key result sets by the descending primary key for the table being queried.
This is the latest code as manufactured by MSS Code Factory 2.9.13673.
Only the primary key index has to be in descending order when PageData="true" is specified. The other indexes over the table can be ascending or descending according to the whims and wishes of the designer.
Clean build of all 2.9 projects as manufactured by MSS Code Factory 2.9.13666-PROD.
CFFreeSwitch 2.10 builds clean.
CFAsterisk 2.10 builds clean.
CFDbTest 2.10 builds clean.
CFCrm 2.10 builds clean.
CFInternet 2.10 builds clean.
CFSecurity 2.10 builds clean.
Initial code from manufacturing run for 2.10 projects by MSS Code Factory 2.9.13658.