Organizations succeed when they are able to make informed decisions across their business. As Information Technology continues to play a vital role in providing the data that management requires to take critical decisions, understanding the application portfolio that generates this data is more important than ever. It is this “knowledge” that enables agile modifications to be made to the portfolio, resulting in rapid responses to business, technology and regulatory changes.
Independent research conducted in October 2014 showed that around 70% of organizations did not have published or referenceable documentation on the current version of their application portfolio. Of this 70%, 63% admitted that if key personnel who held this knowledge were to leave, the information void created would be very difficult to manage.
The results stemming from this lack of documentation have been alarming.
- IT managers have been unable to realistically budget for the cost and time of making changes and bug fixes to their application portfolio because:
- Changes and bug fixes are taking longer to implement than they should
- Code modifications have been error-prone, resulting in delays and cost over-runs
- Operations personnel have been unable to react in a timely manner to changing business conditions and regulatory changes; and
- Management has been unable to effectively plan portfolio rationalization, optimization and modernization
The effect: Reduced efficiencies and slow-down in the adoption of new methodologies and technologies, ultimately leading to increased costs and missed business opportunities.
As an example of the potential slow-down in the adoption of new methodologies, consider the concept of DevOps that is being seriously considered by many organizations to increase co-operation and communications between their IT Department and Operations personnel. DevOps is expected to ensure that both the existing applications, as well as new ones under development, execute business processes and rules that are consistent with company policies; and in a manner that also meets the expectations of operations, users and support personnel. The success of this methodology hinges on the IT Department’s ability to share the logic, processes and rules of all their applications with other departments via robust documentation. This means (i) having documentation available of the current version of all applications and (ii) having this documentation in formats that other departments can review and understand.
What is the primary problem with keeping software documentation current?
Application life-cycles can be 20 to 30 years or longer, and during this time, there are too many changes and fixes made to keep the original documentation updated manually.
EvolveWare’s Intellisys Platform™ automates the extraction of meta-data from software applications to generate:
- An inventory of all files that constitute an application
- Logistical data (such as lines of code, commented lines, executable lines etc.)
- Program complexity metrics down to sub-routine levels
- Details of data entities, including the CRUD activities performed by programs on entities.
- Details of variables
- Identification of dead (unused) and redundant code
- Linkage diagrams between files (JCLs, Screens, Programs, Procedures etc.)
- Linkage diagrams between sub-routines within programs
- Program flow diagram
- Data flow diagram (data manipulation by programs on entities)
- Data model in a relational format
- Business logic in pseudo code that can be annotated in plain English, flowchart, decision tables, XML and CSV formats
Given the myriad of changes made over an application’s lifecycle, what is the solution to keeping software documentation current?
Implement a solution that keeps the documentation updated automatically.
What type of documentation must be generated so that it can be shared across departments, and understood by all?
Personnel in different departments have backgrounds ranging from subject matter experts, to general programmers, to analysts, and to business managers. The documentation should be generated in multiple formats that include pure code for subject matter experts, pseudo code in a neutral language for general programmers, flowcharts and decision tables for analysts, and plain English annotations of pseudo code for business managers.
ADABAS/Natural | ADSO | ASP | Assember | C | C# (.Net) | CA Gen | COBOL (AS/400, HP3000, z/OS, Unisys, etc.) | Easytrieve | Forte | FoxPro | Java |
PACBASE | PL/1 | PL/SQL | PowerBuilder | T SQL | Universe Basic | VB 5.0
& 6.0. | VB (.Net)
Files Inventory | Source Code Logistics | Source Code Complexity | Dead Code Details | Data Dictionary | System Details Diagram | Business Logic Connectivity Diagrams | Data Flow Diagram | Program Flow Diagram | Data Model in SQL/DDL Format | Program Logic | Business Logic.
Rename variables so analysts can understand their roles in the business logic | Rename business logic to better describe their functionality in business processes | Annotate programs & business logic to describe their functionality in plain English.
Why is the Intellisys Platform unique?
While the generation of artifacts listed above provide a detailed view into the functionality and processes of a software application, the Intellisys Platform’s ability to keep documentation updated automatically is even more valuable. The platform allows users to re-process code after every change or fix, automatically append the changes to the existing repository of information, and then re-generate the same artifacts with the updated information. Additionally, the platform generates a report of all the changes identified during code re-processing.
This ability to keep documentation current also provides enormous benefits when training new personnel to support the same applications, and creating more efficient multi-disciplinary Application Management and Development teams.
The benefits do not end there. The Intellisys Platform allows users to (i) modify variable names and business logic titles to reflect their respective roles in the application and (ii) add notes in plain English to provide explanations at both the program level and the business logic levels. This allows the documentation to be shared with analysts and managers so that they understand the applications’ actual functionality.
Finally, the platform provides advanced features that allow users to run impact analysis for required code changes, derive business rules from the extracted logic, optimize the extracted meta-data, and transform the meta-data to modern languages or third-party platforms like BRE and BPM products.
In summary, by using the Intellisys Platform™ organizations can:
Maintain a fully-documented application portfolio
Significantly reduce the time and cost of performing updates and fixes
Provide a detailed insight to all departments of the policies that are being implemented and
Use the platform’s advanced features to rationalize, optimize and modernize their applications to newer and more efficient technologies.
UK Government Agency
Country or Region
Major Areas Served in this Study
Documentation & Migration.
Adabas/Natural – 4000+ Natural objects plus associated maps
The UK Government’s system for licencing and authorization was written over 20 years ago in a language that is difficult to support.
The system utilized work files with local and global datasets interconnecting the applications to execute desired business processes. The business rules were embedded in the code which was primarily data driven, making it difficult for the application to be maintained and extended to serve the changing and growing business needs of the department.
User access to the system was provided through terminals located geographically across the country. System calls to external applications were made a significant feature of the application.
Over the years the system had been maintained by several different groups within and outside the department, further increasing the code complexity. This made the code patchy and difficult to maintain.
Complex as the code was, the cost to maintain the system skyrocketed as the availability of support personnel began to decrease. There was no current documentation for the system and the only way to understand the logic written over the years was by reading any available inline comments.
This unreliability and risk had to be mitigated as quickly as possible by documenting the code’s business rules and database. This task would also be a pre-cursor to moving the application to a modern language with a relational schema.
Other business drivers that supported this decision were the rising cost of the mainframe, a dwindling pool of support staff, the inability to extend the system to other departments within the Government and the need to centralize databases.
In addition to extracting the business rules and the database schema, it became important to review and modernize the extracted rules so that specifications for a new system could be generated. The development of the target system was subject to budgetary approvals.
Extract the business rules and data structures from a legacy application written in Adabas/Natural to a format that was easily understood, maintained and potentially enhanced.
The agency began soliciting for solutions and awarded a contract for the documentation and business rules extraction to a group of service providers. After 2 years the project remained incomplete, having gone through multiple methodologies that included the use of some custom automation tools. Based on undocumented estimates, the cost of this exercise had been in excess of £1 million (U.K. pounds).
EvolveWare was approached to use its Legacy Modernizer product to extract the system’s business rules, optimize and modernize them and export the resultant rules into MS Word and Flowchart formats.
Project Deliverables & Time Lines
The project included the following deliverables:
- Business Rules documentation for 1.2 million lines of code.
- Relational database schema as SQL/DDL scripts.
- Optimization and modernization of extracted business rules to make them business analyst friendly and utilize them to derive target specifications.
- Export business rules in MS Word and Flowchart formats so that they could be archived for audits and meeting compliance requirements.
The code was processed and the business rules extracted, modernized and exported in 6 months. A total of 15 personnel worked with the product’s console to derive, optimize and modernize the extracted business rules. The following statistics provides the calendar time spent on each of the major tasks performed in the course of this re-learn and modernization project.
- Business logic and data model extraction from source code – 4 weeks.
- Derivation, optimization and modernization of business rules to generate target specifications – 14 weeks.
An effort that had not accomplished its objective for almost 2 years was completed in about 6 months. The business logic and data model were extracted from the source code with 100% automation and the product’s web-based console allowed the service provider to use personnel from different locations to derive, optimize and modernize the extracted business logic into business rules and processes.
The cost of licensing the Legacy Modernizer was miniscule as compared to the amount that had been spent for an unsuccessful effort using other methodologies.
The immediate benefit was the reduction of cost to maintain the system. With the ability to view and analyze the system’s business rules, the agency’s ability to make changes to the system increased dramatically. Another advantage was that the agency was now going to be able to meet audit and statutory regulations of the Government.
Perhaps the biggest benefit would be derived in the future when the agency migrated the system to a modern platform. Without the benefits of documentation this would have been an extremely difficult and risky task.
EvolveWare, Inc. has been awarded 5 U.S. Patents and 1 Australian Patent for the technology that is incorporated into Legacy Modernizer and its successor, Intellisys™. The U.S. Patent numbers are 7,769,704, 7,774,290, 7,917,457, 8,051,410 and 8,412,653 respectively. The Australian Patent number is 2007348312. No part of this document may be copied, circulated or reprinted in any form without the prior written permission of EvolveWare, Inc. For questions and additional information please e-mail [email protected] or call +1.408.748.8301
It is amazing how quickly the agency was able to document, modernize and create target specifications for this large mission critical application when over 2 years had been wasted in failed attempts using other methodologies. The speed and cost at which this project was completed surpasses anything we have seen in areas on re-learning and documenting applications.