AFRL: Software and Systems Test Track

Sponsor: Department of the Air Force, Air Force Materiel Command(AFRL)

It is increasingly difficult to create software to deal successfully with increased device and system complexity. Moreover, networked distributed systems create vast increases in the scale and scope of information processing applications, exacerbating the challenges to the system engineers' ability to specify, design, build, verify and test software. This situation is an emerging issue in information technology in general, but the requirement of military systems set them sharply apart from non?military applications in terms of reliability, robustness, security, interoperability and real time operation.

Business, government, and technical endeavors ranging from financial transactions to space missions increasingly require complex software systems to function correctly. The complexity of the software arises from stringent requirements (e.g., for reliability in performance and integrity of the data used), the need to support a range of interactions with the environment in real time, and/or certain structural features. These attributes make software difficult to produce.

Major hardware-software failures in defense acquisition programs have occurred both because components were expected to interoperate properly and they did not, as well as the fact that tools did not work as advertised. Interoperability is required for network centric environments but reality has shown it very difficult to achieve. A scalable, flexible, realistic synthetic testing environment is required for stressing tools against key benchmarks, assessment of the utility of tools by key program offices as well as use as a synthetic environment for testing tools against large systems (tens of millions of SLOC or larger) or systems-of-systems.

Objective:

This program will be acquired in two Phases. Phase I will be the definition phase and will consist of defining, developing, and documenting the Concept of Operations (CONOPS), a user-oriented document that describes system characteristics for a proposed system from the users' viewpoint; and Defining, developing and documenting the architecture and the fundamental organization of the Systems and Software Test Track as embodied in its components, their relationships to each other and the environment, and the principles governing its design and evolution. Initial concepts along with the final CONOPS and architectures will be presented to representatives from Government, Industry and Academia and therefore, the technical data rights proposed need to be consistent with this requirement. Phase I is the area that we are soliciting white papers for now.

Phase II will be the environmental development and operations phase. Additional information will be issued around the January 2007 timeframe concerning Phase II and white papers for Phase II will be solicited at that time.

The overall objective of this Systems and Software Test Track BAA program is to provide an open framework environment where an assortment of experimental tools and products may be deployed and allowed to interact in real-time, interactive, evolutionary and interdependent means, thereby allowing rigorous testing of new technologies, methodologies and theories in support of the Software-Intensive Systems Producibility Initiative. The Systems and Software Test Track will facilitate testing of Software-Intensive Systems Producibility research products and methods, provide an environment for research of DoD embedded systems and software problems, provide an ability for university and industry leverage of technology development, and establish a capability for successful technology transition and transfer.

The environment should be open and available for use by developers as well as independent analysis by the facility operators. This independent analysis allows the facility operators to be supportive to major defense acquisition program offices as well as analyzing the utility of tools. Program offices may bring their unsolved problems to the test track for either help in solving or by looking for needed utility amongst the tools available. Lastly, this synthetic environment should provide a place where big codes can be tested in the loop allowing requirements verification prior to production and deployment. This risk reduction affords the ability to verify and validate functionality of today's complex software intensive systems while providing a realistic environment for researchers to verify their tools against realistic problems.

The Systems and Software Test Track should also provide a place (possibly virtual and not a single physical location) for experimental verification of Software-Intensive Systems Producibility technologies due to their novelty and the potential complexity of the underlying theories. The experimental platforms should incorporate software technology to instrument, monitor and test large-scale applications. Challenge problems for the open experimental platforms should be made accessible for all the research teams. The experimental platform research should include subtasks to conduct large-scale coordination experiments, and to develop methods and tools for evaluating aggregate performance of applications. This environment should provide a full range of collaborative technology challenges, run-time platforms and applications, experiments, evaluations, and demonstrations. A Common infrastructure will enable control and data flow between both kinds of application components for a distributed environment. The open experimentation environment will provide the fundamental reference architecture and underpinnings helping researchers to develop and test their designs as well as facilitates transition of promising technologies into production use.

Research Concentration Areas:

The goal of the Phase 1 research is to

• Define, develop, and document the Concept of Operations (CONOPS), a user-oriented document that describes system characteristics for a proposed system from the users' viewpoint; and
   

• Define, develop and document the architecture and the fundamental organization of the Systems and Software Test Track as embodied in its components, their relationships to each other and the environment, and the principles governing its design and evolution.

The CONOPS document should be written to communicate overall quantitative and qualitative system characteristics to the user, buyer, developer, and other organizational elements. It should describe the system, operational policies, classes of users, interactions among users, and organizational objectives from an integrated systems point of view.

Areas to consider when developing the CONOPS include, but are not limited to:

• Intellectual property issues;
   

• International Traffic in Arms Regulations (ITAR) and export control issues;
   

• Tool Evaluation;
   

• Features and methods to promote and monitor practitioner and researcher interaction as well as providing academic researchers access to large complex software systems;
   

• Technology Transition/Transfer of improved embedded software component integration tools and techniques into military system development programs;
   

• Supporting facilities and staffing, location(s), networking, firewall and policy related issues.
   

• The Architecture definition should provide the ability to understand and control those elements of system design that capture the system's utility, cost, and risk. These elements could be the physical components of the system and their relationships, the logical components or enduring principles or patterns that create enduring structures. The definition shall provide a rigorous definition of what constitutes the fundamental organization of the Systems and Software Test Track embodying all information regarding elemental relations, interactions and interfaces.

Areas to consider when developing the Architecture include, but are not limited to:

• The environment, virtual or not, to host, execute and test technology, both hardware environment (run time platform) and software environment (run time platform);
   

• Interfaces and collaborations;
   

• Mechanisms to support research and analysis of DoD problems including, but not limited to: software artifacts, benchmarks, executables, source code, design documents, requirements documents, examples, models, fault data, lessons learned, software construction files and tools;
   

• Data repositories for results, success stories, benchmarks, quantitative and qualitative results, software disaster studies defining problems and basic research areas, ability to upload/download artifacts;
   

• Measurement Techniques and Software Forensics;
   

• Metrics including reduced development time; ease in which domain experts and software engineers can interact; ease in which different domain experts can specify and design code independently of one another; usability, i.e., the 'naturalness' of the modeling language from which code is generated; ability of test engineers to modify and tune code in the field; accurate automated documentation in design
   

• Mechanism for studying innovative systems and dynamic processes, not static snapshots

Amount: Total funding for this BAA is approximately $18M. The anticipated funding to be obligated under this BAA is broken out by fiscal year as follows: FY 06 - $1.0M; FY 07 - $1.0M; FY 08 - $6.1M; FY 09 - $4.9M; and FY10 - $5.0M. Individual awards will not normally exceed 6 months with dollar amounts ranging between $300K to $400K per year for Phase 1 and will not normally exceed 18 months with dollar amounts ranging between $700K and $3.0M per year for Phase II. Awards of efforts as a result of this announcement will be in the form of contracts, grants, cooperative agreements, or other transactions depending upon the nature of the work proposed.

Deadline: It is recommended that white papers be received by the following dates to maximize the possibility of award: FY 06 should be submitted by 30 March 2006; FY 07 by 31 January 2007; FY 08 by 31 October 2007; FY 09 by 31 October 2008 and, FY 10 by 31 October 2009. White papers will be accepted until 2:00 p.m. Eastern time on 30 March 2010, but it is less likely that funding will be available in each respective fiscal year after the dates cited.

For further information, visit: http://www.fbo.gov/spg/USAF/AFMC/AFRLRRS/Reference-Number-BAA-06-13-IFKA/SynopsisP.html