Email this Article Email   

CHIPS Articles: JAIC Issues Understanding AI Technology Guidance

JAIC Issues Understanding AI Technology Guidance
By CHIPS Magazine - April 22, 2020
The Defense Department’s Joint Artificial Intelligence Center has released “a concise, practical, and readable overview of Artificial Intelligence and Machine Learning technology designed for non-technical managers, officers, and executives,” according to the document introduction.

Written by the JAIC’s Greg Allen, Chief of Strategy and Communications, the document explains AI concepts for the benefit of senior DoD leadership who may be called upon to make decisions about AI implementation without a basic understanding of how AI works and what it can do.

Under the direction of the DoD Chief Information Officer, the JAIC, stood up in 2018, and is charged with accelerating the transformative potential of artificial intelligence technology in support of national security efforts, in areas such as manned and unmanned systems; intelligence, surveillance and reconnaissance; inventory and logistics planning; and much more.

Although AI has been around for decades, breakthroughs over the last 10 years have greatly increased the diversity of applications where AI is practical, powerful, affordable and useful. Most of the recent excitement focuses on advances in machine learning (ML), which is a subfield of AI.

Machine Learning is closely related to statistics and allows machines to learn from data. The best way to understand Machine Learning AI is to contrast it with an older approach to AI, Handcrafted Knowledge Systems, Allen wrote. These systems use traditional, rules-based software to codify subject matter knowledge of human experts into a long series of programmed language rules: “if given x input, then provide y output.”

Machine learning systems are different in that their “knowledge” is not coded by software programmers. Rather, their knowledge “is learned” from data: a machine learning algorithm running on a training dataset produces an AI model, Allen wrote. Largely, machine learning systems program themselves. Still, humans are critical in guiding the learning process. For example, humans choose algorithms, format data, set learning parameters, and troubleshoot problems.

Handcrafted Knowledge Systems are the older of the two AI approaches, nearly as old as electronic computers. At their core, they are merely software developed in cooperation between computer programmers and human domain subject matter experts. Handcrafted Knowledge Systems attempt to represent human knowledge into programmed sets of rules that computers can use to process information, Allen explained.

A well-known example of a Handcrafted Knowledge AI System in widespread use is tax preparation software. By requiring users to input their tax information according to pre-specified data formats and then processing that data according to the formally programmed rules of the tax code (developed in cooperation between human software engineers and accountants), the output can be good enough to pass an IRS audit. When first introduced in the 1980s, tax preparation software was very successfully marketed as artificial intelligence, Allen wrote.

AI is now ripe for further advancement thanks to the ever-increasing availability of massive datasets, massive computing power (from using GPU chips as accelerators), the proliferation of cloud computing, open source code libraries, and software development frameworks, illustrated in Figure 1. Enabled by these advances, the performance and practicality of using machine learning AI systems has increased dramatically, Allen offered.

There are four different families of machine learning algorithms described in Figure 2. It is important to understand the different families because knowing which family an AI system will use has implications for effectively enabling and managing the system’s development.

  1. Supervised Learning uses example data that has been labeled by human “supervisors.” Supervised Learning has incredible performance, but getting sufficient labeled data can be difficult, time-consuming, and expensive.
  2. Unsupervised Learning uses data but doesn’t require labels for the data. It has lower performance than Supervised Learning for many applications, but it can also be used to tackle problems where Supervised Learning isn’t viable.
  3. Semi-Supervised Learning uses both labeled and unlabeled data and has a mix of the pros and cons of Supervised and Unsupervised learning.
  4. Reinforcement Learning has autonomous AI agents that gather their own data and improve based on their trial and error interaction with the environment. It shows a lot of promise in basic research, but so far Reinforcement Learning has been harder to use in the real world. Regardless, technology firms have many noteworthy, real-world success stories.

Utilizing Deep Learning (Deep Neural Networks) is a powerful machine learning technique that can be applied to any of the four above families. It provides the best performance for many applications. However, the technical details are less important for those not on the engineering staff or directly overseeing the procurement of these systems. What matters most for program management is whether or not the system uses machine learning, and whether or not the selected algorithm requires labeled data.

Allen cautioned organizations should not pursue AI for its own sake. Rather, they should have specific metrics for organizational performance and productivity that they are seeking to improve. Merely developing a high-performing AI model will not by itself improve organizational productivity. The model has to be integrated into operational technology systems, organizational processes, and staff workflows.

Almost always, there will be some changes needed to existing processes to take full advantage of the AI model’s capability. Adding AI technology without revising processes will deliver only a tiny fraction of the potential improvements, if any.

Traditional project management wisdom still applies, cautioned Allen. Many AI projects fail not because of the technology, but because of a failure to properly set expectations, integrate with legacy systems, and train operational personnel.

A good example for setting success is to ensure data is authentic, accurate and properly labeled. Figure 3 provides a simple illustration of labeled and unlabeled training data for a classifier of images of cats and dogs. Depending upon whether or not data is labeled, a different family of algorithms applies. For example, if the goal of the AI system is to correctly classify the objects in different images as either “cat” or “dog,” the labeled training data would have image examples paired with the correct classification label. Supervised Learning systems can also be used for identifying the correct labels of continuous numerical outputs. Figure 4 provides an illustration of supervised and unsupervised algorithms.

Talented human capital and access to AI experts are critical factors for success in DoD’s AI strategy, Allen wrote. Still, the basics of AI technology can be understood by anyone who devotes the time to learn. The concepts in this document provide a technical overview that will be adequate for the vast majority of senior leaders to understand what would be required to adopt and utilize AI for their organization.

More in-depth explanations are included in the document as well as additional resources for learning. Those who want to go further and learn more are encouraged to do so, Allen wrote.

Figure 1. Key Factors Driving Recent Improvements in ML Performance by the JAIC
Figure 2: Simplified JAIC Diagram of AI Approaches
Figure 3. JAIC illustration of Labeled and Unlabeled Training Data
Figure 4. JAIC illustrated example of Supervised and Unsupervised Algorithms
Related CHIPS Articles
Related DON CIO News
Related DON CIO Policy

CHIPS is an official U.S. Navy website sponsored by the Department of the Navy (DON) Chief Information Officer, the Department of Defense Enterprise Software Initiative (ESI) and the DON's ESI Software Product Manager Team at Space and Naval Warfare Systems Center Pacific.

Online ISSN 2154-1779; Print ISSN 1047-9988
Hyperlink Disclaimer