Email this Article Email   

CHIPS Articles: NIST’s National Software Reference Library helps digital forensics experts expedite investigations

NIST’s National Software Reference Library helps digital forensics experts expedite investigations
By CHIPS Magazine - September 11, 2018
Most criminals today leave a digital footprint that can be used as evidence, from perpetrators’ laptops and cellphones to internet browsing and social media posts, law enforcement agencies typically can find a goldmine of criminal intent.

The National Institute of Standards and Technology’s National Software Library assists investigators in processing that evidence more quickly.

One of the largest software libraries in the world, NSRL archives copies of the world’s most widely installed software titles, and has expanded to include computer game software from three popular PC gaming distribution platforms—Steam, Origin and Blizzard, NIST said in a release.

The NSRL, which is maintained by computer scientists at NIST, allows cybersecurity and forensics experts to keep track of the vast and ever-growing volume of software on the world’s computers, mobile phones and other digital devices. It is the largest publicly known collection of its kind in the world.

The NSRL does not loan out the software in its collection. However, NIST runs every file in the NSRL through an algorithm that generates a digital “fingerprint” — a 60-character string of letters and numbers, also known as a hash, that uniquely identifies that file. Every quarter, NIST releases an updated list of hashes to the public. The list, which NIST calls the Reference Data Set, or RDS, can be freely downloaded from the agency’s website. The latest RDS contains more than 40 million hashes, including those for the recently added video game files, NIST said.

To professionals who work in the fields of cybersecurity and digital forensics, the world is an ever-increasing “ocean of digital objects,” NIST said. The RDS allows them to navigate that ocean and quickly find what they’re looking for.

If investigators seize a hard drive or mobile phone, for instance, they can quickly hash all the files on that device, then compare that hash list to NIST’s RDS. All the files that match can be typically ignored because they are known software files that wouldn’t contain information relevant to the investigation.

“After they filter out all of the known files, they’re left with everything that’s not recognized,” said Doug White, the NIST computer scientist who runs the NSRL. “Those are the files that might be interesting.”

Digital forensic investigators at all levels of government and in private industry rely on the RDS to efficiently manage their caseload. The NSRL contains operating system software, office software, media players, device drivers — all types of software files that are commonly installed on personal computers. In 2016, the NSRL expanded to include hundreds of thousands of mobile apps, which extended its usefulness to mobile phones.

The recent addition of gaming software to the NSRL reflects the growing popularity of that software category. “We’re not watching what gamers are doing,” White said. “But we need to include gaming software in the NSRL if we want to stay relevant.”

Among the video game titles added to the NSRL are the immensely popular "PlayerUnknown’s Battlegrounds," "World of Warcraft" and "Mass Effect."

Many of the titles were donated to the NSRL by Valve Software, which owns the Steam platform; Electronic Arts, which owns Origin; and Activision Blizzard, which owns Blizzard. Other titles were purchased if their install base was large enough to justify the expense. All titles in the NSRL are properly licensed and acquired.

While the NSRL exists primarily to support cybersecurity and law enforcement efforts, it is also considered a repository of culturally significant digital artifacts. While important books, films and audio recordings are preserved at the Library of Congress, the NSRL functions as a national software archive. Historians value this treasure trove of information because most of modern culture is both produced and consumed using software.

“Think of all the PowerPoints and Word documents that have tremendous historical significance,” said Trevor Owens, head of Digital Content Management at the Library of Congress. He might have added digital artworks, maps and interactive media. “Those documents might be lost, if future historians don’t have access to a comprehensive collection of software.”

An earlier batch of video games was added to the NSRL two years ago, including first editions of "Mario Bros.," "Asteroids" and "Sim City," preserving these retro titles and associated artwork for future generations.

Law enforcement professionals and digital nerds alike share interest in the software library, White said. “We preserve the software and make the RDS available to the public. The more people who find that useful, the better.”

To people who work in cybersecurity and digital forensics, the world is a vast and ever-rising ocean of digital objects. NIST’s Reference Data Set — a list of more than 40 million hashes, or digital "fingerprints” of known software files — helps them quickly find what they’re looking for. Credit: K. Irvine/NIST
Related CHIPS Articles
Related DON CIO News
Related DON CIO Policy

CHIPS is an official U.S. Navy website sponsored by the Department of the Navy (DON) Chief Information Officer, the Department of Defense Enterprise Software Initiative (ESI) and the DON's ESI Software Product Manager Team at Space and Naval Warfare Systems Center Pacific.

Online ISSN 2154-1779; Print ISSN 1047-9988
Hyperlink Disclaimer