In the area of cyber threat intelligence, indicators based on deep understanding of compiled executables usually take a back seat to data artifacts such as log messages, configurations, file hashes, or network flow data. Binary patterns can be an incredibly useful tool to identify threats in executables and other contexts not normally considered.
Recently, New Context built a proof of concept that utilized ELK to store binary data from malicious files. We then used STIX™ patterning as a vendor agnostic repeatable method to describe and search for malicious content in these files.
Approach
- How to effectively use STIX™ patterns to search a database of binary observables.
- How to translate a vendor agnostic STIX™ pattern into a vendor specific format.
Results
- The proof of concept was able to prove that binary cyber observables could be stored in an Elasticsearch database.
- Secondly, the international standard of STIX™ patterns can be programmatically converted to ELK queries and retrieved at an acceptable speed for big data analytics.
Proof of Concept Implementation
New Context created infrastructure for an ElasticSearch, Logstash, and Kibana (ELK) stack. We were provided an example observable of malicious firmware.
In order to introduce randomness and diversity of data, the team created a tool to generate random binary data for the STIX™ artifact payload_bin and periodically included binary data that would match the supplied STIX™ pattern. These observables were submitted to the ELK stack. The modification consisted of unpacking the Base-64 encoded payload_bin and representing it as escaped hex values to allow Elasticsearch to index the data.
Once a collection of observables had been submitted to Elasticsearch, New Context used stix2patterns_translator to convert the STIX™ pattern to Elasticsearch queries to “find” any observable that matched the supplied pattern.
This operation of searching the binary data within ELK using a STIX™ pattern was proved to be fast and repeatable. Furthermore, by utilizing the STIX™ standard the use case shows that STIX™ can be utilized across many different vendor and technology stacks.
More on New Context and STIX™
New Context is considered the foremost authority in extending STIX™ (Structured Threat Information Expression) to support the needs of the electrical industry. Historical submissions to extend the standard for electrical utility uses have included:
- Author of the STIX™ patterning quick reference guide
- Created utility specific STIX™ extensions for DNP3 and ModBus
- Developed tools to represent ICS specific temporal event indicators of compromise
- Enabled mechanisms to perform multi-sensor correlation for OT networks
New Context is committed to the maturation of the STIX™ and TAXII cyber threat intelligence standards as a leading contributor on the OASIS Cyber Threat Intelligence (CTI) Technical Committee. Our integration and research services use these standards to create efficient systems for security teams that enable automation and orchestration for analysts and operations teams.
This article was co-authored by Christian O. Hunt, Principal Security Engineer at New Context (see bio below) and Andrew Storms, VP Product, New Context.
