Strategies in Intelligence Gathering

Juan… But why?

There are many writeups and training strategies for gathering intelligence, but often they are either so specific that the techniques cannot be applied in the same process to other situations. At the other end of the spectrum, some write ups are so abstract there is no guideline and a lack of direction for the reader. I was fortunate to catch Nicole Hoffman’s talk on analysis frameworks at Grimmcon 0x3, just a couple weeks before the first 2021 Trace Labs CTF. Her framework stuck with me, because it could be applied to so much more than raw data analysis. By providing a big-picture of why certain actions are undertaken by the analyst, it can provide better context and lead to better decisions taken. You can watch Nicole introduce her framework at the SANS CTI summit 2021 below and her tweets here: @threathuntergrl.

Using the framework for intel gathering

The framework can be adapted to an intelligence gathering, rather than intelligence analysis by considering how the steps interconntect:

  1. Receive an alert / trigger
  2. Define scope / goals
  3. Enumerate data points
  4. Create information from the data points
  5. Do you have enough information to get a big picture?
    • No: Go back to step 3 with the new perspective
    • Yes: Go on to 6.
  6. Create a hypothesis
  7. Test the hypothesis. Does it pass the test?
    • No: Go back to step 5 with new perspective
    • Yes: Go to step 8
  8. Disseminate the intelligence.

1. Alert / Trigger

This is why you’re doing an investigation. As a pen-tester, it might be part of your scope, if you are doing OSINT it could be a new case, a new source of data… For some reason, you need to investigate the thing.

2. Define scope / goals

This step is crucial, as it defines what you are looking for and thus when your investigation is finished. If you tackle research with no thought as to what the end-goal is, it becomes very easy to enter a wild-goose chase. Something else worth keeping in mind here are the ethics of your actions. As a pen-tester, leaving malware on clients devices is a big no-no. Likewise, when doing OSINT, doxxing bystanders is also a big no-no, and so the choice of equipment, tools etc. becomes crucial to ensure you don’t invade other people’s privacy.

3. Enumerate

Once you have established your goals / objectives for the investigation, you will see your datapoints in that context. By enumerating them all first, you can avoid jumping into rabbit holes and instead prioritise which data points you want to investigate depending on your abilities to process these. Having this list of enumerated datapoints along with how you may use them, also provides additional help further along in the investigation when you may be looking to reduce search scope or rather confirmatory data points.

A major datapoint for enumeration, is the context in which you arrived at the source data. Something shared on social media would have a time stamp and a comment, which may provide context as to wether it was shared in real-time or not. Additionally, this may provide various points to pivot off of further along in your investigation.

4. Create information

From this point on, the investigation enters a loop state of continuously enumerating new data points, as the previous ones are analysed. In this way it is possible to extract additional value from the existing datapoints, or find new possible ways to analyse them. The goal at this step is to extract as much vaue from combinations of data points, which would otherwise not restrict search scope if analysed independently.

5. Check for big picture

Once you have enough elements of information, check wether this provides you with a ‘big-picture’ view of your investigation. That is to say, with the information you have, can you create a hypothesis and test it? If you do not feel confident in the scope your big-picure provides, you can now go back to the enumeration with your current scope in mind. Perhaps you will find dta points which you had previously overlooked, or see them in a new light – can they provide additional value in this new context?

Once you do have a big-picture view of the situation, you can use this to go on to step 6.

6. Create a hypothesis

The hypothesis does not need to be a single hypothesis. It is acceptable to have several hypotheses and test them all out. This might be akin to enumerating but in a very big-picture point of view. That is to say, with the constraints from the previous steps you are looking to find which hopythesis fits best.

7. Test the hypothesis

Depending on the reason for doing this investigation, you may have different levels of confirmation requirements. In the case of a CTF event for example, you would be looking to find additional data points confirming your theory. On the other hand, in the case of sensitive investigations where the dissemination of your uncovered information could have a negative impact on people’s lives if not the exact truth, you would be looking to find any single point of data that contradicts your hypothesis. As part of the investigation you may want to hand it off to another investigator to analyse with an unbiased frame of reference to see if you arrive at the same conclusion.

8. Disseminate

Depending on what you’ve been looking into, it is now time to return it to the appropriate audience. This means that between confirming your hypothesis and transfering that information you need to imbibe it with value for the person receiving it. As an example, in the case of a pen-test, there are 3 key stakeholders:

  1. The person who pays for the test
  2. The person that requires the test
  3. The person who can fix the issues

You could provide all 3 with the information in a single format, but only 1 of the 3 would get any value from it. The person who paid for the service wants to control the costs of what was done & fixing the issues. The person requiring the test, ossibly an auditor needs to know how to verify the issue was fixed. Finally, the person who can fix the issues needs a start of a solution, in order to implement it.

In much the same way, once you are finished gathering your information, you need to transform it into actionable intelligence for the target audience, including your confidence intervals. After all, your report becomes a data source for the reader, who then needs to generate their own added value from past experiences / external sources.

Yes but… why?

By using such a framework it allows several things. Firstly, by establishing a goal, you can see the enumerated data sources under the perspective of the potential information they can provide. Then, by focusing on enumeration, the investigation makes a case of all the data points, rather than having the risk of following certain paths and ommitting the signs that it is contradictory. Additionally, this ensures the investigator’s emotions can only have a limited impact on the perception of the data source.

Additional points

  • Confirmatory step will depend on the intended use for the intelligence. For example, if the investigation will then be brought up as part of a legal case, you would want to look for any signs that the hypothesis is wrong, rather than finding signs of it being right.
  • Google maps is not always up to date, other data sources?
    In my experience, the worst case for satellite view in Google maps was 6 months old. There are tools to find social media posts based on geolocation, choosing what to look for would depend on area (i.e. paragliders are likely near valleys => youtube, snapchat / twitter / instagram in city centres.)
  • Are other browsers as good as Chrome for these challenges?
    I like Firefox, but most modern browsers will support the various search engines & social media without breaking. Always keep a backup.
  • Are other search engines as good as Google?
    Always use multiple search engines. If looking into location specific area, look into what search engines are popular there as you will get better results.
  • Do search engines differ in results?
    Using multiple search engines helps find more data points. Some are more or less respectful of robots.txt so it is definitely worth going through the first 3 or 4 pages of results from multiple search engines and verifying which new data points you can pivot from.
  • Techniques for narrowing down countries or cities?
    This will come down to experience. By focusing on the enumeration steps, you can quickly use multiple data points to reduce possible locations. Then, you need to explore possible locations and find markers that indicate it is, or is not in that area.