Strategies in Intelligence Gathering

Juan… But why?

There are many writeups and training strategies for gathering intelligence, but often they are either so specific that the techniques cannot be applied in the same process to other situations. At the other end of the spectrum, some write ups are so abstract there is no guideline and a lack of direction for the reader. I was fortunate to catch Nicole Hoffman’s talk on analysis frameworks at Grimmcon 0x3, just a couple weeks before the first 2021 Trace Labs CTF. Her framework stuck with me, because it could be applied to so much more than raw data analysis. By providing a big-picture of why certain actions are undertaken by the analyst, it can provide better context and lead to better decisions taken. You can watch Nicole introduce her framework at the SANS CTI summit 2021 below and her tweets here: @threathuntergrl.

Using the framework for intel gathering

The framework can be adapted to an intelligence gathering, rather than intelligence analysis by considering how the steps interconntect:

  1. Receive an alert / trigger
  2. Define scope / goals
  3. Enumerate data points
  4. Create information from the data points
  5. Do you have enough information to get a big picture?
    • No: Go back to step 3 with the new perspective
    • Yes: Go on to 6.
  6. Create a hypothesis
  7. Test the hypothesis. Does it pass the test?
    • No: Go back to step 5 with new perspective
    • Yes: Go to step 8
  8. Disseminate the intelligence.

1. Alert / Trigger

This is why you’re doing an investigation. As a pen-tester, it might be part of your scope, if you are doing OSINT it could be a new case, a new source of data… For some reason, you need to investigate the thing.

2. Define scope / goals

This step is crucial, as it defines what you are looking for and thus when your investigation is finished. If you tackle research with no thought as to what the end-goal is, it becomes very easy to enter a wild-goose chase. Something else worth keeping in mind here are the ethics of your actions. As a pen-tester, leaving malware on clients devices is a big no-no. Likewise, when doing OSINT, doxxing bystanders is also a big no-no, and so the choice of equipment, tools etc. becomes crucial to ensure you don’t invade other people’s privacy.

3. Enumerate

Once you have established your goals / objectives for the investigation, you will see your datapoints in that context. By enumerating them all first, you can avoid jumping into rabbit holes and instead prioritise which data points you want to investigate depending on your abilities to process these. Having this list of enumerated datapoints along with how you may use them, also provides additional help further along in the investigation when you may be looking to reduce search scope or rather confirmatory data points.

A major datapoint for enumeration, is the context in which you arrived at the source data. Something shared on social media would have a time stamp and a comment, which may provide context as to wether it was shared in real-time or not. Additionally, this may provide various points to pivot off of further along in your investigation.

4. Create information

From this point on, the investigation enters a loop state of continuously enumerating new data points, as the previous ones are analysed. In this way it is possible to extract additional value from the existing datapoints, or find new possible ways to analyse them. The goal at this step is to extract as much vaue from combinations of data points, which would otherwise not restrict search scope if analysed independently.

5. Check for big picture

Once you have enough elements of information, check wether this provides you with a ‘big-picture’ view of your investigation. That is to say, with the information you have, can you create a hypothesis and test it? If you do not feel confident in the scope your big-picure provides, you can now go back to the enumeration with your current scope in mind. Perhaps you will find dta points which you had previously overlooked, or see them in a new light – can they provide additional value in this new context?

Once you do have a big-picture view of the situation, you can use this to go on to step 6.

6. Create a hypothesis

The hypothesis does not need to be a single hypothesis. It is acceptable to have several hypotheses and test them all out. This might be akin to enumerating but in a very big-picture point of view. That is to say, with the constraints from the previous steps you are looking to find which hopythesis fits best.

7. Test the hypothesis

Depending on the reason for doing this investigation, you may have different levels of confirmation requirements. In the case of a CTF event for example, you would be looking to find additional data points confirming your theory. On the other hand, in the case of sensitive investigations where the dissemination of your uncovered information could have a negative impact on people’s lives if not the exact truth, you would be looking to find any single point of data that contradicts your hypothesis. As part of the investigation you may want to hand it off to another investigator to analyse with an unbiased frame of reference to see if you arrive at the same conclusion.

8. Disseminate

Depending on what you’ve been looking into, it is now time to return it to the appropriate audience. This means that between confirming your hypothesis and transfering that information you need to imbibe it with value for the person receiving it. As an example, in the case of a pen-test, there are 3 key stakeholders:

  1. The person who pays for the test
  2. The person that requires the test
  3. The person who can fix the issues

You could provide all 3 with the information in a single format, but only 1 of the 3 would get any value from it. The person who paid for the service wants to control the costs of what was done & fixing the issues. The person requiring the test, ossibly an auditor needs to know how to verify the issue was fixed. Finally, the person who can fix the issues needs a start of a solution, in order to implement it.

In much the same way, once you are finished gathering your information, you need to transform it into actionable intelligence for the target audience, including your confidence intervals. After all, your report becomes a data source for the reader, who then needs to generate their own added value from past experiences / external sources.

Yes but… why?

By using such a framework it allows several things. Firstly, by establishing a goal, you can see the enumerated data sources under the perspective of the potential information they can provide. Then, by focusing on enumeration, the investigation makes a case of all the data points, rather than having the risk of following certain paths and ommitting the signs that it is contradictory. Additionally, this ensures the investigator’s emotions can only have a limited impact on the perception of the data source.

Additional points

  • Confirmatory step will depend on the intended use for the intelligence. For example, if the investigation will then be brought up as part of a legal case, you would want to look for any signs that the hypothesis is wrong, rather than finding signs of it being right.
  • Google maps is not always up to date, other data sources?
    In my experience, the worst case for satellite view in Google maps was 6 months old. There are tools to find social media posts based on geolocation, choosing what to look for would depend on area (i.e. paragliders are likely near valleys => youtube, snapchat / twitter / instagram in city centres.)
  • Are other browsers as good as Chrome for these challenges?
    I like Firefox, but most modern browsers will support the various search engines & social media without breaking. Always keep a backup.
  • Are other search engines as good as Google?
    Always use multiple search engines. If looking into location specific area, look into what search engines are popular there as you will get better results.
  • Do search engines differ in results?
    Using multiple search engines helps find more data points. Some are more or less respectful of robots.txt so it is definitely worth going through the first 3 or 4 pages of results from multiple search engines and verifying which new data points you can pivot from.
  • Techniques for narrowing down countries or cities?
    This will come down to experience. By focusing on the enumeration steps, you can quickly use multiple data points to reduce possible locations. Then, you need to explore possible locations and find markers that indicate it is, or is not in that area.
Open Source Security Tech

First Steps with KeePassXC

Why KeePassXC?

Most of my experience with password managers lies with KeePass. For an enterprise environment it is simple to use, easy to deploy and fairly lightweight. For my personal use I can take advantage of its portability and not worry about phishing too much, as I tend to type out websites directly when something sketchy comes around. For work, VMs and other users however, I recommend KeePassXC now. It offers some advantages to KeePass, namely an easier interface, browser integration and TOTP.

The browser integration provides phishing protection => if users click on links asking for credentials, the extension will not recognise the domain and therefore not provide credentials. Along with the easier interface, this means users are more likely to avoid password reuse and less likely to fall for phishing attempts.

First Steps

The KeePassXC team have a great introduction to their password manager, covering details from setting up a database to the configuration of the browser integration. I highly recommend taking a look there before doing the install, even if just to make yourself aware of the different interface. You can find their getting started guide here.

Now, following the steps from the KeePass guide there are some differences in the procedures. The first is the key transformation, instead of telling the system how many iterations you want, KeePassXC suggests choosing how long it takes to decrypt the database.

KeePassXC settings interface, on the security menu, encryption settings tab.

Unlike KeePass, KeePassXC doesn’t provide password templates. nor does it offer the ability to generate a password from the previous password (useful for services with character or length limitations). The password generator takes the last used settings. My recommendation is to check the box to avoid lookalike characters (for those times you need to type it out).

Password creator with the Exclude look-alike characters checkbox ticked

Same as with KeePass, KeePassXC allows entries to autoexpire. This is not checked by default, in my use of the application, it does not stay checked when making several entries. The preset expiry times include 1, 2, 3 weeks, months and years.

KeePassXC new entry window with no expiry date selected.

Same as with KeePass, I recommend renewing your passwords at 6 month intervals, but at least once a year.


Unlike KeePass, KeePassXC comes with TOTP support out of the box. While arguably this is not great as it means you are getting all your authentication tokens through the same medium, it does protect your accounts from brute forcing attacks. You can’t set TOTP from the entry window, but after creating the entry right click => TOTP =>Setup TOTP. When generating your tokens, tell the web service you can’t scan the QR code, and enter the secret into the TOTP window.

Setup TOTP window with an empty Secret Key.

It is worth mentionning that you can get the secret from KeePassXC. Not the most secure solution but I can see this being useful for teams connecting to client environments, as the database can be shared through KeeShare with those people who need access to it. With the browser extension, this means seamless logging in to services as all the credentials come from a single place, (and the database can be locked with a passphrase & hardware token).


KeeShare sort of takes the need for cloud sync away accross devices, however my limited experience with this feature was not great. Most likely due to user error and limited time / infrastructure to play with it, it is worth knowing that the KeePassXC team have documented it quite well over here.

Nice to know

You can save credentials from the browser extension directly to your database. These will be in a default “browser credentials” group. You can order them into whatever folder you want if you actively use the standard interface to find them faster.

By default, autotype is enabled in KeePassXC, but with no assigned shortcut. With the browser extension I have not found any reason to activate it. Specially after the shenanigans with KeePass spouting my credentials into a couple services.

The browser extension requires the database to be unlocked in order to use the credentials. The icon turns green when the connection between the extension and the data base is active, and grey if the connection is not present. I have found it particularly useful to be able to hide the window once it has been unlocked as it lets me keep an uncluttered desktop, and the browser lets me know if anything is the matter.

KeePassXC Application Setting window with ‘Minimize window after unlocking database’ checked

By default, the database will be saved after every change (new entry, password change, etc.). I’ve found this behaviour useful, but worth knowing if you have a long decryption time it can slow down your machine. The behaviour can be medified in the Application Settings => File Management.

KeePassXC Application Settings window with ‘Automatically save after every change’ option checked

Sadly, unlike KeePass, there is no option to show expired / expiring soon entries upon loading up the interface. The entries will have a different icon showing that the entry has expired and the title will be crossed through.

KeePassXC expired entry “Test” with the expired entry icon, & title crossed through

KeePass or KeePassXC?

I think it really comes down to the user. Both offer similar levels of protection for the users’ databases, they differ mainly in what they offer out of the box. Neither solution has been ported to Android or iOS which means regardless of what you pick, there will always be another party accessing your passwords. I believe KeePass will continue to have its place in enterprise environments where it outperforms the XC variant through a slimmer interface. KeePassXC does provide a nicer experience for first time users, with default settings that make it easy to use from the get-go. As long as you don’t use TOTP, you can use both with the same database and decide later on.