Judging?
After the June CTF I was considering participating in a couple more events in a competitive manner. The goal being to get better at the analytics side of things as it is not limited in scope to OSINT for missing people. I like Nicole’s Cognitive Stairways of Analysis as a foundational framework (https://threathuntergirl.com/f/the-cognitive-stairways-of-analysis). However, the intent to be competitive implies preparation, and life got in the way; with 2 team mates down I could only hope for a 3rd time in top 50. So for the Defcon edition of the Tracelabs CTF, I applied to judge.
Preparation
Much like for competitive participation, judging requires some preparation. You want to be the best judge for your team(s). However, the preparation is not as extensive as for competitive. In my case it consisted of catching up with the judge’s briefing & reading the judge’s guide. Following that it was a matter of preparing a way of (easily) keeping track of the accepted flags. Joplin comes pre-installed on the Tracelabs VM, so I created the notes with that tool. I have migrated them to Obsidian as Obsidian uses less metadata within the files. A template for other judges or people participating solo is available on my github. This should be fairly easy to import into Obsidian, Joplin or Notion as it is a simple collection of markdown files. Links may need to be converted if using Joplin instead of Obsidian.
Having the guidelines quickly at hand meant I could easily copy-paste reasons for rejecting a flag, or validate if a flag was in the right category. (Family vs Friends, Basic vs Advanced subject info, etc.). By keeping track of the submitted flags I could quickly spot duplicates.
For future judging I would look into having something else to do, as the flags seem to come in batches wether you have 1, 2 or 3 teams. I found myself often with 10-15 minute interludes between batches of flags, specially at the begining while the teams were getting warmed up.
The process
You might expect the flags to come pouring in with no break – except, of course the tams need to start their research which means until the first flag comes in you’re left finalising your preparations. As you will need to be verifying flags from various social media elements, one element of preparation is ensuring you still have access to the various platforms. I left the various platforms open such that I could then copy-paste the URLs with no need for logging in & avoiding cross-contamination between the isolated tabs (Firefox Containers does wonders for this).
The first flag comes in and suddenly the platform makes a lot more sense to when competing. (Which hopefully means less rejections at the next competition). The flag includes the team name so let the team know I’m judging for them in the comms channel, this way they can discuss any questions more easily than interposed flags & rejections. Test the flag (is it a new flag?) and validate it (does the flag make sense?) accept or reject. Upon rejecting, explain why – having the guidelines handy makes it easier to explain why a flag is rejected or category changed.
You have 2-3 teams. Life happens so some judges will take on 4. If any flags seem odd, or you are unable to verify them other judges can help, this means you can then provide your team with the best possible experience.
Post CTF
The Tracelabs CTFs can expose some of the worst that humanity has to offer. As such, it’s important to create an appropriate disconnection from what you’ve experience before shutting down. See Nicole’s talk at ConINT on strategies for disconnecting:
In my case, it was just catching up on a comedy TV show and crashing for the night. Had there been any significantly traumatic elements having the other judges to discuss and externalise any thoughts was/is reconforting. It is made abundantly clear that if any judge needs to step away at any time for any reason, it is not a fault on their person, and other judges are happy to take on the extra flags. Knowing when and how to seek help is a part of preparation, but the Trace Labs team make sure to remind the judges about their availability should a crisis happen.
Learning outcomes
During competitions I found the platform to make it difficult to explain the value of the flag. You can establish the category, the relevance and the supporting evidence, along with 1 attachment. As a judge it actually made a ot more sense: Submit the URL pointing to the flag. Relevance: explain the flag (i.e. Category => sub category, what cn be done with this piece of information?). Supporting evidence => How did you get to this conclusion? If you got to a profile from a different URL, submit it here. As a jduge, it is a lot easier to follow your reasoning, and during recompilation the flag will be extra useful. Attachment => visual aid. Screenshot of a social media account isn’t ideal, however, being able to show how you can proove it belongs to a person adds value to the flag.
10/10, would judge again