Almost a year from when it started, my Cyber Security Challenge UK journey must sadly come to an end. Recently, Santander hosted the final (at least for now) face-to-face competition, and I'd like to share my experience at the event to help others to prepare for similar competitions.

Qualifiers

At previous events, the qualification tasks have often been a good indication of what will actually happen at the event, but no such hints were available here. The qualifier for the National Crime Agency competition was a disk image, to match the forensics involved in the actual event, and the qualifier for the Bank of England competition was log analysis, which was most of what we did at the competition. As well as being the final face-to-face event, this was also the qualifier for Team UK, and as such, a wide variety of skills needed to be tested in the qualifier, meaning no hints were available. Last year's ECSC Final was hosted at Cyber Re:coded in London, and this year's competition will be in Bucharest, Romania, so places are hard fought after.

Qualifier 1

The first qualifier was somewhat of a red herring as far as the actual event was concerned, as it was a penetration test of a web app. Whilst I obviously can't document the exact solution, it was a nice challenge, and it was interesting to see Rangeforce's take on more offensive tasks (more on Rangeforce later). My only experience with Rangeforce was at the 2018 Masterclass, a defensive challenge, and as with most challenges like this, enumeration was the key, and was what made it take me much longer than it should have.

Qualifier 2

The second qualifier was, whilst closer to the actual event, still a test of other skills. Although technically blue team, qualifier two focused on DFIR, specifically in the form of a network traffic capture and disk image. Whilst I found the network capture to be relatively simple, I must admit I was flummoxed by the disk image, and was very interested to hear some of the solutions used by other competitors.

Ultimately, whilst my performance in these qualifiers was enough to get me through to the actual event, qualifier two in particular highlighted an area I really needed to work on. I will confess that I was hoping for a blue team style event after winning CyberCenturion with TeamWhy2k a few weeks prior (blog post coming about this Soon™), and in this regard, I somewhat got my wish.

Day 1

At this point, there's probably a law that says all I'm allowed to tweet is photos from windows, and as such, that was how this event began.

After arriving at the Leicester Marriot Hotel, I quickly checked in to my room, before heading down to the lobby. Whilst I do know most of the regulars at these events, it was still refreshing to see an influx of new competitors, even at the last event, and a healthy representation of Cyber Discovery hoodies.

After arriving at Santander's head offices, we got a briefing on the events of the coming day, and what we were all fighting for, a place on Team UK. As there is no Masterclass this year, and with the aforementioned trip, a place on the team was everyone's goal.

We then spent some time getting to know the other competitors, and although it started out as a simple quiz, it quickly developed into a difficult logic puzzle, calling on our deduction skills. Whilst we ultimately didn't come out on top, it was a great experience to get us ready for the next day.

After arriving back at the hotel, a large group of us collected in a hotel room and played Fibbage for a while, and it was a good thing that we also had the hotel rooms on both sides, as it turns out 14 people in one room do produce quite a bit of noise! It was also a really nice opportunity to see some of the staff, competitors, and assessors in a less formal environment (although not ball pit levels of informal), and I sat talking until the early hours of the morning in the bar, catching up with people I hadn't seen in months.

Day 2

One of the best ways to evaluate any hotel is the quality of the breakfast, and in this regard, I was certainly not let down.

After stuffing down enough food to satisfy a hippo, I did the obligatory 50 checks of my room to make sure I hadn't forgotten anything, then got in a taxi over to the Santander offices.

A large collection of sweets and a bottle of Ribena Light in hand (I went through 5 bottles of the stuff, God help my insides), we got a briefing from Rangeforce about what we would actually be doing for the competition. We were interns at a firm that made ambulance deployment management software, and the firm was experiencing a major breach. Like Masterclass, we were given a copy of the network topology, but this time with the added complication of "satellites" which had a 5mb/s link to us and the internet. Whilst we were lucky that the machines were mostly up to date and were accessed exclusively through SSH, this is an interesting idea I would really like to see built on. Personally, the most frustrating and least enjoyable competitions are those which are the furthest from an actual situation, and I think Rangeforce strikes a really good balance here, as whilst there are some simulated elements (for example, the scoreboard which was up around the room), they are kept to a minimum, and do a good job of keeping up the pressure.

Speaking of Rangeforce, I would once again like to thank them for putting together the actual challenge for the event. Their challenge design and platform continues to be excellent, and I hope to be involved in an event they run again in the future.

With all that out the way, what was the actual challenge like? Well, as I mentioned, the event was purely defensive, but not in the same style as CyberCenturion. Where Centurion had you patch common Windows and Linux vulnerabilities, at this event, the solely Linux environment had no common issues like these, it was entirely custom services, which really helped improve the experience. Whilst knowing how to secure fresh machines is an important skill, it does get repetitive, so having to quickly learn how custom services work and secure them in a tight time frame is a really fun experience.

Throughout the day, we saw a variety of interesting exploits, and whilst I obviously can't document them in full, one of the most interesting ones I saw related to pam_deny.so, which is the file Linux uses to reject authentication requests, and replacing it with pam_allow.so, essentially meaning that any authentication requests would be accepted. Additionally, I was shown why my beloved fail2ban isn't quite the magic bullet I hoped it would be. For context, fail2ban scans logs from a large number of common applications like nginx and sshd, then writes firewall rules to drop traffic from clients with high error rates. In theory this works really well, and indeed, it did block several exploits on the board, and this was lovely for the 30 seconds it took us to realise it had also blocked the scoring platform. A side effect of this network configuration was that the machines saw all traffic coming from the same source, so when they blocked the malicious traffic, they also blocked the ability for the scoring platform to see if the machine was working.

There were also some less advanced exploits, including things like RCE within the web app, which really highlighted the value of digging through logs for a task like this. If you've got them, tools like Splunk and fluentd are great, but often, things like a command backdoor constantly being tested will stick out like a sore thumb in logs, and spotting them mean you can quickly search the app's code for the function being exploited and remove it. Whilst there is obviously value in manually reviewing source code, especially in time sensitive situations, looking for signs of exploitation and working backwards from there will save you a lot of time, although in real world situations where a log event isn't being triggered every few seconds, this is less appropriate.

Unlike at Masterclass, where I worked on most-all of the reports we submitted, my involvement in the process this time was fairly minimal, as whilst it is something that I enjoy doing, there were people on the team more qualified to do them, both faster and to a higher standard than me. I did appreciate the relevance of the situation to the real world, as one vulnerability that we had to write about was passwords appearing in log files, which both Twitter and GitHub have been caught doing recently.

Just like at Masterclass, we bounced up and down the leaderboard, but in the final hours of the competition, it turned into a fight for uptime with a couple of other teams. We eventually reached the point of taking desperate steps, like the aforementioned fail2ban, or drastic changes to the permissions of system files. In the end though, it was good old fashioned vulnerability hunting (and a couple of people watching for downtime) that pulled us through. Whilst we did finish top of the board, there were still a large number of reports which needed to be marked, with the potential to radically change the scores.

Fortunately, the positions stuck, and we managed to come out on top overall. I'd like to say thank you to the entire team, you were great, and I look forward to being able to work with you again.

This year's Team UK were then announced, and whilst I was obviously disappointed to not make the cut, the team is made up of people from whom I'd seen amazing performances at both this event and others I had attended with them, and are the kind of people I don't have any issue losing to. Everyone on the team has done excellently to get this far, and I wish them good luck in the ECSC competition.

Once again, I'd like to issue a massive thank you to everyone involved in the event, CSC, Santander, the assessors, Rangeforce, and everyone else who was involved in putting together this great event. I feel honoured to have had the involvement with Cyber Security Challenge that I have, and I wish the team there the best of luck with whatever the future may hold.

Unless stated otherwise, photos used are either my own or official event photos published here.