I recently had the honour of participating in the 2018 Cyber Security Challenge UK Masterclass sponsored by Barclays. It was an absolutely insane experience, which I would encourage anyone interested in Cyber Security to get involved with. Hopefully this blog will give a taste of what was involved as well as some hints on how to get the most out of the event if you are lucky enough to qualify.

22 Days To Go

My story actually begins a couple of weeks prior to the Masterclass itself. To help with preparation for the event, CSC sent out a list of pre-reading. I went through most-all of this list over the next few weeks, but for any event like this, the pre-reading can be used as a great way to speculate about the course contents, and work out other helpful skills for the event.

Some of the pre-reading really stuck out to me, for example, we were told to read into National Security, suggesting to me a serious and organised crime focus. In addition, there were several items about different types of DDoS attacks and a mixture of Windows and Linux material. This all suggested that the event would be blue team focused, and whilst I did work on these skills through programs like Cyber Centurion and Immersive Labs, I made sure to work on my red teaming skills in case my assessment was incorrect.

11 Days To Go

A few days before the event began, a very cryptic video was released by CSC. Amongst my friends this inspired some furious inspection (I've watched this trailer more times than I dare to count) and some giggling from some who attended last year who thought they had (incorrectly) predicted what would happen.

Day 1

The first day of the actual event began, as most do, with a tweet of a photo out of the window of a train to London.

I arrived at the Hilton London Bridge Hotel extremely excited, and yet not entirely sure what I was in for. Whilst I had been over the pre-reading, everything I'd heard from previous competitors suggested that the Masterclass was greater than the Bank of England and NCC Group Face to Face events I had attended in both scale and difficulty. They certainly weren't wrong there.

Once I had checked out my room, I made my way down to the lobby and met up with the other contestants. Whilst in previous years the attendance at challenge events, and especially the Masterclass, has been mostly university students, this year had seen a huge influx of sixth form students (making up 12 of the 48 competitors). All of these students (including myself) had participated in Cyber Discovery, and the fact that the Cyber Discovery team had been able to get sixth form students up to this standard was seriously impressive. This also gave me the advantage of knowing a substantial portion of the people attending, which turned out to be both a strength (already having bonded with many of the people present) and a weakness (helps me be introverted as hell).

I'll be honest, there are few things in this world as dramatic as checking in to a hotel and being handed an envelope with the information for the next few days. Once I had unpacked a little I headed downstairs to meet with the other contestants and staff. We talked for a bit before making our ways over to Barclays' Canary Wharf office. On the way we passed through a beautiful looking set of Christmas decorations, at which point I discovered my phone's lack of low light performance, but I managed to get a few usable shots.

Once we were in the venue we were introduced to our teams. In my case, that was formed of one person I already knew, four UK based new people, and one competitor from Singapore. Team building exercises take many forms at CSC events, from puzzles spread around museums to pub quizzes, and this event put yet another unique spin on the formula.

Representatives of the Metropolitan Police organised for us to play a game devised by Bristol University. We were given a situation with a company who operated a power plant, a budget, and a list of potential spending options. For the most part, we did alright, making the systems and facilities more secure and fighting off the majority of attacks. That was until the last round, where we received a £5.5 million fine from the ICO for GDPR violations. But hey, at least we weren't the team who had their power plant blown up. It's the small victories in life...

We also got to experience a small snippet of life within Barclays' SOC, through a virtual reality experience.

Once all that was done, we made our way to the nearby Giant Robot market, where we were bought dinner.

Once we had finished dinner (and Scottie had picked himself up off the floor from laughing at me taking a photo of my food, an admittedly very hipster thing to do), we made our way back to the hotel. I tried to write some of this blog post, but quickly realised the futility of that effort, so I headed off to bed.

Day 2

After a "small" breakfast, we collected in the lobby and made our way over to the venue.

It was then that we were given the full explanation of the coming events. he story was a continuation of the one from the previous year. Nancy Oregon, a senior member of the Research4U firm, had been caught working to bring down the CEO, Rex Buckingham. Nancy was now serving time for Computer Misuse Act offences with the group 29Alpha and we had been hired as security interns at Research4U. It's at this point that I need to thank RangeForce, the technical team behind the event. Whilst at some other events, physical infrastructure had been brought on site, RangeForce instead made use of their powerful cloud learning environment. This meant that each team had a fully simulated network, with Windows and Linux clients, an Active Directory domain, and workstations (as opposed to just servers). The actual challenge was incident response (ensuring systems remained online and secure), and so this setup was perfect.

We were given access to our workstations and a copy of the network topology, and were told to get to work. We spent a few minutes planning how we would spend our time, focusing on our areas and tools of expertise. This was a nice plan in theory, although it broke down quickly. I took the domain controllers as my initial area of focus, as I had gained quite a bit of experience auditing Windows Systems participating in Cyber Centurion. I took the usual basic steps before beginning to search for exploits (reset passwords, check members of the administrators group, check for users who shouldn't be part of the forest).

Throughout the day, evidence related to the overarching story was put up on both the computers and an evidence board, and members of our team made sure to make notes on the new information that was put up. To aid with this, during the day we were given the opportunity to interview Nancy, where we did a passable impression of people who weren't making their questions up on the spot.

We continued work on securing the systems and uncovering the story, and as we approached the end of the day we began compiling reports for both the CEO and CTO of the company. Another member of the team and I worked on these reports (with me focusing on the situation report), and we made sure to check each other's work as we went. In my opinion, this was both one of our greatest strengths and weaknesses. We ended up splitting into pairs for most of the event, and whilst this did mean that we were able to ensure productivity and accuracy, it did lead to reduced communication between the entire team. There were multiple instances where one of us would get several minutes into trying to solve an issue before discovering that someone else on the team was already working on it. We did setup a Discord server as somewhere to share links and credentials (very secure), and I'd highly recommend that any other team at a CSC event do the same. I've sometimes scoffed at teams setting up Slack or Discord groups at events, but given how well it's always worked for them, I have had to eat my words in that regard.

I unfortunately can't include the exact reports, as that would ruin similar challenges for those attempting them in the future, but I can tell you that whilst we did find ourselves copying parts between the reports, we ensured that we used non-technical language in the CEO report, and made sure that there were simple points for him to repeat to press.

We probably spent longer than we should have on these reports, not submitting them until minutes before the deadline, but it was worth it, as we got 70/100 for the technical report, and 90/100 for the situational report. These reports do highlight our coordination issue though. Whilst we did get them in, we started them far too late, and kept retracting them to add new information we had found. Looking back, it would have been a good idea to implement some kind of time management system like Trello, and although we did initially create tokens we handed round to ensure only one person was working on a machine at once, this went out of the window almost immediately.

What the reports don't mention however, is the solution we put in to keep the page off of the site whilst we came up with a real solution. A tmux session running the below code may not seem like the best solution, but as a stop gap, it worked surprisingly well.

Once we had got the reports done, we had a bit more time to work on challenges from the board, and had to prepare a presentation for the team in the USA who we would be handing investigations over to for the night. We made an effort to collect notes and prepare, but this highlighted our communication issues in a similar way to the report, although we were moving in the right direction. We divided up responsibilities into what we had done, what we felt needed doing, any actions we would recommend, and threat intelligence. Finally, once all that was done, the day was rounded out with a check-in from 29Alpha.

We ate dinner before heading back into the challenge room for a whirlwind tour of cryptography. We learnt about various ciphers and heard the story of acquiring an Enigma Machine! Turns out, German border guards don't really like you trying to leave with one, but it all worked out in the end and this machine ended up in a museum in Bletchley.

After all that, we made our way back to the hotel and collapsed. Whilst on the first day I had left the strength to stay up for a bit and talk, I was in no state to do that after a day of incident response, and I was out like a light.

Day 3

Despite being so tired, with all still to play for, I was up bright and early at 6:15. After being one of the first to get Breakfast, we headed back to Barclays. Upon arriving at Barclays, we were given a list of changes that the USA team had made, and their changes were about as helpful as our report to them (not very!). Servers were misconfigured and several previously patched vulnerabilities were now present again. We jumped onto our machines to start patching, but almost immediately hit a road block. Whenever we connected to any of the machines over SSH, we got an escape room to rival Vim. Whilst this was initially really annoying, once we realised what had happened, I did have to stop momentarily as I was laughing hard enough to stop me working.

A quick unalias -a later we were back in business, but because of this quick fix, we never found the true cause, likely losing us points. We continued searching systems for issues, and whilst I wont bore you with a complete list of what we found, there was one example that I think works quite well as a cautionary tale.

One of the systems that we had to maintain was an API built in Ruby and published using nginx. We fixed the issue with the Ruby, but we couldn't get nginx to start. I spent quite a while trying to fix this before discovering that despite initially discounting nginx's error messages as they made references to places I could see there wasn't an issue, looking over them again, I noticed an issue that I'd come across before. Many linters (and Python is especially guilty of this) wont actually throw an error on the line that's not been correctly ended, it'll throw it on the next line when it hits something it isn't expecting. So I'm somewhat ashamed to admit I spent around an hour looking for an issue that was solved by adding a ; to a config file...

Later in the day, we were asked to prep for a press conference. As with the other timed events, we probably didn't start preparing for it early enough, but by the time of the conference, a representative of our team was ready enough to present what we had found so far. Whilst many teams took the opportunity during the press conference to take a small break, our rapidly falling position on the leaderboard left us in no state to do this, and we pressed on. The press conference was concluded with the CEO being taken into custody.

Whilst we had thought that fixing nginx would represent the last of our issues with the API, boy were we wrong. The API continued going down, and whilst we did make our best efforts to fix it, it became clear that it was eating too much of our time, so we took inspiration from the first day, and implemented a stop-gap.

Just as we started to get everything under control, we received word that our network was being blamed for a massive DDoS attack, and that it needed to be stopped immediately or we would be cut off. Fortunately, we had already taken steps against this by disabling UDP on our memcache server, and a few iptables rules later, we had the situation under control.

It was then that it started to come crashing down for 29Alpha, as more evidence began to be uncovered linking Nancy and her brother to the attacks. We had to submit one final report with all the information that we had collected from both the interview and the intel board in the corner of the room, and unlike previous reports, we didn't wait until the last minute to get this one in!

Again, I can't include the actual report here, but in the report, we recommended action for Research4U and law enforcement. We did have to be careful with the second one though, and I had to be corrected by a law student on our team, as I began propose action that was either not appropriate or not something we were in a place to propose.

A few minutes later, we came to the end of the time we had been allocated, and were finally able to relax. This triggered the usual discussion of answers that comes at the end of any CTF, and I had a few key takeaways:

• Don't fight your way through iptables when the box has ufw. If it works, use it.
• Think like a pentester. Whilst we did run linenum on one of the boxes, we stopped looking at it once we spotted a vulnerability, and never really went back. Had we actually looked at it some more, we would have spotted the cron jobs that were causing us so much misery.
• Focus on patching the root cause, as opposed to the actual vulnerability. As the Rangeforce team revealed, all the vulnerabilities existed in at least two places. Patch one and the other would just bring it back. You needed to make sure you were removing all traces of the attackers on the machines, as any foothold was enough to bring the entire system down. This would also have been aided by taking better notes, although we were able to reconstruct most of our actions from memory.

Once we had said our final goodbyes and thank yous to the Barclays team, we headed back over to the hotel to prep for the dinner in the evening. For reasons I don't entirely understand but thoroughly approve of, CSC reserved the hotel's ball pit for the under 18 candidates, and we made full use of it.

I can definitely think of worse ways to catch up on the events of the past few days than partially submerged in a ball pit, but it was eventually time to prepare for the dinner. After several minutes of competitors laughing at each other as we were all clearly not used to wearing formal clothes, we began to mingle amongst the other guests. I can't talk too closely about who I actually spoke to for obvious reasons, but let's just say "various people from various walks of life" and leave it at that. After some business card swapping, we made our way into the hall for dinner.

After polishing off the food, we were then treated to a performance from a very unique dance troop.

The dinner was rounded out with the announcement of the winners of the competitions, the best newcomer award, and the best career changer award, along with some individual prizes. Whilst I earned a Yubikey U2F key and a year's membership to iisp, some participants managed to earn tickets to Blackhat EU and University courses. We spent some more time networking in the dinner room before making our ways out into the bar area, where I took the opportunity to catch up with some of the people that I'd met throughout the experience. It wasn't long however before I spotted a massive group of CSC staff making their way over to the ball pit.

The rest of the evening flew past with some interesting conversations and opportunities to talk to people I would never normally be able to, but the highlight had to be watching the CSC team getting some well earned rest time, Stuart being pelted with balls and returning in kind, and a very different side to people I had only seen in a professional context. I honestly couldn't tell you what time we kept the hotel staff up until keeping an eye on us, but judging from the fact that I tweeted at 3am, it must've been fairly late.

I left my hotel room with a large pile of my business cards, and I came back with a large pile of other people's cards, so I think that counts as a success?

Day 4

After the previous day's festivities, I gave up all semblance of good timekeeping, and rolled out of bed around 9:30, although unsurprisingly I was still very tired. I sauntered down to Breakfast and ate a little, before throwing my messy pile of clothes into my case and starting the long journey home.

I've said it several times, both to their faces and online, but I really can't thank the entire challenge team enough for what they did for me and the other competitors. The Masterclass was like nothing else I'd ever experienced, and will be taking pride of place on my resume. There really is nothing else like a CSC event, and I'd seriously recommend anyone who is even vaguely interested have a go. What have you got to lose?

Advice

Whilst obviously the entire Masterclass was an amazing experience, I can't expect you to read all of that and not get anything useful, so here's some advice for anyone at a future CSC event:

• If someone in a white shirt comes over, explain to them what you are doing. They are assessors, and they are making individual notes on you. Whilst your technical performance is obviously very important, your soft skills are also being closely assessed, so explain what you are doing and ask for help. These people are professionals, and whilst they can't give you the exact solution to the issue you are facing, they deal with this kind of stuff on a daily basis, and can probably give you a push in the right direction.
• Get coordinated. Every event will begin with an icebreaker, and take that opportunity to set up some infrastructure. Get a Slack or Discord group, and for longer events, setup some kind of project board system like Trello.
• Bring a mouse and USB drive. It'll save you so much time. Just trust me on this one.
• If you think something that isn't working should be, let someone know. There's no point sitting there trying to fix something where nothing's actually wrong.
• Actually do the pre-reading. Many people will look at the size of the pre-reading and give up, but it's worth doing. Even if you don't get the whole way through each piece, it's worth doing as much as you can. The reading isn't picked at random, and is generally a pretty good indication of what you'll be doing.
• Once the actual challenge is revealed to you, stop and take stock of what you have. Spend a few minutes coming up with a plan, work on what you're good at, and let the others handle what they're good at. There may be times that you feel that you're out of your depth, but you need to remember that you have a team who are there to help you, and at the end of the day, this is an event. You're behaviour is being evaluated, and being adaptable is a great skill. Even if you don't know exactly what you are doing, a willingness to try and to learn is very important.
• Take notes as you go. You'll usually be asked to produce some kind of report, and there's nothing worse in the world than having to write a report entirely from memory.
• Get business cards. If you have the chance, give them to someone, the worst thing they can do is bin it. If someone offers you their card, take it, but remember to actually follow up, and fairly quickly. Getting an email that says "we met 6 months ago but I've not got round to emailing you" doesn't look great.
• Get LinkedIn. Get it now. I'll be doing a blog post about doing LinkedIn properly at some point, so keep an eye out for that.

Congratulations on getting to the end of that, I know it was a lot. If you have any questions, feel free to email me on [email protected] or on Twitter at @thebeanogamer.