Late in December 2015, I sent the email below to all of "Engineering" across Riot Games. I want to share this externally because it's core to the security culture that we want to build in Riot, i.e. one of accountability and responsibility, where we aren't afraid to talk about our screw-ups.
############################################################
Hey Folks,
############################################################
Hey Folks,
So as most of you know, I'm a bit of a perfectionist with high standards :)
Well recently, I screwed up and didn't come close to meeting my own expectations.
What did I do?
Well, when testing (locally) Netflix's Security Monkey in 2014, I copied over some aws-related scripts I was using and found useful to a local directory on my work laptop, where my Github repo was stored. I also mistakenly copied over a flask configuration file (config-deploy.py) from the local version of Security Monkey that I was testing at the time.
To compound this mistake, prior to committing files and pushing to my Github repo, I performed a "git add" on the sub-directory (aws), not the specific files I'd added. As a result, config_deploy.py was added to the changes to the repo and consequently, committed and pushed up to my personal Github.
What Happened Next?
Nothing for a year until I got a random late-night text from a colleague, nothing unusual there :)
- Bug Bounty Ticket - xxxx
- Internal Vulnerability Ticket - xxxx
However, this text was advised me to check our Bug Bounty system. The shear panic and "shit, shit, shit" went around in my head :-/ So unsurprisingly, I immediately logged on to read the bug report.
What was the impact?
Fortunately, close to nothing:
- the file was a test file with no production credentials of any sort
- we have publicly committed to the Security Monkey repo
- we have publicly spoke about using Security Monkey
- sheer embarrassment and annoyance on my behalf
What's the point of this email?
- I'd like to apologise to you guys for screwing up.
- To let you know that I've removed the personal repo and modified the controls on any public github repo that's been cloned to my system
- This researcher was actually very cool (despite initially claiming it was an awesome vulnerability and significant information disclosure).
- Validation is important on all (vulnerability/bug) reports coming in through our Bug Bounty or any other medium.
- I feel like I got lucky
- Run "git status" before "git push origin master"
- .gitignore doesn't always save you :(
- I got great support from the InfoSec team on this, which meant a lot and reminded me that we're a team at all times (success & failures).
- Hopefully others will realise that it's not the end of the world to make mistakes and it's important that we don't try to hide them but share them and learn from them.
If you've any questions or feedback for me, please let me know.
############################################################
This episode was a huge learning experience for me and I believe I can only improve from it (whilst, yeah, I was lucky). It was ultimately a very humbling experience, a nice reminder that we in #InfoSec make mistakes, that we should not be advocating for security from the our perches up high, feeling perfect.
We have a "no asshole" rule and I believe this is one reason (of many) why I had a pretty awesome and very supportive response from many across Riot, a nice reminder that it's often better to talk about your failures than your successes.
We have a "no asshole" rule and I believe this is one reason (of many) why I had a pretty awesome and very supportive response from many across Riot, a nice reminder that it's often better to talk about your failures than your successes.
Totally true, every time I review code or search for a vulnerability, I am demanding, nonetheless when I found something, I always add "I could have done it, at least now it will be fixed.".
ReplyDeleteYup, exactly :) xD
Delete