Back in 2013 when I was working at Sky News, the person responsible for the social media accounts (with millions of followers in total) stormed into a meeting: "Our Twitter account has been hacked".
This was at a time when many high-profile news Twitter accounts were hacked by so-called "electronic armies" who published damaging tweets. However in our case it was a single obscure "Colin was here" tweet.
We had recently built an internal endpoint in one of the backend apps that takes a string and publishes it straight to the main breaking news Twitter account. This was integrated with a custom UI tool that the news desk people used to quickly break a story across TV, Twitter, the website etc with one click.
I had a suspicion that this endpoint was how that tweet was published, but could not prove it. Many thoughts were going through my head.. “is this an internal job, or did someone hack our backend system and somehow figured this out etc.. “
We quickly returned to our desks, and straight away I greped our logs for "tweeting" as I developed that feature and was sure we logged that when the endpoint is called, but in the heat of the moment forgot that to “-i” as it the log message actually contained "Tweeting" (which cost us a few minutes). In the meantime there was panic around the business, people were putting out PR statements just in case it was a real hack, the tweet was deleted etc.
Finally, with help from colleagues, we tracked down a "Tweeting" log message around the same time the tweet was published along with the HTTP request source IP, and traced it (just like in movies) to our secondary news studio in Central London. This is when one of the managers shouted "I know a Colin who works there, he's a testing team manager!".
We gave Colin a ring to understand what was going on, he had no idea about any of this but said he was doing some DR testing earlier of all tools that editors use, and wasn’t really aware this would go out. As you can imagine, it could have been much worse.
The entertaining bit was the 30 minutes of fame this mysterious Colin enjoyed on the internet, where many people were worried about the welfare of "Colin", and it was picked up by various [1] news [2] websites.
Back in 2013 when I was working at Sky News, the person responsible for the social media accounts (with millions of followers in total) stormed into a meeting: "Our Twitter account has been hacked".
This was at a time when many high-profile news Twitter accounts were hacked by so-called "electronic armies" who published damaging tweets. However in our case it was a single obscure "Colin was here" tweet.
We had recently built an internal endpoint in one of the backend apps that takes a string and publishes it straight to the main breaking news Twitter account. This was integrated with a custom UI tool that the news desk people used to quickly break a story across TV, Twitter, the website etc with one click.
I had a suspicion that this endpoint was how that tweet was published, but could not prove it. Many thoughts were going through my head.. “is this an internal job, or did someone hack our backend system and somehow figured this out etc.. “
We quickly returned to our desks, and straight away I greped our logs for "tweeting" as I developed that feature and was sure we logged that when the endpoint is called, but in the heat of the moment forgot that to “-i” as it the log message actually contained "Tweeting" (which cost us a few minutes). In the meantime there was panic around the business, people were putting out PR statements just in case it was a real hack, the tweet was deleted etc.
Finally, with help from colleagues, we tracked down a "Tweeting" log message around the same time the tweet was published along with the HTTP request source IP, and traced it (just like in movies) to our secondary news studio in Central London. This is when one of the managers shouted "I know a Colin who works there, he's a testing team manager!".
We gave Colin a ring to understand what was going on, he had no idea about any of this but said he was doing some DR testing earlier of all tools that editors use, and wasn’t really aware this would go out. As you can imagine, it could have been much worse.
The entertaining bit was the 30 minutes of fame this mysterious Colin enjoyed on the internet, where many people were worried about the welfare of "Colin", and it was picked up by various [1] news [2] websites.
[1] https://www.buzzfeed.com/lukelewis/an-important-history-of-t... [2] https://www.buzzfeed.com/lukelewis/an-important-history-of-t...