I’m still deep in my lambda love affair. Here’s my talk at the AWS meetup in SG on Lambda layers :
I started the year building out govScan.info, a site that audits .gov.my websites for TLS implementation. Overall I curated a list of ~5000 Malaysian government domains through various OSINT and enumeration techniques and now use that list to scan them…
Sayakenahack was undoubtedly the highlight of my 2017. If you’ve come from sayakenahack.com, I’m sorry but I’ve shutdown the site :(. I learnt so much from it, and it was even my ticket for presenting at Hack In the Box…
Over the past few weeks, I’ve been toying with lambda functions and thinking about using them for more than just APIs. I think people miss the most interesting aspect of serverless functions — namely that they’re massively parallel capability, which can do a lot more than just run APIs or respond to events.
There’s 2-ways AWS let’s you run lambdas, either via triggering them from some event (e.g. a new file in the S3 bucket) or invoking them directly from code. Invoking is a game-changer, because you can write code, that basically offloads processing to a lambda function directly from the code. Lambda is a giant machine, with huge potential.
What could you do with a 1000-core, 3TB machine, connected to a unlimited amount of bandwidth and large number of ip addresses?
Here’s my answer. It’s called potassium-40, I’ll explain the name later
So what is potassium-40
Potassium-40 is an application-level scanner that’s built for speed. It uses parallel lambda functions to do http scans on a specific domain.
Currently it does just one thing, which is to grab the
robots.txt from all domains in the cisco umbrella 1 million, and store the data in the text file for download. (I only grab legitimate robots.txt file, and won’t store 404 html pages etc)
This isn’t a port-scanner like nmap or masscan, it’s not just scanning the status of a port, it’s actually creating a TCP connection to the domain, and performing all the required handshakes in order to get the
Scanning for the existence of ports requires just one SYN packet to be sent from your machine, even a typical banner grab would take 3-5 round trips, but a http connection is far more expensive in terms of resources, and requires state to be stored, it’s even more expensive when TLS and redirects are involved!
Which is where lambda’s come in. They’re effectively parallel computers that can execute code for you — plus AWS give you a large amount of free resources per month! So not only run 1000 parallel processes, but do so for free!
A scan of 1,000,000 websites will typically take less than 5 minutes.
But how do we scan 1 million urls in under 5 minutes? Well here’s how.
Just because you have webhook, doesn’t mean you need a webserver.
With serverless AWS Lambdas you’ve got a free (as in beer) and always on ability to receive webhooks callbacks without the need for pesky servers. In this post, I’ll setup a serverless solution to accept incoming
POST from a GitHub webhook.
DNS Queries on GovScan.Info
This post is a very quick brain-dump of stuff I did over the weekend, in the hopes that I don’t forget it :). Will post more in-depth material if time permits over the weekend.
govScan.info, a site I created as a side hobby project to track TLS implementation across
.gov.my websites — now tracks DNS records as well. For now, I’m only tracking MX, NS, SOA and TXT records (mostly to check for dmarc) but I may put more record types to query.
DNS Records are queried daily at 9.05pm Malaysia Time (might be a minute or two later, depending on the domain name) and will be stored indefinitely. Historical records can be queried via the API, and documentation has been updated.
The security community has been abuzz with an absolutely shocker of story from Bloomberg. The piece reports that the Chinese Government had subverted the hardware supply chain of companies like Apple and Amazon, and installed a ‘tiny chip’ on motherboards manufactured by a company called Supermicro. What the chip did — or how it did ‘it’ was left mostly to the readers imagination.
Supermicro’s stock price is down a whooping 50%, which goes to show just how credible Bloomberg is as a news organization. But besides the Bloomberg story and the sources (all of which are un-named), no one else has come forward with any evidence to corroborate the piece. Instead, both Apple and Amazon have vehemently denied nearly every aspect of the story — leaving us all bewildered.
But Bloomberg are sticking to their guns, and they do have credibility — so let’s wait and see. For now, let’s put this in the bucket called definitely could happen, but probably didn’t happen.
I can only imagine how hard it must be to secure a modern hardware supply chain, but the reason for this post is to share my experience in some supply chain conundrums that occurred to a recent project of mine.
I operate (for fun) a website called GovScan.info, a python based application that scans various
gov.my websites for TLS implementation (or lack thereof). Every aspect of the architecture is written in Python 3.6, including a scanning script, and multiple lambda functions that are exposed via an API, with the entirety of the code available on github.
And thank God for GitHub, because in early August I got a notification from GitHub alerting me to a vulnerability in my code. But it wasn’t a vulnerability in anything I wrote — instead it was in a 3rd-party package my code depended on.
Hosting an S3 site via Cloudflare
From my previous post, you can see that I hosted a slide show on a subdomain on hitbgsec.keithrozario.com. The site is just a keynote presentation exported to html format, which I then hosted on an S3 bucket.
The challenge I struggled with, was how to point the domain which I hosted on Cloudflare to the domain hosting the static content.
The recommended way is to just create a simple CNAME entry and point it to the S3 bucket, but that didn’t work because the ‘crypto’ settings on Cloudflare apply to the entire domain — and not individual subdomains.
And since my website at www.keithrozario.com had a crypto setting of ‘Full’, the regular CNAME entry kept failing. I could have downgraded to ‘Flexible’ but that would mean my blog would be downgraded as well — which wasn’t ideal.
Why downgrade my main blog to accommodate a relatively unimportant sub-domain.
Instead found that the solution is to overlay a CloudFront Distribution in front of S3 Bucket — and then point a CNAME entry to the Distribution.
The solution looks something like this:
I haven’t blogged in a long while — but I have a good(ish!) excuse. I spent most of August prepping for the #HITBGSEC conference in Singapore. It was my first time presenting at a security conference, and I had an…
As Malaysians woke up today, to a brand new cabinet of Ministers, many have already begun expressing their dissatisfaction on the lineup. I know better than to wade into these politically charged discussions — but I will point out that my people have long been overlooked for Ministerial positions.
Who are ‘my people’ you ask…
Or if you prefer a less negative word — Geeks. But for the rest of this post, I’ll use the more accurate term of hacker to refer to technically savvy folks who subscribe to the hacker ethic.
Yes, we in the hacker community have long been overlooked for ministerial positions, and I for one, choose to speak out against this travesty. But before I delve into why I think we’ve not played a bigger part in politics, let me first make the case for why we need hackers in parliament.
Why we need hackers in parliament
As technology becomes more pervasive and ubiquitous in our lives, every policy decision becomes a technology decision, whether it’s in education, finance or defence. Hence it becomes pertinent to ensure that the people making these decisions have the capacity to understand the technology that drives the issues. This is not something you get from a 2-week bootcamp, or a crash course in computers, it involves deep technical knowledge that can only be attain from years (even decades) of experience.
But it’s not enough that policy makers merely understand technology, they also need to subscribe to the hacker ethic , and bring that ethic into the decisions they make.
What is the hacker ethic? Well I’m glad you asked.
The ethic has no hard definition, but it incorporates things like Sharing, Openness, Decentralization and Free access to computers, etc. The ethic further includes attitudes, like pure meritocracy, the idea that hackers should be judged for their hacking (and nothing else), not age, gender, degrees or even position in a hierarchy. So anytime you see some poor sod who claims to be a hacker, but puts CISSP, PMP, CEH at the end of their LinkedIn profile — you know they’re not really hackers.
You can see ethic played out at hacker conferences throughout the world, hackers are ever willing to share what they’ve built with anyone who’ll listen, and they’re accepting of anyone willing to learn, at any age bracket, without any education or formal training.
The Hacker perspective is an interesting one, and like all perspectives, may not always be right or appropriate, but it’s important for it to be present at the decision making process, if nothing more than to add to the diversity of thought.
So why aren’t there more hackers in decision making levels? Well let’s see what it takes to reach the decision making level in the first place.