One Toggle From a Breach: How We Built Keelscan to Scan Code, Live Apps, and Supabase in One Grade

Deals do not die on price. They stall in security review.

A serious customer, or their auditor, asks a simple question: is your app secure? For a small team that shipped fast, often with a lot of AI-generated code, "I think so" is not an answer that closes the deal. You need a real answer, the fix, and something you can send to their security team as proof.

That gap is why we built Keelscan: a continuous security-posture scanner aimed squarely at the painkiller buyer, the founder whose revenue is blocked on a security question they cannot yet answer. This post is the engineering story behind it, including three problems that were more interesting than they first looked.

TL;DR: A real security review checks three surfaces, not one. Keelscan scans your code (secrets, dependencies, the mistakes AI codegen makes), your running app (TLS, security headers, CORS, exposed files), and your cloud data config (Supabase row-level security, Firebase rules), then folds everything into one A-F grade with plain-English fixes. The interesting parts: you can catch the highest-impact Supabase data leak with only the public anon key (and you should read column names, never values); letting users scan any URL means you have built an SSRF machine unless you guard every hop; and a grade is only useful if its severities match what a real reviewer would say, which sometimes means making a finding less severe, not more.

Why one surface is not enough

Most scanners are code scanners. That is a solved, commoditized space: point Semgrep, Gitleaks, or OSV at a repo and you get findings. Useful, but it answers only a third of the question.

When a reviewer actually pokes at your product, they check three different things:

1. The code. Secrets committed to the repo, vulnerable dependencies, client-side auth, and the specific mistakes AI code generators love to make.
2. The running app. What is actually exposed on the internet: TLS and HSTS, security headers, cookie flags, CORS, and files that should never be reachable like /.env or /.git.
3. The data layer. The one that leaks the most PII, and the one generic scanners barely touch: a Supabase table with row-level security switched off, a Firebase database with .read: true.

A tool that only reads your code will hand you a clean bill of health while your patient records are one anonymous HTTP request away from the public internet. So the first architectural decision was that Keelscan had to scan all three, and present them as one posture, not three disconnected reports.

Three surfaces, one grade

The scan engine is built around a single idea: every surface, no matter how different the mechanics, produces the same Finding shape and rolls up into the same A-F grade.

                ┌──────────────── Keelscan scan engine ────────────────┐
  your app  ─►  │   code scan        live-URL scan       config scan    │
                │   secrets/SAST      TLS/headers/CORS    Supabase RLS   │
                │   deps/AI-codegen   exposed files       Firebase rules │
                └───────────────────────┬──────────────────────────────┘
                                        ▼
                     one Finding model  ->  A-F grade + plain-English report

Each surface is a self-contained module (a filesystem scanner, a URL prober, a cloud-config checker), but they all return the same structure: a severity, a category, a title, a location, and a fix. That uniformity is what lets a Supabase misconfiguration and a missing security header live side by side in the same report, under the same grade, and flow through the same "explain this in plain English" layer.

The grade itself is deliberately blunt: any Critical finding fails the whole scan (grade F), because a report that says "mostly fine" while a table leaks PII is worse than useless. No false all-clear.

Surface 1: the code

The code scan is the least novel part, and that is on purpose. Detection here is a commodity, so we wrap best-of-breed open-source scanning (secrets, static analysis, dependency CVEs) rather than reinventing it, and spend our effort on a proprietary rule pack for the mistakes that AI-generated apps ship with over and over:

• A Supabase service_role key referenced in browser-shipped code (it bypasses row-level security; it must never reach the client).
• Authorization decided client-side (if (user.role === "admin") in a React component).
• Secrets hardcoded instead of read from the environment.
• SQL built by string interpolation.
• Tables created in a migration with no row-level security enabled.

Secrets are redacted in the evidence we store, never logged in the clear. The output is raw material for the next layer: the plain-English translation and the fix.

Surface 2: the running app (and the SSRF trap)

The live-URL scan points at your deployed app and checks what an attacker or reviewer sees in the first five minutes: is it HTTPS, does HTTP redirect, is the certificate valid, are HSTS / CSP / X-Frame-Options / X-Content-Type-Options present, are cookies Secure and HttpOnly, is CORS wide open, and are /.env or /.git downloadable. Nearly every early-stage app fails three or four header checks.

Here is the part that is easy to get catastrophically wrong. The moment you let a user type in any URL and have your server fetch it, you have built a Server-Side Request Forgery machine. Without a guard, someone types http://169.254.169.254/ (the cloud metadata endpoint) or http://10.0.0.5:5432 (your internal database) and your scanner happily fetches it and reports back the contents.

So every URL we fetch, the target and every redirect hop, goes through an SSRF guard first:

def validate_target(url):
    host = urlparse(url).hostname
    for info in socket.getaddrinfo(host, port):
        ip = ipaddress.ip_address(info[4][0])
        if (ip.is_private or ip.is_loopback or ip.is_link_local
                or ip.is_reserved or ip.is_multicast):
            raise UnsafeTarget(f"host resolves to a non-public address ({ip})")
    return url

That blocks localhost, private ranges, link-local (which is where the cloud metadata endpoint lives), and anything else off the public internet. Defense in depth backs it up: the scan engine runs on an isolated network that cannot even reach our own database, with all Linux capabilities dropped and a read-only root filesystem. If you build anything that fetches user-supplied URLs, treat the SSRF guard as a load-bearing wall, not a nice-to-have.

Surface 3: cloud data config (the one that actually leaks)

This is the surface we care most about, because it is where a healthcare founder on the AI stack leaks patient data through a single toggle, and it is the surface generic scanners handle worst.

Take Supabase. Its anon key is public by design; it ships in your browser bundle. Row-Level Security (RLS) is what stops that public key from reading your tables. When RLS is off, or a policy is too permissive, the anon key reads everything. The entire check fits in a curl:

# The anon key is public. The question is: what can it read?
curl -s "https://<ref>.supabase.co/rest/v1/patients?select=*&limit=1" \
  -H "apikey: $ANON_KEY" -H "Authorization: Bearer $ANON_KEY"

# []        -> row-level security is doing its job. Good.
# [ {...} ] -> anyone with your public key can download this table.

Keelscan enumerates the tables your PostgREST API exposes (from its OpenAPI description), probes each one anonymously, and any table that hands back a row is a real, provable leak. Two engineering decisions made this safe and useful instead of reckless:

It is value-blind. To prove a table is exposed, we read exactly one row, and only to learn its column names. We never store or echo a single value. For a product whose users may be sitting on protected health information, reading a patient record into your own logs to "prove" it is exposed would be its own breach. So a finding says columns: full_name, dob, diagnosis, never the data behind them.

Severity is PII-aware. A publicly readable blog_posts table might be intentional. A publicly readable patients table is a five-alarm fire. So the classifier escalates: an anonymously readable table is High by default, but Critical when its column names look like personal or health data (email, dob, ssn, patient, diagnosis, and friends). The result reads like a human wrote it:

CRITICAL  Table "patients" exposes personal data to anyone
          1,240 rows readable anonymously - columns: full_name, dob, diagnosis
          Fix: enable Row-Level Security and add access policies:
               ALTER TABLE patients ENABLE ROW LEVEL SECURITY;
               then define policies so users only see their own rows.

The Firebase equivalent is just as blunt: a Realtime Database with open rules serves its entire contents at <db>.firebaseio.com/.json, so we probe it (shallow, node names only) and flag a world-readable database as Critical.

Crucially, that anon key is used in-flight and then discarded. It is never written to our database; the scan record stores only your project's hostname as a label. Least privilege, nothing persisted.

A grade is only useful if it is honest

Here is the lesson that surprised us most, and it is a product lesson disguised as a bug.

Early on, we ran the live-URL scan against a clean, static marketing site hosted on Vercel. It came back a D. The sole finding: "CORS allows any origin," rated High, because the site returned Access-Control-Allow-Origin: *.

That grade was wrong, and a real reviewer would have told us so. A wildcard CORS header on public, non-credentialed content (a static HTML site) is completely normal; anyone can read that content anyway. The genuinely dangerous case is a permissive origin combined with Access-Control-Allow-Credentials: true, which is the account-takeover-class bug. By flagging both the same way, we were punishing a well-configured site and, worse, eroding the one thing the grade exists to build: credibility.

So we calibrated severity by context:

Response	Before	After
Allow-Origin: * with credentials	High	High (the real risk)
Allow-Origin: * without credentials	High -> D	Low -> B

The clean marketing site went from D to B ("passing, minor items"), which is exactly how a security engineer would grade it. The finding still shows up, with an honest note ("fine for public content, but scope it if this ever serves authenticated data"), it just stops tanking the grade.

The principle generalizes: a security grade that cries wolf gets ignored. Sometimes the most important tuning you do is making a finding less severe so the ones that matter are believed.

The moat is not the scanner

It is worth saying plainly: detection is not the hard part. Semgrep, Gitleaks, OSV, and a curl loop can find most of this. What actually helps a non-technical founder pass a security review is everything around the detection:

• Translation. Every finding is rewritten into "here is what an attacker could do, and here is the exact fix," in plain English, no jargon wall.
• Continuity. Your code changes every week. A one-off scan is stale by the next deploy, so Keelscan re-scans on every push and alerts you the moment a new Critical is introduced.
• The artifact. A clean, shareable report you can hand to your customer's security team. That report is the thing that unblocks the deal, and it is the thing that travels to your prospect's reviewer and markets you for free.

The scan engine is the commodity. The plain-English report you can send to a buyer is the product.

What is next

The three surfaces are live today. The roadmap builds toward the other half of the job, closing the deal, not just finding the holes: a hosted Trust Center (the URL you link when a buyer asks whether you are secure), and auto-drafted answers to the security questionnaires (SIG Lite, CAIQ) that buyers send. The goal is a straight line from "scan" to "signed."

Frequently asked questions

What does Keelscan actually check? Three surfaces. Your code (exposed secrets and API keys, vulnerable dependencies, client-side auth, and common AI-generated-code flaws), your running app (TLS, security headers, CORS, exposed files like /.env and /.git), and your cloud data config (Supabase row-level security and public tables, Firebase and Firestore rules). Everything rolls up into one A-F grade.

Do I need to be technical to use it? No, that is the point. Findings are written in plain English with step-by-step fixes, so a non-technical founder can read their top risks and act on them.

Is it safe to give Keelscan my Supabase anon key? Yes. The anon key is the public key that already ships in your app; it is meant to be exposed. Keelscan uses it read-only to check what an anonymous visitor can see, reads at most one row per table (for column names only, never values), and never stores the key.

Will this pass an actual SOC 2 or HIPAA audit? Keelscan gets you review-ready and gives you the report to share; it maps findings to SOC 2 and HIPAA control areas as readiness indicators. It is your continuously-maintained security posture, not a replacement for the auditor.

What stacks does it support? Apps built on the usual AI-coding stacks (Cursor, Lovable, Bolt, Replit, v0) and standard web frameworks. Connect a GitHub repo, paste your app's URL, or link your Supabase or Firebase project.

Wrapping up

The theme running through all three surfaces is the same one that runs through most of our work at CraftRipple: security is not a scan you run once, it is a posture you maintain, and the hard part is making it legible to the people who have to trust you. A tool that finds a leaking Supabase table is useful. A tool that finds it, explains it in a sentence a founder understands, tells them the one-line fix, and hands them a report their customer's security team will accept, that is what actually unblocks revenue.

If you remember one thing from this post: check the surface you are most tempted to ignore. It is almost always the data layer, and it is almost always one toggle away from a very bad day.

Want to see where your app stands? Run a free scan at keelscan.com. Building something in a regulated space and want security done right from the start? Get in touch, we would love to help.