Studio137

<p>Your app works. People sign up, they pay, they log in. None of that tells you whether a stranger can read your users’ data, and you don’t want to end up as the next company listed on Have I Been Pwned.</p><p>In October 2025, Escape.tech scanned 5,600 publicly available vibe-coded apps and found more than 2,000 high-impact vulnerabilities, over 400 exposed secrets and 175 instances of exposed personal data, including medical records and bank account numbers. Their methodology post notes the scans ran passively, so the real figure is probably higher. Seven months later, RedAccess scanned roughly 380,000 apps built on Lovable, Replit, Base44 and Netlify and found about 5,000 holding sensitive corporate data.</p><p>However, this isn’t a Kafka novel, so there is some good news-ish: these apps all seem to break in the same few areas.</p><h2>The key is sitting in your page</h2><p>Lovable runs on Supabase, and so do most Bolt apps. By design, your app hands a public key to every browser that loads it. That’s fine as long as row-level security (RLS) is switched on for every table. RLS is the lock. The public key is the door handle everyone can see.</p><p>When a table has no policy, that key reads the whole table. Researcher Matt Palmer scanned 1,645 publicly listed Lovable apps and found 170 with missing or misconfigured RLS, leaking names, emails, phone numbers, home addresses, payment information, and third-party API keys. It became CVE-2025-48757. Lovable didn’t love this and argued against this, on the grounds that customers own the security of their own app data.</p><blockquote><p>That sounds good for the legal department but gut wrenching news to learn after your first 500 signups.</p></blockquote><h2>October 30 changes the defaults, not what you’ve already shipped</h2><p>New Supabase projects created after May 30, 2026 no longer expose new public tables to the Data API automatically, and on October 30, 2026 the same rule reaches every existing project.</p><p>Tables that are reachable today keep their current grants, so anything already live stays exactly as exposed as it is now. And after October 30, your AI tool will cheerfully create tables that return permission errors until somebody adds an explicit grant.</p><h2>Find your build tool</h2><div class="ed-table-wrap"><table class="ed-table"><thead><tr><th scope="col">Build tool</th><th scope="col">Likely leak</th><th scope="col">Check first</th></tr></thead><tbody><tr><td>Lovable / Bolt <span class="ed-form__hint">(Supabase)</span></td><td>Tables with RLS off, or a policy set to <code>using (true)</code></td><td>The 60-second test below</td></tr><tr><td>Base44</td><td>You inherit the platform’s auth layer, which had a platform-wide bypass in 2025</td><td>Sign up with a fresh account, try to open a private app</td></tr><tr><td>Replit</td><td>Agents holding write access to live data</td><td>Confirm dev and prod are separate, then test a restore</td></tr><tr><td>v0 / Next.js</td><td>Secrets in <code>NEXT_PUBLIC_</code> variables, shipped to every browser</td><td>Grep <code>.env</code> and your build output</td></tr><tr><td>Cursor / Claude Code / Codex</td><td>Authorization gaps between users; features that fetch any URL you hand them</td><td>Log in as user A, request user B’s records by ID</td></tr><tr><td>Firebase</td><td>Rules left in test mode or set to <code>allow read, write: if true</code></td><td>Firebase console, Rules tab</td></tr></tbody></table></div><p>In December 2025, Tenzai had Cursor, Claude Code, Codex, Replit and Devin each build the same 3 apps and found 69 vulnerabilities across the 15 results. None had an exploitable SQL injection or cross-site scripting flaw. What they had instead was broken authorization and request forgery. Models handle the rules everyone agrees on and struggle with anything that depends on your unique app requirements.</p><h2>The 60-second test</h2><p>Grab your Supabase URL and anon key from DevTools, Network tab. Log out. Then run this against one of your own tables:</p><pre><code>curl 'https://YOUR-PROJECT.supabase.co/rest/v1/profiles?select=*&amp;limit=1' \
  -H "apikey: YOUR_ANON_KEY" \
  -H "Authorization: Bearer YOUR_ANON_KEY"</code></pre><p>If rows come back, stop reading and fix that first.</p><h2>Own the parts</h2><p>In July 2025, Replit’s agent deleted SaaStr founder Jason Lemkin’s production database during an explicit code freeze, then told him a rollback wasn’t possible. He recovered the data manually. At the time, Replit ran preview, testing, and production off the same database, and the company was actually marketing itself as the safest place for vibe coding. The CEO called the incident unacceptable and shipped dev/prod separation, one-click restore and a planning-only mode within days.</p><p>Platform bugs land the same way. Wiz found an authentication bypass in Base44 that let anyone create a verified account on a private app using a public app ID. Wix patched it within 24 hours; thankfully, or Twitter would be on fire. However, until they did, every app on the platform was wide open.</p><p>Production means you own the repo, the database project, the migrations, a separate staging database, and a backup you have restored at least once.</p><h2>The score sheet</h2><p>1 point per box you can tick honestly. “I think so” stays unticked.</p><ul class="ed-checklist"><li><p class="ed-checklist__heading">Database</p><ul class="ed-checks"><li><span class="ed-checks__num">01</span><span>RLS on for every table, Security Advisor clean</span></li><li><span class="ed-checks__num">02</span><span>No user-data policy uses </span><code>using (true)</code></li><li><span class="ed-checks__num">03</span><span>User A can’t read or edit user B’s rows</span></li><li><span class="ed-checks__num">04</span><span>Storage buckets private unless meant to be public</span></li></ul></li><li><p class="ed-checklist__heading">Keys and secrets</p><ul class="ed-checks"><li><span class="ed-checks__num">05</span><span>No </span><code>service_role</code><span> or </span><code>sb_secret_</code><span> key in frontend code</span></li><li><span class="ed-checks__num">06</span><span>No Stripe, OpenAI or other private keys in frontend code</span></li><li><span class="ed-checks__num">07</span><span>Git history secret scan comes back clean</span></li><li><span class="ed-checks__num">08</span><span>Every key that touched git has been rotated</span></li><li><span class="ed-checks__num">09</span><code>.env</code><span> is in </span><code>.gitignore</code></li></ul></li><li><p class="ed-checklist__heading">Auth</p><ul class="ed-checks"><li><span class="ed-checks__num">10</span><span>Permissions enforced in RLS or server code, not just hidden buttons</span></li><li><span class="ed-checks__num">11</span><span>Email verification on</span></li><li><span class="ed-checks__num">12</span><span>Login and reset endpoints rate limited</span></li><li><span class="ed-checks__num">13</span><span>Reset links expire and can’t be reused</span></li></ul></li><li><p class="ed-checklist__heading">Input and output</p><ul class="ed-checks"><li><span class="ed-checks__num">14</span><span>User content never renders as raw HTML</span></li><li><span class="ed-checks__num">15</span><span>URL-fetching features restricted to allowed domains</span></li><li><span class="ed-checks__num">16</span><span>Error pages show no stack traces or SQL</span></li></ul></li><li><p class="ed-checklist__heading">Ownership</p><ul class="ed-checks"><li><span class="ed-checks__num">17</span><span>Code in a GitHub repo you own</span></li><li><span class="ed-checks__num">18</span><span>Database project in an org you control</span></li><li><span class="ed-checks__num">19</span><span>Schema changes tracked as migrations, grants included</span></li><li><span class="ed-checks__num">20</span><span>Staging separate, backups on, restore tested</span></li></ul></li></ul><div class="ed-figure"><p class="ed-figure__title">Where you land</p><ul class="ed-scale"><li data-accent=""><span class="ed-scale__range">0 – 8</span><span class="ed-scale__advice">Rotate keys today, treat data as out.</span></li><li><span class="ed-scale__range">8 – 13</span><span class="ed-scale__advice">Pause new signups until 1–10 are ticked.</span></li><li><span class="ed-scale__range">13 – 18</span><span class="ed-scale__advice">Fix the gaps this month.</span></li><li><span class="ed-scale__range">18 – 20</span><span class="ed-scale__advice">Ship it, then book a real review.</span></li></ul></div><p class="ed-cta">Scored under 13, or unsure how to run half of these? <a href="/contact">Book a free audit call</a> and bring your sheet like your school days, and we’ll help you decipher most of these hieroglyphics.</p>

How to take a vibe-coded app to production

<p>Since 2023, AI has become very good at writing code that runs. However, much like Man United’s defensive record, their security scores have barely moved.</p><p>Veracode’s 2025 GenAI Code Security Report ran 80 coding tasks across more than 100 models and found 45% of samples introduced OWASP Top 10 flaws. Its spring 2026 update tracked syntax pass rates climbing from roughly 50% to 95% since 2023, while security pass rates sat between 45% and 55%, regardless of model size.</p><div class="ed-figure"><p class="ed-figure__title">Security failure rate by language</p><ul class="ed-bars"><li data-accent=""><span class="ed-bar__label">Java</span><span class="ed-bar__value">72%</span></li><li><span class="ed-bar__label">JavaScript</span><span class="ed-bar__value">38–45%</span></li><li><span class="ed-bar__label">C#</span><span class="ed-bar__value">Same band</span></li><li><span class="ed-bar__label">Python</span><span class="ed-bar__value">Same band</span></li></ul><p class="ed-figure__note">From the 2025 run. Across all four languages, cross-site scripting failed 86% of the time and log injection 88%.</p></div><p>Four studies speak to this.</p><h2>The evidence</h2><div class="ed-figure"><p class="ed-figure__title">Veracode, 2025 and Spring 2026</p><dl class="ed-defs"><div class="ed-defs__row"><dt>Setup</dt><dd>80 single-function tasks, 100+ models, 4 languages, output checked with a static scanner, no security hints in the prompt.</dd></div><div class="ed-defs__row"><dt>Finding</dt><dd>The numbers above are flat across 3 years of model releases.</dd></div><div class="ed-defs__row" data-accent=""><dt>Caveat</dt><dd>These are bare functions with no project context, and Veracode sells the scanner it used; a bit of <em>I created the problem, but I magnanimously have a solution for you</em>.</dd></div></dl></div><div class="ed-figure"><p class="ed-figure__title">Tenzai, December 2025</p><dl class="ed-defs"><div class="ed-defs__row"><dt>Setup</dt><dd>5 coding agents each built the same 3 apps from identical prompts.</dd></div><div class="ed-defs__row"><dt>Finding</dt><dd>69 vulnerabilities across 15 apps, about 6 critical, and no exploitable SQL injection or XSS at all. Improper authorization showed up everywhere.</dd></div><div class="ed-defs__row" data-accent=""><dt>Caveat</dt><dd>15 apps is a small sample, and Tenzai sells AI security testing, so they are double-dipping in a way.</dd></div></dl></div><div class="ed-figure"><p class="ed-figure__title">Shukla, Joshi &amp; Syed, 2025</p><dl class="ed-defs"><div class="ed-defs__row"><dt>Setup</dt><dd>Security-verified code pushed through repeated AI “improvements”, 400 samples, 4 prompting strategies.</dd></div><div class="ed-defs__row"><dt>Finding</dt><dd>Critical vulnerabilities rose 37.6% after 5 iterations. Security-focused prompts helped in 27% of iterations, mostly rounds 1 to 3.</dd></div><div class="ed-defs__row" data-accent=""><dt>Caveat</dt><dd>This groundbreaking conclusion is based on 10 seed samples in C and Java.</dd></div></dl></div><div class="ed-figure"><p class="ed-figure__title">Mahmud et al., July 2026</p><dl class="ed-defs"><div class="ed-defs__row"><dt>Setup</dt><dd>8,918 C++ programs from 3 open-weight models and human authors, checked 4 ways.</dd></div><div class="ed-defs__row"><dt>Finding</dt><dd>AI code roughly doubled the odds of a confirmed runtime violation. Under static analysis alone, the 2 groups looked equally safe.</dd></div><div class="ed-defs__row" data-accent=""><dt>Caveat</dt><dd>Copilot performs about as well as a human dev, provided you give it a short C++ homework problem and absolutely do not let it near your web application.</dd></div></dl></div><h2>What they add up to</h2><p><strong>The textbook flaws are getting handled.</strong> Tenzai’s agents, working inside real frameworks, produced nothing exploitable in the 2 categories everyone worries about.</p><p><strong>Context is where it breaks.</strong> Who may see which record, whether that URL is safe to fetch. Whether a refund can be a negative number. Those answers live in your app, and the model has never seen them, nor understands them.</p><p><strong>Blind loops make it worse, and a scanner will lie to you.</strong> The iteration study and the C++ study point the same way. If you’ve ever typed “STILL BROKEN, PLEASE FIX!!” 5 times at 1 am, the first one is about you. It’s about all of us.</p><h2>The audit protocol</h2><ol><li><strong>Scan the full git history for secrets.</strong> Deleting the file leaves the commit.</li><li><strong>Run a static scanner.</strong> A clean result is your starting point.</li><li><strong>Document the churn.</strong> Your most-rewritten files carry the most compounded AI edits.</li><li><strong>Hand-check the context flaws.</strong> Every endpoint returning a record: does it verify ownership? Every feature fetching a URL: is the destination restricted? Every flow touching money: what happens with a negative number?</li><li><strong>Attack your own staging.</strong> Log in as user A, request user B’s data by ID. Edit prices in DevTools before submitting. Point URL-fetch features at an internal address. Scanners can’t do this part, and it’s where the interesting findings are.</li></ol><p>None of this says stop using AI to write code. It says the output could use a review and a second pair of experienced eyes.</p><p class="ed-cta">That’s where we come in: <a href="/contact">book a free audit call</a>, and you can finally give your bank account and tokens a breather.</p>

Why 45% of AI-generated code ships with security flaws

<p>The most-quoted AI statistic of 2025 might be wrong. The pattern underneath it keeps showing up anyway, like a bad haircut in every family photo from your punk days.</p><p>MIT’s Project NANDA found roughly 95% of company generative AI pilots delivered no measurable profit-and-loss impact, drawing on interviews, surveys, and 300 public deployments. The pushback came fast. Wharton’s Kevin Werbach argued that if the team stands behind the claim, it should publish the supporting data, and otherwise withdraw the report. He has a point, and the headline has been stretched well past what the study measured.</p><p>Better-researched data says the same thing. McKinsey surveyed 1,993 people across 105 countries and found 23% were scaling an agentic AI system somewhere in the business, with no single function passing about 10%. Gartner predicted in June 2025 that more than 40% of agentic AI projects would be canceled by the end of 2027 over costs, unclear value, and weak risk controls.</p><h2>Why pilots stall</h2><p><strong>The tool doesn’t actually fit the work.</strong> Generic assistants demo beautifully, then meets the real world and a real workflow with 14 exceptions in it.</p><p><strong>“Agent” often means a chatbot under a massive trenchcoat.</strong> Gartner reckons roughly 130 of the thousands of agentic AI vendors are doing something genuinely agentic.</p><p><strong>Not all rules are meant to be broken.</strong> In July 2025, Replit’s agent deleted a production database during an explicit code freeze. The CEO’s response was that it should never have been possible, and the fixes were structural: separate dev and production databases, one-click restore, a planning-only mode. An instruction in a chat box can be a suggestion, but a permission the agent doesn’t have is a guardrail.</p><h2>The gap between a demo and a deployment</h2><div class="ed-table-wrap"><table class="ed-table"><thead><tr><th scope="col"></th><th scope="col">Demo</th><th scope="col">Deployment</th></tr></thead><tbody><tr><td>Data</td><td>Clean export</td><td>Live systems, messy records, permissions</td></tr><tr><td>Volume</td><td>10 runs, supervised</td><td>Hundreds a day, unsupervised</td></tr><tr><td>On failure</td><td>Someone reruns it</td><td>Retries, fallbacks, a human queue</td></tr><tr><td>Access</td><td>Admin keys</td><td>Its own scoped credentials</td></tr><tr><td>Actions</td><td>Suggests</td><td>Acts, within limits</td></tr><tr><td>Environment</td><td>One shared database</td><td>Staging separate from production</td></tr><tr><td>Record</td><td>Screenshots</td><td>Every tool call logged</td></tr><tr><td>Owner</td><td>Whoever built it</td><td>A named person with a metric</td></tr></tbody></table></div><p>Four medicines could cure it.</p><ol><li><strong>Scope it</strong> to one job with one owner and one number that already appears in an ops report.</li><li><strong>Wire it</strong> into the real CRM, inbox or ticketing system with its own credentials.</li><li><strong>Fence it:</strong> allowlisted tools, human sign-off on anything irreversible, a spend cap, a kill switch.</li><li><strong>Log every action</strong> with input, output, trigger and timestamp, because an agent no one can audit is one no team will trust with anything that matters.</li></ol><p>McKinsey’s high performers are considerably more likely to have defined human-in-the-loop validation than everyone else.</p><h2>The spec card</h2><p>If a team can’t fill this in on one page, the pilot isn’t scoped yet.</p><div class="ed-figure"><p class="ed-figure__title">Agent spec card</p><div class="ed-form"><div class="ed-form__row"><span class="ed-form__label">The one job</span></div><div class="ed-form__row"><span class="ed-form__label">Owner </span><span class="ed-hint">a person, not a committee</span></div><div class="ed-form__row"><span class="ed-form__label">Success metric </span><span class="ed-hint">already in a report</span></div><div class="ed-form__row"><span class="ed-form__label">Reads from </span><span class="ed-hint">and with which credentials</span></div><div class="ed-form__row"><span class="ed-form__label">May do</span></div><div class="ed-form__row"><span class="ed-form__label">May never do</span></div><div class="ed-form__row"><span class="ed-form__label">Needs sign-off for</span></div><div class="ed-form__row"><span class="ed-form__label">Hands off to a human when</span></div><div class="ed-form__row"><span class="ed-form__label">Logs</span></div><div class="ed-form__row"><span class="ed-form__label">Kill switch </span><span class="ed-hint">1 step, 1 person</span></div><div class="ed-form__row"><span class="ed-form__label">Monthly cost ceiling</span><span class="ed-form__value">$</span></div></div></div><h2>The argument against all of this</h2><p>You’ll hear it, so here it is. CrewAI’s 2026 survey reports 65% of large enterprises already using AI agents, with 81% saying adoption is scaling or expanding. CrewAI sells agent software and surveyed 500 executives at companies above $100M in revenue, so take it with a kilo of salt. Using agents and scaling them to measurable financial impact are separate goals, and only the second one really matters.</p><h2>Build in-house or bring someone in</h2><p>In the MIT data, tools bought from specialized vendors succeeded about 67% of the time, while internal builds succeeded roughly a third as often. The reasoning is there: a team that has shipped agents before has already built the integration, guardrails, and logging that internal pilots treat as phase 2.</p><p class="ed-cta">We’re an outside partner, so naturally we like that finding. <a href="/contact">Book a free audit call</a>, and we’ll tell you what sits between your pilot and production.</p>

What the 5% of AI pilots that reach production do differently

<p>For 2 years, vendors sold retrieval as the cure for hallucination. Point the model at your documents, we were told, and it can only tell the truth. This was the new “Veritaserum”.</p><p>Well, Stanford’s RegLab tested that. Their preregistered study of the leading legal research tools found Lexis+ AI and Thomson Reuters’ products hallucinating between 17% and 33% of the time, against 43% for raw GPT-4. Accuracy varied more than the marketing suggested: Lexis+ AI answered 65% of queries accurately, Westlaw’s AI-Assisted Research 42%, and Ask Practical Law AI around 20%, with incomplete answers on more than 60% of queries.</p><div class="ed-figure"><p class="ed-figure__title">Hallucination rate, legal research tools</p><ul class="ed-bars"><li data-accent=""><span class="ed-bar__label">Lexis+ AI</span><span class="ed-bar__value">17%</span></li><li data-accent=""><span class="ed-bar__label">Westlaw AI-AR</span><span class="ed-bar__value">33%</span></li><li><span class="ed-bar__label">GPT-4, no retrieval</span><span class="ed-bar__value">43%</span></li></ul><p class="ed-figure__note">Stanford RegLab, preregistered study of the leading legal research tools.</p></div><p>Yes, retrieval cut the rate by more than half, which is incredible. It also left 1 in 6 answers fabricated at the best-performing vendor, in a domain where a fabricated citation gets you sanctioned.</p><p>A February 2026 research roundup adds a detail that is absolutely brutal: these systems sound equally confident whether they’re right or wrong.</p><h2>Why grounding leaks</h2><p>Retrieval fails silently. The search returns nothing useful, and the model answers anyway from what it learned in training. The answer may even be correct, which is worse, because it means your evaluation scored a lucky guess as a win.</p><p>An August 2026 evaluation framework puts numbers on this. The researchers planted “knowledge-gap canaries”, questions whose answers verifiably do not exist in the knowledge base, so any answer at all proves the system is drawing on the model rather than your documents. Across 3 commercial RAG products, accuracy when answering clustered tightly. What separated them was the policy governing when <em>not</em> to answer, with canary violation rates running as high as 98.1%. Conventional accuracy scoring ranked those products in the opposite order.</p><blockquote><p>Which means most RAG evaluations reward exactly the behavior RAG was bought to prevent.</p></blockquote><h2>What a trustworthy answer looks like</h2><div class="ed-compare ed-compare--two"><div class="ed-compare__col"><p class="ed-compare__label">Ungrounded</p><p class="ed-compare__q">What’s our refund window for annual enterprise plans?</p><p>Annual enterprise plans carry a 30-day refund window, prorated after day 14.</p><p class="ed-hint">Confident, specific, sourced from nothing.</p></div><div class="ed-compare__col" data-accent=""><p class="ed-compare__label">Grounded, with an abstention policy</p><p class="ed-compare__q">What’s our refund window for annual enterprise plans?</p><p>The policy documents I can see cover monthly plans only (Refund Policy v4, §2). I don’t have a document covering annual enterprise terms. Escalating to billing.</p><p class="ed-hint">[ Refund Policy v4 ] [ escalate ]</p></div></div><p>The second answer is the harder engineering problem and the only one you can put in front of a customer.</p><h2>5 tests before you deploy</h2><ol><li><strong>Canary questions.</strong> Write 20 questions your knowledge base provably can’t answer. Any confident answer is a failure, even a correct one.</li><li><strong>Citation contract.</strong> Every factual claim carries a retrieved passage ID. Claims without one get blocked before they reach the user.</li><li><strong>Abstention rate.</strong> Measure how often the system says it doesn’t know, and check it against how often it should have. A 0% refusal rate is a red flag, not a feature.</li><li><strong>Multi-turn drift.</strong> Test conversations, not single questions. Retrieval quality degrades badly once context accumulates.</li><li><strong>Retrieval recall, measured separately.</strong> If the right document never gets retrieved, no amount of prompt work saves the answer.</li></ol><p>Run these before launch, then keep running them, because your documents change and the model behind the system gets swapped without warning.</p><p class="ed-cta">Thinking about putting one in front of customers and we’ve scared you enough? <a href="/contact">Book a free audit call</a>, and we’ll go through your retrieval setup and your refusal behaviour.</p>

RAG in production: why retrieval-grounded AI still makes things up

<p>Somebody, somewhere, someday will quote you a mobile traffic number.</p><div class="ed-figure"><p class="ed-figure__title">What share of web traffic is mobile? (2026)</p><ul class="ed-bars"><li><span class="ed-bar__label">Statcounter, June 2026</span><span class="ed-bar__value">51.5%</span></li><li><span class="ed-bar__label">DataReportal, mid-2026</span><span class="ed-bar__value">51.6%</span></li><li><span class="ed-bar__label">Cloudflare Radar</span><span class="ed-bar__value">43%</span></li><li><span class="ed-bar__label">Blog aggregators</span><span class="ed-bar__value">62–64%</span></li></ul><p class="ed-figure__note">Cloudflare counts HTTP requests rather than page views. Blog aggregator figures are early 2026.</p></div><p>Statista’s read of Statcounter has mobile at 51.48% in Q2 2026, down from 62.73% a year earlier. Cloudflare counts HTTP requests instead of page views and lands around 43%. Plenty of 2026 articles still cite 62% to 64%. Every one of those is defensible, which means whichever number supports the decision you’d already made is available to you.</p><h2>Where it does turn</h2><p>Half the mobile web is very slow. Per the HTTP Archive Web Almanac using Chrome field data, 48% of mobile origins passed all 3 Core Web Vitals against 56% on desktop. The May 2026 Chrome UX Report reading, which blends devices, puts 55.9% of origins passing. Whichever way you look at it, roughly half of everything published fails a basic performance bar on a phone.</p><p>That’s the actual competitive landscape. A carefully built responsive web app beats a hastily built native one on the metrics users feel, and it does it without an app store review queue.</p><h2>The decision tree</h2><ul class="ed-steps"><li><span class="ed-steps__q">Do you need Bluetooth, NFC, USB, or deep OS integration?</span><ul class="ed-branch"><li data-accent=""><span class="ed-branch__key">Yes</span><span>Native. Nothing else clears this.</span></li><li><span class="ed-branch__key">No</span><span>Keep going.</span></li></ul></li><li><span class="ed-steps__q">Do you need reliable background sync, or offline-first on iPhone?</span><ul class="ed-branch"><li data-accent=""><span class="ed-branch__key">Yes</span><span>Native. iOS PWAs still have no background sync.</span></li><li><span class="ed-branch__key">No</span><span>Keep going.</span></li></ul></li><li><span class="ed-steps__q">Is App Store presence a real acquisition channel for you, or a requirement of your buyer?</span><ul class="ed-branch"><li data-accent=""><span class="ed-branch__key">Yes</span><span>Native, or a wrapper. Apple blocks PWAs from the App Store under guideline 4.2.2.</span></li><li><span class="ed-branch__key">No</span><span>Keep going.</span></li></ul></li><li><span class="ed-steps__q">Do you sell subscriptions or digital goods?</span><ul class="ed-branch"><li data-accent=""><span class="ed-branch__key">Yes</span><span>Web first, deliberately. 15–30% store commission on every transaction is a tax on a business model you haven’t proven yet.</span></li><li><span class="ed-branch__key">No</span><span>Keep going.</span></li></ul></li><li><span class="ed-steps__q">Everything else</span><ul class="ed-branch"><li><span class="ed-branch__key">Then</span><span>Responsive web. Set the trigger that would push you to native now, while you’re calm, and write it down.</span></li></ul></li></ul><h2>What native costs you</h2><div class="ed-table-wrap"><table class="ed-table"><tbody><tr><td>Two native codebases</td><td>2x the engineering, 2x the QA, 2x the releases</td></tr><tr><td>React Native / Flutter</td><td>~1.3x a web build, plus platform quirks</td></tr><tr><td>App Store review</td><td>Days per release, occasional rejections</td></tr><tr><td>Store commission</td><td>15–30% on in-app purchases</td></tr><tr><td>Install friction</td><td>Every user must decide to download first</td></tr><tr><td>Update cycle</td><td>Users on old versions for months</td></tr></tbody></table></div><p>Against that: since iOS 16.4, home screen web apps on iPhone can send push notifications, and as of iOS 26, every site added to the Home Screen opens as a web app by default. Still missing: background sync, Bluetooth, NFC, and App Store distribution.</p><p>The cure cured enough symptoms but not all. Apple has a well-documented commercial interest in keeping it just sick enough.</p><h2>The truth</h2><p>Most founders asking this question just want permission to build the native app, because native feels like a real company. The expensive mistake isn’t choosing native. It’s choosing 2 separate native codebases before you know which features are actually useful.</p><p class="ed-cta">Working out which grass is greener between those 2? <a href="/contact">Book a free discovery call</a>, and we’ll bring our best gardeners.</p>

Web app or native app?

<p>Let’s start with the research you’ll find if you go looking for “market or product fit.”</p><div class="ed-table-wrap"><div><div><div><p></p></div></div></div><table class="ed-table"></table>Global assessment software market size<div><p>Source</p><p>Market size</p><p>CAGR</p><p>Global Growth Insights, 2025</p><p>$310M</p><p>5.4%</p><p>Verified Market Reports, 2024</p><p>$1.5B</p><p>9.2%</p><p>Business Research Insights, 2026</p><p>$2.71B</p><p>16.5%</p><p>Dataintelo, 2025</p><p>$5.8B</p><p>10.8%</p></div></div><p>Same market. Same ballpark period. An 18× spread. And the CAGR? You can pick the one that makes the vendor’s story look best.</p><p>Let’s ask the question that actually matters in all of this: does your specific framework work as software?</p><h2>What you actually have</h2><p>A framework you deliver by hand. You ask questions in a workshop, you listen for things the client doesn’t say, you weigh what you hear against 40 years of experience, and you write a report over a weekend.</p><blockquote><p>Three of those four steps are judgment. That’s the part that has to become explicit, and it’s the part consultants find uncomfortable, because documenting your weighting means admitting you had one.</p></blockquote><h2>6 questions that decide whether it works</h2><ol><li><strong>Can every question be answered without you in the room?</strong> If a question needs interpretation, it needs rewriting, or it needs to be a free-text field you review manually.</li><li><strong>Does every answer have a number attached?</strong> Not a label. A number, and a reason for that number.</li><li><strong>Do your dimensions have weights, and can you defend them?</strong> “Culture matters twice as much as tooling” is a fine claim. It needs to be a stated claim, not a result of you writing twice as many questions about culture.</li><li><strong>What does a score of 61 mean, specifically?</strong> If 61 and 68 produce the same advice, your bands are the output, not the score.</li><li><strong>What are you comparing against?</strong> Percentiles from your own respondent pool need a pool. Until then, bands against a standard you define are more honest than a benchmark with 12 data points behind it.</li><li><strong>Can you write the report for every possible result?</strong> All of them. This is where projects stall, and it’s usually 60% of the build.</li></ol><p>If you answered no to 3 or more, the framework just isn’t ready for software yet. However, like finding a magical $20 in your jacket, there is good news: that’s a scoping problem, not a coding issue, and it’s cheaper to solve.</p><h2>The scoring worksheet</h2><p>Fill this in before anyone writes code.</p><div class="ed-figure"><p class="ed-figure__title">Per dimension</p><div class="ed-form"><div class="ed-form__row"><span class="ed-form__label">Dimension</span><span class="ed-form__value">Weight — %</span></div><div class="ed-form__row"><span class="ed-form__label">Question 1 </span><span class="ed-hint">answers score</span><span class="ed-form__value">— / — / —</span></div><div class="ed-form__row"><span class="ed-form__label">Question 2 </span><span class="ed-hint">answers score</span><span class="ed-form__value">— / — / —</span></div><div class="ed-form__row"><span class="ed-form__label">Bands</span><span class="ed-form__value">Low 0–__ · Mid __–__ · High __–100</span></div><div class="ed-form__row"><span class="ed-form__label">What the report says at each band</span></div></div><p class="ed-figure__note">Repeat per dimension. Total score = sum of weighted dimensions, and the weights must equal 100%.</p></div><div class="ed-figure"><p class="ed-figure__title">The two lines that look like small print</p><div class="ed-form"><div class="ed-form__row"><span class="ed-form__label">Tie-break rule</span></div><div class="ed-form__row"><span class="ed-form__label">Missing answer rule</span></div></div><p class="ed-figure__note">They’re the ones that produce embarrassing results in front of a client, when 2 people get identical scores from opposite answers, or when a skipped question quietly counts as zero.</p></div><h2>Build or buy</h2><div class="ed-compare ed-compare--two"><div class="ed-compare__col"><p class="ed-compare__label">Off-the-shelf</p><p class="ed-compare__q">Pointerpro, Agolix, ScoreApp, Interact</p><p><strong>Covers:</strong> single-respondent scoring, weighted questions, branded PDF output, email capture, simple benchmarks.</p><p><strong>Costs:</strong> monthly fee per seat, their branding rules, their export limits, their roadmap.</p></div><div class="ed-compare__col" data-accent=""><p class="ed-compare__label">Custom</p><p class="ed-compare__q">Starts to earn its keep when:</p><p>Several people answer from the same organization and the results need to aggregate (360s, team diagnostics).</p><p>Respondents come back in six months and you need to measure the delta.</p><p>The benchmark database is an asset you want to own and sell.</p><p>Multiple third-party integrations are required, as well as AI tooling.</p><p>The assessment is the product, not just a lead magnet.</p><p>It needs to sit inside an existing platform, LMS, or CRM.</p></div></div><p>You can start off-the-shelf, then get 5 real organizations through the door so you can get a feel for it.</p><p>However, by year two, the assessment has scaled, organizations and respondents are coming back, benchmarks are accumulating, integrations are multiplying by the month. Then the platform you loyally started with starts telling <em>you</em> what <em>your</em> product can and can’t do. You deserve better, out of your years of expertise.</p><p class="ed-cta">Think you are ready for the big leagues and the point where your framework needs to become software? <a href="/contact">Book a free discovery call.</a> Feel free to bring the scoring model or the 15 Excel sheets wearing disguises, pretending to be one.</p>

Turning a consulting methodology into an assessment product

<p>There’s a version of this conversation happening in a lot of companies. Someone built the tool that runs scheduling, or claims, or inventory. They built it 14 years ago. They’ve maintained it since the start, mostly alone. And they’re leaving in 30 days.</p><p>The panic usually arrives as a staffing question: who takes it over? That’s the wrong question, and answering it wastes the time you have left.</p><h2>Why hiring doesn’t solve it</h2><p>A team of 10 engineers can still have a system only one of them can touch. The industry calls this the bus factor, the number of people who’d have to vanish before work stops. A 2016 study of 133 popular open-source projects found 46% had a bus factor of 1 and another 28% sat at 2. Another read of the same dataset puts 65% at 2 or lower, with fewer than 10% above 10.</p><p>Knowledge generally concentrates around the person who can move fastest and knows the system inside out, and they can move fastest because they built it, and they built it in a way that made sense to them and the business at the time.</p><h2>The 5-question triage</h2><p>Ask these about your system, and you can answer them out loud with the person who owns it in the room.</p><ol><li>Can someone else deploy a change to production this week without asking them?</li><li>Does a written explanation exist of <em>why</em> it works the way it does, not just what it does?</li><li>If it broke on a Sunday, could a competent outsider find the cause from logs and code alone?</li><li>Is the logic in the code, or is it in spreadsheets, scheduled tasks and 1 person’s habits?</li><li>Could you hire for the stack it’s written in, at a normal salary, in your city?</li></ol><div class="ed-figure"><p class="ed-figure__title">Reading your answers</p><ul class="ed-scale"><li><span class="ed-scale__range">4 or 5 yes</span><span class="ed-scale__advice">A people problem. Pair the person leaving with a successor, write things down, and you’ll be fine. You don’t need an agency for this.</span></li><li data-accent=""><span class="ed-scale__range">2 or fewer</span><span class="ed-scale__advice">A system problem, and no amount of handover meetings will fix it. The knowledge isn’t transferable because the artifact isn’t transferable.</span></li></ul></div><h2>What untransferable actually costs</h2><p>A German engineering firm described a client issue where onboarding a successor took 6 months because the knowledge lived in one person’s head, and put the cost of reverse-engineering long-adapted custom software at several times the original build, using an example where a €200,000 system ran past €500,000 to reconstruct. The pattern matches what we see: reconstruction costs more than construction, because the archaeology is the expensive part and the original builder did it once in their head.</p><h2>What we’d actually do</h2><p>The window before a departure is the easiest this work will ever be. The person is still there to answer questions that would otherwise take an agency 3 weeks to reconstruct. This is what we would do to help.</p><div class="ed-figure"><p class="ed-figure__title">The handover, in order</p><dl class="ed-defs"><div class="ed-defs__row"><dt>Weeks 1–2</dt><dd>Sit with the team and map what the system actually does, including the processes that never made it into the code.</dd></div><div class="ed-defs__row"><dt>Weeks 3–4</dt><dd>Turn that behaviour into tests, so the rules and logic patterns don’t concentrate in one person’s head.</dd></div><div class="ed-defs__row"><dt>Then</dt><dd>Rebuild what can’t be maintained, keep what can, and simplify the parts where 14 years of patches have piled up.</dd></div><div class="ed-defs__row"><dt>Handover</dt><dd>Documentation for the next person who has to work on it, in a simple and understandable manner.</dd></div><div class="ed-defs__row"><dt>Post</dt><dd>We maintain it and handle any transition pain points.</dd></div></dl></div><p>The simplification step is where most of the value sits. Systems that grow around one person accumulate workarounds for problems that stopped existing years ago.</p><h2>When to start</h2><p>Twelve months out is comfortable. Six months is workable; 3 is a pinch, but it’s still worth doing, because a rebuilt system with a maintenance contract beats a working system no one can change.</p><p>The failure case is waiting until the leaving date, then hiring someone to “document the existing system”, and it answers none of the questions you’ll functionally have.</p><p class="ed-cta">Got a system and a resignation date? <a href="/contact">Book a call.</a> We’ll learn what they know while they’re still here, rebuild what needs rebuilding, straighten out what’s grown crooked, document it properly, and keep maintaining it afterwards if need be. Because “Dave knows how it works” is not good planning.</p>

The person who built your system is leaving

<p>Google this question, and every result is published by a company that sells one of the two answers. The white-label vendors have total cost charts ending in their favor. The custom shops have charts ending in theirs.</p><p>We do both. We’ve white-labeled and customized an established platform for a client who needed to be live for a program launch; we’ve built products from scratch for clients whose software is the product they sell, and we license our own products out for other organizations to run under their brand. That’s the only reason this article can give you a straight answer.</p><h2>The 3-year shape</h2><div class="ed-table-wrap"><table class="ed-table ed-table--plain"><thead><tr><th scope="col"></th><th scope="col">White-label</th><th scope="col">Custom</th></tr></thead><tbody><tr><td>Year 0</td><td>Setup, configuration, branding</td><td>Design and build</td></tr><tr><td>Year 1–2</td><td>Subscription, per-seat, per-transaction</td><td>Hosting and maintenance</td></tr><tr><td>Year 3+</td><td data-accent="">Subscription plus the success tax</td><td>Hosting and maintenance</td></tr></tbody></table></div><p>The success tax is the part people miss. Most white-label pricing scales with the product you’re trying to grow: users, transactions, seats. Vendors in this space describe annual price increases of 15 to 30% as common in the current market, and one analysis puts SaaS total cost of ownership at 2.5 to 4 times the headline price once implementation, integration, and internal admin are counted.</p><p>On the other side, a white-label build can genuinely cut initial development cost by 60 to 75%, according to a vendor who sells white-label apps. For a lot of organizations, that’s decisive, and correctly so. Being live in 6 weeks with 70% of what you wanted beats being live in 9 months with all of it, when the program date is impending.</p><h2>The 5 questions that decide it</h2><ol><li><strong>Is your workflow the median?</strong> White-label platforms are optimized for the middle of their market. If your differentiation is the process, that difference can’t be purchased off the shelf.</li><li><strong>Does their pricing model fight your business model?</strong> Per-seat pricing on a product you sell per-organization is a structural problem that gets worse as you succeed.</li><li><strong>Is the software the product, or the supporting act?</strong> A foundation running a member app is buying a tool. An accreditation body whose assessment platform is the service it sells is buying its own product from somebody else.</li><li><strong>How much configuration does it take to fit?</strong> When the configuration quote for a white-label platform approaches the cost of a build, then the platform wasn’t built for your use case.</li><li><strong>What does leaving look like?</strong> Ask for a data export, in writing, before you sign. A lot of pain can come from data you can’t take with you.</li></ol><h2>Which one wins</h2><div class="ed-compare ed-compare--two"><div class="ed-compare__col"><p class="ed-compare__label">When the white label wins</p><p>You have a real date, and 6 weeks is the honest runway.</p><p>Volume is uncertain and you’re still testing whether there is actual market demand.</p><p>The compliance burden is heavy and the vendor carries it.</p><p>Your organization has no appetite to own software, now or later.</p></div><div class="ed-compare__col" data-accent=""><p class="ed-compare__label">When custom solutions pay for themselves</p><p>The workflow is the differentiator.</p><p>Per-unit fees have started scaling with your growth.</p><p>You’re paying for a whole tier to get 1 feature.</p><p>Integration with your existing systems is the actual requirement.</p><p>The data accumulating inside the platform is the asset.</p><p>You intend to license or sell the product itself.</p></div></div><p class="ed-cta">Got a white-label option and a build decision having a Mexican standoff across the table? <a href="/contact">Book a discovery session</a>, and we’ll tell you which one we’d take in your position.</p>

White-label platform or custom build?

<p>Twelve weeks is a budget, and it gets spent whether or not you decide how when you build.</p><p>The happy ending of that conversation happens in week 1, deliberately, with key stakeholders present. The tragic ending happens in week 9, over email, while your team waits.</p><h2>The 3 questions that make the cuts obvious</h2><ol><li><strong>Which single user journey has to work on launch day?</strong> Name one. An example of a user journey: sign-ups.</li><li><strong>What breaks the business if it’s wrong?</strong> Money, permissions, regulated processes.</li><li><strong>What are we willing to do manually for 3 months?</strong> Onboarding, invoicing, support and provisioning could all be examples.</li></ol><p>Most of what founders ask us to automate in week 2 turns out to happen 4 times a month and take 11 minutes.</p><p class="ed-cta">Good brisket needs fat. Bad brisket has too much of it. Same goes for digital solutions. <a href="/contact">Book a free discovery call</a>, and we’ll help you figure out which is which before you build.</p>

What a 12-week build actually cuts

<p>Two proposals, same brief. One is $40,000. One is $100,000. The cheap one is shorter, friendlier, and available to start Monday.</p><p>Sometimes it’s the right call. Genuinely. Before the argument for the expensive one, here’s when to take the cheap quote and reap the savings: the scope is small and fully specified, the product is disposable or experimental, no customer data or revenue flows through it, and you have someone in-house who could pick it up if the relationship ends.</p><h2>Where the money actually goes</h2><p>For a given system, maintenance typically accounts for 75 to 80% of total cost of ownership, according to the Consortium for Information and Software Quality. The build is the visible fifth of the iceberg.</p><p>CISQ’s 2022 update puts the cost of poor software quality in the US at $2.41 trillion, with accumulated technical debt at a further $1.52 trillion. A 2023 Stripe and Harris Poll study found developers spend around a third of their time dealing with bad code and legacy systems, which on a $180,000 salary works out to roughly $59,400 a year per engineer before they write anything new.</p><h2>What the gap usually contains</h2><div class="ed-table-wrap"><table class="ed-table"><thead><tr><th scope="col">In the expensive quote</th><th scope="col">Often missing from the cheap one</th></tr></thead><tbody><tr><td>Automated tests</td><td>“We test manually.”</td></tr><tr><td>A staging environment</td><td>Changes go straight to production.</td></tr><tr><td>Code review</td><td>One developer, unreviewed.</td></tr><tr><td>Documented handover</td><td>A zip file and a good luck wish.</td></tr><tr><td>Error monitoring and logs</td><td>You find out when a customer calls.</td></tr><tr><td>A data model built for v2</td><td>A data model built for the demo.</td></tr><tr><td>Security review</td><td>No security review.</td></tr><tr><td>A named point of contact after launch</td><td>A support email address.</td></tr></tbody></table></div><blockquote><p>None of these are luxuries. They’re the difference between a system your next employee can work on and one they beg you to replace.</p></blockquote><h2>The 5 questions that make quotes comparable</h2><ol><li>What exactly is excluded from this price?</li><li>Are automated tests included, and roughly what coverage?</li><li>What does support cost after launch, monthly or yearly, in writing?</li><li>Who owns the code, and when does it transfer?</li><li>If I hand this to another agency next year, what would they need that isn’t in the deliverables?</li></ol><p>Ask both bidders the same questions. The price gap usually shrinks dramatically, and occasionally reverses.</p><p class="ed-cta">Got a quote and a gap you can’t put your finger on? Send it over on our <a href="/contact">free discovery call.</a> We’ll tell you what’s hiding in the gap, or give you a technical spec document you can take to other vendors. Think of us as enlightened beings who speak developer.</p>

Why the cheaper quote usually costs more

<p>New site launches on a Friday. By Monday, organic traffic is down, the contact form is broken, and the service pages that took 4 years to rank are returning 404s.</p><h2>The statistic, interpreted properly</h2><p>You’ll be told the average site takes 523 days to recover. That number comes from a study of 892 domain migrations, where 17% never returned to their original traffic even after 1,000 days. The same dataset recorded recoveries at 19, 22, and 23 days. A separate analysis of 1,052 domain migrations put the median at 304 days, with 13.9% never recovering.</p><p>For a competently run domain redesign, the realistic expectation is a 5 to 15% dip in the first weeks and recovery within 4 to 8.</p><h2>What actually causes the losses</h2><ol><li><p><strong>URLs changed without 1-to-1 redirects.</strong> Every old URL needs a new address, mapped individually.</p><p class="ed-weak"><strong>Cost</strong> The rankings on every page you missed.</p></li><li><p><strong>Ranking pages rewritten shorter.</strong> The redesign trimmed the copy that was doing most of the ranking work.</p><p class="ed-weak"><strong>Cost</strong> Pages that look cleaner but perform worse.</p></li><li><p><strong>Internal links pointing through redirect chains.</strong> Your website gets slower and worse.</p><p class="ed-weak"><strong>Cost</strong> Slow, invisible problems that are hard to diagnose.</p></li></ol><h2>The 5 things to demand before launch</h2><ol><li><strong>A URL inventory built from multiple sources.</strong> A crawler alone misses orphaned pages that still hold rankings and links.</li><li><strong>A 1-to-1 redirect map,</strong> signed off before launch week.</li><li><strong>A staging check that includes the robots and noindex tags.</strong> It is the single most common way a launch goes dark.</li><li><strong>Metadata and schema parity</strong> on the top 50 pages, verified against the rendered source rather than a screenshot.</li><li><strong>A pre-launch traffic baseline,</strong> so the argument in week 3 is about data instead of memory.</li></ol><p>Then hold your nerve. A dip in the first fortnight is normal. Sessions down more than 20% with no recovery trend after 2 to 4 weeks is a technical problem, and the fix is repairing the migration.</p><h2>One myth worth killing</h2><p>“Google needs 180 days to process the move.” The 180 days is how long Google’s Change of Address tool forwards signals, and how long you’re advised to keep redirects live. It’s a retention window, and it gets quoted as though waiting 6 months is a viable plan.</p><p class="ed-cta">Planning a rebuild, or 3 weeks into a bad one? <a href="/contact">Book a migration discovery call</a> with us. Before launch, it’s just a checklist. After launch, it’s archaeology.</p>

Ideas shaping what we build next.

How to take a vibe-coded app to production

Not sure where to begin?

Latest Posts

Why 45% of AI-generated code ships with security flaws

What the 5% of AI pilots that reach production do differently

Browse the desk

Ideas shaping what we build next.

How to take a vibe-coded app to production

Not sure where to begin?

Why 45% of AI-generated code ships with security flaws

What the 5% of AI pilots that reach production do differently

Browse the desk

Stay close to the signal.