Lingua Detective vs Build It In-House: the real cost of DIY language QA
Verdict: franc, fastText, and CLD3 are excellent, fast, genuinely accurate libraries for classifying a string of text — there's no reason to replace them if that's all you need. The real cost isn't the library, it's everything around it: the crawler, the Chromium renderer, template stripping, scheduling, and reporting. Lingua Detective bundles all of that for $6 on a 500-page site; building and maintaining the equivalent in-house rarely costs less, even before counting your team's time.
| Build it in-house | Lingua Detective | |
|---|---|---|
| What it is | OSS detector (franc/fastText/CLD3) + your own crawler, renderer, template stripper, and reports | Managed crawl + offline detection + per-page verdicts, ready to use |
| Setup | Days to weeks: crawler, headless Chromium, dedup, scheduling, storage, dashboards | Minutes: supply URLs, or let Lingua Detective spider the site |
| Pricing | Library is free; you pay in dev time, hosting, and maintenance | $0.015 → $0.006/page in bulk, or $0.002/call via pay-per-call API |
| Privacy / offline | Fully offline if you keep the whole stack in-house | Offline detection by default — nothing sent to a third party |
| Output | Raw language codes, unless you build verdict logic yourself | Per-page pass/fail verdict, AI arbitration on edge cases, ready-made reports |
| Best for | Teams with spare engineering capacity and unusual, specific requirements | Teams who want an audited answer without owning a crawler |
Pick Build it in-house if…
- You already maintain a crawler and a Chromium rendering pipeline for other purposes, and detection is a small addition to it.
- Your requirements are specific enough — proprietary formats, unusual language rules — that no off-the-shelf tool fits well.
- You operate at a scale where per-page pricing would genuinely exceed years of engineering and hosting cost, not just the first sprint.
- Nothing can ever leave your network, including calls to a third-party arbitration service.
- You have a team willing to own WAF and anti-bot handling, plus scheduled re-runs, for the long haul — not just the first version.
Pick Lingua Detective if…
- You want a pass/fail verdict per page, not a language code you still have to turn into a decision yourself.
- You'd rather spend $6-$17 than 3-5 developer-days building a crawler, a renderer, and a template stripper.
- Your site sits behind a WAF or anti-bot service and you don't want to build and maintain workarounds yourself.
- You want ambiguous or mixed-language pages arbitrated by AI instead of silently mis-scored by a fast heuristic.
- You need this occasionally or on a schedule, not as a new internal service someone has to keep patched and running.
Pricing math
Auditing a 500-page site:
- In-house: the detection library is free, but you still need a crawler, a JS renderer, template and boilerplate stripping, storage, and a reporting layer. A conservative first build — crawl + render + detect + report — runs 3-5 developer-days, call it $2,000-$5,000 in loaded engineering time, before a single page is audited. Every re-run afterward still costs server time and someone's attention when the WAF changes or a library update breaks something.
- Lingua Detective: 500 pages = 200 × $0.015 + 300 × $0.010 = $3 + $3 = $6, crawling and reports included. Add spider crawling (+$0.005/page) for $8.50 total if you don't already have the URL list.
FAQ
Q: Isn't franc/fastText free, so cheaper than Lingua Detective?
The library is free; the crawler, renderer, and reports around it aren't. That usually costs far more than $6 for 500 pages.
Q: Can an in-house setup handle JS-rendered sites?
Only if you also build and maintain headless Chromium rendering — a real, ongoing cost most estimates leave out.
Q: When does building in-house actually make sense?
At sustained, very large scale with spare engineering capacity to own it long-term — not for occasional or one-off audits.