Direct answer: Ask a compact checklist of 12 questions across IP, staffing, delivery, security, and ops; insist on named senior staffing, a clear IP assignment, escrow for source with a 0.5–1% fee, and payment milestones that cap upfront risk (10/40/50 as industry practice). If a vendor’s proposal costs under $100k for a mid-complexity B2B platform, treat that as a red flag. These checks identify vendors who will actually own outcomes for the next 3 years.
Why this matters: a 5‑engineer product team costs roughly $900k–$1.1M/year fully loaded (US market, 2026 compensation). Paying a third-party $200k–$400k/year for a component that saves you one to two engineers can be justified; paying $100k because a shop 'will figure it out' usually isn't. Your procurement decision is less about hourly rate and more about avoiding a 2–4× rework tax over three years.
A poor vendor choice shows up as three predictable losses: 1) ownership ambiguity that forces you to re-implement a feature later (migration cost: $20k–$200k depending on data and integrations), 2) senior-staffing bait-and-switch that raises bug backlog and delays (senior substitution rates above 30% correlate with schedule slips >30%), and 3) operational debt because the vendor never delivered runbooks or SLOs (SRE-style work that costs $150k+/yr to remediate).
Questions to ask a software development company before hiring — the checklist
Below are the questions you must ask and the good-answer vs. red-flag answer to expect. Each question maps to a concrete decision: hire, run a paid spike, or walk away. These are not negotiation talking points; they are acceptance criteria you can test during a 2–4 week paid validation sprint.
- IP and ownership
- Named staffing and substitution policy
- Delivery milestones, acceptance criteria, and payment terms
- References and code samples from living deployments
- Security and compliance posture (pen test, SOC2, GDPR readiness)
- Operational readiness (runbooks, monitoring, SLOs)
- Pricing model and 3‑year TCO
- Exit terms, escrow, and transition plan
1) Who owns the code and IP after delivery? Good answer: "All code and work-for-hire deliverables will be assigned to you; source code is placed in your repo; we agree to a vendor-source escrow in clause X with a 0.5–1% annual fee to cover facilitation." Red flag: "We retain ownership of foundational components; you get a license." Any ambiguity here should be a deal-breaker for product-core work.
2) Who exactly will do the work? Good answer: named leads (Engineering Manager — name, Principal Engineer — name) with a clear % allocation (e.g., Principal 20% capacity, EM 15%). Red flag: a title-only org chart and a clause that allows unspecified "team members" to work on your project. Expect senior substitution rates under 10% in the first 6 months; anything above 25–30% correlates with knowledge loss.
3) How do you price and how are payments staged? Good answer: fixed-price for milestones with measurable acceptance tests; typical split is 10% kickoff, 40% after an agreed alpha with acceptance tests, 50% on production handoff. Red flag: 50%+ upfront or daily/hourly retainer with no deliverable-based milestones.
4) Show me two live references for similar scope and access to the codebase. Good answer: three references including a CTO who will let you review a production commit history, CI pipeline, and incident report. Red flag: curated case studies with no contactable references or references that were project-managed by someone not on engineering leadership.
5) What are your security and compliance controls? Good answer: recent pen test (within 12 months), SOC2 Type II or an explicit remediation plan, logging retention (30–90 days), and a data classification map. Red flag: "We'll follow best practices" with no artifacts and no documented response SLO for vulnerabilities (e.g., 72 hours for critical fixes).
6) What operational artifacts do you deliver? Good answer: runbooks, deployment scripts, alerting thresholds, tracing instrumentation, and a 90‑day operational warranty with an SLO of 1‑hour SEV1 response. Red flag: delivery of 'the code' only and an expectation that your team reverse-engineers operational behavior.
7) What's the 3‑year TCO and who pays for runbooks, maintenance, and license egress? Good answer: an itemized 3‑year forecast — development, infrastructure (e.g., expected $1.5k–$5k/month for staging+prod small SaaS), support retainer, and estimated egress costs. Red flag: a one-line rate card with no TCO and no migration/egress assumptions.
8) What happens on termination or acquisition? Good answer: an exit schedule, transfer-of-knowledge timeline (30 days), and source-escrow activation conditions. Red flag: no exit terms, or the vendor claims escrow only for an additional fee that exceeds 2% of contract value annually.
Ask for named people, not titles; ask for source in your repo, not a ZIP file; and insist payment ties to measurable acceptance criteria — those three checks alone eliminate most risky vendors.
How to interpret vendor answers: red flags and good answers
If a vendor gives you scripted answers, challenge them. Ask for a 2‑week paid spike ($20k–$60k typical) with a 2‑page deliverable: working API endpoints, a CI pipeline pointing at your staging, and a runbook for deploy and rollback. If they refuse a paid spike and prefer long spec-writing phases, they want to lock you into scope creep and time-and-materials billing.
Use small experiments to validate three hypotheses: senior involvement, operational competence, and IP posture. Run a code review session with their proposed Principal Engineer and an internal engineer — if they refuse, treat it as a trust gap. Demand to see a commit history and CI run times — one sign of production discipline is automated tests with <10 minute pipeline runs and >80% of mainline commits passing CI on first run.
Quantify your risk. If vendor staffing shows senior allocation under 20% across a 6‑month engagement, budget 15–25% schedule slippage and a 20–40% higher bug backlog. Translate that into dollars: for a $300k engagement, expect $45k–$90k remediation later if the vendor understaffs seniors.
Vendor scorecard and how to use it
Below is a minimal scorecard you can copy into a spreadsheet and use during vendor calls. Weight the categories by your risk tolerance: IP and staffing usually deserve the highest weights for product-core work.
- IP & escrow (weight 25%): assign 0–3 points; 3 = clear assignment + escrow clause 0.5–1%
- Named staffing (weight 20%): 0–3 points; 3 = named senior leads with % allocations and CVs
- Delivery & payment terms (weight 15%): 0–3 points; 3 = milestone-based 10/40/50 with acceptance tests
- Operational readiness (weight 15%): 0–3 points; 3 = runbooks, SLOs, monitoring in place
- Security & compliance (weight 10%): 0–3 points; 3 = recent pen test, SOC2 or equivalent artifacts
- References & code access (weight 10%): 0–3 points; 3 = contactable references and commit history access
Score each vendor across the six categories and compute a weighted score. Use 60% as a minimum pass threshold for product-core work. If a vendor scores 70–80% and bids significantly below market, probe whether they are under-scoping non-functional requirements or planning to substitute juniors for seniors.
If you'd prefer a ready-made CSV template, download the vendor scorecard and onboarding checklist from our engagement page or have us run a 2‑week vendor validation sprint. The validation sprint is a paid short-term engagement that verifies staffing, code quality, and operational handoff patterns before you commit to a larger contract. See custom software development for how Eltherion runs these validation sprints.
For guidance on modeling longer-term cost, compare the vendor's proposal against a 3‑year TCO model — development + hosting + support. Our framework for that comparison is detailed in "Custom software TCO" and if you're asking AI-specific questions about vendors, our companion checklist "Questions to ask an AI development company" covers data, models, and eval criteria.
What this means for a CTO or technical founder
You should treat vendor selection as a technical risk reduction exercise, not a procurement cost-cutting exercise. Allocate budget for a 2‑4 week paid spike ($20k–$60k) to validate the three core hypotheses: named senior participation, operational handoff, and clear IP. This upfront expense typically saves 2–4× in rework and remediation later.
Don't accept opaque staffing clauses. Insist on a substitution policy that caps senior substitution at 10% in the first 3 months and requires notice and equivalent replacement thereafter. Make acceptance tests part of payment; make source placement into your repo a condition for final payment.
If you're deciding between building in-house vs. hiring a vendor, use the scorecard to translate qualitative answers into dollars: assign a probability of rework and multiply by expected remediation cost. If remediation risk exceeds 25% of contract value, budget to build in-house or hire a senior consultancy instead.
- Insist on named senior staff and a substitution cap under 10% for the first 3 months.
- Make IP assignment explicit and require source-in-your-repo plus a vendor-source escrow clause (0.5–1%).
- Stage payments to milestone-based acceptance tests (recommended 10/40/50) and run a 2‑week paid spike before committing.
- Score vendors with a weighted checklist and require references that give commit history and CI access.
- Budget for 3‑year TCO, not just the upfront price, and translate staffing risks into expected remediation dollars.
Final path: if this checklist feels tedious, that's intentional. The goal is to convert trust into testable artifacts: names, repos, runbooks, and references. If you need a short vendor validation sprint or help turning answers into a binding SOW, our production custom software development team runs exactly this process and can deliver a validated vendor report in 2–4 weeks.
