Demo2Test: Turn Demo Scripts into CI Smoke Tests
A framework that converts a project's demo or example scripts into lightweight end-to-end smoke tests automatically, so the bug-catching power of a live demo is preserved in CI.
Maintainers of open-source libraries and SDKs that ship demos or examples
- Reads existing demo scripts and instruments them to capture assertions
- Generates a smoke test per demo that runs on every PR with real credentials optional
- Alerts on contract drift between the script's expected and the live API's response
- Badge in README showing live-demo health for downstream consumers
The dev.to smash story argues a demo script is the cheapest end-to-end test you'll ever write — and most projects already have demos lying around that nobody has wired into CI.
Smoke testing in CI is well-established (CircleCI, Microsoft DevOps docs, Virtuoso guides all promote it), but the specific niche of 'auto-converting demo scripts' has minimal search footprint and the cited dev.to smash story returned zero search results — indicating unvalidated demand.Smoke testing in CI/CD pipelines - CircleCI ↗Integrating Smoke Tests into your Continuous Delivery Pipeline ↗
Adjacent incumbents already convert doc/example snippets to tests: Python's built-in doctest, pytest doctest integration, and pytest-doctestplus (supports .rst/.md/.tex examples) — leaving limited whitespace for a 'demo script to CI' wrapper unless it tackles multi-language demos or non-Python scripts.pytest-doctestplus · PyPI ↗doctest — Test interactive Python examples — Python 3.14.6 documentation ↗
OSS maintainers are notoriously reluctant to pay for testing tooling — pytest, doctest, and most CI plugins are free. No evidence of OSS maintainers paying for this category; dev.to article has 0 reactions, signaling weak willingness-to-pay signal.Best Open Source Testing Tools and Frameworks for 2025 ↗
Smoke testing as a category is durable, but AI coding agents (OpenAI Agents SDK, Codex OSS maintainer kits) are increasingly absorbing 'auto-generate tests from examples' into broader workflows — risking feature absorption rather than a standalone product.Using skills to accelerate OSS maintenance | OpenAI Developers ↗
Parsing arbitrary demo scripts across multiple languages, extracting assertions, generating CI configs, and handling fixtures is non-trivial — feasible for a single language (Python) but a true multi-language auto-converter is a substantial engineering effort.code-with-engineering-playbook smoke-testing README ↗