Shortformify turns a long video — a talk, podcast, or YouTube URL — into short, captioned, vertical clips ready for TikTok, Reels, and Shorts. Upload once, and it finds the 3–5 moments worth clipping, cuts them to 9:16, and burns in word-by-word captions automatically.

I built this solo, end to end: product decisions, backend pipeline, frontend, infra, and deployment. It was a chance to ship something with the shape of a real SaaS product — billing, tiers, background processing, a third-party API surface — rather than a scoped client feature.

From upload to viral clips

A video goes through three stages, each a queue of its own: transcription, then AI analysis, then video transformation. The API server only ever enqueues a job and returns — it never blocks on work that can take minutes.

Transcription sends the audio to AssemblyAI and stores word-level timestamps. Analysis hands the full transcript to Gemini with a prompt tuned for finding self-contained, hook-and-payoff segments, and gets back scored, ranked clip candidates. Transformation slices the source with ffmpeg, crops to vertical, and burns in captions generated from the original word timestamps — re-aligned to be relative to each clip, so one transcription pass serves every clip cut from it.

Curious how the pipeline is actually wired up? The backend is public — browse the shortformify-be repo on GitHub.

Decisions that mattered

A few choices that shaped how this holds up under real use, not just a demo:

• Queues instead of a fat request handler. BullMQ + Redis means transcription and encoding retry independently, and a failed step doesn't restart the whole job.
• Idempotent transcription. Before paying AssemblyAI for a transcript, the worker checks if one already exists or is mid-flight, and reuses it — retries don't double the bill or the wait.
• Server-enforced tiers. Export resolution (720p vs. original quality) is decided from the user's billing status inside the worker itself, never trusted from the client.
• Credits as an append-only ledger. Every charge and refund is its own row tied to a video, not a mutated balance — so usage is auditable and a failed job can be refunded without losing history.
• Direct-to-storage uploads. Large video files go straight from the browser to Cloudflare R2 via presigned multipart URLs — they never pass through the API server at all.

Giving creators control, not just output

Fully automated clipping only works if people trust the result enough to post it. So the app surfaces the AI's reasoning — each clip shows why it was picked and how it scored — and gives creators a real choice of caption styles (Classic, MrBeast, Alex Hormozi, Minimal), previewed live over the actual clip thumbnail before anything renders.

The unglamorous part: making YouTube imports reliable

Importing directly from a YouTube URL (a Pro-only path) meant working around bot detection in production — yt-dlp needs an authenticated session, and cookies quietly expire. That turned into its own small system: a headless, scheduled renewal job that keeps a logged-in browser profile alive and re-exports fresh cookies, so imports don't silently start failing days after a deploy.

---

Built and deployed solo — product, backend, frontend, and infra — as a self-directed project rather than client work. Currently in beta, running against a real Postgres/Redis/Docker stack on a production VPS.