Windsurf prompt
Windsurf: Python ETL Data Pipeline
Extract, transform, and load pipeline with pandas, data validation, error handling, and scheduling.
Build a Python ETL data pipeline for {{pipeline_name}}. ## Source: {{data_source}} ## Destination: {{data_destination}} ## Schedule: {{schedule}} ## Data volume: {{data_volume}} ### Pipeline Architecture **Extract** Connect to {{data_source}}: {{connection_method}}. Handle pagination or incremental extraction (only fetch new/changed records). Retry logic: exponential backoff on connection failures (3 attempts, 1s/4s/16s). Log all extraction statistics: records fetched, duration, errors. **Transform (pandas)** Load raw data into DataFrame. Data validation: schema check (column names + types), null checks on required fields, range/format validation. Cleaning: strip whitespace, normalize case, parse dates with explicit format, handle nulls. Transformations specific to {{pipeline_name}}: {{transformations}}. Create audit columns: processed_at, source, pipeline_version. **Load** Write to {{data_destination}}. Upsert logic: update existing records, insert new ones (use merge/ON CONFLICT). Transaction wrapping: all-or-nothing per batch. Batch size: {{batch_size}} records per write. **Error Handling** Failed records go to dead letter queue (separate table or file). Log error details: record, error type, timestamp. Send alert on failure: {{alert_method}}. Pipeline continues processing remaining records after single-record failures. **Scheduling and Orchestration** {{schedule}} schedule using APScheduler or Airflow DAG. Idempotent runs: safe to re-run without duplicates. Run log with: start time, end time, records processed, status. **Configuration** All connection strings and secrets via environment variables. Pipeline config in YAML file (not code). Logging to stdout in JSON format for log aggregation.
Fill in: {{pipeline_name}}, {{data_source}}, {{data_destination}}, {{schedule}}, {{data_volume}}, {{connection_method}}, {{transformations}}, {{batch_size}}, {{alert_method}}. In Loopy you fill these in a form and copy the finished prompt in one click.
More automation prompts
- CursorCursor: Create GitHub Actions CI PipelineComplete GitHub Actions CI/CD pipeline with parallel jobs for testing, linting, type checking, security scanning, and deployment.
- n8nn8n: Content Publishing PipelineWrite → review → approve → publish → announce automation across platforms.
- n8nn8n: Customer Onboarding SequenceSign up → welcome → email sequence → task creation → check-in automation workflow.
- n8nn8n: Error Alerting SystemMonitor → detect → alert → create ticket workflow for system error management.
- n8nn8n: Lead Capture to CRM AutomationWebhook → lead enrichment → CRM creation → team notification workflow.
- n8nn8n: Social Media SchedulerContent calendar → format per platform → schedule → post → track engagement workflow.

