DrFirst
Data Integration
2025-10 → present · Senior Software Engineer · Remote
Data pipelines on AWS for a healthcare technology platform — Glue scripts orchestrated by Step Functions — and the QuickSight dashboards they feed.
Composition
- ExtractionAWS Glue scripts
- OrchestrationAWS Step Functions
- ReportingAmazon QuickSight
Mission & Constraints
The pipelines are Glue scripts under Step Functions orchestration, which is where the compute time went: over two hours per run down to twenty to thirty minutes.
Dashboard performance was the second front. QuickSight went from a three-hour refresh to forty to fifty minutes.
The compute saving is the measurable part, but the shorter runs also take management overhead out of operating the pipelines day to day — fewer long jobs to watch, wait on, and restart.
Implementation
Extraction
AWS Glue scripts
Orchestration
AWS Step Functions
Reporting
Amazon QuickSight
Measured Outcomes
Pipeline compute time
−75 %2h+ to20–30 min
DrFirst — AWS Glue scripts orchestrated by Step Functions
QuickSight dashboard load time
−72 %3 h to40–50 min
DrFirst — dashboard performance optimisation
Questions about this work?
Happy to go deeper on the architecture, the constraints, or how the numbers were measured.