Cloud Analytics Platform

The Problem: The data science team's Python models were written to run sequentially — one computation after another. A full analysis run with multiple parameter combinations (recognizers, market segments, coefficient sets) could take up to an hour within the platform itself, because the work was still happening in a single thread.

The Solution: Primotly worked with the data science team to identify natural parallelization boundaries within each model. The general pattern: data preparation (single process) → parallel computation (multiple independent sub-jobs, one per parameter combination or market segment) → result aggregation (single process). Primotly re-architected the backend to orchestrate this fan-out/fan-in pattern, launching parallel workers for each sub-job and collecting results once all workers completed. This required close collaboration with the data science team to understand how the algorithms could be decomposed without affecting output correctness — a non-trivial design challenge given the mathematical dependencies between some steps.

The Result: computation times have been progressively reduced throughout the platform's development. Most analytical jobs now run in under ten minutes, and the overall journey from the pre-platform era — when analyses took hours to days on local machines — represents a reduction of over 98% in computation time.

The Problem: Each analytical module depends on a Python library maintained by the client's data science team in a separate repository. These libraries evolve independently, have their own release cycles, and are not under Primotly's control. Keeping the platform in sync with library updates without breaking existing functionality required a structured dependency management approach.

The Solution: Each module has its own backend service in Primotly's codebase that imports the corresponding data science library as a Python package. The interface between Primotly's code and the data science library is kept deliberately thin — Primotly calls methods, not rewrites algorithms. Authorization and user management are handled via .NET integration with the central research data hub, keeping access control consistent across all internal platforms.

The Result: clean separation between platform engineering (Primotly) and statistical methodology (client data science). Updates to algorithms can be rolled out by updating the library dependency without touching Primotly's orchestration code.

The Problem: Analytics workloads are inherently different from typical web application workloads — they are compute-intensive, long-running, and bursty. A standard monolithic web architecture would either over-provision resources for idle periods or under-provision for peak analytical demand.

The Solution: The platform uses a monolith-with-microservices-elements architecture. The main application is a monolith for simplicity of deployment and shared data access. Heavy analytical computations are offloaded to isolated worker processes that can be scaled independently. This gives the benefits of monolithic simplicity for most of the application while handling compute-intensive workloads efficiently.

The Result: efficient resource utilization across both idle and peak periods, and individual analytical modules can be scaled independently based on demand — compute-heavy modules get more resources without affecting lighter workloads.