Automating cloud scaling based on workloads and utilization while decreasing node spin-up time more than 20x

Bridging the gap between HPC and AI

Brightskies empowers organizations to digitally transform and accelerate innovation with secure, automated enterprise solutions that use state-of-the-art technology, robust processes and skilled resources. The global company was founded in 2012 and has offices in Alexandria, Cairo, Dubai, Germany and the Kingdom of Saudi Arabia. It enables scientists to effectively use computing resources to achieve excellence in their fields, seamlessly bridging the gap between high-performance computing (HPC), software design, artificial intelligence (AI) and parallel programming.

The Brightskies team is recognized for its expertise in designing, implementing and provisioning HPC clusters, on-premises and in the cloud, and creating software solutions for AI and HPC users to achieve optimal performance and accelerate customer innovation with research and development (R&D) across fields that require computation-intensive AI applications. Through advanced technologies and deep domain expertise, it harnesses the power of AI to build a smarter, more sustainable future.

Enabling scientific research in the cloud

When a university customer reached out to Brightskies for help setting up powerful computing resources at a new location without an on-premises data center, Brightskies leveraged its deep technical expertise to research, test and collaborate with Siemens Digital Industries Software to implement HPCWorks™ Navops™ software on Amazon Web Services (AWS), enabling scalable, high-performance research in the cloud. The university’s team already used HPCWorks PBS Professional™ software to manage workloads, so HPCWorks Navops was a natural fit. Requirements included cloud autoscaling, support for different Intel and AMD graphics processing unit (GPU) nodes and the ability for users to easily set up budgets and control costs. HPCWorks is part of the Siemens Xcelerator business platform of software, hardware and services.

Using HPCWorks Navops optimizes cloud HPC environments to maximize business impact and enable global operations. Users can dynamically scale on-demand cloud resources with their choice of cloud providers and workload managers, prioritize critical workloads and control cloud spending via an intuitive interface.

Autoscaling and node spin-up

Brightskies’ university customer faced challenges with peak-time workloads. It did not want computing resources to sit idle and needed them to be fully operational and available to researchers during high-demand periods, automatically scaling up and down based on workloads and utilization. HPCWorks Navops delivered that flexibility and elasticity via customizable, automated rules and seamless integration with HPCWorks PBS Professional.

The university also needed to quickly spin up compute nodes so users could innovate faster. Manually spinning up nodes was time-consuming, but Brightskies successfully configured and optimized HPCWorks Navops to add new nodes to the HPCWorks PBS Professional cluster with just one click. Instead of taking about 45 minutes, the time required to spin up a new node was reduced to just one or two minutes. Once a job has completed and the node is idle, it terminates without causing inconsistencies, streamlining operations and enhancing user productivity.

Controlling cloud spending

Cost optimization was a priority for the university team, which wanted to eliminate the need for additional hardware and associated on-premises management overhead. It also needed to ensure cloud resources were not allocated and running without being effectively utilized. By implementing HPCWorks Navops, Brightskies enabled users to view burn rate (cost per hour) and accrued cost per period (for example, a day or a week), with estimated cloud costs displayed throughout the interface to keep spending front and center.

Autoscaling with HPCWorks Navops ensures compute resources are provisioned precisely when needed and deprovisioned when idle, dynamically adjusting to significantly reduce overprovisioning and underutilization. Paying only for resources that are actively consumed leads to substantial cost savings. Users can manage and track budgets across multiple teams and projects.

Leveraging efficient cloud HPC

Using HPCWorks Navops empowered Brightskies to deliver an agile, efficient and cost-effective cloud HPC solution to its university customer and set the stage for future collaboration. It transformed complex, time-intensive processes into streamlined, automated workflows, allowing users to focus on research.

“One of the highlights when working with HPCWorks Navops is hybrid and multi-cloud support and flexible deployment options, with integration across different cloud providers and on-premises environments,” says Alaa Hassan, HPC cloud engineer, Brightskies.

Brightskies cited elasticity, easy provisioning and autoscaling as key factors in successful implementation, along with expert Siemens support.

“The support we received from Siemens made the experience smooth and easy, especially in the implementation phase, addressing all our questions and incorporating our ideas,” says Nour Abou-Elnadar, HPC cloud engineer, Brightskies.

The Brightskies team also appreciated clear documentation and an intuitive, user-friendly interface. By deploying HPCWorks Navops, Brightskies equipped its university customer with powerful cloud computing resources, positioning it for sustained innovation and growth.