Computing and Library Services - delivering an inspiring information environment

Autonomous Discovery and Management in Virtual Container Clusters

Higgins, Joshua, Holmes, Violeta and Venters, Colin (2017) Autonomous Discovery and Management in Virtual Container Clusters. The Computer Journal, 60 (2). pp. 240-252. ISSN 0010-4620

Metadata only available from this repository.


Global software stacks on scientific cluster computing resources are required to provide a homogeneous software environment which is typically inflexible. Efforts to integrate Virtual Machines (VMs), in order to abstract the software environment of various scientific applications, suffer from performance limitations and require systems administration expertise to maintain. However, the motivation is clear; in addition to increasing resource utilization, the burden of supporting new software installations on existing systems can be reduced. In this paper, we introduce the Virtual Container Cluster (VCC) that encapsulates a typical HPC software environment within Docker containers. The novel component cluster–watcher enables context aware discovery and configuration of the virtual cluster. Containers offer a lightweight alternative to VMs that more closely match the native performance, and presents a solution that is more accessible to customization by the average user. Combined with a Software Defined Networking (SDN) technology, the VCC enables dynamic features such as transparent scaling and spanning across multiple physical resources. Although SDN introduces an additional performance limitation, within the context of a parallel communication network, the benchmarking demonstrates that this cost is application dependent. The Linpack benchmarking shows that the overhead of container virtualization and SDN interconnect is comparable to the native performance.

Item Type: Article
Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Q Science > QA Mathematics > QA76 Computer software
Schools: School of Computing and Engineering > High-Performance Intelligent Computing > High Performance Computing Research Group
School of Computing and Engineering
Related URLs:
Depositing User: Joshua Higgins
Date Deposited: 08 Mar 2017 14:49
Last Modified: 28 Aug 2021 12:14


Downloads per month over past year

Repository Staff Only: item control page

View Item View Item

University of Huddersfield, Queensgate, Huddersfield, HD1 3DH Copyright and Disclaimer All rights reserved ©