Infrastructure Group

Infrastructure Group

The Infrastructure Group are a team of multi-discipline software engineers, system administrators and researchers. They design and build the testbeds in use in TSSG to meet current and future research needs.

The Infrastructure Group design and build the testbeds in use in TSSG to meet current and future research needs. Their role is to support and enable experimenters get the most out of the computing resources and testbeds they use. They also operate and manage TSSG’s High-Density Water-Cooled Data Centre facility which is home to many Testbed Infrastructures such as Unified Communications, IoT, Cloud, SDN/NFV, and SDR Testbeds, as well as an AI cluster and ICHEC’s Super Computer.

In addition to their testbed work the Infrastructure Group work on a range of EI, EU, InterReg and SFI projects ranging from security, radio technologies, advanced networking protocols, automation and virtualisation, as well as building prototypes for SMEs.

The Infrastructure Group take a DevOps ethos to everything they do and as such heavily use Software Defined Infrastructure (SDI), including Software Defined Networking (SDN), Software Defined Storage (SDS), and Orchestration as means of delivering services quickly and effectively.

The TSSG house and operate their own Data Centre facility to support over 50 concurrently active ICT research projects through the provisioning of Internet services, Cloud Computing resources, an AI cluster, and project bespoke testbeds (such as Unified Communications, SDN/NFV, Internet of Things, etc.) and research equipment.  The TSSG Data Centre also houses research infrastructure for other research groups and centres, such as ICHEC’s Super Computer ‘Kay’ and CONNECT’s Pervasive Nation Testbed.

The Data Centre was designed for functional modularity, with separate rooms for servers, networking equipment, UPSs and batteries, with the generator and cooling equipment located outside. Each room in the Data Centre is designed to use the best cooling option for that function. For example, the UPSs and batteries are located in exterior rooms using natural ventilation and small fans for cooling.

As a result, the Server Room of the Data Centre has no hot or cold aisles. All the heat/cooling is contained inside the server cabinets, making the server room both comfortable and quiet.

The Data Centre has an IT power load of 300kW, with cabinets engineered to house 30kW of IT equipment. 1+1 300kW UPSs and an 800kW Generator provide the backup power. The Data Centre uses Rittal LCP technology, which uses chilled water (at 15°C) and provides free cooling 80% of the time.

As the Data Centre is Water-cooled, higher densities than traditional air-cooled systems can be achieved allowing a smaller overall footprint as more servers can be packed into the cabinets and there is no need for the traditional CRAC units in the server room.

The TSSG Data Centre currently supports over 160 physical servers, providing more than 1,000 cpu cores for processing and 400 virtual servers for cloud computing.  In addition, there is over ½ PB (that 512 TB) of Data Storage, and ~3,000 network ports. All of which provide a high-level of interconnectivity and flexibility for TSSG’s research projects.

The TSSG has their own Publically accessible Data Centre that houses their Testbed Infrastructure, such as their Unified Communications, IoT, Cloud, and SDN/NFV and SDR Testbeds and their AI cluster.

The Unified Communications testbed provides a highly comprehensive, flexible and powerful platform for tests, validation, and evaluation of algorithms, protocols, tools etc.  This testbed can be used to specifically refine and tune algorithms and knowledge-based systems which can’t be achieved through simulations. The size of the testbed and the users connected to it, make it possible to run tests that can evaluate scalability aspects which can be difficult if not possible to model and/or identify through simulations of models and algorithms. This testbed allows for evaluations of aspects of interoperability which is a key feature for systems that are to cope with heterogeneity and the challenges to bridge systems across domain boundaries.

A Smart Agriculture IoT testbed is located at TSSG. It is deployed in a distributed manner and consist of a distributed set of IoT sensors, IoT-enabling radio basestations (and associated antennas), gateway servers enabling edge processing capabilities, backhauling solutions (e.g. links from basestation sites to core IoT services datacentre), and server capacity hosting IoT resource management services and application enabling software services.

TSSG have a large-scale (40+ server) multiple OpenStack cloud deployments, that leverages a highly performant SDN network, capable of 160Gbps per leaf node. In addition, storage is provided via a 300TB Ceph cluster. These cloud platforms are used to provide compute resources, VNFs, virtual networks, and big data clusters; and are fully accessible by the other testbed resources.

The SDN/NFV platform provides testing, monitoring, control and virtualisation resources to enable the conformance, performance, interoperability and proof of concept testing of new and existing SDN and NFV technologies. In addition it leverages the orchestration features of the cloud computing platform for rapid prototyping and deployment.

The TSSG SDR platform is one of the only large-scale academic SDR testbed deployments in Ireland. It combines the natural surroundings of the Carriganore Campus at Waterford Institute of Technology with its variable terrain (many species of trees, rocks, hills, flat terrain, bodies of water) over 72 hectares (720000 square meters). It hosts programmable USRPs on easy to configure masts with single mode fibre connectivity to an on-site data centre. In the data centre, this radio platform is connected to private cloud infrastructure with allowing for scalable deployment of supporting components. This cloud infrastructure has already been used for experiments and deployments involving SDN, NFV, data analytics and pre-commercial projects with industrial partners.

The AI cluster comprises of 16 nodes each with 40 CPU cores and 2 GPUs. These nodes are integrated within the TSSG cloud platform and are also available in a  bare-metal configuration for dedicated workflows.

All of the infrastructure is supported by the Infrastructure Group – a team of multi-discipline software engineers, system administrators and researchers. Their role is to support and enable experimenters get the most out of the resources they use.