Edinburgh Research Explorer

Science and Innovation: Numerical Algorithms and Intelligent Software for the Evolving HPC Platform

Project: Funded ProjectResearch

  • Leimkuhler, Benedict (Principal investigator)
  • Duncan, Dugald (Co-Investigator (External))
  • Barrenchea, Gabriel (Co-Investigator (External))
Total award£3,725,770.00
Funding organisationEPSRC
Funder project referenceep/g036136/1
Project websitehttp://www.nais.org.uk

Key findings

What are the scientific advances you have made which would not have been possible without the S&I grant
Edinburgh - Maths
We highlight the achievements of Richtarik and Maddison. During his appointment Richtarik has written and submitted 18 papers in the areas of optimization and machine learning. Two papers, on randomized coordinate descent methods, scalable to problems with billions of variables, attracted major international best paper awards. In a third paper, joint with M. Takac, he developed the Hydra algorithm a variation of which is now implemented behind the YouTube engine, and at Amazon. Research by Maddison on automated model development for time dependent problems extends the high level automated code generation system FEniCS while providing adjoint methods for sensitivity studies. Parallelisation of optimised forward and adjoint models also follows immediately (via FEniCS). This technology will be used in a soon-to-start NERC project for ocean eddy parameterisation. The FEniCS system, when coupled to the adaptivity library of Fluidity-ICOM, enables the first dynamic-mesh-adaptive simulations of a baroclinic large-scale ocean. A performance analysis has been completed, indicating that the dynamic-mesh-adaptive finite element ocean model has a comparable absolute efficiency to standard finite difference methods, with far fewer degrees of freedom, but a much higher computational cost per degree of freedom, making it a desirable strategy for future high performance computing systems.
Edinburgh - EPCC
NAIS funding enabled research into Soft Matter that appeared as a cover story in the RSC journal 'Soft Matter'. The research into complex bulk flow behaviour of a Blue Phases liquid crystal, which has unique optical properties making them very attractive for novel photonic materials and optical switch gear, was enabled through advances to the simulation code Ludwig. Further investigations into these liquid crystals by EPCC and collaborators have recently been published in Nature Communications. Our collaboration with Cray, and others, has enabled cutting-edge research into the challenges involved with designing, utilising, and programming Exascale computers. For instance, new programming languages or techniques for parallel and high performance computing, such as OpenACC and MDMP have been enabled. The research has been disseminated via numerous publications, presentations and training (including prestigious joint EPCC/Cray tutorials at a number of the supercomputing conferences) plus activity within global standards organisations for all the main parallel computing programming mechanism (EPCC is now an active member of the standards bodies for MPI, OpenMP, and OpenACC).
Edinburgh – Informatics
Nagarajan's research group in memory models and consistency schemes is a major legacy of the award, which we expect to continue to expand. Fensch's contribution during and after his award period led to funded collaborations with Samsung, and supervision of strong PhD work in the areas of multicore and GPU programming abstractions and scheduling.
Loisel has written and submitted 13 papers during the NAIS appointment. Some of these were published in or accepted to leading journals and conference proceedings in the areas of domain decomposition, statistics and psychometrics, with applications in engineering. The new algorithms solve problems with billions of unknowns on tens of thousands of processors. Loisel is also training a NAIS Ph.D. student in the area of high performance computing. Loisel supervised two postdocs and made significant progress with them in two developing areas of domain decomposition: Waad Subber on domain decomposition for uncertainty quantification; and with Hieu Nguyen on the analysis of the Bank-Holst paradigm for heterogeneous grids with local refinements.
Apart from the training of PhD students, we have done some significant scientific advances on how much the use of HPC techniques and algorithms can help the computation of realistic approximations to interesting and challenging problems. The work of Ainsworth and Rankin is one particular example. In the series of papers they wrote together (some in collaboration with Allendes and Barrenechea), the parallelisation of adaptive finite element schemes was extensively studied, and the possibility of getting much farther in terms of size of the problem was achieved. Also, Riaz’s thesis provided a different way of designing finite element software, in which the parallel structure is somehow hard-wired into it. The possibilities of auto-tuning of this code are also a very promising area for future research.
What new collaborations have you been able to establish with (i) other UK research groups (ii) industry and commerce (iii) internationally
(i) Other UK research groups
The NAIS initiative has led to increased collaboration both within the partner institutions as well as with NA-HPC communities across the UK, especially the NAIS extended network partners (Bath, Leicester and Warwick). NAIS co-organised the NA-HPC networks, based in Manchester. The collaboration between academics was promoted throughout the lifetime of the NAIS programme, supported through sponsored seminar series, workshops/meetings and research visits. This culminated in a series of meetings in the final 6 months of the programme. Either organised or co-supported by NAIS, these meetings covered a broad range of topics. These events were held at the NAIS centres at Edinburgh and Glasgow, as well as at NAIS Network venues throughout the UK (Cambridge, Durham, Bath and Dundee). With over 500 delegates, these meetings represent an excellent opportunity for cross-discipline networking, research ideas exchange. As a consequence of these meetings, and earlier activities, we are already aware of joint projects being initiated and NAIS members arranging future research visits.
(ii) Industry and commerce
During the NAIS programme there has been a wide range of external (industry/commerce) involvement in the programme through, participation in workshops/meetings, student placements, specific research collaboration, fellowships/awards for NAIS researchers and Industry visits/briefings. Areas of collaboration have included HPC/Software/Hardware (Cray, Agilent, Intel, OpenAcc Standards Consortium), Engineering/Defence/Aerospace (Dstl/Selex/QinetiQ/Airbus/ Cobham/Sandia/Naval Research laboratory), Drug Design/Molecular Dynamics (Plebiotic, Accelrys, NVIDIA, IBM), Industrial Fellowships/Awards/Student Placements (Google/Intel/IBM), Marketing/Social Media/Big Data (Bloom Media/IQcodex/Schuh/Tag Digital/Merchant Soul/NHS/Scottish Government/HRL Labs), Finance (Bloomberg LP, Lloyds Banking Group), Optimisation (Amazon, SAS Institute, IBM, Baidu, Western General Hospital, Arup), Medical Imaging (SINAPSE), and Oil and Gas (DNV-GL).
(iii) International collaboration
Through the meetings/workshops discussed previously, sponsored research visits, and appointment of staff with established connections, the NAIS programme has led to substantially enhanced international collaborations. This includes links/collaborations with researchers based in the US (Rice, Chicago, Texas, Wisconsin Madison University, UC Berkeley), across Europe, (CERFACS (France), Besancon (France), Paris (France), Geneva (Switzerland), Louvain-la-Neuve (Belgium), Bonn (Germany) and the Swiss National Supercomputer Centre), as well as the rest of the world (Singapore). Additionally, many of the industrial connections detailed in (ii) were international.
(iv) Collaboration leading to building capability
Collaborations with other UK research groups established in this programme will be key to ongoing research excellence/success, e.g. the links between the Informatics Group and EPCC are a key feature of the CDT in Pervasive Parallelism, links between EPCC and UoE Maths were key to obtaining a joint NSF-EPSRC software infrastructures grant. Networks, established and co-organised under this programme will provide long-term opportunities for collaboration of UK NAIS members.
The established industrial/commerce links are being used in on-going and future research projects. For example, Google, SAS, Amazon and Baidu are involved as a project partner in an EPSRC bid, SAS is a project partner in a funded EPSRC grant, Selex and Dstl were used in shaping the industrial participation for the successful MIGSAA CDT. Furthermore, there are examples where Industrial Collaborators re using tangible outputs from research e.g. Google and Amazon are using parallel and distributed algorithms developed in Richtarik’s group. These tangible outputs and collaborations are a key success of the programme and will feed directly the future IMPACT case studies from the partner institutions.
The international collaborations have provided enhanced scope for research visits. The ability to attract world-leading academics to the partner institutions for seminars/workshops/research visits was a real asset of the NAIS programme. These collaborations have enhanced the international standing of the research as well as providing opportunities to attract world-leading expertise to the UK/partner institutions.
Have the research outcomes from your S&I award irrevocably changed (i.e. been incorporated into) the overall future research strategy at your research organisation and if so in what ways?
Edinburgh - Maths
NAIS has profoundly altered the scale and intensity research being conducted in the area of computational mathematics. There are many more active researchers, more grant funding, and steady streams of speakers addressing themes such as numerical analysis, computational modelling and parallel numerical operational research-related methods. Computational fluid dynamics is being developed with a rich, algorithm-intensive approach. Moreover, NAIS funding has helped us to move forward in research areas such as exploratory data-guided molecular sampling, large scale optimization methods, and other areas that are on the forefront of data-centric modelling. New hires in the HPC area are now established within a School that actively encourages and values their contributions.
Edinburgh - EPCC
We have built ties with the maths departments within, and associated, with this project, and also with important commercial companies involved with either producing hardware that is used for computational simulation or who are actively using simulation in their business. Particularly, the establishment of an Intel Parallel Computing Centre at EPCC and the involvement of EPCC in the OpenACC standard have both grown out of NAIS work. Furthermore, NAIS has enabled EPCC to be a leader in Exascale software and technologies, which has driven our involvement in a wide range of different projects and initiatives. NAIS funding enabled EPCC to embed academic researchers within the department, moving beyond the model of simply focussing on parallel and HPC enabling and EPCC and collaborating with application scientists, to bring application scientists within EPCC so they can work much more closely together, giving the application scientists a more detailed understanding of HPC issues and challenges for their work and giving HPC experts a more detailed understanding of the requirements of specific scientific fields and the simulation codes they use.
Edinburgh – Informatics
ICSA, our research institute, has expanded by around 35% in size over the course of NAIS, gaining critical mass for a substantial future. The case for this expansion, most directly including Nagarajan's post, has been supported by the increased research activity in the area (all of our NAIS funded work sits squarely within ICSA). Parallel computing is now firmly embedded within the research agenda of both ICSA and the School, and indeed we are now able to play a full role in the growing strength of parallelism research within central Scotland, leading to further collaborative actions and proposals.
The NAIS project has enabled us to enlarge and diversify our computational mathematics group and our focus is now firmly on the development of approximation methods for PDEs, SDEs and BIEs that are both provably reliable as well as being efficient to implement on current and likely future computer systems. Diversification into the linear algebra needed for efficient PDE solvers has come through the appointment of Sebastien Loisel and his continuing group of students. We have built excellent new connections with EPCC, strengthened links with the other NAIS partners and developed a mutually beneficial arrangement with our own CS colleagues sharing HPC equipment and expertise.
The Department made major commitments with this project, including the cluster and partial funding for scholarships. The expertise built in the department with the hiring of Dolean, plus the topics of the PhD studentships and PDRAs has helped to shift the direction of the Numerical Analysis and Scientific Computing group, and also of other groups in the Department. As it was stated before, the Population Modelling Group, in particular the Marine Population Modelling (sub)group, have been intensively using the cluster to run finite differences models of the Clyde River. This has created the awareness that accurate and fast methods are necessary, and then there will be an increase of collaboration between the NASC and MPM groups in the future. Moreover, the involvement of D. Higham in problems involving Big Data, and the two NAIS-funded workshops organised by him, have had a sizeable impact in the future directions within the NASC group.