What does it take to build a national big data superhighway?


Big data is a major subject of modern science in today’s world and engineers and scientists all over the country are focused on the technology which enables it. A workshop was held on 7th and 8th August at the Montana State University which was attended by about 125 engineers and scientists. Future of infrastructure which is needed in order to support science was planned in the first NRP (National Research Platform) workshop.

Implementation strategies were discussed in the workshop regarding building of a national big data superhighway. This is basically deploying exchangeable Science DMZs all over the nation. Sessions of the workshop focused on the needs for data transfer at high-speed, discussing their pros and cons.

It is known that physicists are the original users of big data. Data is produced at CERN in terms of exabyte and petabyte (exabyte = one billion gigabytes). Harvey Newman, a physicist who is known as the largest data user, was present at the workshop to remind everyone that newer instruments are coming into existence which will create massive sizes of data that make today’s instruments look pale in comparison.What does it take to build a national big data superhighway?

Not only physicists

Larry Smarr, computer scientist who was an organizer of the event, works with archaeologists to develop methods for analyzing and digitizing archaeological sites in order to preserve, visualize and analyze them. Climate scientists gather detailed data which support improved modelling of weather and climate. This process involves better networks, more computing and increased data storage and analysis. Academic researches increase the scope of computer science further. Networks driven by software change how scientists gain access to computing, data managing and analyzing their results.


Institutions in North Carolina are leading the way. At Renaissance Computing Institute (RENCI), Dr. Claris Castillo, leader of SciDAS which helps researchers in developing a more flexible and fluid cyber infrastructure for analyzing and working with large-scale data. The SciDAS project was discussed at the  NRP workshop.

National Carolina Research and Education Network (NCREN) is providing communications at high capacity which supports the Breakable Experimental Network, also known as BEN. NCREN supports community college and K-12, hence they provide unique opportunities for education at university level for advanced research.

Benefits of the NRP workshop

A cooperative agreement was funded by the National Science Foundation for five years which will enable the Pacific Research Platform (PRP) improve high speed data transfer, end to end, big-data science in 20 institutions. The NRP is dedicated to facilitate social engineering to a diverse group which includes R&E network, science, IT leaders and provide them with end-to-end networking which has been proven. To build an effective partnership, cyber infrastructure experts need to work with scientists and understand what their desired outcomes are. Identification of common functionality can influence science applications and make the National Research Platform partnership become more effective and efficient. Prioritizing access to high performance supercomputer centres is key.

This will alter how the Internet is today.  The techniques discussed at the workshop will soon be implemented to support cutting edge research and transform the Internet which we use at our homes.