UCR Research Computing Office Hours

An open forum for all UCR Researchers, Staff, and Students to join and hear some Research Computing updates, ask questions, learn and have an open discussion.

Weekly Fridays 9:30 to 10:00. More sessions soon to follow.

Join Zoom Meeting https://ucr.zoom.us/j/93872021612?pwd=bFE2Y3ZCZmphRkRGZ0dhLzVQWFMxUT09

Meeting ID: 938 7202 161

Passcode: RCOS21

Dial by your location

+1 669 900 6833 US (San Jose)

+1 253 215 8782 US (Tacoma)

+1 346 248 7799 US (Houston)

+1 312 626 6799 US (Chicago)

+1 646 876 9923 US (New York)

+1 301 715 8592 US (Washington DC)

Meeting ID: 938 7202 1612

The High-Performance Computing Center can support and consult with various HPC and Research Software related hardware and software issues.

They have an excellent HPC Manual and Linux Manual.

They hold regular training classes on topics such as:

• The cluster scheduler, Slurm
• Basic Linux commands
• Linux File systems
• Cluster policies and best practices
• Basic job submissions to the cluster

UCR High-Performance Computing Center

We have a very strong partnership with UCR's Information Security Team and can provide consulting on securing your Research.

Commonly we assist researchers with the following:

1. Understand the research and identify any regulations, requirements, and controls.
2. Help develop a Data Security Plan for the research project. (the DSP outlines Roles, Responsibilities, Policies, Processes and Controls)
3. Implement any plans or controls we develop.
4. Review regularly.

Please connect with the Research Comptuing Team to learn more and get started.

Some additional Resources:

Data Security Plan Template for use at UCR.

• UCR Data Security Plan Template

Protection Level Classification of Information and IT Resources at UC's established by UCOP

• Protection Level classifications

UC AWS EA and BAA 

• UC AWS Enterprise Agreement (EA) and HIPAA Business Associate Agreement (BAA)

UC Cloud Contract and Guidance 

• UC Cloud Contract and Guidance

The UCR Library serves as an information commons and intellectual center for the campus and is the focal point for research and study at UCR.

Training classes on software and data management.

• Workshops and Classes
• UCR Library Main Page​

Some online training resources for AWS.

• AWS Training
• AWS Learning Paths
• AWS Online Tech Talks
• AWS Hands-on Tutorials

Internet2 NET+ Introduction to AWS for Researchers Virtual Event

• Access the session recording
• Access the AWS and NET+ slides
• AWS regularly hosts events for the researcher community. If you’d like to be invited, please click here

Deploying a Burstable and Event-driven HPC Cluster on AWS Using SLURM
Link

AWS Education Research Seminar Series
Join us

Each month, Amazon Web Services (AWS) brings together a community of leading researchers for informational sessions focused on how research spurs scientific and technological evolutions. You’re invited to our next webinar session. Read below to learn more:

Register today »

The National Institutes of Health (NIH) Science and Technology Research Infrastructure for Discovery, Experimentation, and Sustainability (STRIDES) Initiative aims to modernize the biomedical research ecosystem by fostering innovation in biomedical research using technological advancements in the cloud. NIH-funded biomedical researchers are eligible to leverage the NIH STRIDES Initiative to access cutting-edge cloud services and tools, which includes access to valuable datasets and favorable pricing on cloud environments. Join this session where we will have three separate presentations from representatives from the NIH, Amazon Web Services (AWS), and Four Points Technology, LLC.
We hope you can join the discussion on Wednesday, August 26. If you have to miss, don’t worry. We will share a recording with all registrants. Feel free to invite your colleagues.

Serverless Applications in Python

Tuesday, February 16, 2021 | 9:00 AM - 10:00 AM PT

Registration: https://pages.awscloud.com/GLOBAL-public-OE-Python-2021-reg-event.html

January 2021 AWS Announcements

Learn how to establish an Amazon Web Services (AWS) account under the UC Agreement.

A UC-wide agreement for Amazon Web Services (AWS) has been established. The agreement provides a Data Egress fee waiver for up to 15% of total monthly AWS fees.

Please follow the instructions below to ensure that an AWS account for UCR business is covered under this agreement and in compliance with UC policy and the law.

Understand Appropriate Use

Review the following to understand the applicable terms and conditions, and allowable data use for AWS:

• University of California AWS Enterprise Customer Agreement
  • Applies to all AWS accounts UC-wide, except those used for:
    • Commercial Web Hosting
    • Media Streaming
    • Massive Open Online Courses (MOOCs)
  • 80%+ of data egress must be via an approved National Research and Education Network (NREN). This includes CENIC and Internet2, so normal UC usage meets this requirement.
• Determine whether the data you are working with should be hosted in the cloud.
• HIPAA Business Associate Agreement (BAA)
  • In order to cover your locations AWS accounts under the terms of the UC AWS Enterprise Agreement (EA) and HIPAA Business Associate Agreement (BAA), follow the instructions found on the UCOP Website (PDF).

Connect AWS Account to UC Agreement and PO

Send an email to UCs AWS account representative, Devinder Narula aws-uc-procurement@amazon.com to:

• Activate your new AWS account (if pertinent)
• Include your AWS account under the terms of the UC-wide AWS agreement
• Connect your AWS account to your Purchase Order number

To accomplish this, the email needs to include the following information:

• AWS 12-Digit Account Number
• Company Name = University of California, Riverside (UCR)
• Your Name

AWS Account Activation

You will receive an email response from AWS confirming that your AWS account is now set up for invoicing under the UC AWS agreement and providing final instructions to activate your account. Follow the instructions in the AWS activation email and your account will be active and ready for use.

The mission of the Extended Collaborative Support Services (ECSS) program is to improve the productivity of the XSEDE user community through successful, meaningful collaborations which optimize applications, improve work and data flows, increase effective use of the XSEDE digital infrastructure and broadly expand the XSEDE user base by engaging members of underrepresented communities and domain areas.

Through ECSS, users have access to a variety of cyberinfrastructure expertise. ECSS experts, many with advanced degrees in domain areas, are available for collaborations, lasting months to one year, to help researchers fundamentally advance their use of XSEDE resources.

Expertise is available over a wide range of areas:

• performance analysis
• petascale optimization
• efficient use of accelerators
• I/O optimization
• data analytics
• visualization
• use of XSEDE by science gateways
• workflows

XSEDE Extended Collaborative Support Services

NEXT Training:

Please join us for our next ECSS Symposium.

Tuesday, October 20, 10:00 AM Pacific / 1:00 PM Eastern

Join from PC, Mac, Linux, iOS or Android: https://zoom.us/j/114343187

Or iPhone one-tap : US: +16468769923,,114343187## or +16699006833,,114343187##

Or Telephone: Dial(for higher quality, dial a number based on your current location): US: +1 646 876 9923 or +1 669 900 6833 or +1 408 638 0968

Meeting ID: 114 343 187

International numbers available: https://zoom.us/zoomconference?m=Kzg5xp93d84wOReXpCZzGCWHpKACTSLh

More information about the Symposium series and links to previous presentation can be found here

https://www.xsede.org/for-users/ecss/ecss-symposium

Speaker: Shawn Brown, Director of PSC and Bridges-2 PI

Title: Introducing Bridges-2

Abstract: Bridges-2, PSC's newest supercomputer, will provide transformative capability for rapidly evolving, computation-intensive and data-intensive research, creating opportunities for collaboration and convergent research. It will support both traditional and non-traditional research communities and applications. Bridges-2 will integrate new technologies for converged, scalable HPC, machine learning and data; prioritize researcher productivity and ease of use; and provide an extensible architecture for interoperation with complementary data-intensive projects, campus resources, and clouds. 

Speaker: Sergiu Sanielevici, Neocortex co-PI and Associate Director of PSC User Support

Title: Introducing Neocortex

Abstract: Neocortex will be a highly innovative resource at PSC that will accelerate AI-powered scientific discovery by vastly shortening the time required for deep learning training, foster greater integration of artificial deep learning with scientific workflows, and provide revolutionary new hardware for the development of more efficient algorithms for artificial intelligence and graph analytics.

Association for Computing Machinery

Practice and Experience in Advanced Research Computing (PEARC) Proceedings

• https://dl.acm.org/conference/pearc/proceedings

PEARC20

• Cyberinfrastructure of a Multi-Petabyte Microscopy Resource for Neuroscience Research
• Grendel: Bare Metal Provisioning System for High Performance Computing
• Cloud and on-premises data center usage, expenditures, and approaches to return on investment: A survey of academic research computing organizations
• Implementing a Loosely-Coupled Integrated Assessment Model in the Pegasus Workflow Management System
• Cluster Usage Policy Enforcement Using Slurm Plugins and an HTTP API
• The Hawai‘i Rainfall Analysis and Mapping Application (HI-RAMA): Decision Support and Data Visualization for Statewide Rainfall Data
• Extracting and Analyzing Deep Learning Features for Discriminating Historical Art: Deep Learning Features and Art
• Research Workflows - Towards reproducible science via detailed provenance tracking in Open Science Chain
• Simulating Large-scale Models of Brain Neuronal Circuits using Google Cloud Platform

CLOUD LEARNING AND SKILLS SESSIONS

• FAQ

CLASS: Cloud Learning and Skills Sessions
Empowering the R&E Workforce to Leverage the Transformational Capabilities of Cloud Computing
 
Internet2 has explored the many challenges of fully leveraging cloud computing platforms with U.S. research and education (R&E) community stakeholders including CIOs, campus research computing center leaders, researchers, and funding agencies. These discussions have consistently highlighted the lack of suitable and sustainable training for our large, diverse, and distributed workforces as a significant factor hindering the effective use of cloud platforms.

nanoHUB.org is the premier open and free platform for computational research, education, and collaboration in nanotechnology, materials science, and related fields. Our site hosts a rapidly growing collection of simulation tools that run in the cloud and are accessible through a web browser. In addition, nanoHUB provides online presentations, nanoHUB-U short courses, animations, teaching materials, and more. These resources instruct users about our simulation tools as well as general nanoelectronics, materials science, photonics, data science, and other topics. A good starting page for those new to nanoHUB is the Education page.

Our site offers researchers a venue to explore, collaborate, and publish content as well. Many of these collaborative efforts occur via workspaces, user groups, and projects. Uncertainty Quantification (UQ) is now automatically available for most nanoHUB tools and adds powerful analytical and predictive capabilities for researchers.

Learn More:

• About
• Model & Simulate
• Teach & Learn
• Develop Software
• Share & Publish
• Resources
• Explore
• NANOHUB-U
• Tools

Resources:

Training:

• ICE Cube Computing Grid (Video)
  • Presentation covering how the IceCube computing efforts have evolved over the last 15 years from mostly home-grown solutions to a globally distributed computing infrastructure. It highlights the advantages and disadvantages of such an approach for an experiment with broad science goals ranging from astrophysics to particle physics to geophysics, and what we see in our future as we engage in more external collaborations in computing.
• Distributed Computing CS Lecture Series (Video)
  • Classroom lecture videos for CS 436, University of Waterloo
•  

Resources:

Training:

• Edge Computing Explained (Article)
• What is Edge Computing (Video)
• What is Edge Computing .1 (Video)

Resources:

• Ask.cyberinfrastructure
  • Q & A platform for people who do research computing - researchers, facilitators, research software engineers, cyberinfrastructure engineers, system administrators, and others. Ask.CI provides a centralized, searchable, archived site to aggregate expert knowledge that is widely distributed throughout the research computing community. A joint project of the Northeast Cyberteam and the Campus Champions, it was launched in July, 2018 at PEARC18.
• Authentication
  • OAuth Authorization for SSH
    • OAuth SSH delivers use of federated identities to the SSH platform providing a uniform security infrastructure for institutions.
    • https://github.com/xsede/oauth-ssh

Training:

• HPC Carpentry
  • HPC Carpentry is a set of teaching materials designed to help new users take advantage of high-performance computing systems. No prior computational experience is required - these lessons are ideal for either an in-person workshop or independent study.
• HPC University
  • The HPC University (HPCU) is a virtual organization whose primary goal is to provide a cohesive, persistent, and sustainable on-line environment to share educational and training materials for a continuum of high performance computing environments that span desktop computing capabilities to the highest-end of computing facilities offered by HPC centers.
• Hardware
  • Overview of HPC Interconnects (Video)
    • In this video from ATPESC 2019, Ken Raffenetti from Argonne presents an overview of HPC interconnects.
    • Slides
• Software
  • OSU InfiniBand Network Analysis and Monitoring tool - OSU INAM
    • Monitors IB clusters in real-time by querying various subnet management entities in the network.
  • MVAPICH: MPI over InfiniBand, Omni-Path, Ethernet/iWARP, and RoCE
    • MVAPICH is a high performance implementation of MPI-1 over InfiniBand and RDMAoE based on MPICH1. MVAPICH2 is a high performance MPI-2 implementation for InfiniBand, Omni-Path, Ethernet/iWARP, and RoCE based on MPICH2. High performance, scalable and fault-tolerance support are provided in these packages. These packages provide multiple underlying support (such as OpenFabrics/Gen2, OpenFabrics/Gen2-Hybrid, OpenFabrics-RDMAoE, QLogic-InfiniPath, and the standard TCP/IP) for portability across multiple networks
  • Spack is a package manager for supercomputers, research clusters, Linux, and macOS. It makes installing scientific software easy. Spack isn’t tied to a particular language; you can build a software stack in Python or R, link to libraries written in C, C++, or Fortran, and easily swap compilers or target specific microarchitectures. Learn more here.
    • Stay connected with the community via Slack
    • SPACK on AWS tutorial (useful to your cluster as well)
      • Spack Tutorial
      • Spack with AWS ParallelCluster
         

Resources:

• Machine Learning Roadmap 2020

Training:

Summer Webinar Series 

PAST

Greetings from NCGAS!

Here in Indiana, the leaves are falling, midterms are wrapping up, so it's time... for another round of our Introduction to R for Biologists course!  We will be hosting this course online again this round, through IU's Expand/Canvas platform, starting Oct 27 and running through Nov 12.  

This course has filled every time we've offered it, so register early if you are interested! Registration deadline is October 25th at 5pm EST!  Register here:

https://ittraining.iu.edu/events/event-registration.html?preview=true&scheduleid=e266d8bb-a3fb-6dae-b112-29b21bbfbac2

Format:

All participants in the course will be directed to a pre-made Virtual Machine with RStudio installed and all necessary packages loaded.  You are free to use your own installation of RStudio, however, we will not be troubleshooting individual installations as that is tremendously time-consuming.

The course materials are all available in pdf textbook format with links to video lectures on YouTube.  These parts will be self-guided.  The labs are also available as textbook chapters.  All answers to problems in the labs can be filled in on Canvas quizzes to get immediate feedback.  The answers are also available in pdf if you prefer that route.

Instructors will be having live office hours via Piazza on Tues from 1-4:30 PM EST to answer questions and provide insight on the lecture and lab sections.  We will also be answering questions throughout the week, so if the time does not work for your schedule, you can still take the course!  We will also be adding supplemental videos for the course to answer common questions, additional tips, etc.  The schedule is as follows:

Tuesday, Oct 27:  Chapter 1 - Intro to R Language and Data Types; Chapter 2 - Lab I: DNA words (practice for vectors and syntax)

Tuesday, Nov 3:   Chapter 3 - Intro to Graphing and Visualization in R; Chapter 4 - Lab II: Metagenomic Analysis with PCA/PCoA (practice for visualization)

Tuesday, Nov 10:  Chapter 5 - Intro to Reading and Writing Functions in R; Chapter 6 - Lab III: Writing a Sliding Window Analysis Function (practice for functions and all other skills)

You will be free to work at your own pace in the course, but we will focus on answering questions/running discussions for the course as per the schedule (i.e. if you get to Chapter 6 on Oct 28, we won't likely get to your questions as fast as we do for people who are still on Chapter 1).  

We designed this so you could do as much as possible offline (using pdfs and your own RStudio install) or on public machines (using Canvas and VMs, without software installation) to help mitigate any access issues you may have.  We have run this course online before, and it was a big hit, so we're hoping this will go smoothly as well!

Feel free to email me any questions you may have!  Also, feel free to send this to anyone you think might be interested, as this course is open to students at any university.

Dr. Sheri Sanders

Manager, Bioinformatic Analyst

National Center for Genome Analysis Support (NCGAS)

NCGAS is a management unit of the Research Technologies division of UITS; NCGAS is affiliated with the Indiana University Pervasive Technology Institute.

CI Engineering Lunch & Learn Series

Google Cloud Public Sector Summit

We will showcase fresh ideas and innovation from academic, research, and gov leaders alongside Google executives. 
 
Here is a sampling of topics that may be of interest to you:
•    Running Astronomical Pipelines with HTCondor
•    AI Fuels Smarter Medical Imaging and Better Outcomes
All sessions will have YouTube closed captioning option for language preference.
 
Explore the session catalog. We’re constantly adding sessions, so we encourage you to check in regularly.
 
Register Today →
 

 Michener, William K. 2015. “Ten Simple Rules for Creating a Good Data Management Plan (Links to an external site.).” PLoS Computational Biology 11(10): e1004525.

 Strasser, Carly. 2015. “Research Data Management: A Primer Publication of the National Information Standards Organization (Links to an external site.).” National Information Standards Organization (NISO). 

 Griffin, Philippa C. et al. 2017. "Best Practice Data Life Cycle Approaches for the Life Sciences (Links to an external site.)." bioRxiv 167619.

 Ziemann, Mark, Yotam Eren, and Assam El-Osta. 2016. "Gene name errors are widespread in the scientific literature (Links to an external site.)." Genome Biology 17(1): 177-179. 

 DataONE. Data Management Best Practices 

Kubernetes

Cloud Native Computing Foundation

https://landscape.cncf.io/

Shodor

Established in Durham, NC in 1994, Shodor is a nonprofit organization serving students and educators by providing materials and instruction relating to computational science (scientific, interactive computing).

With an Internet presence producing 3 to 4 million page views per month, Shodor has an international impact. Its award-winning, free online education tools such as Interactivate are popular with students and educators alike.

Shodor is transforming learning through computational thinking. In the Raleigh-Durham, NC area, Shodor offers workshops, apprenticeships and internships for youth and teens to build excitement for science, technology, engineering, and mathematics (STEM) through interactive explorations using hands-on and computer-enhanced activities, giving them the experience they will need to pursue a technology-intensive career path.

Data-Facing Track (first Tuesdays)

Tuesday, January 5, 1p ET/ 12p CT/ 11a MT/ 10a PT

Python for Big Data

Presenter: Bala Desinghu, Rutgers University

Python is a popular programming language for developing software and data science applications. Its popularity stems from many factors such as simplicity, readability, portability, etc. As such, Python is slow compared to C or Fortran and it does not manage memory well. These limitations in speed and memory management may not be significant when analyzing small data sets, but it becomes a bottleneck when analyzing big data sets. Techniques based on vectorization, parallelization, just in time compilation, and distributed task executions have been widely adopted by the Python community to address these challenges associated with big data. This presentation will address a few techniques suitable for large scale data analysis and answer the following questions: What to do when the data set size exceeds the available physical memory? How to speed up the data analysis? How to distribute the workloads when doing machine learning for big data sets?

Researcher-Facing Track (second Thursdays)

Thursday, January 14, 12p ET/ 11a CT/ 10a MT/ 9a PT

All about CaRCC (... beyond the R-F Track)

The Researcher-Facing Track would like to take the opportunity to start 2021 by pausing, reflecting, and asking "Who are we?" CaRCC has grown and evolved over the past two to three years -- the People Network alone currently has over 900 persons on its mailing list; and the number of working groups has approximately doubled. Many persons within CaRCC and part of the  Ecosystem are hard at work "boundary spanning" -- bringing groups together, sharing ideas, and helping both the RCD profession and the research & researchers we support. We will give attendees an overview & update on the organization; highlight some key, example activities and outputs of a couple Working Groups, and conclude with Engagement. This is a great time to invite new colleagues to come and learn about CaRCC, its activities, and how to Get Started or Get Involved!

Emerging Centers Track (third Wednesdays)

Wednesday, January 20, 12p ET/ 11a CT/ 10a MT/ 9a PT

EPOC and OSG support for CC* Proposals

Carrying on our popular meeting with NSF program directors on the CC* Program, the Emerging Centers Track January call will bring representatives from the EPOC (Engagement and Performance Operations Center) and OSG on support of potential CC* proposals.  Jennifer Schopf and Lauren Michael will contribute.  Learn more about each of these programs interoperation with CC* activities and how you can improve your proposal by engaging these organizations.

Systems-Facing Track (third Thursdays)

Thursday, January 21, 12p ET/ 11a CT/ 10a MT/ 9a PT

HPC Cluster Operating Systems Options

With the recent announcement from IBM/Redhat that CentOS 8 will be EOL at the end 2021 and CentOS 7 in 2024 many HPC systems professionals now face the prospect of having to migrate off the downstream distribution and either adopt CentOS 8 Stream or purchase licenses for RHEL. There are some institutions where clusters run other operating systems, most notably Cray has deployed SuSE as part of their Cray environment and NVIDIA DGX boxes notably run more smoothly with Ubuntu.  Our January meeting we will have a panel discussion on this topic and explore experiences using/migrating to/from other distributions on their HPC resources.