# Frequently Asked Questions ```{tip} See our [Navigating CURC Documentation](./navigating_docs) page for information on core areas of our documentation each user should become familiar with. ``` ## How do I acknowledge use of CURC Resources? ::::{dropdown} Show :icon: note On our [Acknowledging CURC Resources](./acknowledge_curc_resources.md) page, we provide material on how to acknowledge CURC resources. :::: ## Duo Multi-Factor Authentication ### How do I setup Duo? :::{dropdown} Show :icon: note - CU Boulder users can follow these steps: 1. Download the Duo mobile app to your phone (available via Apple App Store or Google Play Store). 2. Once installed, visit to enroll. - CSU users please see [CSU's documentation on Duo 2-factor authentication](https://it.colostate.edu/duo-two-factor-authentication/) - CSU users can manage their account at [https://it.colostate.edu/duo-two-factor-authentication/](https://it.colostate.edu/duo-two-factor-authentication/) - AMC or RMACC users please see [Duo's documentation on 2-factor authentication](https://guide.duo.com/) - AMC and RMACC users can manage their account at [https://cilogon.org/?selected_idp=httsp%3A%2F%2Faccess-ci.org%2Fidp&skin=access](https://cilogon.org/?selected_idp=httsp%3A%2F%2Faccess-ci.org%2Fidp&skin=access) ::: ### As a CU Boulder user, how can I login with Duo? ::::::{dropdown} Show :icon: note Duo offers a variety of methods to log into your account. Depending on what you select when setting up your Duo account, you should have several different methods of 2-factor authentication when logging into RC Resources. (tabset-ref-ucb-duo-login-methods)= `````{tab-set} :sync-group: tabset-ucb-duo-login-methods ````{tab-item} Duo Mobile Push :sync: ucb-duo-login-methods-push **Duo Mobile Push is Research Computing's recommended method of 2-factor authentication.** Because Duo Push is tied to your physical smart device instead of a phone number or account, Duo Push provides a more secure method of 2-factor authentication than either SMS or phone call. 1. Type: `ssh @login.rc.colorado.edu` into the command line. For example, if my username is jodo2018 I would type `ssh jodo2018@login.rc.colorado.edu`. 2. Enter your identikey password when prompted. 3. Wait for a push to your phone. ![](./faq_images/duo_app2.png) ![](./faq_images/duo_app1.png) ```{note} Duo push is tied to your device so [you will need to add or remove your device if you get a new phone.](#as-a-cu-boulder-user-how-can-i-manage-my-duo-devices) There is also an option to generate a temporary passcode from within the Duo app. This passcode can be used to log into . ``` ```` ````{tab-item} Duo SMS :sync: ucb-duo-login-methods-sms If you prefer to not use the Duo app or if you don't have a smart device, then Duo offers an SMS method of 2-factor authentication: 1. Type: `ssh @login.rc.colorado.edu` into the command line. For example, if my username is jodo2018 I would type `ssh jodo2018@login.rc.colorado.edu`. 2. Enter your password when prompted, as `mypassword,sms`. For example, if my password is Ih3artdu0 I would type: `Ih3artdu0,sms`. 3. A list of one-time-passwords (OTPs) will be texted to you via SMS. Another login prompt will appear on your screen. Using the first OTP in the list, login with your password as `mypassword,OTP`. For example, if my password is Ih3artdu0 and my OTP is 330456 I would type (without quotes): `Ih3artdu0,330456`. 4. Note that the next time you login, you can either request a new list of OTPs using Step 2 and then enter the first OTP via Step 3, or you can just use the next OTP in the list, and skip directly to Step 3. ```` ````{tab-item} Duo Phone Call :sync: ucb-duo-login-methods-call Duo also provides a phone call solution for 2-factor authorization if you only have a land line, or prefer to not use Push or SMS: 1. Type: `ssh username@login.rc.colorado.edu` into the command line. For example, if my username is jodo2018 I would type `ssh jodo2018@login.rc.colorado.edu`. 2. Enter your password when prompted, as `mypassword,phone`. For example, if my password is Ih3artdu0 I would type: `Ih3artdu0,phone`. 3. Wait for a phone call. Answer the call, select option #1, and you will automatically be logged in. ```` ````{tab-item} Duo Token :sync: ucb-duo-login-methods-token This documentation is for customers who have [registered a Yubikey with CU Boulder](https://oit.colorado.edu/tutorial/register-yubikey-otp). 1. Type: `ssh username@login.rc.colorado.edu` into the command line. For example, if my username is jodo2018 I would type `ssh jodo2018@login.rc.colorado.edu`. 2. Enter your password when prompted, as `mypassword,6-digit-number`. The 6-digit number is given to you on the Duo token. For example, if my password is Ih3artdu0, and I press the button on the token and it is 123456, I would type: `Ih3artdu0,123456`. ```` ````` :::::: ### I have a new phone. As a CU Boulder user, how do I move my Duo onto it? ::::{dropdown} Show :icon: note You can add a new device to your duo account by visiting https://duo.colorado.edu. After a CU authorization page you will be directed to a Duo authentication page. **Ignore the Duo Push prompt and instead click "Add a new device":** ![](faq_images/duo_new_device1.png) Duo will then try to authenticate your account by push notification to verify your identity. Cancel this push notification... ![](faq_images/duo_new_device2.png) ...and click on "Enter a Passcode", or "Call Me". - If you select "Call Me" the simply receive the call and press 1. - If you select "Enter a Passcode" then click "Text me new codes" and you will be sent a list of one time passwords. Type in any one of the codes and you will be authenticated. Once you have verified your identity, follow the instructions provided by Duo to add your device. If you cannot authenticate your account (e.g. do not have your old device), we suggest reviewing the [Duo FAQ page](https://oit.colorado.edu/services/identity-access-management/multi-factor-remote-access/faq) or contacting . :::: ### As a CU Boulder user, how can I manage my Duo devices? ::::{dropdown} Show :icon: note Users can manage their own Duo devices by visiting . This enables users to add or remove activated devices at their discretion. After a CU authorization page, you will be directed to a Duo authentication page. Do not respond to the Push notification and instead click the "Settings" button: ![](./faq_images/duo-management1edit.png) This will bring up a menu that provides several options on device management and general help. Clicking "Add a new device" will allow you to add a new smart phone, tablet, or land-line. Simply select the option you wish to add and follow the steps provided by Duo to complete setting up your new device. Selecting "My Settings & Devices" provides a more detailed list of all devices you have registered. From here you can also add a new device, set your default device, and change your default authentication method when you attempt to log in. :::: ### None of the FAQs for Duo resolved my issue, how do I proceed as a CU Boulder user? ::::{dropdown} Show :icon: note If none of the above resolved the issues you were experiencing, then your Duo account may have expired. The university purges Duo accounts after six months of non-use. You can re-enroll by visiting . If that did not resolve your issue, then we suggest contacting the University helpdesk at or calling 303-735-4357. :::: ## General High-Performance Computing ### What is Arbiter2? ::::{dropdown} Show :icon: note [Arbiter2](https://github.com/chpc-uofu/arbiter2) is a tool created by the University of Utah that enables monitoring of non-compute node resources for undesirable behavior. For an in-depth explanation of how Arbiter2 works, please see the official paper ["Arbiter: Dynamically Limiting Resource Consumption on Login Nodes"](https://dl.acm.org/doi/10.1145/3332186.3333043). In the context of Arbiter2, "undesirable behavior" refers to processes that consume substantial CPU or memory (RAM) resources on low resource hosts, such as CURC's [login nodes](../compute/node-types.md#login-nodes). If Arbiter2 flags a user's process(es) as undesirable behavior, the user's account will be placed into a penalty state. Once in a penalty state, the amount of CPU and memory resources available to the user on the host will be reduced and the user will be sent a no-reply warning email. Examples of processes that can consume substantial resources include installing/compiling/running software applications and manipulating large files (e.g. compressing, uncompressing, and/or moving files). For a list of all hosts that Arbiter2 is deployed on, see the `Host` column in the table below. ```{important} - A user's penalty state will be the same across all of the login nodes. This means that if your account is placed in a penalty state on one login node, it will be in the same penalty state on the other login nodes. - Where and how Arbiter2 is deployed may change over time. This is due to adjustments we may need to make so that Arbiter2 fits the needs of the system and users. Please refer to this FAQ for the most up-to-date information. - Once a user's processes consume more than the allotted amount of CPU resources, their processes will be automatically throttled. - If a user's processes use more memory than they are allotted, the processes will be automatically killed. ``` `````{admonition} Attention :class: danger Please note that the maximum values listed are intended to support intermittent spikes in demand for resources, not sustained usage. These hosts are a shared resource, so please be considerate of others and use the compute nodes whenever possible. ````` ```{eval-rst} .. raw:: html :file: ./faq_html/arbiter_penalty_table.html ``` :::: ### How can I reset my .bashrc? ::::{dropdown} Show :icon: note Every Research Computing account has a .bashrc file in their /home directory. This file ensures your account's environment is properly configured whenever you login or run a job on a compute node. We do not recommend editing this file, unless you know what you are doing. If you accidentally delete or edit the file, you can restore your .bashrc by taking the following steps: 1. Navigate to your home directory : `cd ~` 2. Open or create the .bashrc : `vim .bashrc` or `nano .bashrc` 3. Copy + paste the following code snippet into your .bashrc and then save the file, replacing any existing text in the file (if any). ``` # .bashrc if [ -f /etc/bashrc ]; then . /etc/bashrc fi ``` :::: ### How can I add users to a Linux group? ::::{dropdown} Show :icon: note Before requesting users be added to a Linux group, please fully read the [Workspace Sharing](../compute/filesystems.md#workspace-sharing) section, which provides an alternative approach for workspace sharing. If you would like to proceed with adding someone to a Linux group, please submit a [support request form](https://colorado.service-now.com/req_portal?id=ucb_sc_rc_form) using the following: 1. Select `Storage` for `Nature of request` 2. Select `I would like to add a collaborator to my directory` for `Issue type` 3. Fill out the requested additional information 4. Provide any justification or additional details in `Detailed description` :::: ### When will my job start? ::::{dropdown} Show :icon: note You can pull up information on your job's start time using the `squeue` command: ``` squeue --user=your_rc-username --start ``` Note that Slurm's estimated start time can be a bit inaccurate. This is because Slurm calculates this estimation off the jobs that are currently running or queued in the system. Any job that is added in later with a higher priority may delay your job. For more information on the `squeue` command, [take a look at our Useful Slurm Commands tutorial.](../running-jobs/slurm-commands.md) Or visit the Slurm page on [squeue](https://slurm.schedmd.com/squeue.html) Note that you can also see system level wait times and how they change through time by visiting the [CURC metrics portal](../compute/monitoring-resources.md) at [https://xdmod.rc.colorado.edu](https://xdmod.rc.colorado.edu) :::: ### How can I get metrics about CURC systems such as how busy they are, wait times, and account usage? ::::{dropdown} Show :icon: note Please refer to our [XDMoD](../compute/monitoring-resources.md#xdmod) documentation, which is a portal that allows users to easily see CURC system metrics. :::: ### How much memory did my job use? ::::{dropdown} Show :icon: note You can check how much memory your job utilized by using the `sacct` command and refering to the `MaxRSS` metric. This is done as follows where you can replace `YYYY-MM-DD` with the date you ran the job and specify your JobID: ``` sacct --starttime=YYYY-MM-DD --jobs=your_job_id --format=User,JobName,JobId,MaxRSS ``` If you'd like to monitor memory usage on jobs that are currently running, use the `sstat` command: ``` sstat --jobs=your_job_id --format=User,JobName,JobId,MaxRSS ``` For more information on `sstat` or `sacct` commands, [take a look at our Useful Slurm Commands tutorial.](../running-jobs/slurm-commands.md) Or visit the Slurm reference pages on [sstat](https://slurm.schedmd.com/sstat.html) and [sacct](https://slurm.schedmd.com/sacct.html). You can also view information related to service unit (SU) usage and CPU & RAM efficiency [using slurm commands](../compute/monitoring-resources.md#monitoring-through-slurm-commands). Note that CPU & RAM efficiency statistics will be included in emails sent when a job completes, if requested. :::: ### Why am I getting unexpected results for my GPU memory or utilization metrics? ::::{dropdown} Show :icon: note Currently, GPU metrics are collected by Slurm. Due to system limitations, there are some scenarios where GPU metric collection is not possible. Thus, the output of `sacct` or `seff` may not match your expectation or what is provided by `nvidia-smi` or `amd-smi`. Below we provide important considerations when reviewing GPU metrics collected by Slurm. - GPU metrics are available for most NVIDIA GPUs across Alpine and Blanca. At this time, GPU memory and utilization metrics are not available on the following GPU types: AMD, GH200s, P100s, and A40s. - GPU metrics can only be collected if your code utilizes CUDA 12 or newer. - GPU metrics are partially available on MIG-enabled NVIDIA GPUs. You can get GPU memory usage values, but GPU utilization metrics are not reported for individual MIG partitions. - Core cluster GPUs (e.g., core-gpu[0-4], viz1, viz2), which power Core Desktop and MATLAB GUI, do not currently support GPU memory or utilization metrics. - Users running jobs on unsupported GPUs or older CUDA versions will see zeros or N/A or infinite values for GPU memory and utilization fields. Make sure your jobs are running on compatible hardware to obtain meaningful GPU metrics. :::: ### How can I see my current FairShare priority? ::::{dropdown} Show :icon: note There are a couple ways you can check your FairShare priority: 1. Using the `levelfs` command. `levelfs` shows the current fair share priority of a specified user. Using `levelfs` on your username: ``` $ levelfs $USER ``` * A value of 1 indicates average priority compared to other users in an account. * A value of < 1 indicates lower than average priority (longer than average queue waits) * A value of > 1 indicates higher than average priority (shorter than average queue waits)

2. Using the `sshare` command: ``` sshare -U -l ``` The `sshare` command will print out a table of information regarding your usage and priority on all allocations. The `-U` flag will specify the current user and the `-l` flag will print out more details in the table. The field we are looking for is the _LevelFS_. The LevelFS holds a number from 0 to infinity that describes the fair share of an association in relation to its other siblings in an account. Over-serviced accounts will have a LevelFS between 0 and 1. Under-serviced accounts will have a LevelFS greater than 1. Accounts that haven't run any jobs will have a LevelFS of infinity (inf). For more information on fair share the `sshare` command, [take a look at Slurm's documentation on fair share](https://slurm.schedmd.com/fair_tree.html) Or [check out the Slurm reference page on sshare](https://slurm.schedmd.com/sshare.html) :::: ### Why is my job pending with reason `ReqNodeNotAvail`? ::::{dropdown} Show :icon: note The `ReqNodeNotAvail` message usually means that your node has been reserved for maintenance during the period you have requested within your job script. This message often occurs in the days leading up to our regularly scheduled maintenance, which is performed the first Wednesday of every month. So, for example, if you run a job with a 72 hour wall clock request on the first Monday of the month, you will receive the `ReqNodeNotAvail` error because the node is reserved for maintenance within that 72-hour window. You can confirm whether the requested node has a reservation by typing `scontrol show reservation` to list all active reservations. If you receive this message, the following solutions are available: 1. Run a shorter job or modify your current job's time so that it does not intersect with the maintenance window. One can modify your current job's time by using the `scontrol` command: ```bash $ scontrol update jobid= time=