Update 2023-09-25-Eddie.md

_posts/2023-09-25-Eddie.md

@@ -44,22 +44,22 @@ This is the shared group space in UoE. \
 ## **Basic Commands on Eddie**
 To start an interactive session run:
 ```shell
-$ qlogin 
+qlogin 
 # The default maximum runtime is 48 hours. The default resources: 1 core, 1 GB memory.
 
-$ qlogin -l h_vmem=1G 
+qlogin -l h_vmem=1G 
 # request memory for an interactive session
 
-$ qlogin -pe gpu 1 
+qlogin -pe gpu 1 
 # request 1 gpu (need for torch and memory usage is at least 2GB)
 
-$ qlogin -q staging 
+qlogin -q staging 
 # choose the staging environment
 ```
 
 Staging data (transfer data between Eddie and DataStore):
 ```shell
-$ qsub stagein/out.sh 
+qsub stagein/out.sh 
 # stagein/out.sh file content can be seen in Eddie handbook (need UUN login)
 ```
 > ##### TIP
@@ -68,103 +68,103 @@ $ qsub stagein/out.sh
 
 To view current running jobs
 ```shell
-$ qstat
+qstat
 ```
 To cancel a job,
 ```shell
-$ qdel <JOB IDENTIFIER>
+qdel <JOB IDENTIFIER>
 ```
 Detailed information about completed jobs is available with the qacct command:
 ```shell
-$ qacct -j <JOB IDENTIFIER>
+qacct -j <JOB IDENTIFIER>
 ```
 
 ## **Set up the Anaconda Environment**
 
 ### Add path and create a virtual environment
 Configure the path for your environments directory, i.e., the directory where all your conda environments will be stored, and packages (pkgs) directory used as a cache for your conda packages. By default cache for packages will be placed under `~/.conda/pkgs` and will very likely cause your home directory to go over quota as a result. We recommend using the group space you have used for your conda environments instead. **Note that this should be an existing directory, so you need to create it first if it doesn't already exist, before running the command below.**
 ```shell
-$ module load anaconda # version 5.0.1
+module load anaconda # version 5.0.1
 ```
 Skip this step if already created the virtual environment
 ```shell
-$ conda config --add envs_dirs /exports/<COLLEGE>/eddie/<SCHOOL>/groups/<GROUP NAME>/anaconda/envs
+conda config --add envs_dirs /exports/<COLLEGE>/eddie/<SCHOOL>/groups/<GROUP NAME>/anaconda/envs
 
-$ conda config --add pkgs_dirs /exports/<COLLEGE>/eddie/<SCHOOL>/groups/<GROUP NAME>/anaconda/pkgs
+conda config --add pkgs_dirs /exports/<COLLEGE>/eddie/<SCHOOL>/groups/<GROUP NAME>/anaconda/pkgs
 ```
 Once you have configured your `envs_dirs` and `pkgs_dirs`, you can create an environment. The example below creates an environment specifying particular Python versions (also the version used for this project):
 ```shell
-$ conda create -n %env_name% python=3.9
+conda create -n %env_name% python=3.9
 ```
 To list your conda environments:
 ```shell
-$ conda info --envs
+conda info --envs
 ```
 
 ### Activate your environment
 Switch to the created environment and deactivate the environment when no longer needed:
 ```shell
-$ source activate %env_name%
-$ source deactivate
+source activate %env_name%
+source deactivate
 ```
 
 ### Install required packages
 ```shell
-$ conda install pip
-$ pip --no-cache-dir install -r requirements.txt 
+conda install pip
+pip --no-cache-dir install -r requirements.txt 
 # `--no-cache-dir` is to ensure torch installed successfully
 ```
 **Some package management commands:**
 ```shell
-$ conda list 
+conda list 
 # To see what packages are in the active environment
-$ conda search matplotlib 
+conda search matplotlib 
 # To search for a package in the Anaconda repositories
-$ conda install matplotlib=1.4.1 
+conda install matplotlib=1.4.1 
 # To install a package e.g. matplotlib version 1.4.1
 ```
 
 #### Using pip
 If a package is not available in the Anaconda repositories it can still be installed with `pip` in the usual way. First install `pip` into your active Anaconda environment. You must install pip into your environment first before running a pip command. Otherwise, `pip install` will use the pip installed in the Anaconda directory (which users do not have write access to) and your installation will fail. Once you have installed pip in your environment, then you can install a package using pip into your environment:
 ```shell
-$ conda install pip
-$ pip install matplotlib
+conda install pip
+pip install matplotlib
 ```
 
 ## **Set up R Environment**
 
 ### Configure R packages path
 ```shell
-$ module load cmake/3.5.2
+module load cmake/3.5.2
 # some packages may need this to be loaded first
-$ module load igmm/apps/R/4.1.3
-$ export R_LIBS=/exports/<COLLEGE>/eddie/<SCHOOL>/groups/<GROUP NAME>/rlibs
+module load igmm/apps/R/4.1.3
+export R_LIBS=/exports/<COLLEGE>/eddie/<SCHOOL>/groups/<GROUP NAME>/rlibs
 ```
 
 **Install required packages in R**
 ```R
-> install.packages("tidyverse")
+install.packages("tidyverse")
 ```
 
 ## **Run Code on Eddie**
 ### Through job submission
 **Example**
 ```shell
-$ qsub run_job.sh 
+qsub run_job.sh 
 # job submission files are available in shell folder
 ```
 
 ### Through interaction session
 First start an interaction session, specify max run time, memory usage (gpu) and activate the environment then run according to the following:
 **Run Jupyter notebook**
 ```shell
-$ jupyter nbconvert --to notebook --inplace --execute jupyter-notebook.ipynb
+jupyter nbconvert --to notebook --inplace --execute jupyter-notebook.ipynb
 ```
 Note that `nbconvert` will fail if any cell takes longer than 30s to run, then add the following option to disable timing
 ```
 --ExecutePreprocessor.timeout=-1
 ```
 View jupyter notebook ouptputs
 ```shell
-$ nbpreview jupyter-notebook.ipynb
+nbpreview jupyter-notebook.ipynb
 ```