Troubleshooting on Sapelo2 - Revision history

Shtsai: /* Unrecognized lines following backslash line continuation */

2023-09-06T02:14:48Z

Unrecognized lines following backslash line continuation

← Older revision		Revision as of 22:14, 5 September 2023
Line 111:		Line 111:
	<span style="background:#fae406;">module load ml BLAST+/2.13.0-gompi-2022a </span><br>		<span style="background:#fae406;">module load ml BLAST+/2.13.0-gompi-2022a </span><br>
	<span style="background:#fae406;">blastn -num_threads 4 \</span><br>		<span style="background:#fae406;">blastn -num_threads 4 \</span><br>
	<span style="background:#fae406;">-db /db/ncbiblast/nrte/latest/nt \ \ </span><br>		<span style="background:#fae406;">-db /db/ncbiblast/nrte/latest/nt \  </span><br>
	<span style="background:#fae406;">-query my_fasta.fa</span><br>		<span style="background:#fae406;">-query my_fasta.fa</span><br>
	</div>		</div>

Shtsai: /* Unrecognized lines following backslash line continuation */

2023-09-06T02:14:33Z

Unrecognized lines following backslash line continuation

← Older revision		Revision as of 22:14, 5 September 2023
Line 111:		Line 111:
	<span style="background:#fae406;">module load ml BLAST+/2.13.0-gompi-2022a </span><br>		<span style="background:#fae406;">module load ml BLAST+/2.13.0-gompi-2022a </span><br>
	<span style="background:#fae406;">blastn -num_threads 4 \</span><br>		<span style="background:#fae406;">blastn -num_threads 4 \</span><br>
	<span style="background:#fae406;">-db /db/ncbiblast/nrte/latest/nt \ </span><br>		<span style="background:#fae406;">-db /db/ncbiblast/nrte/latest/nt \ \ </span><br>
	<span style="background:#fae406;">-query my_fasta.fa</span><br>		<span style="background:#fae406;">-query my_fasta.fa</span><br>
	</div>		</div>

Shtsai: /* Unrecognized lines following backslash line continuation */

2023-09-06T02:13:49Z

Unrecognized lines following backslash line continuation

← Older revision		Revision as of 22:13, 5 September 2023
Line 109:		Line 109:
	<nowiki>#</nowiki>SBATCH --mail-type=ALL<br>		<nowiki>#</nowiki>SBATCH --mail-type=ALL<br>

	<span style="background:#fae406;">module load ml BLAST+/2.10.1-gompi-~~2019b~~ </span><br>		<span style="background:#fae406;">module load ml BLAST+/2.13.0-gompi-2022a </span><br>
	<span style="background:#fae406;">blastn -num_threads 4 \</span><br>		<span style="background:#fae406;">blastn -num_threads 4 \</span><br>
	<span style="background:#fae406;">-db /db/ncbiblast/nrte/latest/nt \ </span><br>		<span style="background:#fae406;">-db /db/ncbiblast/nrte/latest/nt \ </span><br>
Line 117:		Line 117:
	There is a space after the backslash on the line "-db /db/ncbiblast/nrte/latest/nt \ ". This will cause error as "-query my_fasta.fa" would be interpreted an individual command. The blast command would throw its help prompt since there is no input file found. The other error could be like "Error: Too many positional arguments ..." due to the operand parameter line.		There is a space after the backslash on the line "-db /db/ncbiblast/nrte/latest/nt \ ". This will cause error as "-query my_fasta.fa" would be interpreted an individual command. The blast command would throw its help prompt since there is no input file found. The other error could be like "Error: Too many positional arguments ..." due to the operand parameter line.

	The solution is to simply remove the extra space(s).		The solution is to simply remove the extra space(s).

	===Job Runs out of Memory===		===Job Runs out of Memory===

Ben at 18:07, 30 November 2021

2021-11-30T18:07:12Z

← Older revision		Revision as of 14:07, 30 November 2021
Line 27:		Line 27:
	* Accidentally referencing a file with absolute path when you meant to reference it with a relative path or vice versa		* Accidentally referencing a file with absolute path when you meant to reference it with a relative path or vice versa
	* That the file is actually where you think it is		* That the file is actually where you think it is

			===Lmod has detected the following error: The following module(s) are unknown:===

			This error means that a module or modules you are trying to load do not exist. You may have made a typo when typing the module name, but you should look for the correct module names with the ml spider command. For example: <code>ml spider Python</code>.

	===Command not found===		===Command not found===

Ben at 15:55, 27 October 2020

2020-10-27T15:55:46Z

← Older revision		Revision as of 11:55, 27 October 2020
Line 117:		Line 117:
	===Job Runs out of Memory===		===Job Runs out of Memory===

	In some cases a job on an HPC cluster can fail because an application being used runs out of memory. When this happens, the operating system will kill the running process. You'll very often see the word "Killed" in the error message when this happens. You may also see the acronym "OOM," which stands for out of memory. ~~One thing to keep in mind in this case is~~ that ~~while~~ the ~~memory usage reported by Slurm may seem to indicate~~ the job ~~didn~~'~~t run out of memory, it~~ is ~~very possible that there was a sudden spike in~~ memory ~~that happened in between~~ the ~~intervals at which~~ Slurm ~~checks resource consumption~~.		In some cases a job on an HPC cluster can fail because an application being used runs out of memory. When this happens, the operating system will kill the running process. You'll very often see the word "Killed" in the error message when this happens. You may also see the acronym "OOM," which stands for out of memory. Additionally, if you look at the state of your job via the <code>seff</code> or <code>sacct</code> command, you'll see that the value for the job's state is OUT_OF_MEMORY. The solution to this error will typically be to allocate more memory to your job via either the --mem or --mem-per-cpu Slurm headers.

	[[#top\|Back to Top]]		[[#top\|Back to Top]]

Ben at 14:51, 27 October 2020

2020-10-27T14:51:16Z

← Older revision		Revision as of 10:51, 27 October 2020
Line 114:		Line 114:

	The solution is to simply remove the extra space(s).		The solution is to simply remove the extra space(s).

			===Job Runs out of Memory===

			In some cases a job on an HPC cluster can fail because an application being used runs out of memory. When this happens, the operating system will kill the running process. You'll very often see the word "Killed" in the error message when this happens. You may also see the acronym "OOM," which stands for out of memory. One thing to keep in mind in this case is that while the memory usage reported by Slurm may seem to indicate the job didn't run out of memory, it is very possible that there was a sudden spike in memory that happened in between the intervals at which Slurm checks resource consumption.

	[[#top\|Back to Top]]		[[#top\|Back to Top]]

Shtsai: /* General Troubleshooting Guidelines */

2020-10-27T13:20:56Z

General Troubleshooting Guidelines

← Older revision		Revision as of 09:20, 27 October 2020
Line 12:		Line 12:
	* '''Use Google'''. It is cliché advice, but too important not to mention. Perhaps you may find the relevant error in an output file, but it makes no sense to you. One of the first things to do in this situation is to copy the error and paste it into Google. More often than not, you will find someone who encountered the same or a similar error and a solution to it.		* '''Use Google'''. It is cliché advice, but too important not to mention. Perhaps you may find the relevant error in an output file, but it makes no sense to you. One of the first things to do in this situation is to copy the error and paste it into Google. More often than not, you will find someone who encountered the same or a similar error and a solution to it.
	* '''Be conscientious of the case of the text you type'''. One of the main things that takes time to adjust to if you're new to using Linux is getting used to being very particular about using upper or lowercase text. The case of text matters when referencing files or directories in Linux. For example, <code>File1.txt</code> and <code>file1.txt</code> are two different file names. There are many errors that can occur by using the incorrect case of text in Linux.		* '''Be conscientious of the case of the text you type'''. One of the main things that takes time to adjust to if you're new to using Linux is getting used to being very particular about using upper or lowercase text. The case of text matters when referencing files or directories in Linux. For example, <code>File1.txt</code> and <code>file1.txt</code> are two different file names. There are many errors that can occur by using the incorrect case of text in Linux.
	* '''Be conscientious of your current directory and relative file paths'''. Sometimes it may seem that a file or directory is missing when in fact an incorrect path was referenced or care was not given to where things are located. For example, it is easy to make the mistake of trying to access your scratch directory, but forgetting to type the root directory, in which the scratch directory is located (<code>/scratch/MyID</code> vs. <code>scratch/MyID</code>). When submitting a job to Slurm, the ~~jobs~~ execution directory will default to the directory from which you submitted the job. Therefore, make sure your input files are in that directory or have their location correctly specified.		* '''Be conscientious of your current directory and relative file paths'''. Sometimes it may seem that a file or directory is missing when in fact an incorrect path was referenced or care was not given to where things are located. For example, it is easy to make the mistake of trying to access your scratch directory, but forgetting to type the root directory, in which the scratch directory is located (<code>/scratch/MyID</code> vs. <code>scratch/MyID</code>). When submitting a job to Slurm, the job's execution directory will default to the directory from which you submitted the job. Therefore, make sure your input files are in that directory or have their location correctly specified.
	* '''Try troubleshooting job steps in an interactive session'''. It can be helpful when encountering an error with a job on an HPC cluster to start an interactive session, load necessary modules, and try to execute job steps one by one in real time on the command line. Being able to see exactly what is happening with each step of your job like this can provide insight into any errors that may be occurring.		* '''Try troubleshooting job steps in an interactive session'''. It can be helpful when encountering an error with a job on an HPC cluster to start an interactive session, load necessary modules, and try to execute job steps one by one in real time on the command line. Being able to see exactly what is happening with each step of your job like this can provide insight into any errors that may be occurring.
	* '''Take advantage of the job mail features'''. You are able to get detailed information about your job when it starts and/or finishes by using the <code>#SBATCH --mail-type=</code> and <code>#SBATCH --mail-user=</code> options in your submissions scripts. This will give you detailed information about your job, including resource usage, which can be relevant if not enough resources were provided to a job. These job emails can help provide insight into many different types of errors.		* '''Take advantage of the job mail features'''. You are able to get detailed information about your job when it starts and/or finishes by using the <code>#SBATCH --mail-type=</code> and <code>#SBATCH --mail-user=</code> options in your submissions scripts. This will give you detailed information about your job, including resource usage, which can be relevant if not enough resources were provided to a job. These job emails can help provide insight into many different types of errors.

Shtsai: /* Unrecognized lines following backslash line continuation */

2020-10-27T13:18:16Z

Unrecognized lines following backslash line continuation

← Older revision		Revision as of 09:18, 27 October 2020
Line 111:		Line 111:
	</div>		</div>

	There is a space after the backslash on the line "-db /db/ncbiblast/nrte/latest/nt \ ". This will cause error as "-query my_fasta.fa" would be interpreted an individual command. The blast command would throw its help prompt since there is no input file found. The other error could be ~~"error as~~ "Error: Too many positional arguments ..." due to the operand parameter line.		There is a space after the backslash on the line "-db /db/ncbiblast/nrte/latest/nt \ ". This will cause error as "-query my_fasta.fa" would be interpreted an individual command. The blast command would throw its help prompt since there is no input file found. The other error could be like "Error: Too many positional arguments ..." due to the operand parameter line.

	The solution is to simply remove the extra space(s).		The solution is to simply remove the extra space(s).

	[[#top\|Back to Top]]		[[#top\|Back to Top]]

Shtsai at 13:15, 27 October 2020

2020-10-27T13:15:42Z

← Older revision		Revision as of 09:15, 27 October 2020
Line 5:		Line 5:
	==General Troubleshooting Guidelines==		==General Troubleshooting Guidelines==

	We aim to provide a comprehensive resource for various errors that may come up when working on an HPC cluster, but occasionally you may encounter errors that are not specifically outlined here. These are just a few general troubleshooting guidelines that ~~you~~ will help you resolve issues more quickly.		We aim to provide a comprehensive resource for various errors that may come up when working on an HPC cluster, but occasionally you may encounter errors that are not specifically outlined here. These are just a few general troubleshooting guidelines that will help you resolve issues more quickly.

	* '''Check job output files'''. By default, when you submit a job to the cluster, it will produce an output file with the naming format of slurm-''JobID''.out in the directory from which you submitted your job. It is important to check this file in order to diagnose any error that may come up.		* '''Check job output files'''. By default, when you submit a job to the cluster, it will produce an output file with the naming format of slurm-''JobID''.out in the directory from which you submitted your job. It is important to check this file in order to diagnose any error that may come up.
	* '''Read job output very carefully'''. Whether it is the default Slurm job output file or an output file produced by the software being used, it is worth taking the time to carefully read any output. Often these files may have a lot of text to parse through, but by taking the time to read them carefully, you may just find that one line that gives you the information you need to understand why an error is occurring.		* '''Read job output very carefully'''. Whether it is the default Slurm job output file or an output file produced by the software being used, it is worth taking the time to carefully read any output. Often these files may have a lot of text to parse through, but by taking the time to read them carefully, you may just find that one line that gives you the information you need to understand why an error is occurring.
	* '''At least look at the end of a stack trace'''. A stack trace ~~the~~ often lengthy list of commands that happened when some exception occurred. It can be intimidating and confusing to look at, but very often the root cause of the problem will be at the bottom of the stack trace, often revealing the problem is something simple that you have the ability to fix.		* '''At least look at the end of a stack trace'''. A stack trace is an often lengthy list of commands that happened when some exception occurred. It can be intimidating and confusing to look at, but very often the root cause of the problem will be at the bottom of the stack trace, often revealing the problem is something simple that you have the ability to fix.
	* '''Use Google'''. It is cliché advice, but too important not to mention. Perhaps you may find the relevant error in an output file, but it makes no sense to you. One of the first things to do in this situation is to copy the error and paste it into Google. More often than not, you will find someone who encountered the same or a similar error and a solution to it.		* '''Use Google'''. It is cliché advice, but too important not to mention. Perhaps you may find the relevant error in an output file, but it makes no sense to you. One of the first things to do in this situation is to copy the error and paste it into Google. More often than not, you will find someone who encountered the same or a similar error and a solution to it.
	* '''Be conscientious of the case of the text you type'''. One of the main things that takes time to adjust to if you're new to using Linux is getting used to being very particular about using upper or lowercase text. The case of text matters when referencing files or directories in Linux. For example, <code>File1.txt</code> and <code>file1.txt</code> are two different file names. There are many errors that can occur by using the incorrect case of text in Linux.		* '''Be conscientious of the case of the text you type'''. One of the main things that takes time to adjust to if you're new to using Linux is getting used to being very particular about using upper or lowercase text. The case of text matters when referencing files or directories in Linux. For example, <code>File1.txt</code> and <code>file1.txt</code> are two different file names. There are many errors that can occur by using the incorrect case of text in Linux.

Ben at 18:05, 26 October 2020

2020-10-26T18:05:44Z

← Older revision		Revision as of 14:05, 26 October 2020
Line 10:		Line 10:
	* '''Read job output very carefully'''. Whether it is the default Slurm job output file or an output file produced by the software being used, it is worth taking the time to carefully read any output. Often these files may have a lot of text to parse through, but by taking the time to read them carefully, you may just find that one line that gives you the information you need to understand why an error is occurring.		* '''Read job output very carefully'''. Whether it is the default Slurm job output file or an output file produced by the software being used, it is worth taking the time to carefully read any output. Often these files may have a lot of text to parse through, but by taking the time to read them carefully, you may just find that one line that gives you the information you need to understand why an error is occurring.
	* '''At least look at the end of a stack trace'''. A stack trace the often lengthy list of commands that happened when some exception occurred. It can be intimidating and confusing to look at, but very often the root cause of the problem will be at the bottom of the stack trace, often revealing the problem is something simple that you have the ability to fix.		* '''At least look at the end of a stack trace'''. A stack trace the often lengthy list of commands that happened when some exception occurred. It can be intimidating and confusing to look at, but very often the root cause of the problem will be at the bottom of the stack trace, often revealing the problem is something simple that you have the ability to fix.
	* '''Use Google'''. It is cliché advice, but too important not to mention. Perhaps you may find the relevant error in an output file, but it makes no sense to you. One of the first ~~thing~~ to do in this situation is to copy the error and paste it into Google. More often than not, you will find someone who encountered the same or a similar error and a solution to it.		* '''Use Google'''. It is cliché advice, but too important not to mention. Perhaps you may find the relevant error in an output file, but it makes no sense to you. One of the first things to do in this situation is to copy the error and paste it into Google. More often than not, you will find someone who encountered the same or a similar error and a solution to it.
	* '''Be conscientious of the case of the text you type'''. One of the main things that takes time to adjust to if you're new to using Linux is getting used to being very particular about using upper or lowercase text. The case of text matters when referencing files or directories in Linux. For example, <code>File1.txt</code> and <code>file1.txt</code> are two different file names. There are many errors that can occur by using the incorrect case of text in Linux.		* '''Be conscientious of the case of the text you type'''. One of the main things that takes time to adjust to if you're new to using Linux is getting used to being very particular about using upper or lowercase text. The case of text matters when referencing files or directories in Linux. For example, <code>File1.txt</code> and <code>file1.txt</code> are two different file names. There are many errors that can occur by using the incorrect case of text in Linux.
	* '''Be conscientious of your current directory and relative file paths'''. Sometimes it may seem that a file or directory is missing when in fact an incorrect path was referenced or care was not given to where things are located. For example, it is easy to make the mistake of trying to access your scratch directory, but forgetting to type the root directory, in which the scratch directory is located (<code>/scratch/MyID</code> vs. <code>scratch/MyID</code>). When submitting a job to Slurm, the jobs execution directory will default to the directory from which you submitted the job. Therefore, make sure your input files are in that directory or have their location correctly specified.		* '''Be conscientious of your current directory and relative file paths'''. Sometimes it may seem that a file or directory is missing when in fact an incorrect path was referenced or care was not given to where things are located. For example, it is easy to make the mistake of trying to access your scratch directory, but forgetting to type the root directory, in which the scratch directory is located (<code>/scratch/MyID</code> vs. <code>scratch/MyID</code>). When submitting a job to Slurm, the jobs execution directory will default to the directory from which you submitted the job. Therefore, make sure your input files are in that directory or have their location correctly specified.