+
Skip to content

Minor fixes spotted in v1.7 template #425

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Oct 16, 2019
Merged

Minor fixes spotted in v1.7 template #425

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
0d0f1db
Highlight citation
drpatelh Oct 14, 2019
baa33dc
Update igenomes files
drpatelh Oct 14, 2019
8e1f6fc
Add max_multiqc_email_size parameter
drpatelh Oct 14, 2019
29b3503
Add max_multiqc_email_size parameter
drpatelh Oct 14, 2019
92a8e79
Add max_multiqc_email_size parameter
drpatelh Oct 14, 2019
57e8f70
Update CHANGELOG
drpatelh Oct 14, 2019
24e388f
Add new para
drpatelh Oct 14, 2019
f3a83c4
Add -- before param
drpatelh Oct 14, 2019
78a8380
Export Conda environment in Dockerfile
drpatelh Oct 15, 2019
71b69fa
Update CHANGELOG
drpatelh Oct 15, 2019
cf9c662
Reorder commands
drpatelh Oct 15, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,10 @@

### Template

* Fixed incorrect paths in iGenomes config as described in issue [418](https://github.com/nf-core/tools/issues/418)
* Fixed incorrect paths in iGenomes config as described in issue [#418](https://github.com/nf-core/tools/issues/418)
* Add UCSC genomes to `igenomes.config` and add paths to all genome indices
* Change `maxMultiqcEmailFileSize` parameter to `max_multiqc_email_size`
* Export conda environment in Docker file [#349](https://github.com/nf-core/tools/issues/349)

### Other

Expand Down
1 change: 1 addition & 0 deletions nf_core/pipeline-template/{{cookiecutter.name_noslash}}/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,5 @@ LABEL authors="{{ cookiecutter.author }}" \

COPY environment.yml /
RUN conda env create -f /environment.yml && conda clean -a
RUN conda env export --name {{ cookiecutter.name_noslash }}-{{ cookiecutter.version }} > {{ cookiecutter.name_noslash }}-{{ cookiecutter.version }}.yml
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested this with the atacseq and chipseq Singularity images and it seems to be working fine.

ENV PATH /opt/conda/envs/{{ cookiecutter.name_noslash }}-{{ cookiecutter.version }}/bin:$PATH
3 changes: 2 additions & 1 deletion nf_core/pipeline-template/{{cookiecutter.name_noslash}}/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,4 +64,5 @@ For further information or help, don't hesitate to get in touch on [Slack](https
<!-- If you use {{ cookiecutter.name }} for your analysis, please cite it using the following doi: [10.5281/zenodo.XXXXXX](https://doi.org/10.5281/zenodo.XXXXXX) -->

You can cite the `nf-core` pre-print as follows:
Ewels PA, Peltzer A, Fillinger S, Alneberg JA, Patel H, Wilm A, Garcia MU, Di Tommaso P, Nahnsen S. **nf-core: Community curated bioinformatics pipelines**. *bioRxiv*. 2019. p. 610741. [doi: 10.1101/610741](https://www.biorxiv.org/content/10.1101/610741v1).

> Ewels PA, Peltzer A, Fillinger S, Alneberg JA, Patel H, Wilm A, Garcia MU, Di Tommaso P, Nahnsen S. **nf-core: Community curated bioinformatics pipelines**. *bioRxiv*. 2019. p. 610741. [doi: 10.1101/610741](https://www.biorxiv.org/content/10.1101/610741v1).
473 changes: 339 additions & 134 deletions nf_core/pipeline-template/{{cookiecutter.name_noslash}}/conf/igenomes.config
View file
Open in desktop

Large diffs are not rendered by default.

4 changes: 4 additions & 0 deletions nf_core/pipeline-template/{{cookiecutter.name_noslash}}/docs/usage.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
* [`--outdir`](#--outdir)
* [`--email`](#--email)
* [`--email_on_fail`](#--email_on_fail)
* [`--max_multiqc_email_size`](#--max_multiqc_email_size)
* [`-name`](#-name)
* [`-resume`](#-resume)
* [`-c`](#-c)
Expand Down Expand Up @@ -215,6 +216,9 @@ Set this parameter to your e-mail address to get a summary e-mail with details o
### `--email_on_fail`
This works exactly as with `--email`, except emails are only sent if the workflow is not successful.

### `--max_multiqc_email_size`
Theshold size for MultiQC report to be attached in notification email. If file generated by pipeline exceeds the threshold, it will not be attached (Default: 25MB).

### `-name`
Name for the pipeline run. If not specified, Nextflow will automatically generate a random mnemonic.

Expand Down
76 changes: 38 additions & 38 deletions nf_core/pipeline-template/{{cookiecutter.name_noslash}}/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ def helpMessage() {
--outdir The output directory where the results will be saved
--email Set this parameter to your e-mail address to get a summary e-mail with details of the run sent to you when the workflow exits
--email_on_fail Same as --email, except only send mail if the workflow is not successful
--maxMultiqcEmailFileSize Theshold size for MultiQC report to be attached in notification email. If file generated by pipeline exceeds the threshold, it will not be attached (Default: 25MB)
--max_multiqc_email_size Theshold size for MultiQC report to be attached in notification email. If file generated by pipeline exceeds the threshold, it will not be attached (Default: 25MB)
-name Name for the pipeline run. If not specified, Nextflow will automatically generate a random mnemonic.

AWSBatch options:
Expand Down Expand Up @@ -75,17 +75,17 @@ if (params.fasta) { ch_fasta = file(params.fasta, checkIfExists: true) }
// this has the bonus effect of catching both -name and --name
custom_runName = params.name
if (!(workflow.runName ==~ /[a-z]+_[a-z]+/)) {
custom_runName = workflow.runName
custom_runName = workflow.runName
}

if ( workflow.profile == 'awsbatch') {
// AWSBatch sanity checking
if (!params.awsqueue || !params.awsregion) exit 1, "Specify correct --awsqueue and --awsregion parameters on AWSBatch!"
// Check outdir paths to be S3 buckets if running on AWSBatch
// related: https://github.com/nextflow-io/nextflow/issues/813
if (!params.outdir.startsWith('s3:')) exit 1, "Outdir not on S3 - specify S3 Bucket to run on AWSBatch!"
// Prevent trace files to be stored on S3 since S3 does not support rolling files.
if (workflow.tracedir.startsWith('s3:')) exit 1, "Specify a local tracedir or run without trace! S3 cannot be used for tracefiles."
// AWSBatch sanity checking
if (!params.awsqueue || !params.awsregion) exit 1, "Specify correct --awsqueue and --awsregion parameters on AWSBatch!"
// Check outdir paths to be S3 buckets if running on AWSBatch
// related: https://github.com/nextflow-io/nextflow/issues/813
if (!params.outdir.startsWith('s3:')) exit 1, "Outdir not on S3 - specify S3 Bucket to run on AWSBatch!"
// Prevent trace files to be stored on S3 since S3 does not support rolling files.
if (workflow.tracedir.startsWith('s3:')) exit 1, "Specify a local tracedir or run without trace! S3 cannot be used for tracefiles."
}

// Stage config files
Expand All @@ -101,19 +101,19 @@ if (params.readPaths) {
.from(params.readPaths)
.map { row -> [ row[0], [ file(row[1][0], checkIfExists: true) ] ] }
.ifEmpty { exit 1, "params.readPaths was empty - no input files supplied" }
.into { read_files_fastqc; read_files_trimming }
.into { ch_read_files_fastqc; ch_read_files_trimming }
} else {
Channel
.from(params.readPaths)
.map { row -> [ row[0], [ file(row[1][0], checkIfExists: true), file(row[1][1], checkIfExists: true) ] ] }
.ifEmpty { exit 1, "params.readPaths was empty - no input files supplied" }
.into { read_files_fastqc; read_files_trimming }
.into { ch_read_files_fastqc; ch_read_files_trimming }
}
} else {
Channel
.fromFilePairs( params.reads, size: params.singleEnd ? 1 : 2 )
.ifEmpty { exit 1, "Cannot find any reads matching: ${params.reads}\nNB: Path needs to be enclosed in quotes!\nIf this is single-end data, please specify --singleEnd on the command line." }
.into { read_files_fastqc; read_files_trimming }
.into { ch_read_files_fastqc; ch_read_files_trimming }
}

// Header log info
Expand All @@ -133,17 +133,17 @@ summary['Working dir'] = workflow.workDir
summary['Script dir'] = workflow.projectDir
summary['User'] = workflow.userName
if (workflow.profile == 'awsbatch') {
summary['AWS Region'] = params.awsregion
summary['AWS Queue'] = params.awsqueue
summary['AWS Region'] = params.awsregion
summary['AWS Queue'] = params.awsqueue
}
summary['Config Profile'] = workflow.profile
if (params.config_profile_description) summary['Config Description'] = params.config_profile_description
if (params.config_profile_contact) summary['Config Contact'] = params.config_profile_contact
if (params.config_profile_url) summary['Config URL'] = params.config_profile_url
if (params.email || params.email_on_fail) {
summary['E-mail Address'] = params.email
summary['E-mail on failure'] = params.email_on_fail
summary['MultiQC maxsize'] = params.maxMultiqcEmailFileSize
summary['E-mail Address'] = params.email
summary['E-mail on failure'] = params.email_on_fail
summary['MultiQC maxsize'] = params.max_multiqc_email_size
}
log.info summary.collect { k,v -> "${k.padRight(18)}: $v" }.join("\n")
log.info "-\033[2m--------------------------------------------------\033[0m-"
Expand Down Expand Up @@ -179,7 +179,7 @@ process get_software_versions {
}

output:
file 'software_versions_mqc.yaml' into software_versions_yaml
file 'software_versions_mqc.yaml' into ch_software_versions_yaml
file "software_versions.csv"

script:
Expand All @@ -203,10 +203,10 @@ process fastqc {
saveAs: { filename -> filename.indexOf(".zip") > 0 ? "zips/$filename" : "$filename" }

input:
set val(name), file(reads) from read_files_fastqc
set val(name), file(reads) from ch_read_files_fastqc

output:
file "*_fastqc.{zip,html}" into fastqc_results
file "*_fastqc.{zip,html}" into ch_fastqc_results

script:
"""
Expand All @@ -223,12 +223,12 @@ process multiqc {
input:
file multiqc_config from ch_multiqc_config
// TODO nf-core: Add in log files from your new processes for MultiQC to find!
file ('fastqc/*') from fastqc_results.collect().ifEmpty([])
file ('software_versions/*') from software_versions_yaml.collect()
file ('fastqc/*') from ch_fastqc_results.collect().ifEmpty([])
file ('software_versions/*') from ch_software_versions_yaml.collect()
file workflow_summary from create_workflow_summary(summary)

output:
file "*multiqc_report.html" into multiqc_report
file "*multiqc_report.html" into ch_multiqc_report
file "*_data"
file "multiqc_plots"

Expand Down Expand Up @@ -267,7 +267,7 @@ workflow.onComplete {
// Set up the e-mail variables
def subject = "[{{ cookiecutter.name }}] Successful: $workflow.runName"
if (!workflow.success) {
subject = "[{{ cookiecutter.name }}] FAILED: $workflow.runName"
subject = "[{{ cookiecutter.name }}] FAILED: $workflow.runName"
}
def email_fields = [:]
email_fields['version'] = workflow.manifest.version
Expand All @@ -293,12 +293,12 @@ workflow.onComplete {
email_fields['summary']['Nextflow Build'] = workflow.nextflow.build
email_fields['summary']['Nextflow Compile Timestamp'] = workflow.nextflow.timestamp

// TODO nf-core: If not using MultiQC, strip out this code (including params.maxMultiqcEmailFileSize)
// TODO nf-core: If not using MultiQC, strip out this code (including params.max_multiqc_email_size)
// On success try attach the multiqc report
def mqc_report = null
try {
if (workflow.success) {
mqc_report = multiqc_report.getVal()
mqc_report = ch_multiqc_report.getVal()
if (mqc_report.getClass() == ArrayList) {
log.warn "[{{ cookiecutter.name }}] Found multiple reports from process 'multiqc', will use only one"
mqc_report = mqc_report[0]
Expand Down Expand Up @@ -326,29 +326,29 @@ workflow.onComplete {
def email_html = html_template.toString()

// Render the sendmail template
def smail_fields = [ email: email_address, subject: subject, email_txt: email_txt, email_html: email_html, baseDir: "$baseDir", mqcFile: mqc_report, mqcMaxSize: params.maxMultiqcEmailFileSize.toBytes() ]
def smail_fields = [ email: email_address, subject: subject, email_txt: email_txt, email_html: email_html, baseDir: "$baseDir", mqcFile: mqc_report, mqcMaxSize: params.max_multiqc_email_size.toBytes() ]
def sf = new File("$baseDir/assets/sendmail_template.txt")
def sendmail_template = engine.createTemplate(sf).make(smail_fields)
def sendmail_html = sendmail_template.toString()

// Send the HTML e-mail
if (email_address) {
try {
if ( params.plaintext_email ){ throw GroovyException('Send plaintext e-mail, not HTML') }
// Try to send HTML e-mail using sendmail
[ 'sendmail', '-t' ].execute() << sendmail_html
log.info "[{{ cookiecutter.name }}] Sent summary e-mail to $email_address (sendmail)"
if ( params.plaintext_email ){ throw GroovyException('Send plaintext e-mail, not HTML') }
// Try to send HTML e-mail using sendmail
[ 'sendmail', '-t' ].execute() << sendmail_html
log.info "[{{ cookiecutter.name }}] Sent summary e-mail to $email_address (sendmail)"
} catch (all) {
// Catch failures and try with plaintext
[ 'mail', '-s', subject, email_address ].execute() << email_txt
log.info "[{{ cookiecutter.name }}] Sent summary e-mail to $email_address (mail)"
// Catch failures and try with plaintext
[ 'mail', '-s', subject, email_address ].execute() << email_txt
log.info "[{{ cookiecutter.name }}] Sent summary e-mail to $email_address (mail)"
}
}

// Write summary e-mail HTML to a file
def output_d = new File( "${params.outdir}/pipeline_info/" )
if (!output_d.exists()) {
output_d.mkdirs()
output_d.mkdirs()
}
def output_hf = new File( output_d, "pipeline_report.html" )
output_hf.withWriter { w -> w << email_html }
Expand All @@ -361,9 +361,9 @@ workflow.onComplete {
c_red = params.monochrome_logs ? '' : "\033[0;31m";

if (workflow.stats.ignoredCount > 0 && workflow.success) {
log.info "${c_purple}Warning, pipeline completed, but with errored process(es) ${c_reset}"
log.info "${c_red}Number of ignored errored process(es) : ${workflow.stats.ignoredCount} ${c_reset}"
log.info "${c_green}Number of successfully ran process(es) : ${workflow.stats.succeedCount} ${c_reset}"
log.info "${c_purple}Warning, pipeline completed, but with errored process(es) ${c_reset}"
log.info "${c_red}Number of ignored errored process(es) : ${workflow.stats.ignoredCount} ${c_reset}"
log.info "${c_green}Number of successfully ran process(es) : ${workflow.stats.succeedCount} ${c_reset}"
}

if (workflow.success) {
Expand Down
2 changes: 1 addition & 1 deletion nf_core/pipeline-template/{{cookiecutter.name_noslash}}/nextflow.config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ params {
multiqc_config = "$baseDir/assets/multiqc_config.yaml"
email = false
email_on_fail = false
maxMultiqcEmailFileSize = 25.MB
max_multiqc_email_size = 25.MB
plaintext_email = false
monochrome_logs = false
help = false
Expand Down
点击 这是indexloc提供的php浏览器服务，不要输入任何密码和下载