-
Notifications
You must be signed in to change notification settings - Fork 206
Fix collectFile cache issue causing software version inconsistencies #3654
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: dev
Are you sure you want to change the base?
Conversation
Add cache: false to collectFile() call for software versions collection to prevent inconsistencies when using Nextflow resume function. When collectFile uses storeDir with caching enabled, it can lead to missing or additional processes in software version reports across multiple pipeline runs with resume. Fixes nf-core#3653 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
This comment was marked as resolved.
This comment was marked as resolved.
@nf-core-bot changelog |
@ewels Thanks for creating the pull request! This fixed the issue for me. |
This comment was marked as resolved.
This comment was marked as resolved.
Yes - It will make any consumers of the Generally there aren't any processes downstream of MultiQC, it's typically used as a final step that summarises the run, but you're right that it is something that we should be intentional about. I think that we used to have the cache disabled for MultiQC anyway, but I can't find that config now so maybe it was dropped. I'll raise this in Slack on the |
It makes it worse.. |
Why not make it a native process? |
I'm assuming this will be greatly improved by usage of topics and workflow output |
Something I just discovered in my own pipeline, but many pipelines make use of |
Summary
cache: false
parameter tocollectFile()
call for software versions collection in pipeline templateProblem
When
collectFile()
usesstoreDir
with caching enabled, it can lead to missing or additional processes listed in software version reports across multiple pipeline runs with resume. This happens because:--skip_gprofiler
) creates new cached versionsSolution
Adding
cache: false
to thecollectFile()
call ensures software versions are always collected fresh and consistent with the actual processes that ran.Test plan
Fixes #3653
🤖 Generated with Claude Code