+
Skip to content

Lint refactoring #809

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 73 commits into from
Dec 14, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
993ef82
Rename test functions to exclude check_ prefix
ewels Dec 5, 2020
eafa914
Start playing around with improving the sphinx docs for linting
ewels Dec 5, 2020
70dd0d4
Fight with sphinx a bit more
ewels Dec 5, 2020
a99b7f4
Make each lint test return its results instead of updating the object
ewels Dec 5, 2020
7a52fef
Don't use numeric test IDs
ewels Dec 5, 2020
15d4f5e
Added ability to optionally load a pipeline lint config file
ewels Dec 5, 2020
3b494ce
Config: skip test if named and is false
ewels Dec 5, 2020
c01e9e4
Rename .nf-core-lint-config.yml to .nf-core-lint.yml
ewels Dec 5, 2020
89df655
Log / report ignored tests
ewels Dec 5, 2020
44ad230
Make module directory for linting
ewels Dec 5, 2020
0f7f023
Break lint functions out into their own files
ewels Dec 5, 2020
0abffb4
Testing and tidying of the newly split linting code
ewels Dec 5, 2020
f9093cb
Add sphinx docs .rst files for all new lint test functions
ewels Dec 5, 2020
558c1d6
More bugfixes as I start working on tests
ewels Dec 5, 2020
0a4319c
Deleted lint examples, started rewriting lint tests
ewels Dec 5, 2020
99ad455
Remove somewhat pointless lint test 'docker'
ewels Dec 5, 2020
abd4544
Spinx docs - move function reference into a subdir called 'API Refere…
ewels Dec 6, 2020
664895e
Lint pytests - comment out old tests
ewels Dec 6, 2020
6046bde
Lint tests - more core tests
ewels Dec 6, 2020
094ca66
Lint tests - more core functions, slight simplification
ewels Dec 6, 2020
3c3debd
Update lint function names in nf-core licences
ewels Dec 6, 2020
02aa6dd
Clean up missing self. references in simplification of test code
ewels Dec 6, 2020
1f0e4cf
Fix old references to lint_pipeline() function
ewels Dec 6, 2020
0eec578
Set the pipeline name in the helper function, not a test
ewels Dec 6, 2020
43cd4ba
Bump versions - run all linting load functions
ewels Dec 6, 2020
9603afb
Try to disentangle nf_core.lint from rest of nf_core package
ewels Dec 6, 2020
fe268d1
Rewrite bump-version to not using linting code.
ewels Dec 6, 2020
dc60c20
Rewrite tests for bump-version
ewels Dec 6, 2020
f833c8e
Make test_download.py use created wf instead of PATH_WORKING_EXAMPLE
ewels Dec 6, 2020
9eed44a
Update lint pytests for refactored code and fix a bug in the process
ewels Dec 6, 2020
851abe5
Move some pytests from lint to utils
ewels Dec 6, 2020
2fd086a
Added / rewrote pytests for first couple of lint tests
ewels Dec 6, 2020
7b5a0b3
Split lint pytest tests into files too
ewels Dec 6, 2020
cef17b7
Wrote helper script to auto-generate sphinx docs .rst files for lint …
ewels Dec 6, 2020
cc43e6a
Added documentation to function docstrings for awstest lint tests
ewels Dec 6, 2020
b4c7177
Added ignoring option for 'files_exist' test
KevinMenden Dec 7, 2020
500fcaf
Lint tests - add test to check that all sphinx .rst files are present…
ewels Dec 7, 2020
527766c
updated changes
KevinMenden Dec 7, 2020
025b268
Port over lint documentation for actions_branch_protection
ewels Dec 7, 2020
25414a7
Added 'files_exist' tests
KevinMenden Dec 7, 2020
440a619
Remove docs from errors.md that has been ported into docstrings
ewels Dec 7, 2020
541a2c0
Added pytests for actions_branch_protection lint test
ewels Dec 7, 2020
37418e8
Auto-black formatting didn't work. Try again..
ewels Dec 7, 2020
7fddb25
actions_ci lint - return ignored message if file not found
ewels Dec 7, 2020
d1bdeb6
Moved docs to docstring for actions_ci test
ewels Dec 7, 2020
d1e4fed
Write pytests for actions_ci lint test
ewels Dec 7, 2020
e459973
actions_ci pytests - more granular
ewels Dec 7, 2020
03ac603
small edit
KevinMenden Dec 8, 2020
aac25e9
added logging of ignored files/configs
KevinMenden Dec 8, 2020
c5c07da
added licence tests
KevinMenden Dec 8, 2020
c5b7489
added branch protection test
KevinMenden Dec 8, 2020
efd82b1
Merge branch 'lint-refactoring' into lint-refactoring
KevinMenden Dec 8, 2020
d6309e7
Apply suggestions from code review
KevinMenden Dec 8, 2020
7228e3d
removed nextflow_config; added passing test (files_exist)
KevinMenden Dec 8, 2020
2986826
Merge pull request #2 from KevinMenden/lint-refactoring
ewels Dec 8, 2020
e668d85
Run black for ewels/nf-core-tools#2
ewels Dec 8, 2020
fe9ab0a
Lint test actions_lint - docs + functionality
ewels Dec 10, 2020
cb14fc7
Add requirements.txt for Sphinx docs generation
ewels Dec 10, 2020
6268eb8
Docs: Small intro for tools docs homepage
ewels Dec 10, 2020
5368414
Ported lint error docs to docstring: conda_dockerfile
ewels Dec 10, 2020
e8c34f0
errors to docstrings - conda_env_yaml
ewels Dec 10, 2020
364a340
Fixup more lenient make_lint_rst.py helper script
ewels Dec 10, 2020
c4d9b59
Errors to docstring - cookiecutter_strings
ewels Dec 10, 2020
bf58a60
Files exist docs - minor tweaks
ewels Dec 11, 2020
daae3dd
Errors to docstring - licence lint test
ewels Dec 11, 2020
d399079
Errors to docstring - pipeline name conventions
ewels Dec 11, 2020
b28eb12
Errors to docstring - pipeline todos
ewels Dec 11, 2020
d2f63cd
Errors to docstring - readme lint test
ewels Dec 11, 2020
117ba8f
Errors to docstring - schema lint
ewels Dec 11, 2020
f9bef03
Errors to docstring - schema params
ewels Dec 11, 2020
b6e64b4
Errors to docstring - version consistency
ewels Dec 11, 2020
1a8b4b4
Errors to docstring - nextflow config
ewels Dec 11, 2020
135fb8b
Remove docs/lint_errors.md
ewels Dec 11, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ We aim to write function docstrings according to the [Google Python style-guide]
You can find this documentation here: [https://nf-co.re/tools-docs/](https://nf-co.re/tools-docs/)

If you would like to test the documentation, you can install Sphinx locally by following Sphinx's [installation instruction](https://www.sphinx-doc.org/en/master/usage/installation.html).
Once done, you can run `make clean` and then `make html` in the root directory of `nf-core tools`.
Once done, you can run `make clean` and then `make html` in the `docs/api` directory of `nf-core tools`.
The HTML will then be generated in `docs/api/_build/html`.

## Tests
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

.coverage
.pytest_cache
docs/api/_build

# Byte-compiled / optimized / DLL files
__pycache__/
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
9 changes: 9 additions & 0 deletions docs/api/_src/api/index.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
API Reference
=============

.. toctree::
:maxdepth: 2
:caption: Tests:
:glob:

*
2 changes: 1 addition & 1 deletion docs/api/_src/launch.rst → docs/api/_src/api/launch.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
nf_core.launch
============
==============

.. automodule:: nf_core.launch
:members:
Expand Down
File renamed without changes.
14 changes: 14 additions & 0 deletions docs/api/_src/api/lint.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
nf_core.lint
============

.. seealso:: See the `Lint Tests <../lint_tests/index.html>`_ docs for information about specific linting functions.

.. automodule:: nf_core.lint
:members: run_linting
:undoc-members:
:show-inheritance:

.. autoclass:: nf_core.lint.PipelineLint
:members: _lint_pipeline
:private-members: _print_results, _get_results_md, _save_json_results, _wrap_quotes, _strip_ansi_codes
:show-inheritance:
File renamed without changes.
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
nf_core.modules
============
===============

.. automodule:: nf_core.modules
:members:
Expand Down
2 changes: 1 addition & 1 deletion docs/api/_src/schema.rst → docs/api/_src/api/schema.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
nf_core.schema
============
==============

.. automodule:: nf_core.schema
:members:
Expand Down
File renamed without changes.
File renamed without changes.
3 changes: 2 additions & 1 deletion docs/api/_src/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,8 @@
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
#
html_theme = "nature"
# html_theme = "nature"
html_theme = "sphinx_rtd_theme"

# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
Expand Down
9 changes: 8 additions & 1 deletion docs/api/_src/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,19 @@ Welcome to nf-core tools API documentation!
===========================================

.. toctree::
:hidden:
:maxdepth: 2
:caption: Contents:
:glob:

*
lint_tests/index.rst
api/index.rst

This documentation is for the ``nf-core/tools`` package.

Primarily, it describes the different `code lint tests <lint_tests/index.html>`_
run by ``nf-core lint`` (typically visited by a developer when their pipeline fails a given
test), and also reference for the ``nf_core`` `Python package API <api/index.html>`_.

Indices and tables
==================
Expand Down
8 changes: 0 additions & 8 deletions docs/api/_src/lint.rst

This file was deleted.

4 changes: 4 additions & 0 deletions docs/api/_src/lint_tests/actions_awsfulltest.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
actions_awsfulltest
===================

.. automethod:: nf_core.lint.PipelineLint.actions_awsfulltest
4 changes: 4 additions & 0 deletions docs/api/_src/lint_tests/actions_awstest.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
actions_awstest
===============

.. automethod:: nf_core.lint.PipelineLint.actions_awstest
4 changes: 4 additions & 0 deletions docs/api/_src/lint_tests/actions_branch_protection.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
actions_branch_protection
=========================

.. automethod:: nf_core.lint.PipelineLint.actions_branch_protection
4 changes: 4 additions & 0 deletions docs/api/_src/lint_tests/actions_ci.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
actions_ci
==========

.. automethod:: nf_core.lint.PipelineLint.actions_ci
4 changes: 4 additions & 0 deletions docs/api/_src/lint_tests/actions_lint.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
actions_lint
============

.. automethod:: nf_core.lint.PipelineLint.actions_lint
4 changes: 4 additions & 0 deletions docs/api/_src/lint_tests/conda_dockerfile.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
conda_dockerfile
================

.. automethod:: nf_core.lint.PipelineLint.conda_dockerfile
6 changes: 6 additions & 0 deletions docs/api/_src/lint_tests/conda_env_yaml.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
conda_env_yaml
==============

.. automethod:: nf_core.lint.PipelineLint.conda_env_yaml
.. automethod:: nf_core.lint.PipelineLint._anaconda_package
.. automethod:: nf_core.lint.PipelineLint._pip_package
4 changes: 4 additions & 0 deletions docs/api/_src/lint_tests/cookiecutter_strings.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
cookiecutter_strings
====================

.. automethod:: nf_core.lint.PipelineLint.cookiecutter_strings
4 changes: 4 additions & 0 deletions docs/api/_src/lint_tests/files_exist.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
files_exist
===========

.. automethod:: nf_core.lint.PipelineLint.files_exist
9 changes: 9 additions & 0 deletions docs/api/_src/lint_tests/index.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
Lint tests
============================================

.. toctree::
:maxdepth: 2
:caption: Tests:
:glob:

*
4 changes: 4 additions & 0 deletions docs/api/_src/lint_tests/licence.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
licence
=======

.. automethod:: nf_core.lint.PipelineLint.licence
4 changes: 4 additions & 0 deletions docs/api/_src/lint_tests/nextflow_config.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
nextflow_config
===============

.. automethod:: nf_core.lint.PipelineLint.nextflow_config
4 changes: 4 additions & 0 deletions docs/api/_src/lint_tests/pipeline_name_conventions.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
pipeline_name_conventions
=========================

.. automethod:: nf_core.lint.PipelineLint.pipeline_name_conventions
4 changes: 4 additions & 0 deletions docs/api/_src/lint_tests/pipeline_todos.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
pipeline_todos
==============

.. automethod:: nf_core.lint.PipelineLint.pipeline_todos
4 changes: 4 additions & 0 deletions docs/api/_src/lint_tests/readme.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
readme
======

.. automethod:: nf_core.lint.PipelineLint.readme
4 changes: 4 additions & 0 deletions docs/api/_src/lint_tests/schema_lint.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
schema_lint
===========

.. automethod:: nf_core.lint.PipelineLint.schema_lint
4 changes: 4 additions & 0 deletions docs/api/_src/lint_tests/schema_params.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
schema_params
=============

.. automethod:: nf_core.lint.PipelineLint.schema_params
4 changes: 4 additions & 0 deletions docs/api/_src/lint_tests/version_consistency.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
version_consistency
===================

.. automethod:: nf_core.lint.PipelineLint.version_consistency
32 changes: 32 additions & 0 deletions docs/api/make_lint_rst.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
#!/usr/bin/env python

import fnmatch
import os
import nf_core.lint

docs_basedir = os.path.join(os.path.dirname(os.path.abspath(__file__)), "_src", "lint_tests")

# Get list of existing .rst files
existing_docs = []
for fn in os.listdir(docs_basedir):
if fnmatch.fnmatch(fn, "*.rst") and not fnmatch.fnmatch(fn, "index.rst"):
existing_docs.append(os.path.join(docs_basedir, fn))

# Make .rst file for each test name
lint_obj = nf_core.lint.PipelineLint("", True)
rst_template = """{0}
{1}

.. automethod:: nf_core.lint.PipelineLint.{0}
"""

for test_name in lint_obj.lint_tests:
fn = os.path.join(docs_basedir, "{}.rst".format(test_name))
if os.path.exists(fn):
existing_docs.remove(fn)
else:
with open(fn, "w") as fh:
fh.write(rst_template.format(test_name, len(test_name) * "="))

for fn in existing_docs:
os.remove(fn)
2 changes: 2 additions & 0 deletions docs/api/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Sphinx>=3.3.1
sphinx_rtd_theme>=0.5.0
Loading
点击 这是indexloc提供的php浏览器服务,不要输入任何密码和下载