NotebookManagementTools.jl

NotebookManagementTools

Module providing tools for generating markdown from Literate.jl-compliant julia scripts, after testing those scripts.

Intended to facilitate the separate execution and generation of notebooks that are part of a larger Documenter.jl documentation deployment. In this scenario, notebook "ground truth" is always a julia script (with Literate.jl-compatible narrative). Furthermore, when used with appropriate continuous integration workflows, notebooks can be generated on a need-to basis, to speed up document generation and the development of new tutorials.

The kind of repository we have in mind will be structured using Julia's "workspace" package management structure. For a stand-alone collection of tutorials following Julia's workspace package pattern, this might look something like this:

├── Project.toml     # <---- root project
├── docs
│   ├── make.jl
│   ├── Project.toml # <--- project for generating documentation wrapping the notebooks
│   └── src
│       └── notebooks
│           ├── tutorial1
│           │   ├── Project.toml  # <--- tutorial-specific project
│           │   ├── runtests.jl
│           │   └── notebook.jl
│           └── tutorial2
│               ├── Project.toml
│               ├── runtests.jl
│               ├── notebook.jl
│               └── notebook.md

Tools

• generate: The main tool, used to generate markdown from annotated julia script.

• notebook_dirs: Used in continuous integration to list notebook directories.

• notebook_dirs_containing: Used in continuous integration to find tutorial directories containing files known to have changed in a pull-request.

Advanced Tools

• set_path_to_literate: For pointing the generator to a version of Literate.jl different from the one in the NotebookManagementTools project.

• path_to_literate: For inspecting the above.

set_path_to_literate(path)

Point TutorialsTools to the location of a directory containing a Project.toml file with a [deps] entry for "Literate". Typically, this is the absolute path to a repository "docs" directory.

julia> pwd()
"/Users/anthony/GoogleDrive/Julia/ClassImbalanceTutorials.jl"

julia> set_path_to_literate("~/GoogleDrive/Julia/ClassImbalanceTutorials.jl/docs/")

Use path_to_literate() to inspect the path once set.

path_to_literate()

Return the path to a Project.toml file with a [deps] entry for "Literate". Use set_path_to_literate to set or change.

generate(notebook_dir, path_to_literate=path_to_literate(); tests=true)

Attempt to generate a new markdown file from the file named "notebook.jl" and contained in notebook_dir, using Literate.jl and the project at notebook_dir, after activating and instantiating the Julia project in notebook_dir. A version of Literate.jl is used consistent with the project specified at path_to_literate (which is pushed to LOAD_PATH to make it available).

Literate.jl executes all code cells and wraps the input cell in regular julia fencing. In that way, when the markdown is included in a Documenter.jl project, no code is re-executed.

Testing

If tests==true, then, before attempting markdown generation, run, in a new Julia process, the code appearing in the file "runtests.jl" appearing in the directory notebook_dir, which should additionally contain the julia script notebook.jl.

Return value

Returns the path to the generated markdown, unless the Julia process exits abnormally, in which case an empty string is returned.

generate(notebook_dirs::AbstractVector, path_to_literate=path_to_literate(); tests=true)

Attempts to generate each notebook with directory in the provided list notebook_dirs. Returns paths to re-generated markdown files, or nothing if there is a failed notebook test or markdown generation failure.

notebook_dirs(root=pwd())

Return all subdirectories of root (possibly nested) that contain a file called "notebook.jl". Directories are returned as full paths.

notebook_dirs_containing([paths]; root=pwd())

Return, as full paths, a list of all subdirectores of root that contain a file called "notebook.jl" and one or more of the files with full paths specified by paths.

shell> pwd
/MLJTutorial/NotebookManagementTools/test/dummy_tutorials

shell> tree
.
├── bad_tutorial
│   ├── notebook.jl
│   ├── Project.toml
│   └── runtests.jl
├── good_tutorial
│   ├── notebook.jl
│   ├── Project.toml
│   └── runtests.jl
├── nested
│   └── good_tutorial
│       ├── notebook.jl
│       ├── Project.toml
│       └── runtests.jl
├── tutorial_without_script
│   ├── Project.toml
│   └── runtests.jl
└── tutorial_without_tests
    ├── notebook.jl
    └── Project.toml

julia> path1 = joinpath(pwd(), "nested", "good_tutorial", "Project.toml")
julia> path2 = joinpath(pwd(), "tutorial_without_scripts", "Project.toml")
julia> notebook_dirs_containing([path1, path2])
1-element Vector{String}:
 "MLJTutorial/NotebookManagementTools/test/dummy)tutorials/nested/good_tutorial"