Skip to content

Introduction

fw-curation is a Flywheel-maintained library which simplifies the process of running curation tasks on the Flywheel hierarchy. For data managers and developers who need to take bulk curation actions on their projects, fw-curation provides useful classes and methods designed to cut down on repeated, boiler-plate code, traverse the Flywheel hierarchy in a configurable manner, handle multiprocessing to speed up task completion, and cleanly aggregate reports during processing.

fw-curation is built to assist with higher-level processes than the Flywheel API or the flywheel-sdk, which is a lightweight Python wrapper around the Flywheel API and is an optional dependency of fw-curation. fw-curation is focused specifically on data curation, whereas flywheel-sdk provides more generalized methods for interacting with Flywheel.

fw-curation can be utilized independently of gear runtime and context, as with a standalone Python script, or can be utilized within a gear. For gear development, fw-curation can be paired with fw-gear, which is a Flywheel-maintained Python package designed to simplify the development of Flywheel gears.

Note

By curation, we mean the process of selecting, inspecting, organizing, and reporting on containers and files within the Flywheel Hierarchy with user-defined actions.

This library provides three main modules that make curation easier:

  • walker: A generalized, configurable, and parallelizable "walker" which traverses the Flywheel hierarchy
  • curator: An abstract class which exposes methods for the user to implement that handle (perform) curation on a given container or file
    • HierarchyCurator: A subclass of Curator which exposes methods for each level of the hierarchy, meant to be used in combination with the Walker
    • FileCurator: A subclass of Curator which exposes only one method to handle File objects in Flywheel.
  • reporters: Reporter objects which can be attached to a curator to allow for easy reporting on visited containers during curation.

License

fw-curation is developed under an MIT-based license.

Installation

The package can be installed using pip or poetry using python 3.10 or later.

  pip install fw-curation
  # or
  poetry add fw-curation

Optional Dependencies

fw-curation can optionally install the flywheel-sdk, but will also work without the flywheel-sdk and will work with flywheel-sdk installed seperately >= 21.0.0.

To install the sdk with fw-curation, install the sdk extra with either pip or poetry:

  pip install 'fw-curation[sdk]'
  # or
  poetry add fw-curation -E sdk

Example

A very minimal example which simply logs the name of each acquisition it visits:

from fw_curation.curator import HierarchyCurator
from fw_curation.walker import Walker
import flywheel
import logging

log = logging.getLogger(__name__)


class Curator(HierarchyCurator):
    def curate_acquisition(acq):
        log.info(acq.label)


if __name__ == '__main__':
    fw = flywheel.Client()
    project = fw.lookup('example/test')
    walker = Walker.from_container(project)
    curator = Curator()
    curator.curate(walker)