gear

Usage

Gear option commands

Enable/Disable

Use these options by passing a gear name, gear name and version or gear id. This will enable or disable the matching gear and if multiple are matched will render a table from which you can choose the gears to enable/disable.

Even if a gear is disabled, its unique name and version combination are reserved behind the scenes. Alter the version to upload any changes, as any new gear is prevented from using an existing name and version combination.

NOTE: Enable/Disable don't search for gears like ls, they will only match a specific gear name (+version) or id.

Version

This option can be used as a flag and will simply print out the version from the gear in the current directory as found in the manifest. You can also use this option with one argument and it will update the gear in the current directory to the specified version.

Validate Manifest

Use this option as a flag to validate the manifest in the current directory. This can be helpful in debugging failed gear uploads.

Common gear workflows

Create a gear

This tutorial walks through creating a very simple gear using fw-beta

Setup

Say you have a script that takes a DICOM archive, modifies one tag across the archive, and re-saves that DICOM.

That script could look like this in python using fw-file.

modify_dicom.py:

import sys
import zipfile
from fw_file.dicom import DICOMCollection
from pathlib import Path

def main(dicom_path):
    if zipfile.is_zipfile(dicom_path):
        # Load dicoms
        dcms = DICOMCollection.from_zip(dicom_path)
        # Modify PatientName
        dcms.set('PatientName', 'Anonymized')
        # Overwrite existing archive
        dcms.to_zip(dicom_path)
        print('success')
    else:
        # Single dicom
        dcm = DICOM(dicom_path)
        # Modify PatientName
        dcm.PatientName = 'Anonymized'
        # Overwrite existing DICOM
        dcm.save(dicom_path)
        print('success')


if __name__ == '__main__':
    if len(sys.argv) < 2:
        print("Usage: modify_dicom.py [path]")
        sys.exit(1)
    path = Path(sys.argv[1:])
    if not (path.exists() and path.is_file()):
        print("Usage: modify_dicom.py [path]")
        sys.exit(1)

    main(path)

This is a simple script that just replaces the PatientName attribute of the DICOM(s) to be Anonymized.

Initializing the gear template with gear create

In our directory where we want to create the gear, we can simply run gear create and enter the prompts we want. NOTE: I can also pass in anything I know already as a flag, such as author below.

This created a directory with the following structure:

.
├── Dockerfile
├── manifest.json
├── requirements.txt
└── run.py

I can build this directly, but first I want to modify my script to work as a gear.

Modifying run.py

I want to get my modify_dicom.py script to work as a gear. Since modify_dicom.py is pretty small, I'll just add it to the run.py script. In order to that, I'll need to do a few things:

• The only thing I really care about in this script is the main function which actually modifies the DICOM. The rest is all set upSo I'll change this function name to modify_dicom and add it to the run.py file
• I'll also copy over my imports from modify_dicom.py, namely fw_file and zipfile
• And finally, I'll modify the main function to pass my input file into modify_dicom

These leave run.py looking like this:

#!/usr/bin/env python
from fw_gear import GearContext
# See docs at https://flywheel-io.gitlab.io/scientific-solutions/lib/fw-gear/

import zipfile
from fw_file.dicom import DICOMCollection
from pathlib import Path

def modify_dicom(dicom_path):
    if zipfile.is_zipfile(dicom_path):
        # Load dicoms
        dcms = DICOMCollection.from_zip(dicom_path)
        # Modify PatientName
        dcms.set('PatientName', 'Anonymized')
        # Overwrite existing archive
        dcms.to_zip(dicom_path)
        print('success')
    else:
        # Single dicom
        dcm = DICOM(dicom_path)
        # Modify PatientName
        dcm.PatientName = 'Anonymized'
        # Overwrite existing dicom
        dcm.save(dicom_path)
        print('success')


def main(context):
    # Get input defined in manifest
    input_file = context.get_input_path("input-file")
    modify_dicom(input_file)

if __name__ == '__main__':
    # Initialize Gear Context
    with GearContext() as context:
        context.init_logging()
        main(context)

Building the gear

The one other thing I'll need to do is add fw-file to my requirements.txt so that gets installed in the docker container.

$ cat -p requirements.txt
flywheel-gear-toolkit
fw-file

Now that I've updated run.py, and since I don't need to update the manifest (the template has one input and my gear requires one input), I'm ready to build the gear, which I can do with gear build:

Running the gear locally

Now that I've built my gear, I want to run it locally to debug and test it.

Before running a gear locally, I need to create and populate a config.json file. This file tells the gear where input files are located and stores configuration options. I first create an new config.json file.

fw-beta gear config --new

I next populate my config.json with any input files (if any). My example gear takes in one input file named input-file. On my local machine, the test file I want to use is located at /Users/user/Desktop/test-dicom.dcm.

fw-beta gear config -i input-file="/Users/user/Desktop/test-dicom.dcm"

After the config.json is populated with any file inputs and/or config options I need to setup the proper directory structure. To do this, I run the following command in the gear directory.

The output of this command tells me where the gear directory was created on my local machine. I'll copy this directory path to my clipboard and am now ready to run my gear.

fw-beta gear run /var/folders/by/b710ry_x51vcbjxfs1zf9fjw0000gp/T/gear/dicom-modifier_0.1.0

Uploading the gear

Finally, now that the gear is built, I can upload it to my site:

Debugging a failed gear run

This is a really common scenario in gear development. You've released a version of the gear that works, but in practice runs into an issue you didn't expect, or you've released a version with a bug that you later notice.

The problem

I have a gear dicom-qc which runs various quality control rules on a DICOM file and reports the results in the file custom information under the qc key. However, I noticed that running this gear was removing the modality from the file it ran on.

This particular gear updates file information using the Output Metadata method. I have this gear enabled to print the output metadata before finishing for debugging purposes. For this specific issue, I looked at the job log and saw that it was indeed setting the file Modality to null:

Pull the job configuration

In order to debug this, I want to step through the code with a debugger like PDB, the VSCode debugger, or the Pycharm debugger.

First, I'll pull the job configuration from the site with job pull

This creates a folder /tmp/dicom-qc-0.4.1-rc.1-6217b1cd9d3a8b718d837bf4 which contains all I need to run the gear.

$ ls --tree /tmp/dicom-qc-0.4.1-rc.1-6217b1cd9d3a8b718d837bf4
/tmp/dicom-qc-0.4.1-rc.1-6217b1cd9d3a8b718d837bf4
├── config.json
├── input
│  ├── dicom
│  │  └── T1_AX_SE.dcm.zip
│  └── validation-schema
│     └── empty-validation.json
├── manifest.json
├── output
├── run.sh
└── work

Run the job locally

From this directory, I can then use gear run, but first I'll need to add an API key, since Flywheel redacts the API key in the job configuration for security reasons. I can do this with gear config

I can verify this was added by looking at the config.json:

$ cat config.json | jq '.inputs."api-key"'
{
  "base": "api-key",
  "key": "ga.ce.flywheel.io:<redacted>"
}

Now I can try running the gear:

Debugging

From the above output I can see I had the same issue locally, which means I can debug.

I'll run the gear interactively and set a breakpoint where I expect the issue could be:

$ fw-beta gear run -i --entrypoint=/bin/bash
...
root@0494044dc388:/flywheel/v0# poetry run python -m pdb run.py
Skipping virtualenv creation, as specified in config file.
> /flywheel/v0/run.py(2)<module>()
-> """The run script."""
(Pdb) b fw_gear_dicom_qc/utils.py:22
Breakpoint 1 at /flywheel/v0/fw_gear_dicom_qc/utils.py:22
(Pdb) c
[2271ms   INFO     ]  Log level is INFO
[2272ms   INFO     ]  Checking format of provided schema
[2273ms   INFO     ]  Validating file.info.header
[2273ms   INFO     ]  Did not find 'dicom' or 'dicom_array' in properties, falling back to legacy validation
[2273ms   INFO     ]  Determining rules to run.
[2273ms   INFO     ]  Evaluating qc rules.
[2292ms   INFO     ]  check_zero_byte PASSED
[2300ms   INFO     ]  Found 24 slices in archive
[2408ms   INFO     ]  bed_moving NOT APPLICABLE 'ORIGINAL' Image Type not in all frames, assuming not axial.
[2717ms   INFO     ]  Skipping group_by step.
[2737ms   INFO     ]  Splitting collection series-1101_MR_T1 AX  SE
[3576ms   INFO     ]  embedded_localizer PASSED
[3578ms   INFO     ]  instance_number_uniqueness PASSED
[3584ms   INFO     ]  series_consistency PASSED
[3594ms   INFO     ]  slice_consistency PASSED
[3608ms   WARNING  ]  Error - </FrameOfReferenceUID(0020,0052)> - Missing attribute for Type 1 Required - Module=<FrameOfReference>
Error - </PositionReferenceIndicator(0020,1040)> - Missing attribute for Type 2 Required - Module=<FrameOfReference>
Error - </ImageType(0008,0008)> - A value is required for value 3 in MR Images
Error - </EchoTrainLength(0018,0091)> - Missing attribute for Type 2 Required - Module=<MRImage>
[3912ms   WARNING  ]  dciodvfy FAILED Found 96 errors and 144 warnings across archive.
> /flywheel/v0/fw_gear_dicom_qc/utils.py(22)create_metadata()
-> context.update_file_metadata(
(Pdb) l
 17         file_name = file_["location"]["name"]
 18         existing_info = copy.deepcopy(file_["object"]["info"])
 19
 20         existing_info.update(info)
 21
 22 B->     context.update_file_metadata(
 23             file_name, {"modality": file_.get("modality")}, info=existing_info, tags=tags
 24         )
[EOF]
(Pdb) file_.keys()
dict_keys(['hierarchy', 'object', 'location', 'base'])
(Pdb) file_.get('object').get('modality')
'MR'

Here I can see that I'm updating the modality to file_.get('modality') but modality isn't stored as a top level key of file_ so file_.get('modality') is None. In Flywheel, modality is stored on the object of the file.

Patching the gear

Because of what I found above, I see I can simply update line 23 in fw_gear_dicom_qc/utils.py to update modality to file_.get('object').get('modality').

Once I have this code change in place, I can bump the gear version and re-upload it.

From my development directory:

$ fw-beta gear version
Current version: dicom-qc:0.4.1-rc.1
$ fw-beta gear version 0.4.1-rc.2
Bumping gear version from 0.4.1-rc.1 to 0.4.1-rc.2
$ fw-beta gear upload
....
(ga.ce.flywheel.io/dicom-qc:0.4.1-rc.2) Registering gear on server...
(ga.ce.flywheel.io/dicom-qc:0.4.1-rc.2) Uploaded gear with id 6217b6319d3a8b718d837bf7
You should now see your gear in the Flywheel web interface or find it with `gear list`

Now if I re-run this gear, my issue will be fixed!