Title: Getting and Reloading Containers
Date: 18-10-2020
Description:

Requirements

Install and import dependencies

Flywheel API Key and Client

Get a API_KEY. More on this in the Flywheel SDK doc here.

Instantiate the Flywheel API client

Show Flywheel logging information


Get a specific type of container

If you know what kind of container you want, using fw.get_<container>() will be the most efficient/proper way to do so.

Here, we are trying to get a specific project container by usingfw.get_project() method which takes in project_id as an argument.

Example Output:

{'analyses': [],
 'created': datetime.datetime(2020, 5, 5, 1, 22, 6, 229000, tzinfo=tzutc()),
 'description': None,
 'editions': {'lab': False},
 'files': [],
 'group': 'scien',
 'id': '1111111111111111111',
 'info': {'BIDS': {'Acknowledgements': '',
                   'Authors': [],
                   'BIDSVersion': '1.0.2',
                   'DatasetDOI': '',
                   'Funding': '',
                   'HowToAcknowledge': '',
                   'License': '',
                   'Name': 'AnxietyStudy01',
                   'ReferencesAndLinks': [],
                   'template': 'project'}},
 'info_exists': None,
 'label': 'AnxietyStudy01',
 'modified': datetime.datetime(2020, 10, 22, 0, 13, 21, 622000, tzinfo=tzutc()),
 'notes': [{'created': datetime.datetime(2020, 10, 22, 0, 13, 21, 622000, tzinfo=tzutc()),
            'id': '1111111111111111111',
            'modified': datetime.datetime(2020, 10, 22, 0, 13, 21, 622000, tzinfo=tzutc()),
            'text': 'This is a tutorial project',
            'user': 'example02@flywheel.io'}],
 'parents': {'acquisition': None,
             'analysis': None,
             'group': 'scien',
             'project': None,
             'session': None,
             'subject': None},
 'permissions': [{'access': None,
                  'id': 'example01@flywheel.io',
                  'role_ids': ['1111111111111111111']},
                 {'access': None,
                  'id': 'example02@flywheel.io',
                  'role_ids': ['1111111111111111111']}],
 'providers': {'compute': None, 'storage': None},
 'revision': 6,
 'tags': [],
 'templates': None}

What if unsure the type of the container?

Another way to get a container is to use fw.get().

Warning: This method is the least efficient way to get a container, and we recommend only use this method only if the type of id is unknown.

Example Output:

{'analyses': [],
 'created': datetime.datetime(2020, 5, 5, 1, 22, 6, 229000, tzinfo=tzutc()),
 'description': None,
 'editions': {'lab': False},
 'files': [],
 'group': 'scien',
 'id': '1111111111111111111',
 'info': {'BIDS': {'Acknowledgements': '',
                   'Authors': [],
                   'BIDSVersion': '1.0.2',
                   'DatasetDOI': '',
                   'Funding': '',
                   'HowToAcknowledge': '',
                   'License': '',
                   'Name': 'AnxietyStudy01',
                   'ReferencesAndLinks': [],
                   'template': 'project'}},
 'info_exists': None,
 'label': 'AnxietyStudy01',
 'modified': datetime.datetime(2020, 10, 22, 0, 13, 21, 622000, tzinfo=tzutc()),
 'notes': [{'created': datetime.datetime(2020, 10, 22, 0, 13, 21, 622000, tzinfo=tzutc()),
            'id': '1111111111111111111',
            'modified': datetime.datetime(2020, 10, 22, 0, 13, 21, 622000, tzinfo=tzutc()),
            'text': 'This is a tutorial projects',
            'user': 'example02@flywheel.io'}],
 'parents': {'acquisition': None,
             'analysis': None,
             'group': 'scien',
             'project': None,
             'session': None,
             'subject': None},
 'permissions': [{'access': None,
                  'id': 'example01@flywheel.io',
                  'role_ids': ['1111111111111111111']},
                 {'access': None,
                  'id': 'example02@flywheel.io',
                  'role_ids': ['1111111111111111111']}],
 'providers': {'compute': None, 'storage': None},
 'revision': 6,
 'tags': [],
 'templates': None}

Can I get a container with the label name?

Yes, you can use fw.lookup() to get the container by the label name instead of id.

Here, we will demostrate how to get the Project container by using fw.lookup() method. This method requires path arguement. In our case, it will be group-name/project-name.

Example Output:

{'analyses': [],
 'created': datetime.datetime(2020, 5, 5, 1, 22, 6, 229000, tzinfo=tzutc()),
 'description': None,
 'editions': {'lab': False},
 'files': [],
 'group': 'scien',
 'id': '1111111111111111111',
 'info': {'BIDS': {'Acknowledgements': '',
                   'Authors': [],
                   'BIDSVersion': '1.0.2',
                   'DatasetDOI': '',
                   'Funding': '',
                   'HowToAcknowledge': '',
                   'License': '',
                   'Name': 'AnxietyStudy01',
                   'ReferencesAndLinks': [],
                   'template': 'project'}},
 'info_exists': None,
 'label': 'AnxietyStudy01',
 'modified': datetime.datetime(2020, 10, 22, 0, 13, 21, 622000, tzinfo=tzutc()),
 'notes': [{'created': datetime.datetime(2020, 10, 22, 0, 13, 21, 622000, tzinfo=tzutc()),
            'id': '1111111111111111111',
            'modified': datetime.datetime(2020, 10, 22, 0, 13, 21, 622000, tzinfo=tzutc()),
            'text': 'This is a tutorial projects',
            'user': 'huiqiantan@flywheel.io'}],
 'parents': {'acquisition': None,
             'analysis': None,
             'group': 'scien',
             'project': None,
             'session': None,
             'subject': None},
 'permissions': [{'access': None,
                  'id': 'example01@flywheel.io',
                  'role_ids': ['1111111111111111111']},
                 {'access': None,
                  'id': 'example02@flywheel.io',
                  'role_ids': ['1111111111111111111']}],
 'providers': {'compute': None, 'storage': None},
 'revision': 6,
 'tags': [],
 'templates': None}

You can also get the Subject container with fw.lookup() method. In the example below, we are getting the Subject with a label name, anx_s1.

Example Output:

{'age': None,
 'analyses': [],
 'code': 'anx_s1',
 'cohort': 'Control',
 'created': datetime.datetime(2020, 5, 5, 20, 10, 49, 813000, tzinfo=tzutc()),
 'ethnicity': 'Unknown or Not Reported',
 'files': [],
 'firstname': 'John',
 'id': '1111111111111111111',
 'info': {'a_complicated_nested_dict': {'key1': [1, 2, 3, 4],
                                        'key2': [{'an': 'other',
                                                  'list': 'with'},
                                                 {'dictionaries': ['in',
                                                                   'it']}]}},
 'info_exists': None,
 'label': 'anx_s1',
 'lastname': 'Doe',
 'master_code': None,
 'modified': datetime.datetime(2020, 10, 22, 0, 12, 4, 83000, tzinfo=tzutc()),
 'notes': [],
 'parents': {'acquisition': None,
             'analysis': None,
             'group': 'test',
             'project': '1111111111111111111',
             'session': None,
             'subject': None},
 'permissions': [{'access': None,
                  'id': 'example01@flywheel.io',
                  'role_ids': ['1111111111111111111']},
                 {'access': None,
                  'id': 'example02@flywheel.io',
                  'role_ids': ['1111111111111111111']}],
 'project': '1111111111111111111',
 'race': 'More Than One Race',
 'revision': 45,
 'sex': 'male',
 'species': None,
 'strain': None,
 'tags': ['tutorial', 'blue'],
 'type': 'human'}

You can also use the finder module to get the container by using the label name.

In the example below, we will be using find_first() method to get the Subject container. This method takes in one argument for this instance, it is the label of the container.

Example Output:

{'age': None,
 'analyses': None,
 'code': 'anx_s1',
 'cohort': 'Control',
 'created': datetime.datetime(2020, 5, 5, 20, 10, 49, 813000, tzinfo=tzutc()),
 'ethnicity': None,
 'files': [],
 'firstname': None,
 'id': '1111111111111111111',
 'info': {},
 'info_exists': True,
 'label': 'anx_s1',
 'lastname': None,
 'master_code': None,
 'modified': datetime.datetime(2020, 10, 22, 0, 12, 4, 83000, tzinfo=tzutc()),
 'notes': [],
 'parents': {'acquisition': None,
             'analysis': None,
             'group': 'test',
             'project': '1111111111111111111',
             'session': None,
             'subject': None},
 'permissions': [{'access': None,
                  'id': 'example01@flywheel.io',
                  'role_ids': ['1111111111111111111']},
                 {'access': None,
                  'id': 'example02@flywheel.io',
                  'role_ids': ['1111111111111111111']}],
 'project': '1111111111111111111',
 'race': None,
 'revision': 45,
 'sex': None,
 'species': None,
 'strain': None,
 'tags': ['tutorial', 'blue'],
 'type': 'human'}

What is the difference between fw.lookup() method and fw.get_<container>() method?


Reloading a Container

One thing you might notice from the output above(subj_01_finder) is that, the info attribute return as an empty list while the subj_01_lookup's info attribute contains some value. This is because different method will return some generic/common metadata but not all due to performance reasons. So, in order to load the complete version of metadata in the container you will need to use reload() method.

What is reload()?

A <container>.reload() method can be used in two situation.

Is it nessecary to use reload() method?

Yes. This is because some API endpoints contain a very large amount of information. So for performance reasons, different method will return a lot of (but not all) the same attributes. Therefore, reload() is helpful to retrieve a consistent view of the data within a container.

When to use reload()?

One way to identify when you should use reload() is when <container>.info_exists() is True. You should also use it when you are expecting the Container to return a specific attributes.

Common Attributes: analyses, file, info, container's metadata (eg:ethnicity, firstname, lastname, race, sex)

Let's see some example

We will be comparing the Subject container for subject anx_s1 with and without reload().

First, we will get the Subject container by using find_first() method.

Let's see what is being retrieve from the Subject Container.

Now, let's see what happend when we use <container>.reload() method.

As you can see from the two screenshots above, some of the attributes were being returned as None or an empty list/dictionary. For the subject container without reload() method, the info_exists attribute return as True whereas with reload() method, the info_exists attribute return as None.

Key Takeaway

It will be always helpful to keep reload() method handy, so that you can retrieve the complete version of the metadata in the container.