Title: Getting and Reloading Containers
Date: 18-10-2020
Description:
# Install specific packages required for this notebook
!pip install flywheel-sdk
# Import packages
from getpass import getpass
import logging
import os
import flywheel
from permission import check_user_permission
# Instantiate a logger
logging.basicConfig(level=logging.INFO, format='%(asctime)s %(levelname)s %(message)s')
log = logging.getLogger('root')
Get a API_KEY. More on this in the Flywheel SDK doc here.
API_KEY = getpass('Enter API_KEY here: ')
Instantiate the Flywheel API client
fw = flywheel.Client(API_KEY if 'API_KEY' in locals() else os.environ.get('FW_KEY'))
Show Flywheel logging information
log.info('You are now logged in as %s to %s', fw.get_current_user()['email'], fw.get_config()['site']['api_url'])
If you know what kind of container you want, using fw.get_<container>() will be the most efficient/proper way to do so.
Here, we are trying to get a specific project container by usingfw.get_project() method which takes in project_id as an argument.
anxiety_proj = fw.get_project(project_id='<-enter-project-id-here->')
anxiety_proj
Example Output:
{'analyses': [],
'created': datetime.datetime(2020, 5, 5, 1, 22, 6, 229000, tzinfo=tzutc()),
'description': None,
'editions': {'lab': False},
'files': [],
'group': 'scien',
'id': '1111111111111111111',
'info': {'BIDS': {'Acknowledgements': '',
'Authors': [],
'BIDSVersion': '1.0.2',
'DatasetDOI': '',
'Funding': '',
'HowToAcknowledge': '',
'License': '',
'Name': 'AnxietyStudy01',
'ReferencesAndLinks': [],
'template': 'project'}},
'info_exists': None,
'label': 'AnxietyStudy01',
'modified': datetime.datetime(2020, 10, 22, 0, 13, 21, 622000, tzinfo=tzutc()),
'notes': [{'created': datetime.datetime(2020, 10, 22, 0, 13, 21, 622000, tzinfo=tzutc()),
'id': '1111111111111111111',
'modified': datetime.datetime(2020, 10, 22, 0, 13, 21, 622000, tzinfo=tzutc()),
'text': 'This is a tutorial project',
'user': 'example02@flywheel.io'}],
'parents': {'acquisition': None,
'analysis': None,
'group': 'scien',
'project': None,
'session': None,
'subject': None},
'permissions': [{'access': None,
'id': 'example01@flywheel.io',
'role_ids': ['1111111111111111111']},
{'access': None,
'id': 'example02@flywheel.io',
'role_ids': ['1111111111111111111']}],
'providers': {'compute': None, 'storage': None},
'revision': 6,
'tags': [],
'templates': None}
Another way to get a container is to use fw.get().
unknown_container = fw.get(id='<-enter-project-id-here->')
unknown_container
Example Output:
{'analyses': [],
'created': datetime.datetime(2020, 5, 5, 1, 22, 6, 229000, tzinfo=tzutc()),
'description': None,
'editions': {'lab': False},
'files': [],
'group': 'scien',
'id': '1111111111111111111',
'info': {'BIDS': {'Acknowledgements': '',
'Authors': [],
'BIDSVersion': '1.0.2',
'DatasetDOI': '',
'Funding': '',
'HowToAcknowledge': '',
'License': '',
'Name': 'AnxietyStudy01',
'ReferencesAndLinks': [],
'template': 'project'}},
'info_exists': None,
'label': 'AnxietyStudy01',
'modified': datetime.datetime(2020, 10, 22, 0, 13, 21, 622000, tzinfo=tzutc()),
'notes': [{'created': datetime.datetime(2020, 10, 22, 0, 13, 21, 622000, tzinfo=tzutc()),
'id': '1111111111111111111',
'modified': datetime.datetime(2020, 10, 22, 0, 13, 21, 622000, tzinfo=tzutc()),
'text': 'This is a tutorial projects',
'user': 'example02@flywheel.io'}],
'parents': {'acquisition': None,
'analysis': None,
'group': 'scien',
'project': None,
'session': None,
'subject': None},
'permissions': [{'access': None,
'id': 'example01@flywheel.io',
'role_ids': ['1111111111111111111']},
{'access': None,
'id': 'example02@flywheel.io',
'role_ids': ['1111111111111111111']}],
'providers': {'compute': None, 'storage': None},
'revision': 6,
'tags': [],
'templates': None}
Yes, you can use fw.lookup() to get the container by the label name instead of id.
Here, we will demostrate how to get the Project container by using fw.lookup() method. This method requires path arguement. In our case, it will be group-name/project-name.
project = fw.lookup('<group-name>/<project-name>')
project
Example Output:
{'analyses': [],
'created': datetime.datetime(2020, 5, 5, 1, 22, 6, 229000, tzinfo=tzutc()),
'description': None,
'editions': {'lab': False},
'files': [],
'group': 'scien',
'id': '1111111111111111111',
'info': {'BIDS': {'Acknowledgements': '',
'Authors': [],
'BIDSVersion': '1.0.2',
'DatasetDOI': '',
'Funding': '',
'HowToAcknowledge': '',
'License': '',
'Name': 'AnxietyStudy01',
'ReferencesAndLinks': [],
'template': 'project'}},
'info_exists': None,
'label': 'AnxietyStudy01',
'modified': datetime.datetime(2020, 10, 22, 0, 13, 21, 622000, tzinfo=tzutc()),
'notes': [{'created': datetime.datetime(2020, 10, 22, 0, 13, 21, 622000, tzinfo=tzutc()),
'id': '1111111111111111111',
'modified': datetime.datetime(2020, 10, 22, 0, 13, 21, 622000, tzinfo=tzutc()),
'text': 'This is a tutorial projects',
'user': 'huiqiantan@flywheel.io'}],
'parents': {'acquisition': None,
'analysis': None,
'group': 'scien',
'project': None,
'session': None,
'subject': None},
'permissions': [{'access': None,
'id': 'example01@flywheel.io',
'role_ids': ['1111111111111111111']},
{'access': None,
'id': 'example02@flywheel.io',
'role_ids': ['1111111111111111111']}],
'providers': {'compute': None, 'storage': None},
'revision': 6,
'tags': [],
'templates': None}
You can also get the Subject container with fw.lookup() method. In the example below, we are getting the Subject with a label name, anx_s1.
subj_01_lookup = fw.lookup('<group-name>/<project-name>/<subj-label>')
subj_01_lookup
Example Output:
{'age': None,
'analyses': [],
'code': 'anx_s1',
'cohort': 'Control',
'created': datetime.datetime(2020, 5, 5, 20, 10, 49, 813000, tzinfo=tzutc()),
'ethnicity': 'Unknown or Not Reported',
'files': [],
'firstname': 'John',
'id': '1111111111111111111',
'info': {'a_complicated_nested_dict': {'key1': [1, 2, 3, 4],
'key2': [{'an': 'other',
'list': 'with'},
{'dictionaries': ['in',
'it']}]}},
'info_exists': None,
'label': 'anx_s1',
'lastname': 'Doe',
'master_code': None,
'modified': datetime.datetime(2020, 10, 22, 0, 12, 4, 83000, tzinfo=tzutc()),
'notes': [],
'parents': {'acquisition': None,
'analysis': None,
'group': 'test',
'project': '1111111111111111111',
'session': None,
'subject': None},
'permissions': [{'access': None,
'id': 'example01@flywheel.io',
'role_ids': ['1111111111111111111']},
{'access': None,
'id': 'example02@flywheel.io',
'role_ids': ['1111111111111111111']}],
'project': '1111111111111111111',
'race': 'More Than One Race',
'revision': 45,
'sex': 'male',
'species': None,
'strain': None,
'tags': ['tutorial', 'blue'],
'type': 'human'}
You can also use the finder module to get the container by using the label name.
In the example below, we will be using find_first() method to get the Subject container. This method takes in one argument for this instance, it is the label of the container.
subj_01_finder = fw.subjects.find_first('label=<subject-label>')
subj_01_finder
Example Output:
{'age': None,
'analyses': None,
'code': 'anx_s1',
'cohort': 'Control',
'created': datetime.datetime(2020, 5, 5, 20, 10, 49, 813000, tzinfo=tzutc()),
'ethnicity': None,
'files': [],
'firstname': None,
'id': '1111111111111111111',
'info': {},
'info_exists': True,
'label': 'anx_s1',
'lastname': None,
'master_code': None,
'modified': datetime.datetime(2020, 10, 22, 0, 12, 4, 83000, tzinfo=tzutc()),
'notes': [],
'parents': {'acquisition': None,
'analysis': None,
'group': 'test',
'project': '1111111111111111111',
'session': None,
'subject': None},
'permissions': [{'access': None,
'id': 'example01@flywheel.io',
'role_ids': ['1111111111111111111']},
{'access': None,
'id': 'example02@flywheel.io',
'role_ids': ['1111111111111111111']}],
'project': '1111111111111111111',
'race': None,
'revision': 45,
'sex': None,
'species': None,
'strain': None,
'tags': ['tutorial', 'blue'],
'type': 'human'}
fw.lookup() method and fw.get_<container>() method?¶fw.get_<container>() method knows that it is returning a specific container while the fw.lookup() method only know that it is returning a container. With that being said, fw.lookup() method will only return a generic/common container details but not nessecarily all the unique per layer attributes.
One thing you might notice from the output above(subj_01_finder) is that, the info attribute return as an empty list while the subj_01_lookup's info attribute contains some value. This is because different method will return some generic/common metadata but not all due to performance reasons. So, in order to load the complete version of metadata in the container you will need to use reload() method.
reload()?¶A <container>.reload() method can be used in two situation.
fw.get() or fw.get_<container>() did not return.reload() method?¶Yes. This is because some API endpoints contain a very large amount of information. So for performance reasons, different method will return a lot of (but not all) the same attributes.
Therefore, reload() is helpful to retrieve a consistent view of the data within a container.
reload()?¶One way to identify when you should use reload() is when <container>.info_exists() is True. You should also use it when you are expecting the Container to return a specific attributes.
We will be comparing the Subject container for subject anx_s1 with and without reload().
First, we will get the Subject container by using find_first() method.
Let's see what is being retrieve from the Subject Container.
Now, let's see what happend when we use <container>.reload() method.
As you can see from the two screenshots above, some of the attributes were being returned as None or an empty list/dictionary. For the subject container without reload() method, the info_exists attribute return as True whereas with reload() method, the info_exists attribute return as None.
It will be always helpful to keep reload() method handy, so that you can retrieve the complete version of the metadata in the container.