Workflow package

Submodules

Workflow.Workflow_Func module

Workflow.Workflow_Func.guide_YN(prompt: str, skip=False, default_value='Y')[source]

terminal guidance for choosing yes or no Users can type Y, y, Yes, yes, N, n, No, or no

Parameters:

  • prompt – a string for prompt

  • skip – False or True to skip the setting

  • default_value – a default value when skip is enabled

Returns:

input_YN, ‘Y’ or ‘N’

Yuncong Ma, 10/5/2023

Workflow.Workflow_Func.guide_choice(prompt: str, list_choice: tuple, skip=False, default_value=None)[source]

Terminal guidance for getting a directory Use sequence ratio to find the most similar option in the list_choice to the user input

Parameters:

  • prompt – a string for prompt

  • list_choice – a list of choices

  • skip – False or True to skip the setting

  • default_value – None or value

Returns:

choice

Yuncong Ma, 10/5/2023

Workflow.Workflow_Func.guide_dir(prompt: str)[source]

terminal guidance for getting a directory

Parameters:

prompt – a string for prompt

Returns:

input_dir

Yuncong Ma, 10/5/2023

Workflow.Workflow_Func.guide_file(prompt: str, existed=True, extension=None)[source]

terminal guidance for setting up an existing or a new file

Parameters:

  • prompt – a string for prompt

  • existed – True or False, check the existence of the file if True.

  • extension – None or a str, or a tuple of strings, specifying the file extension

Returns:

input_file

Yuncong Ma, 10/5/2023

Workflow.Workflow_Func.guide_number(prompt: str, data_type='Int', data_range=None, skip=False, default_value=0)[source]

terminal guidance for setting up a value

Parameters:

  • prompt – a string for prompt

  • data_type – ‘Int’ or ‘Float’

  • data_range – None or (1, 2)

  • skip – False or True to skip the setting

  • default_value – a default value when skip is enabled

Returns:

input_value

Yuncong Ma, 10/5/2023

Workflow.Workflow_Func.workflow(dir_pnet_result: str, file_scan: str, dataType='Surface', dataFormat='HCP Surface (*.cifti, *.mat)', file_subject_ID=None, file_subject_folder=None, file_group_ID=None, file_Brain_Template=None, templateFormat='HCP', file_surfL=None, file_surfR=None, file_maskL=None, file_maskR=None, file_mask_vol=None, file_overlayImage=None, maskValue=0, file_surfL_inflated=None, file_surfR_inflated=None, method='SR-NMF', sr_param=[2, 30], K=17, init='random', sampleSize='Automatic', nBS=5, nTPoints=99999, Combine_Scan=False, file_gFN=None, FN_model_parameter=None, Parallel=False, Computation_Mode='CPU_Torch', N_Thread=1, dataPrecision='double', outputFormat='Both', synchronized_view=True, synchronized_colorbar=True)[source]

Run the workflow of pNet, including Data Input, FN Computation, Quality Control and Visualization This function is for running pNet in a single job

Parameters:

  • dir_pnet_result – directory of the pNet result folder

  • dataType – ‘Surface’, ‘Volume’, ‘Surface-Volume’

  • dataFormat – ‘HCP Surface (.cifti, *.mat)’, ‘MGH Surface (.mgh)’, ‘MGZ Surface (.mgz)’, ‘Volume (.nii, .nii.gz, *.mat)’, ‘HCP Surface-Volume (.cifti)’, ‘HCP Volume (*.cifti)’

  • file_scan – a txt file that stores directories of all fMRI scans

  • file_subject_ID – a txt file that store subject ID information corresponding to fMRI scan in file_scan

  • file_subject_folder – a txt file that store subject folder names corresponding to fMRI scans in file_scan

  • file_group_ID – a txt file that store group information corresponding to fMRI scan in file_scan

  • file_Brain_Template – file directory of a brain template file in json format

  • templateFormat – ‘HCP’, ‘FreeSurfer’, ‘3D Matrix’

  • file_surfL – file that stores the surface shape information of the left hemisphere, including vertices and faces

  • file_surfR – file that stores the surface shape information of the right hemisphere, including vertices and faces

  • file_maskL – file that stores the mask information of the left hemisphere, a 1D 0-1 vector

  • file_maskR – file that stores the mask information of the right hemisphere, a 1D 0-1 vector

  • file_surfL_inflated – file that stores the inflated surface shape information of the left hemisphere, including vertices and faces

  • file_surfR_inflated – file that stores the inflated surface shape information of the right hemisphere, including vertices and faces

  • file_mask_vol – file of a mask file for volume-based data type

  • file_overlayImage – file of a background image for visualizing volume-based results

  • maskValue – 0 or 1, 0 means 0s in mask files are useful vertices, otherwise vice versa. maskValue=0 for medial wall in HCP data, and maskValue=1 for brain masks

  • method – ‘SR-NMF’ or ‘GIG-ICA’

  • K – number of FNs

  • Combine_Scan – False or True, whether to combine multiple scans for the same subject

  • file_gFN – None or a directory of a precomputed gFN in .mat format

  • FN_model_parameter – advanced parameters for FN models ‘SR-NMF’, ‘GIG-ICA’. Default is set to None, otherwise a dict. Details are in setup_SR_NMF and setup_GIG_ICA

  • init – ‘nndsvda’: NNDSVD with zeros filled with the average of X (better when sparsity is not desired) #updated on 08/03/2024 ‘random’: non-negative random matrices, scaled with: sqrt(X.mean() / n_components) ‘nndsvd’: Nonnegative Double Singular Value Decomposition (NNDSVD) initialization (better for sparseness) ‘nndsvdar’ NNDSVD with zeros filled with small random values (generally faster, less accurate alternative to NNDSVDa for when sparsity is not desired)

  • Parallel – False or True, whether to enable parallel computation

  • Computation_Mode – ‘CPU_Numpy’, ‘CPU_Torch’

  • N_Thread – positive integers, used for parallel computation

  • dataPrecision – ‘double’ or ‘single’

  • outputFormat – ‘MAT’, ‘Both’, ‘MAT’ is to save results in FN.mat and TC.mat for functional networks and time courses respectively. ‘Both’ is for both matlab format and fMRI input file format

  • synchronized_view – True or False, whether to synchronize view centers for volume data between gFNs and pFNs

  • synchronized_colorbar – True or False, whether to synchronize color bar between gFNs and pFNs

Yuncong Ma, 2/8/2024

Workflow.Workflow_Func.workflow_cluster(dir_pnet_result: str, file_scan: str, dataType='Surface', dataFormat='HCP Surface (*.cifti, *.mat)', file_subject_ID=None, file_subject_folder=None, file_group_ID=None, file_Brain_Template=None, templateFormat='HCP', file_surfL=None, file_surfR=None, file_maskL=None, file_maskR=None, file_mask_vol=None, file_overlayImage=None, maskValue=0, file_surfL_inflated=None, file_surfR_inflated=None, method='SR-NMF', K=17, init='random', sampleSize='Automatic', nBS=5, nTPoints=99999, Combine_Scan=False, file_gFN=None, sr_param=[2, 30], FN_model_parameter=None, outputFormat='Both', Computation_Mode='CPU_Torch', dataPrecision='double', synchronized_view=True, synchronized_colorbar=True, dir_pnet=None, dir_env=None, dir_python=None, submit_command='qsub -terse -j y', thread_command='-pe threaded ', memory_command='-l h_vmem=', log_command='-o ', computation_resource={'memory_bootstrap': '50G', 'memory_fusion': '10G', 'memory_pFN': '10G', 'memory_qc': '10G', 'memory_visualization': '20G', 'thread_bootstrap': 4, 'thread_fusion': 4, 'thread_pFN': 1, 'thread_qc': 1, 'thread_visualization': 1})[source]

Run the workflow of pNet, including Data Input, FN Computation, Quality Control and Visualization This function is for running pNet using multiple jobs to facilitate computation in a cluster environment This script can be re-run to restart the desired workflow from where it stops

Parameters:

  • dir_pnet_result – directory of the pNet result folder

  • dataType – ‘Surface’, ‘Volume’, ‘Surface-Volume’

  • dataFormat – ‘HCP Surface (.cifti, *.mat)’, ‘MGH Surface (.mgh)’, ‘MGZ Surface (.mgz)’, ‘Volume (.nii, .nii.gz, *.mat)’, ‘HCP Surface-Volume (.cifti)’, ‘HCP Volume (*.cifti)’

  • file_scan – a txt file that stores directories of all fMRI scans

  • file_subject_ID – a txt file that store subject ID information corresponding to fMRI scan in file_scan

  • file_subject_folder – a txt file that store subject folder names corresponding to fMRI scans in file_scan

  • file_group_ID – a txt file that store group information corresponding to fMRI scan in file_scan

  • file_Brain_Template – file directory of a brain template file in json format

  • templateFormat – ‘HCP’, ‘FreeSurfer’, ‘3D Matrix’

  • file_surfL – file that stores the surface shape information of the left hemisphere, including vertices and faces

  • file_surfR – file that stores the surface shape information of the right hemisphere, including vertices and faces

  • file_maskL – file that stores the mask information of the left hemisphere, a 1D 0-1 vector

  • file_maskR – file that stores the mask information of the right hemisphere, a 1D 0-1 vector

  • file_surfL_inflated – file that stores the inflated surface shape information of the left hemisphere, including vertices and faces

  • file_surfR_inflated – file that stores the inflated surface shape information of the right hemisphere, including vertices and faces

  • file_mask_vol – file of a mask file for volume-based data type

  • file_overlayImage – file of a background image for visualizing volume-based results

  • maskValue – 0 or 1, 0 means 0s in mask files are useful vertices, otherwise vice versa. maskValue=0 for medial wall in HCP data, and maskValue=1 for brain masks

  • K – number of FNs

  • Combine_Scan – False or True, whether to combine multiple scans for the same subject

  • file_gFN – None or a directory of a precomputed gFN in .mat format

  • method – ‘SR-NMF’ or ‘GIG-ICA’

  • FN_model_parameter – advanced parameters for FN models ‘SR-NMF’, ‘GIG-ICA’. Default is set to None, otherwise a dict. Details are in setup_SR_NMF and setup_GIG_ICA

  • outputFormat – ‘MAT’, ‘Both’, ‘MAT’ is to save results in FN.mat and TC.mat for functional networks and time courses respectively. ‘Both’ is for both matlab format and fMRI input file format

  • Computation_Mode – ‘CPU_Numpy’, ‘CPU_Torch’

  • dataPrecision – ‘double’ or ‘single’

  • synchronized_view – True or False, whether to synchronize view centers for volume data between gFNs and pFNs

  • synchronized_colorbar – True or False, whether to synchronize color bar between gFNs and pFNs

  • dir_pnet – directory of the pNet toolbox

  • dir_env – directory of the desired virtual environment

  • dir_python – absolute directory to the python folder, ex. /Users/YuncongMa/.conda/envs/pnet/bin/python

  • submit_command – command to submit a cluster job

  • thread_command – command to setup number of threads for each job

  • memory_command – command to setup memory allowance for each job

  • log_command – command to specify the logfile

  • computation_resource – a dict to specify the number of threads and memory allowance for jobs in each predefined step

Yuncong Ma, 2/12/2024

Workflow.Workflow_Func.workflow_guide()[source]

This is a step-by-step guidance for configuring a workflow of pNet in command line It will generate a Python script to run the desired workflow with comments Yuncong Ma, 2/5/2024

Workflow.Workflow_Func.workflow_simple(dir_pnet_result: str, dataType: str, dataFormat: str, file_scan: str, file_Brain_Template: str, method='SR-NMF', K=17, init='random', sampleSize='Automatic', nBS=5, nTPoints=99999, Combine_Scan=False, file_gFN=None, sr_param=[2, 30])[source]

Run the workflow of pNet, including Data Input, FN Computation, Quality Control and Visualization This is a minimal version of run_workflow for fast deployment using a single job

Parameters:

  • dir_pnet_result – directory of the pNet result folder

  • dataType – ‘Surface’, ‘Volume’

  • dataFormat – ‘HCP Surface (.cifti, *.mat)’, ‘MGH Surface (.mgh)’, ‘MGZ Surface (.mgz)’, ‘Volume (.nii, .nii.gz, *.mat)’, ‘HCP Surface-Volume (.cifti)’, ‘HCP Volume (*.cifti)’

  • file_scan – a txt file that stores directories of all fMRI scans

  • file_Brain_Template – file directory or content of a brain template file in json format

  • method – ‘SR-NMF’ or ‘GIG-ICA’

  • K – number of FNs

  • Combine_Scan – False or True, whether to combine multiple scans for the same subject

  • file_gFN – directory of a precomputed gFN in .mat format

Yuncong Ma, 2/8/2024

Module contents